Commit Graph

222 Commits

Author SHA1 Message Date
Richard Purdie
6fecb1a3e0 bitbake: cooker/utils: Drop custom 'interactive' variables list and store environment in BB_ORIGENV instead
Maintining hardcoded lists of user environmental variables is hacky,
replace these with the savedenv datastore. Allow access to that
through the BB_ORIGENV variable.

(Bitbake rev: 0a99563a4ea270594fd9a61da46f9387fb79dc66)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:13:01 +00:00
Cristiana Voicu
f0aef9953d bitbake: bitbake & hob: implement functions to assure consistency for configuration files
Added a new command in bitbake to save a variable in a file; added a function
in cooker which is called by this command.

Added new command in bitbake to enable/disable data tracking.

The function saveConfigurationVar from cooker.py saves a variable in the file that
is received by argument. It checks all the operations made on that variable, using the history.
If it's the first time when it does some changes on a variable,it comments the lines where
an operation is made on it, and it sets it in a line to the end of file. If it's not
the first time(it has a comment before), it replaces the line.

Made some changes in hob to save the variables from bblayers.conf and local.conf
using the bitbake command.

[YOCTO #2934]
(Bitbake rev: 55b814ccfa413d461d12956896364ab63eed70a8)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-31 12:46:19 +00:00
Richard Purdie
dd335b0908 bitbake: utils.py: Add function for processing profile output
(Bitbake rev: 0df64810e8d40e7761cfd5059c0617dda31a6641)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-28 14:49:05 +00:00
Peter Seebach
fe30cbc2cb bitbake: bitbake: data_smart.py and friends: Track variable history
This patch adds tracking of the history of variable assignments.
The changes are predominantly localized to data_smart.py and
parse/ast.py. cooker.py and data.py are altered to display the
recorded data, and turn tracking on for the bitbake -e case.
The data.py update_data() function warns DataSmart.finalize()
to report the caller one further back up the tree.

In general, d.setVar() does what it used to do. Optionally,
arguments describing an operation may be appended; if none
are present, the operation is implicitly ignored. If it's
not ignored, it will attempt to infer missing information
(name of variable, value assigned, file and line) by examining
the traceback. This slightly elaborate process eliminates a
category of problems in which the 'var' member of the keyword
arguments dict is set, and a positional argument corresponding
to 'var' is also set. It also makes calling much simpler for
the common cases.

The resulting output gives you a pretty good picture of what
values got set, and how they got set.

RP Modifications:

a) Split from IncludeHistory to separate VariableHistory
b) Add dedicated copy function instead of deepcopy
c) Use COW for variables dict
d) Remove 'value' loginfo value and just use 'details'
e) Desensitise code for calling order (set 'op' before/after
   infer_caller_details was error prone)
f) Fix bug where ?= "" wasn't shown correctly
g) Log more set operations as some variables mysteriously acquired
   values previously
h) Standardise infer_caller_details to be triggered from .record()
   where at all possible to reduce overhead in non-enabled cases
i) Rename variable parameter names to match inference code
j) Add VariableHistory emit() function to match IncludeHistory
k) Fix handling of appendVar, prependVar and matching flag ops
l) Use ignored=True to stop logging further events where appropriate

(Bitbake rev: f00524a3729000cbcb3317fee933ac448fae5e2d)

Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-18 12:35:19 +00:00
Peter Seebach
4dd6d9139c bitbake: bitbake: data_smart.py and friends: Track file inclusions for bitbake -e
This code adds inclusion history to bitbake -e output, showing
which files were included, in what order. This doesn't completely
resolve timing questions, because it doesn't show you which lines
of a file were processed before or after a given include, but it
does let you figure out what the path was by which a particular
file ended up in your build at all.

How it works: data_smart acquires a .history member, which is an
IncludeHistory; this represents the inclusion of a file and all its
inclusions, recursively. It provides methods for including files,
for finishing inclusion (done as an __exit__), and for
dumping the whole tree.

The parser is modified to run includes inside a with() to push
and pop the include filename.

RP Modifications:

a) Split Include and Variable tracking
b) Replace deepcopy usage with dedicated copy function
c) Simplify some variable and usage

(Bitbake rev: b2dda721262da8abb7dc32d019e18fbc32ed8860)

Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-18 12:35:19 +00:00
Cristiana Voicu
f9d9cc87c5 bitbake: hob/bitbake: custom image is now using the base image
Till now, a custom image made in Hob was using only the packages from
the base image. Now it is using everything declared in the base image.
Also next to hob-image.bb, it creates another .bb file which is used
in building process. Those images are ignored by git.

[YOCTO #2601]
(Bitbake rev: 22007cf71a00fbb03b40f97201a6eb06c9aebd16)

Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-14 13:08:49 +00:00
Richard Purdie
5de7744a49 bitbake: parse/cache/cooker: Preserve order in the file inclusion list
The data returned by get_file_depends() may me used in contexts like
checksums where order is important. The current usage of sets means
that some of the checksums can change in circumstances they should not.

This patch changes to use lists, thereby removing the problem.

(Bitbake rev: a44285fc4109236ab89f7aad0a1fc9220eec19b6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-11-20 15:31:56 +00:00
Robert Yang
41125ddc86 bitbake: print clear message for "bitbake -e ASSUME_PROVIDED"
"bitbake -e ASSUME_PROVIDED" should fail, but the error message wasn't
clear enough in the past:

$ bitbake -e bzip2-native
[snip]
ERROR: Command execution failed: Traceback (most recent call last):
  File "/buildarea/lyang1/poky/bitbake/lib/bb/command.py", line 94, in
runAsyncCommand
    commandmethod(self.cmds_async, self, options)
  File "/buildarea/lyang1/poky/bitbake/lib/bb/command.py", line 323, in
showEnvironmentTarget
    command.cooker.showEnvironment(None, pkg)
  File "/buildarea/lyang1/poky/bitbake/lib/bb/cooker.py", line 325, in
showEnvironment
    fnid = taskdata.build_targets[targetid][0]
KeyError: 0
[snip]

With this patch, the massage will be:
[snip]
ERROR: bzip2-native is in ASSUME_PROVIDED
ERROR: Command execution failed: Exited with 1

Summary: There were 2 ERROR messages shown, returning a non-zero exit code.

[YOCTO #3392]

(Bitbake rev: f31447dac92454c822d4ebb7dd48e96c6c69dde4)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-11-13 14:21:16 +00:00
Richard Purdie
f5acdd87bd bitbake: cooker.py: Don't dump the environment into the console log file
Dumping the environment data into the console log files directory is
invariably not what the user wants or expects and leads to confusion
when looking at the log directory.

This change forces the logs to be disabled by default when using
the -e option.

(Bitbake rev: 5d825b31d1133e41d3982db1b94f6a30a6fb99f7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-11-01 11:46:21 +00:00
Paul Eggleton
31f0de42fc bitbake: cooker: adjust layer dependency error messages
Make these a little easier to understand.

(Bitbake rev: 84ab874c8818484d37ee438aab27486fff497705)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-09-27 16:45:27 +01:00
Trevor Woerner
f77488e565 bitbake: cooker: Clarify package/recipe wording
When the '-s' option is run, change the heading above the list of recipes
to say "Recipe Name" instead of "Package Name".

(Bitbake rev: d1c3a9de875fb488a56ab5cb1d2f8e2f24f31d69)

Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-09-20 11:56:32 +01:00
Richard Purdie
149c121209 bitbake: cooker: Fix priority for virtual recipes
When making filename comparisons for recipes for priority calculations,
we need to split off any virtual prefix. Without this, BBCLASSEXTEND
version of recipes don't follow the priority settings they should.

[YOCTO #2933]

(Bitbake rev: 055b72a230e6b0b1cababd65372c62d9ddce385e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-09-14 09:50:36 +01:00
Paul Eggleton
21816355c0 bitbake: cooker: fix handling of exceptions during exception handling
If an exception occurs during handling another exception we were
getting a useless traceback such as the following, after which
BitBake froze:

ERROR: Command execution failed: Traceback (most recent call last):
  File "/home/user/poky/poky/bitbake/lib/bb/command.py", line 84, in runAsyncCommand
    self.cooker.updateCache()
  File "/home/user/poky/poky/bitbake/lib/bb/cooker.py", line 1207, in updateCache
    if not self.parser.parse_next():
  File "/home/user/poky/poky/bitbake/lib/bb/cooker.py", line 1694, in parse_next
    logger.error('Unable to parse %s', value.recipe,
AttributeError: 'exceptions.TypeError' object has no attribute 'recipe'

Fix this to print an actual traceback of the exception and exit
gracefully (well, as gracefully as possible under the circumstances).

The general fix for [YOCTO #2977].

(Bitbake rev: 675b237a284dff84e972546774b69e2f89afb360)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-09-10 13:14:08 +01:00
Richard Purdie
ae71aa0fbb bitbake: cooker: Add explict handling of BBHandledException for parsing failures
This removes some unnecessary tracebacks

(Bitbake rev: db0ff7b4d47fce8322dd2350a2b1a6f60ef61d25)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-23 09:22:43 +01:00
Richard Purdie
4f3d48653a bitbake: cooker: Ensure parsing failures stop the build
Currently parsing failures still allow bitbake to continue on and try
and execute a build. This is clearly a bad idea and this patch adds in
more correct error handling and stops the build.

The use of sys.exit is nasty but this patches other usage in this function
so is at least consisent and its better than the current situation of
trying to execure a half parsed set of recipes. There are probably better
ways this could be improved to use to stop the build.

(Bitbake rev: 22756e9c0f1da33ba2c6e881b214577a610b7986)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-23 09:22:43 +01:00
Paul Eggleton
fddc16989d bitbake: cooker: fix traceback when using -b with skipped recipe
If a recipe is skipped during parsing for whatever reason, check and
report this as an error rather than trying to use the data that is sent
back and failing.

Fixes [YOCTO #2976].

(Bitbake rev: a324df40243fa55ccc89fd5970d46f25330d0a0d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-23 09:22:42 +01:00
Paul Eggleton
a46fac74c3 bitbake: bitbake/cooker: minor refactor of parseConfiguration()
All calls to parseConfiguration set self.status and call
self.handleCollections() afterwards, so just do this inside
parseConfiguration() itself.

(Bitbake rev: 3c2322d8a6ce15e20adb07a61aa321d884a9bcca)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-22 13:59:56 +01:00
Martin Jansa
cecf2b48be bitbake: cooker: fix OVERRIDES in BB_SIGNATURE_HANDLER (to override value from DISTRO.conf from local.conf)
We really need to pass the finalised data store into the parser init function
(and hence the siggen init function). This ensures any value changes get
passed into the correct code.

(Bitbake rev: 19efc6081c15a59bb8f5aaf8478650a2732cafe8)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-02 23:04:27 +01:00
Richard Purdie
897d7d171e bitbake: cooker.py: Ensure we track parser errors and only show bbappend issues on complete parse
Currently if any parsing failure occurs, there can be a long list of
bbappends that are "dangling" based on the fact that recipes were not
parsed. This change firstly ensures the error counter is incremented
and secondly that the bbappends list is only shown on a completed
parse list.

(Bitbake rev: 358952f0d874b869d361bbd72a5ea317bf939cd3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-24 10:35:33 +01:00
Robert Yang
041a0e8dff bitbake: bitbake/cooker: Print which pkgs would be built in -g output
This is for giving the user a clear list to show which pkg would be
built, we have the "bitbake -g", but it is not easy to read for people,
it is for "dot". Improve the "bitbake -g" to also save a pn-buildlist:

$ bitbake -g core-image-sato
...
NOTE: PN build list saved to 'pn-buildlist'
[snip]

The contents of pn-buildlist:

busybox
shadow-native
pth
sysfsutils
qemu-helper-native
curl-native
ncurses-native
gdbm
xserver-xorg
linux-libc-headers
[snip]

[YOCTO #2404]

(Bitbake rev: 18aff925aece774d0172894e25584353519ca03f)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-09 17:19:32 +01:00
Paul Eggleton
5bd11a9bf3 bitbake: bitbake: ensure -f causes dependent tasks to be re-run
If -f is specified, force dependent tasks to be re-run next time. This
works by changing the force behaviour so that instead of deleting the
task's stamp, we write a "taint" file into the stamps directory, which
will alter the taskhash randomly and thus trigger the task to re-run
next time we evaluate whether or not that should be done as well as
influencing the taskhashes of any dependent tasks so that they are
similarly re-triggered. As a bonus because we write this file as
<stamp file name>.taskname.taint, the existing code which deletes the
stamp files in OE's do_clean will already handle removing it.

This means you can now do the following:

bitbake somepackage
[ change the source code in the package's WORKDIR ]
bitbake -c compile -f somepackage
bitbake somepackage

and the result will be that all of the tasks that depend on do_compile
(do_install, do_package, etc.) will be re-run in the last step.

Note that to operate in the manner described above you need full hashing
enabled (i.e. BB_SIGNATURE_HANDLER must be set to a signature handler
that inherits from BasicHash). If this is not the case, -f will just
delete the stamp for the specified task as it did before.

This fix is required for [YOCTO #2615] and [YOCTO #2256].

(Bitbake rev: f7b55a94226f9acd985f87946e26d01bd86a35bb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-06-21 13:10:32 +01:00
Paul Eggleton
bb81db6f61 cooker: report recipe being parsed when ExpansionError occurs
When an ExpansionError occurs during parsing it is useful to know which
recipe was being parsed when it occurred.

(Bitbake rev: a2a8e8c9f5a1a30aacab696fd2b434939ee2c4e2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-30 17:23:31 +01:00
Paul Eggleton
f0809ed788 cooker: fix UnboundLocalError when exception occurs during parsing
Fix a recent regression where we see the following additional error
after an error occurs during parsing:

ERROR: Command execution failed: Traceback (most recent call last):
  File "/home/paul/poky/poky/bitbake/lib/bb/command.py", line 84, in runAsyncCommand
    self.cooker.updateCache()
  File "/home/paul/poky/poky/bitbake/lib/bb/cooker.py", line 1202, in updateCache
    if not self.parser.parse_next():
  File "/home/paul/poky/poky/bitbake/lib/bb/cooker.py", line 1672, in parse_next
    self.virtuals += len(result)
UnboundLocalError: local variable 'result' referenced before assignment

(Bitbake rev: 1ae0181ba49ccfcb2d889de5dd1d8912b9e49157)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-30 17:23:31 +01:00
Kang Kai
f28209d9d3 cooker.py: terminate the Parser processes
[Yocto 2142]

Force to exit HOB when hob is parsing recipes, the bitbake doesn't stop.
It hangs on function BitBakeServerConnection::terminate in file
server/process.py:
    else:
        self.procserver.join()
It is waiting for the children process quit.

In stage of parse recipes BBCooker spawns Parser processes as many as
cpu numbers. When quit the Parser processes they make their internal
Queue to call cancel_join_thread() to avoid block but don't work at
this time.
So force to terminate the Parser processes.

(Bitbake rev: bebef58b21bdff7a3ee1fa2449b7df19144f26fd)

Signed-off-by: Kang Kai <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-25 11:39:33 +01:00
Richard Purdie
2d3ff5e6e1 bitbake/cooker: Ensure matchFile returns full pathnames
We should always be passing full pathnames around within bitbake. If
a file was referenced as a relative path to the current working
directory, it might not get passed through the abspath call and
hence the cwd would not get added as a prefix.

This change adds a second pass at ensuring we only return absolute paths.

(Bitbake rev: 72fc62ca124a24e2dbe404a3c83a49608a7c7931)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-25 11:15:11 +01:00
Paul Eggleton
8b8be74ed2 bitbake: implement checksums for local files in SRC_URI
Gathers a list of paths to have checksums calculated at parse time, and
processes these when calculating task hashes. Checksums are cached with
the file's current mtime. Thus, changing any local file in SRC_URI will
now cause the do_fetch taskhash to change, thus forcing a rebuild.

This change adds very roughly about an 8% increase in parse time (a few
seconds) and maybe a few seconds during runqueue generation, so a fairly
moderate performance hit.

Note that since paths are resolved at parse time, this will not force
a rebuild when files are introduced which would cause that resolved path
to be different - for example, where a machine-specific version of a file
was added without otherwise changing the recipe. This will need to be
handled in a future update.

Code to hook this into the signature generator was courtesy of
Richard Purdie <richard.purdie@linuxfoundation.org>.

Implements [YOCTO #2044].

(Bitbake rev: c993b7c457f8b7776e8a5dff253bfa0724bc2cae)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-23 11:35:04 +01:00
Richard Purdie
d55db7b058 bitbake/cooker.py: Bring into sync with upstream bitbake (trivial difference)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-31 08:07:49 +01:00
Dongxiao Xu
ef2e881c1b cooker: add inherit information when generating package tree
Inherit information could be used to judge if a recipe is image type
or not.

(From Poky rev: 203f112a84b3971dc71da3dc455fd054d1c16a12)

(Bitbake rev: fe52af99918f61abec8f779efb02926b713f2aac)

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-22 14:40:40 +00:00
Richard Purdie
d3d9a37aa7 parse/ast: Expand inherit statements before splitting them
This means that statements that expand to more then one entry
such as:

CLASSES = "a b"
inherit ${CLASSES}

work correctly instead of trying to inherit a class called "a b".

(Bitbake rev: 2568e9ace6e6f483e1bf2a9ef2f4d8318d6c85b7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-15 22:56:16 +00:00
Dongxiao Xu
8c6ce63433 cooker: add package rdepends and rrecommends info
When calculating the image installation dependency, we need the
rdepends and rrecommends information of certain packages.

(Bitbake rev: 5eab31ad9e9431e27725dcefa38cbd7f2e37d394)

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-15 10:35:15 +00:00
Richard Purdie
c003a30860 cooker.py: Allow parsing shutdown to be called more than once
The parsing shutdown call can occur more than once. Currently
if this happens the code will hang. Add some code to prevent this.

[YOCTO #2039 partial]

(Bitbake rev: 7614c5bdcaf9d70614b4fad2ca02c7e6eaa92f1b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-12 15:52:33 +00:00
Richard Purdie
ff73b02a72 meta/classes: Convert to use appendVar and appendVarFlags
(From OE-Core rev: 3b57de68e70e77dbc03c0616a83a29a2e99e40b4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-05 10:36:53 -08:00
Dongxiao Xu
ca566412b1 cooker: fix calculating of depends and rdepends
Remove part of the original logic that stores pkg dependency and pkg
recomends, which will not be used in Hob.

Add the judgement for preferred fn provider of a certain package,
which maps package name to its providing recipe name.

The above approaches correct the build dependency calculation,
and they also reduced the total depend_tree size, which speed up the
tree data generation time.

(Bitbake rev: d668eb1300b3b3115964e98127b1bef554caae17)

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-01 15:51:34 +00:00
Dongxiao Xu
2d23943209 cooker.py: Fix a bug due to variable name
fnid is an outer variable and shouldn't be overrided, change the inner variable
name from 'fnid' to 'id'.

(Bitbake rev: c6dc8d6310af22c95adb06c64339e7ec9eaeb315)

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-01 15:51:34 +00:00
Dongxiao Xu
612173b48f command.py: remove the resolve parameter in generateTargetsTree
Remove the "resolve" parameter since the original resolve=False
option is no longer be used.

(Bitbake rev: dadce609149cfb09ecdc53bfe1f416a3f57a5033)

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-01 15:51:34 +00:00
Paul Eggleton
0e39107047 bitbake: implement BB_VERBOSE_LOGS
Enable configuring whether "set +x" is added to all shell tasks rather
than forcing it; this is enabled by setting BB_VERBOSE_LOGS to 1.

(Bitbake rev: 659411b6bb30e1a8355afc1c29b8170a8f2b55ac)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-28 13:47:10 +00:00
Lianhao Lu
959981da53 bitbake: Added BBINCLUDED variable.
Added new variable BBINCLUDED indicating the file dependency
information. It exposes the internal variable '__base_depends' and
'__depends'.

(Bitbake rev: af524a656fce32d01687481b86c31bef00eb9fc3)

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-28 12:27:21 +00:00
Paul Eggleton
c9e95d3363 bitbake/cooker: terminate when errors found in layer configuration
If we find an error in the layer configuration (such as an unsatisfied
item in LAYERDEPENDS) then exit by raising an exception at the end of
handleCollections() (without producing a backtrace).

(Bitbake rev: c7486a09310fe63b1aa1b7b0bb9450f306b6093b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-24 23:30:27 +00:00
Shane Wang
8fa33800ef bitbake: change for adding progress bar in Hob2.
The changes include:
- Clean some events in event.py
- Fire essential events for Hob2 to handle with more information.
- knotty changes

(Bitbake rev: 9ede881620c501574f014e600cea6947ea908ac2)

Signed-off-by: Shane Wang <shane.wang@intel.com>
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-23 22:52:17 +00:00
Dongxiao Xu
de77b9752a command.py: add resolve option for generateTargetsTree API
Currently we have generateTargetsTree API, which is used to get
dependency information. However in that tree, there will be
"virtual/xxx" in depends fields. Therefore we add the resolve option
to replace it with its real providers.

Besides, for packages that provided by multiple recipes, we will find
their preverred provider.

(Bitbake rev: 28501612efdfc6ee47576cc90deb6e897883e7f5)

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-23 22:52:17 +00:00
Dongxiao Xu
8e737db4fc cache: Use configuration's hash value to validate cache
Previously we use the file time stamp to judge if a cache is valid.
Here this commit introduce a new method, which calculates the total
hash value for a certain configuration's key/value paris, and tag
it into cache filename, for example, bb_cache.dat.xxxyyyzzz.

This mechanism also ensures the cache's correctness if user
dynamically setting variables from some frontend GUI, like HOB.

(Bitbake rev: 1c1df03a6c4717bfd5faab144c4f8bbfcbae0b57)

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-23 22:52:15 +00:00
Paul Eggleton
99d326a818 bitbake: add file and line number to ParseError
Ensure that a file and line number are reported for ParseError where
possible. This helps particularly in the case of inherit and require
which previously did not report either of these upon failure.

(Bitbake rev: f588ba69622a2df35417ced184e56c79ac1b40d5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-23 22:52:15 +00:00
Paul Eggleton
ebc0d4252a bitbake: fix parse errors not being reported
Fixes a regression introduced in BitBake rev
c9f58ef6b897d3fa5b0d23734b5f2cb3dabb057a which prevents errors during
parsing from being fully reported because BitBake shuts down before it
can print them. Move the error printing to before the shutdown in order
to fix it.

Also remove a couple of superfluous calls to sys.exit() since these
will never be called (as self.shutdown(clean=False) does this itself).

(Bitbake rev: 356c42ed00541dea2c382c57f768a4f431d22db0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-23 22:52:15 +00:00
Paul Eggleton
5210cc40db bitbake: show appropriate warnings for universe target
When building the universe target:
* Show a warning about universe likely producing errors
* Multiprovider errors are now shown as warnings

Also remove an unused "error" variable in runqueue.py.

Fixes [YOCTO #1936]

(Bitbake rev: 293c796e8a3d1f74ca1e51017b6dede261612281)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-22 20:25:30 +00:00
Caleb Crome
0aedc60b02 Fix parser hang at 0% with single-core machines
The number of threads is self.num_processes - 1 due to the range(1,
self.num_processes). Fixed by changing to range(0, self.num_processes).

(Bitbake rev: adc041fd9e3def29cdf9c1ae4849c5383bac46e5)

Signed-off-by: Caleb Crome <caleb@signalessence.com>
Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-22 20:25:30 +00:00
Christopher Larson
cf60f95d9f cooker: roll our own process pool
This fixes the hang issue encountered with parse errors. The underlying issue
seems to have been the pool.terminate(). This sends SIGTERM to each of the
multiprocessing pool's processes, however, a python process terminating in
this fashion can corrupt any queues it's interacting with, causing a number of
problems for us (e.g. the queue that sends events to the UI).

So instead of using multiprocessing's pool, we roll our own, with the ability
to cancel the work. In the very long term, the python concurrent.futures
module introduced in python 3.2 could be used to resolve this as well.

(Bitbake rev: 7c39cfd8e060cca8753ac4114775447b18e13067)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-22 20:25:29 +00:00
Christopher Larson
754847f34b Revert the switch to futures for now
Without it, we get random hangs on parse failure. With it, some folks have
seen hangs even on successful cases. The former is clearly less problematic.
This is temporary, until I can finish investigating the root causes of both
issues.

(Bitbake rev: db689a99beffea1a285cdfc74a58fe73f1666987)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-10 17:00:54 +00:00
Shane Wang
a95445cf7b bitbake: Add description into the cache and clean up cache duplication
With the addition of new cache domains, the summary, license and section
information is no longer requred in the core cache since its only used by
the hob UI. This patch removes the duplicated entries.

It also adds the DESCRIPTION field to the cache for the benefit of hob2.

(Bitbake rev: 33ffb2e99825cb643b148b3462c2d4cf33ff5f58)

Signed-off-by: Shane Wang <shane.wang@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-02 16:55:28 +00:00
Christopher Larson
38daf8b7eb cooker: add missing shutdowns to the exception handling
(Bitbake rev: c9f58ef6b897d3fa5b0d23734b5f2cb3dabb057a)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-01 15:08:42 +00:00
Paul Eggleton
526264e7d7 bitbake-layers: improve show-overlayed output
Make the following improvements to the show-overlayed subcommand:

* Show recipes that are overlayed when the version is higher or lower,
  not just when it is the same. This gives a much better picture of the
  influence each layer is having over the metadata used for building.
  This can be disabled with the -s option if you just want to see
  recipes with the same version as before.
* Default to showing name (PN), layer and version rather than the full
  path and filename. The old style formatting can be used by specifying
  the -f option.
* Mark skipped recipes as such in the output, and print them in the
  correct sorted place in the list rather than at the end
* Prefix/suffix title line with === so it can be filtered out easily in
  shell scripts if desired

(Bitbake rev: 43b473275d3cb2e60a14e4a52cdc4654b3f4e5e7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-01 15:08:41 +00:00