This ensures that the time spent loading the cache from disk occurs with the
progress bar up. Though the progress bar stays at 0% during this period, I
think this is an improvement over the multi-second stall which occurred
previously before the progress bar came up. Ideally, we'd integrate cache
loading from disk into the progress display, but this is a first step.
(Bitbake rev: f6d0a5c219f9deb84f702450d30d868ba6271f77)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Previously, the cache was actually being loaded from disk twice whenever using
-b or -e -b. This also moves the bb_cache instance into the CookerParser, as
it's not needed by the cooker itself at all.
(Bitbake rev: dd0ec2f7b18e2a9ab06c499b775670516bd06ac8)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
This version uses a thread rather than a process, to avoid problems with
waitpid handling. This gives slightly less overall build time reduction than
the separate process for it did (this reduces a -c compile coreutils-native by
about 3 seconds, while the process reduced it by 7 seconds), however this time
is quite insignificant relative to a typical build.
The biggest issue with non-backgrounded syncing is the perceived delay before
work begins, and this resolves that without breaking anything, or so it seems.
(Bitbake rev: 5ab6c5c7b007b8c77c751582141afc07c183d672)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Rather than adding nocache items to the cache, then copying the cache and
removing them to sync it, don't add them in the first place. Also use 'with'
for the cachefile.
(Bitbake rev: 343b6f6255ad020c39e30742175a241f0859a5a6)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
This utilizes python's multiprocessing module. The default number of threads
to be used is the same as the number of available processor cores, however,
you can manually set this with the BB_NUMBER_PARSE_THREADS variable.
(Bitbake rev: c7b3ec819549e51e438d293969e205883fee725f)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
If the only recipes's we reparsed this run were those flagged as not to be
cached, there's no point in re-saving the cache, as those items won't be
included anyway.
(Bitbake rev: 1e0c4dbcbec886a30b89f8b4bb365c3c927ef609)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
This class holds the particular pieces of information about a recipe which are
needed for runqueue to do its job.
By using it, I think we improve code clarity, reduce method sizes, reduce
overuse of primitive types, and prepare for parallel parsing. In addition,
this ditches the leaky abstraction whereby bb.cache attempted to hide the
difference between cached data and a full recipe parse. This was a remnant
from the way things used to be done, and the code using it had to know the
difference anyway. If we choose to reimplement caching of the full recipes,
we can do it in bb.parse, in a completely transparent way.
(Bitbake rev: 992cc252452221f5f23575e50eb67528b2838fdb)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
range() allocates an actual list when called. xrange() is just an iterator
and creates the next range item on demand. This provides a slight
performance increase.
In python 3, range will do what xrange does currently, but the upgrade will
be handled by the 2to3 tool.
(Bitbake rev: 73b40f06444cb877a5960b2aa66abf7dacbd88f0)
Signed-off-by: Bob Foerster <robert@erafx.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
1) too spammy
2) can be implemented in the metadata instead
This reverts commit 8da9744fcdf856abebcfbe9e3bc1b8cf07bc317b.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
as noted by rp in ac00ca89a4e43cd4f38ba86455079d31be78e644
(Bitbake rev: 8da9744fcdf856abebcfbe9e3bc1b8cf07bc317b)
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
no functional changes
(Bitbake rev: e88834fb7c6821cc29c12d296f2edd51f6eb3746)
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Several fetcher need a way to strip leading slashes off a local path.
This helper-function consolidates all such occurances.
(Bitbake rev: 823a02185ed109054c6c1ae366221aaed0353f24)
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
osc had it already spelled correctly?!
(Bitbake rev: b8bb4433de7a981c6826173e926ca34705c4ac70)
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
If a recipe depends on a file, and that file is out of date, we show a
message, but if that file was removed, we do not, until now.
(Bitbake rev: 67984ba0ac2db79874541bc031f2e3e9ff7a6c32)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Parallel processes interacting with the persist_data db can quite easily
explode without this.
(Bitbake rev: b3d5432cff0ff28f4c8a5bcf10efa3e383b4fd4d)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
This avoids alot of misleading log-messages like "Removing FOO from cache"
if FOO was not in the cache and as such is not a removal candidate.
(Bitbake rev: de34a403e206867e09410ad4925c7b9cff04fee6)
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
The pysh we're using is modified, and we don't want to risk it conflicting
with one from elsewhere.
(Bitbake rev: 1cbf8a9403b4b60d59bfd90a51c3e4246ab834d6)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
There are occasions when developing when I want a package always to
grab the latest copy of a package. Witht eh CVS fetcher you can do
this by setting the `date' tag to `now'. This patch adds similar
functionality to the mercurial fetcher: if the revision to fetch is
`tip' then always grab from the server, and don't use the cached
tarball.
Oh, and I fixed a typo in the Class comment.
(Bitbake rev: 01b85608d8a37f8af66dfd80133e950120679079)
Signed-off-by: Peter Chubb <peter.chubb@nicta.com.au>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Use bb.utils.explode_deps to break up the rdepends and rrecommends strings.
This fixes the same issue which was fixed by a number of patches floating
around, but uses explode_deps rather than regular expressions.
(Bitbake rev: 83cdb23f8b89453a3527a276bd0b4deb85d63deb)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* without this fix, we get :
updating working directory
74 files updated, 0 files merged, 0 files removed, 0 files unresolved
abort: There is no Mercurial repository here (.hg not found)!
(Bitbake rev: 75ea005ac8fc05b2b3afca803d77a6b5f558efee)
Signed-off-by: Eric Bénard <eric@eukrea.com>
Tested-by: Frans Meulenbroeks <fransmeulenbroeks@gmail.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
This was inadvertantly removed when trying to reduce the amount of duplicated
information the user sees when a failure occurs.
(Bitbake rev: 850d6158ea9daa58e896fd6b258d586df797dcf4)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Ensure it raises KeyError for a missing key, this is required to use this as a
mapping in various places, e.g. as locals in an eval.
(Bitbake rev: 8d661ce0c303e8d69f17c1d095545d5ed086d1d5)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
This option will exclude the SCM metadata from tar files.
Tested with gcc where svn tar which used to be 156M for gcc 4.5
is now 77M
(Bitbake rev: f264cb6d43472525ad787b0887764ea696ec52ba)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
- Queue up any events fired to the UI before the UI exists
- At exit, check if UIs exist, and if not, flush the queue of LogRecords to
the console directly.
- When establishing a connection from the UI to the server, flush the queue of
events to the queue in the server connection, so the UI will receive them
when it begins its event loop.
(Bitbake rev: 73488aeb317ed306f2ecf99cc9d3708526a5933c)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
- Don't store key/value pairs when the value is None
- Delete the depends_cache when we're done with it
This reduces the memory usage after sync on initial parse by roughly 11.5% on
this machine.
(Bitbake rev: c7eb4c989459d182fdf9c81a627d32b7ef11626b)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
A SystemExit from a python function wasn't being raised as a FuncFailed, which
resulted in it not being caught by the exception handlers in the runqueue for
the worker process, which resulted in a SystemExit exit, rather than os._exit,
which causes all manner of problems when used in a forked process. This fixes
it by ensuring we raise a FuncFailed when seeing exceptions which aren't
instances of Exception.
(Bitbake rev: dafe92fe9f387450d9f9e9ff41c99388998b7495)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
