Commit Graph

785 Commits

Author SHA1 Message Date
Bernhard Reutner-Fischer
4df0d6adca fetch: be more pythonic
no functional changes

(Bitbake rev: e88834fb7c6821cc29c12d296f2edd51f6eb3746)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:41 +00:00
Bernhard Reutner-Fischer
b7d667f252 fetch: use os.path.join
(Bitbake rev: c360b01df18d90a513a3d61d395f905102e7568e)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:41 +00:00
Bernhard Reutner-Fischer
87b6f7d27a fetch: add common helper _strip_leading_slashes()
Several fetcher need a way to strip leading slashes off a local path.
This helper-function consolidates all such occurances.

(Bitbake rev: 823a02185ed109054c6c1ae366221aaed0353f24)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:41 +00:00
Bernhard Reutner-Fischer
d0d67a9fe2 fetch: rename suppports_srcrev to supports_srcrev
osc had it already spelled correctly?!

(Bitbake rev: b8bb4433de7a981c6826173e926ca34705c4ac70)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:41 +00:00
Chris Larson
814c80f53c cache: Add debug msg for a nonexistant dep file
If a recipe depends on a file, and that file is out of date, we show a
message, but if that file was removed, we do not, until now.

(Bitbake rev: 67984ba0ac2db79874541bc031f2e3e9ff7a6c32)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:41 +00:00
Chris Larson
84f02831a7 cache: use new style classes
(Bitbake rev: e084430446be2544dd1a6b627088f888c37cc7f0)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:40 +00:00
Chris Larson
c4d939079e cache: pyflakes/pep8/pylint cleanup
(Bitbake rev: 06420ff839ffc37de3e42474b8b0e47c4608a985)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:40 +00:00
Chris Larson
77d52cb1b8 cache: use set for clean, checked
(Bitbake rev: 3a776ef947e23b1a04bdd9e78a3a0c4616249b0e)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:40 +00:00
Chris Larson
f207bb7c93 cache: make use of defaultdict
(Bitbake rev: 7171019b11ad656d0edb979564941fcf92ab0a02)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:40 +00:00
Chris Larson
9c7c18d06f Pre-explode rundeps/runrecs in CacheData
(Bitbake rev: 0c8e0f3191252ccedb83c431c48e5c7d905e2717)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:40 +00:00
Chris Larson
3a2176fe73 persist_data: handle locked db for SELECT
Parallel processes interacting with the persist_data db can quite easily
explode without this.

(Bitbake rev: b3d5432cff0ff28f4c8a5bcf10efa3e383b4fd4d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Bernhard Reutner-Fischer
f565258368 BBHandler: use os.path in inherit()
(Bitbake rev: 5b85de2c71973ba490b95a5d9ab634635f395142)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Bernhard Reutner-Fischer
b44100eafc BBHandler: simplify supports()
(Bitbake rev: 858d704d713d15bf97053eb1374758c74b0d8874)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Bernhard Reutner-Fischer
1d8a9b8856 bb.cache: only log if the respective action was taken
This avoids alot of misleading log-messages like "Removing FOO from cache"
if FOO was not in the cache and as such is not a removal candidate.

(Bitbake rev: de34a403e206867e09410ad4925c7b9cff04fee6)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Chris Larson
7e1b709231 bb.pysh: add Case support to format_commands
(Bitbake rev: 6e24f573a0e95068eb9237c1d264ad1148b2f690)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Chris Larson
32b4bd9224 bb.pysh: fix writing pyshtables all over the place
(Bitbake rev: ff4753e362714a3c4c759c2fad8a9e5b8fe5bef5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Chris Larson
4addbd191d Move the pysh package into the bb package
The pysh we're using is modified, and we don't want to risk it conflicting
with one from elsewhere.

(Bitbake rev: 1cbf8a9403b4b60d59bfd90a51c3e4246ab834d6)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Peter Chubb
489d17596d Allow mercurial fetcher to follow tip
There are occasions when developing when I want a package always to
grab the latest copy of a package.  Witht eh CVS fetcher you can do
this by setting the `date' tag to `now'.  This patch adds similar
functionality to the mercurial fetcher: if the revision to fetch is
`tip' then always grab from the server, and don't use the cached
tarball.

Oh, and I fixed a typo in the Class comment.

(Bitbake rev: 01b85608d8a37f8af66dfd80133e950120679079)

Signed-off-by: Peter Chubb <peter.chubb@nicta.com.au>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:38 +00:00
Chris Larson
17ea152990 Fix GraphViz .dot output for rdepends and rrecs
Use bb.utils.explode_deps to break up the rdepends and rrecommends strings.
This fixes the same issue which was fixed by a number of patches floating
around, but uses explode_deps rather than regular expressions.

(Bitbake rev: 83cdb23f8b89453a3527a276bd0b4deb85d63deb)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:38 +00:00
Chris Larson
14889f9cca Make 'cache is clean' message debug
(Bitbake rev: 92637a355d55cb66de91b4314bc0e7cf1ac64ade)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:38 +00:00
Eric BENARD
3f7000cd3a bitbake: lib/bb/fetch/hg: fix fetching from a mercurial repository
* without this fix, we get :
updating working directory
74 files updated, 0 files merged, 0 files removed, 0 files unresolved
abort: There is no Mercurial repository here (.hg not found)!

(Bitbake rev: 75ea005ac8fc05b2b3afca803d77a6b5f558efee)

Signed-off-by: Eric Bénard <eric@eukrea.com>
Tested-by: Frans Meulenbroeks <fransmeulenbroeks@gmail.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:38 +00:00
Chris Larson
493f0cff71 Resurrect display of failed files
This was inadvertantly removed when trying to reduce the amount of duplicated
information the user sees when a failure occurs.

(Bitbake rev: 850d6158ea9daa58e896fd6b258d586df797dcf4)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:38 +00:00
Chris Larson
283b0a20c2 Access metadata vars as locals in python snippets
Example:
FOO = "bar"
BAR = "${@FOO + '/baz'}"

${BAR} == "bar/baz"

(Bitbake rev: 606fa1fd97cbd47a6a7ebdc7a2e6aa93a8f65cf5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:38 +00:00
Chris Larson
f3406dd288 Fix __getitem__ for DataSmart
Ensure it raises KeyError for a missing key, this is required to use this as a
mapping in various places, e.g. as locals in an eval.

(Bitbake rev: 8d661ce0c303e8d69f17c1d095545d5ed086d1d5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:37 +00:00
Khem Raj
6491ed9e2e fetchers: Use tar --exclude pattern to remove SCM files
This option will exclude the SCM metadata from tar files.

Tested with gcc where svn tar which used to be 156M for gcc 4.5
is now 77M

(Bitbake rev: f264cb6d43472525ad787b0887764ea696ec52ba)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:37 +00:00
Chris Larson
c81cd1be64 Fix syntax issue and don't include 'd' in ui_queue
(Bitbake rev: bed8e09971dc577f5443ad3d89aa14634c54eb16)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:37 +00:00
Chris Larson
26eda93337 Queue up events before the UI is spawned
- Queue up any events fired to the UI before the UI exists
- At exit, check if UIs exist, and if not, flush the queue of LogRecords to
  the console directly.
- When establishing a connection from the UI to the server, flush the queue of
  events to the queue in the server connection, so the UI will receive them
  when it begins its event loop.

(Bitbake rev: 73488aeb317ed306f2ecf99cc9d3708526a5933c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:37 +00:00
Chris Larson
30cef6bade Fix another little bug with undefined var from cache
(Bitbake rev: f29ceb22b6a79ff62287a1eb398811fd81ec5c18)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:37 +00:00
Chris Larson
a964761283 bb.cache: fix bug from last commit with loading from cache
(Bitbake rev: 99c324fe7395a44da78403c615797104413503a5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:36 +00:00
Chris Larson
3f40a2efe8 Reduce bb.cache memory usage a bit
- Don't store key/value pairs when the value is None
- Delete the depends_cache when we're done with it

This reduces the memory usage after sync on initial parse by roughly 11.5% on
this machine.

(Bitbake rev: c7eb4c989459d182fdf9c81a627d32b7ef11626b)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:36 +00:00
Chris Larson
666513982f Fix FuncFailed construction
(Bitbake rev: 085e66f9c14123ea2c0f1e34f7737cf77071f86d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:36 +00:00
Chris Larson
136a631b2e Restore FuncFailed argument compatibility
(Bitbake rev: c07cc08f7fd503ac3013ccc43c79198c4c3b7b29)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:36 +00:00
Chris Larson
311c9f5042 Only reference logfiles that exist
(Bitbake rev: 760f647ba044009150ee219869fc9dea171a7535)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:36 +00:00
Chris Larson
c90bfa57f5 Fix bitbake -k issue introduced by build exception cleanup
A SystemExit from a python function wasn't being raised as a FuncFailed, which
resulted in it not being caught by the exception handlers in the runqueue for
the worker process, which resulted in a SystemExit exit, rather than os._exit,
which causes all manner of problems when used in a forked process.  This fixes
it by ensuring we raise a FuncFailed when seeing exceptions which aren't
instances of Exception.

(Bitbake rev: dafe92fe9f387450d9f9e9ff41c99388998b7495)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:36 +00:00
Chris Larson
bfe4bec8ad Don't show tracebacks for SystemExit or KeyboardInterrupt
(Bitbake rev: d71984b3934c3dd9791c3bc00f332b79a1985a05)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:35 +00:00
Chris Larson
7afe34e2c2 Fix exit code display for task failure
Per the python documentation, os.waitpid returns the exitcode shifted up by 8
bits, and we weren't compensating, resulting in a display of 'failed with 256'
when a worker process exits with a code of 1.

(Bitbake rev: 90c2b6cb24dc9c82f0a9aa9d23f2d1ed2e6ff301)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:35 +00:00
Richard Purdie
7011ae3f78 FIXUP: runqueue bb.debug refs
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:35 +00:00
Chris Larson
9897d56861 Simplify build exception handling
- Drop EventException
- Use FuncFailed as the primary function failure exception, using TaskFailed
  for the event (leaving it up to the process running exec_{func,task} to
  display the more detailed information available in the exception).
- Switch InvalidTask to an exception rather than an event, as that's a
  critical issue.
- Reduce the number of messages shown to the user when a task fails -- they
  don't need to be told it fails 12 times.  Work remains in this area though.

(Bitbake rev: 06b742aae2b8013cbb269cc30554cff89e3a5667)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:35 +00:00
Chris Larson
1e7204a7b5 Turn Event into a new style class
(Bitbake rev: b42221cabeb1193ade134d1d3c0318203ab8eb93)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:35 +00:00
Chris Larson
39d61dc498 Work around issue with pickling of traceback objects
(Bitbake rev: 82928613256bad92fde9f4071244a53e20fc89ee)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:34 +00:00
Chris Larson
a4b195f180 Apply fix for issue where the loggers were constructed with the wrong class
(Bitbake rev: 1b21daf052c49f3126dac001712ec01ad63c5f60)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:34 +00:00
Chris Larson
9c2bcae35e Fix providerlog NameError
(Bitbake rev: 992e460f24d4da707c76d6e6d74d3684c9646279)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:34 +00:00
Chris Larson
fc27334d1a Drop 'fn' arguments to bb.msg functions
(Bitbake rev: 8341458e3d21b45db84e46bd32f8ad270000ce3c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:34 +00:00
Chris Larson
55c0b36e36 Add pending deprecation warnings to the bb.msg functions
(Bitbake rev: 60293a42b5500b6139bcd912bf294f862ef9936b)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:34 +00:00
Chris Larson
ecc68fa4fb Switch bitbake internals to use logging directly rather than bb.msg
We use a custom Logger subclass for our loggers

This logger provides:
- 'debug' method which accepts a debug level
- 'plain' method which bypasses log formatting
- 'verbose' method which is more detail than info, but less than debug

(Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Chris Larson
d3a45c7d41 Use logging in the knotty ui, and pass the log record across directly
This kills firing of Msg* events in favor of just passing along LogRecord
objects.  These objects hold more than just level and message, but can also
have exception information, so the UI can decide what to do with that.

As an aside, when using the 'none' server, this results in the log messages in
the server being displayed directly via the logging module and the UI's
handler, rather than going through the server's event queue.  As a result of
doing it this way, we have to override the event handlers of the base logger
when spawning a worker process, to ensure they log via events rather than
directly.

(Bitbake rev: c23c015cf8af1868faf293b19b80a5faf7e736a5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Chris Larson
4855548ffb Use the python logging module under the hood for bb.msg
(Bitbake rev: 47ca82397bc395b598c6b68b24cdee9e0d8a76d8)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
C Michael Sundius
3e57e63b2d Allow %20 in a file name in the SRC_URI
(Bitbake rev: f7c181a0f6ab0b4d33bf80a0e24a788de441f82b)

Signed-off-by: C Michael Sundius <msundius@sundius.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Chris Larson
2dd8c01513 Make DataSmart inherit the MutableMapping ABC
Provide __len__, __iter__, and the getitem/setitem/delitem methods, and its
mixed in versions of keys(), values(), items(), etc will automatically behave,
making the DataSmart act more like a real mapping.

(Bitbake rev: 89b5351c656d263b0ce513cee043bc046d20a01e)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:32 +00:00
Chris Larson
ebe3850bee Split out 'find next buildable task' into a separate generator function
It needs to be a generator, so scheduler subclasses have the option to skip
buildable tasks and return a later one.

(Bitbake rev: a8c61e41bc6277222e4cde667ad0b24bd1597aa0)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:32 +00:00