- Queue up any events fired to the UI before the UI exists
- At exit, check if UIs exist, and if not, flush the queue of LogRecords to
the console directly.
- When establishing a connection from the UI to the server, flush the queue of
events to the queue in the server connection, so the UI will receive them
when it begins its event loop.
(Bitbake rev: 73488aeb317ed306f2ecf99cc9d3708526a5933c)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
- Don't store key/value pairs when the value is None
- Delete the depends_cache when we're done with it
This reduces the memory usage after sync on initial parse by roughly 11.5% on
this machine.
(Bitbake rev: c7eb4c989459d182fdf9c81a627d32b7ef11626b)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
A SystemExit from a python function wasn't being raised as a FuncFailed, which
resulted in it not being caught by the exception handlers in the runqueue for
the worker process, which resulted in a SystemExit exit, rather than os._exit,
which causes all manner of problems when used in a forked process. This fixes
it by ensuring we raise a FuncFailed when seeing exceptions which aren't
instances of Exception.
(Bitbake rev: dafe92fe9f387450d9f9e9ff41c99388998b7495)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Per the python documentation, os.waitpid returns the exitcode shifted up by 8
bits, and we weren't compensating, resulting in a display of 'failed with 256'
when a worker process exits with a code of 1.
(Bitbake rev: 90c2b6cb24dc9c82f0a9aa9d23f2d1ed2e6ff301)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
- Drop EventException
- Use FuncFailed as the primary function failure exception, using TaskFailed
for the event (leaving it up to the process running exec_{func,task} to
display the more detailed information available in the exception).
- Switch InvalidTask to an exception rather than an event, as that's a
critical issue.
- Reduce the number of messages shown to the user when a task fails -- they
don't need to be told it fails 12 times. Work remains in this area though.
(Bitbake rev: 06b742aae2b8013cbb269cc30554cff89e3a5667)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
We use a custom Logger subclass for our loggers
This logger provides:
- 'debug' method which accepts a debug level
- 'plain' method which bypasses log formatting
- 'verbose' method which is more detail than info, but less than debug
(Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
This kills firing of Msg* events in favor of just passing along LogRecord
objects. These objects hold more than just level and message, but can also
have exception information, so the UI can decide what to do with that.
As an aside, when using the 'none' server, this results in the log messages in
the server being displayed directly via the logging module and the UI's
handler, rather than going through the server's event queue. As a result of
doing it this way, we have to override the event handlers of the base logger
when spawning a worker process, to ensure they log via events rather than
directly.
(Bitbake rev: c23c015cf8af1868faf293b19b80a5faf7e736a5)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Only shows warnings that come from bb, oe, or <string>
(Bitbake rev: 57018687f60b222ab220dd904c4bf870780171e9)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Provide __len__, __iter__, and the getitem/setitem/delitem methods, and its
mixed in versions of keys(), values(), items(), etc will automatically behave,
making the DataSmart act more like a real mapping.
(Bitbake rev: 89b5351c656d263b0ce513cee043bc046d20a01e)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
It needs to be a generator, so scheduler subclasses have the option to skip
buildable tasks and return a later one.
(Bitbake rev: a8c61e41bc6277222e4cde667ad0b24bd1597aa0)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
If you create a runqueue scheduler class in a python module, available in the
usual python search path, you can now make it available to bitbake via the
BB_SCHEDULERS variable, and the user can then select it as they select any
other scheduler.
Example usage:
In a test.py I placed appropriately:
import bb.runqueue
class TestScheduler(bb.runqueue.RunQueueScheduler):
name = "myscheduler"
In local.conf, to make it available and select it:
BB_SCHEDULERS = "test.TestScheduler"
BB_SCHEDULER = "myscheduler"
(Bitbake rev: 4dd38d5cfb80f9bb72bc41a629c3320b38f7314d)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
SIGINT should be from the user, not a script. It also doesn't work as
reliably to shut down processes, as it's not always interpreted as a
termination request. In addition, it causes KeyboardInterrupt exceptions in
the worker processes, which can interfere with our exception handling.
(Bitbake rev: e5f6e0e9de4c6d1dfdd269d2bf7f83c00c415a27)
Signed-off-by: Chris Larson <clarson@kergoth.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Bug 606 report that if $DL_DIR is read-only, do_fetch will
simply hang without any error message.
The root cause is that: bb.fetch.go()->bb.utils.lockfile()
will try to lock file ${DL_DIR}/xxxxx.lock. Since ${DL_DIR}
is read-only, it will cause IOError exception. Although
lockfile() can catch the exception, currently code simply
ignore all the exception and continue the loop. it make
sense if the exception is caused by locking contention,
but in the read-only $DL_DIR case, it cause endless waiting
unfortunately.
So this patch add read-only check for lockfile to avoid the
silent hang.
Fix [BUGID #606]
Signed-off-by: Yu Ke <ke.yu@intel.com>
Stupdi typo/thinko from me had depexp exiting once recipes had parsed
as I'd used a return the while loop where I'd meant a continue...
Signed-off-by: Joshua Lock <josh@linux.intel.com>
In previous exec() model, cooker is re-initialized from scratch with environmental
variable exported accordingly. Now in fork() model, environmental variables are
not exported again, and thus original method to export BB_TASKHASH doesn't apply
now which breaks all sstate packages. Now we can set data variable directly instead.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
If the checksum check failed, the .md5 stamp file would still have been created
meaning subsequent builds would proceed with the corrupt file. Reorder the calls
to avoid this. Also raise a specific error for the checksum not specified error
case.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
This patch add the per-recipe SRC_URI checksum verification.
- SRC_URI format
The format of SRC_URI checksum follow OE definition:
1. SRC_URI has single src
SRC_URI = "http://some.domain/file.tar.gz"
SRC_URI[md5sum] = "xxxxxxxxxxxxxxx"
SRC_URI[sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx"
2. SRC_URI has multiple src, every src need specify name
SRC_URI = "http://some.domain/file1.tar.gz;name=name1 \
http://some.domain/file2.tar.gz;name=name2 "
SRC_URI[name1.md5sum] = "xxxxxxxxxxxxxxx"
SRC_URI[name1.sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx"
SRC_URI[name2.md5sum] = "xxxxxxxxxxxxxxx"
SRC_URI[name2.sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx"
- SRC_URI checking invocation:
the checksum checking is invoked in do_fetch phase,
so it can be invoked manually by
# bitbake -f -c fetch <recipe_name>
if recipes has no SRC_URI checksum item, bitbake will show warning:
"
WARNING: Missing SRC_URI checksum for xxxx.tar.gz, consider to add
SRC_URI[md5sum] = "5c69f16d452b0bb3d44bc3c10556c072"
SRC_URI[sha256sum] = "f4e0ada8d4d516bbb8600a3ee7d9046c9c79e38cd781df9ffc46d8f16acd1768"
"
thus recipe author can add it to recpie file after SRC_URI
- control variable BB_STRICT_CHECKSUM
when SRC_URI checksum is missing, this variable decide pass or not
if BB_STRICT_CHECKSUM = "1", bitbake should fatal in this case, otherwise bitbake just pass
Signed-off-by: Yu Ke <ke.yu@intel.com>
sstate hash validation is done at initialization of RunQueueExecuteScenequeue.
However the index of 'valid' list returned from the validation doesn't
correspond to setscene task ID. It's just an intermediate namespace between
runqueue and sstate hash func. Use it as setscene task ID fully mess the flow.
Previously this doesn't cause trouble because all setscene tasks are passed. Commit
58396a5d24 add 'noexec' concept to setscene
tasks which grabs some tasks out of the list and thus trigger this problem
Without this fix there're ~50 recipes (gzip-native, glib, ...) rebuilt weirdly
with a minimal build, even though existing sstate packages could accelerate them.
there's another typo using wrong task ID in a debug message which further hide
this issue
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
