The commit [991f92b4d bb.ui: delete __init__.py to make bb.ui a
namespace package] caused `bitbake -h' failed
...
$ bitbake -h
Traceback (most recent call last):
File "/buildarea/raid5/hjia/community/poky/bitbake/bin/bitbake", line 35, in <module>
sys.exit(bitbake_main(BitBakeConfigParameters(sys.argv),
File "/buildarea/raid5/hjia/community/poky/bitbake/lib/bb/cookerdata.py", line 27, in __init__
self.options, targets = self.parseCommandLine(argv or sys.argv)
File "/buildarea/raid5/hjia/community/poky/bitbake/lib/bb/main.py", line 297, in parseCommandLine
options, targets = parser.parse_args(argv)
File "/usr/lib64/python3.6/optparse.py", line 1387, in parse_args
stop = self._process_args(largs, rargs, values)
File "/usr/lib64/python3.6/optparse.py", line 1431, in _process_args
self._process_short_opts(rargs, values)
File "/usr/lib64/python3.6/optparse.py", line 1536, in _process_short_opts
option.process(opt, value, values, self)
File "/usr/lib64/python3.6/optparse.py", line 785, in process
self.action, self.dest, opt, value, values, parser)
File "/usr/lib64/python3.6/optparse.py", line 807, in take_action
parser.print_help()
File "/usr/lib64/python3.6/optparse.py", line 1647, in print_help
file.write(self.format_help())
File "/usr/lib64/python3.6/optparse.py", line 1635, in format_help
result.append(self.format_option_help(formatter))
File "/usr/lib64/python3.6/optparse.py", line 1615, in format_option_help
result.append(OptionContainer.format_option_help(self, formatter))
File "/usr/lib64/python3.6/optparse.py", line 1061, in format_option_help
result.append(formatter.format_option(option))
File "/buildarea/raid5/hjia/community/poky/bitbake/lib/bb/main.py", line 54, in format_option
valid_uis = list_extension_modules(bb.ui, 'main')
File "/buildarea/raid5/hjia/community/poky/bitbake/lib/bb/main.py", line 72, in list_extension_modules
pkgdir = os.path.dirname(pkg.__file__)
AttributeError: module 'bb.ui' has no attribute '__file__'
...
This reverts commit 991f92b4d15b0571b6a540964e5216d1b9728539
(Bitbake rev: 22b8c53205f8915b33d1e0ad6a666dcacc01491d)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster has been failing to start correctly when in interactive mode. The
issue is due to setEventMask being called (which triggers parsing) before
the environment has been sent from the UI over to the server. This means
PATH isn't setup, which causes the sanity checks on HOSTTOOLS to fail
in base.bbclass.
The fix is to ensure the environment is sent to the server before
other commands are run.
The pain in debugging this highlights other improvements to the logging
are needed.
[YOCTO #14079]
Reviewed-by: Tim Orling <timothy.t.orling@linux.intel.com>
Tested-by: Tim Orling <timothy.t.orling@linux.intel.com>
(Bitbake rev: a8efff5c83cd5a25f4b6720e6414a7aa35d04bc7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After parseprogress.finish() it was intended to
set parseprogress to None, but a typo means this
is not happening.
(Bitbake rev: f504d6f6598f62aa20fbf69c30fea95569858edb)
Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The introspected API for GtkTreeModel and friends had some unexpected
quirks which have now been fixed, see[1] for details. However, for
example Ubuntu 20.04 has the fixed GTK but not an updated pygobject which
means taskexp raises an exception on startup.
Solve by manually looking at what functions are present and calling the
right one.
[ YOCTO #14055 ]
[1] 9cdbc56fba
(Bitbake rev: ac7d1114a7e99e6efd6a37b03d170faf678513fb)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new server startup code means exceptions can happen when we aren't
setup to show them to the user correctly, leading to ugly tracebacks.
Add in some special case handling of BBHandledException to at least
ensure that common case doesn't traceback and the user sees meaningful
output.
In the future, the logging setup can likely be improved, as can the way
runCommand handles exceptions, they all should likely become real
exceptions again on the UI side.
[YOCTO #14022]
[YOCTO #14033]
(Bitbake rev: 6059d0e77f60ddb679049bd34478f41b1ab7995d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Correctly import, and inherit functions, and variables.
Also fix some typos and remove some Python 2 code that isn't recognised.
(Bitbake rev: b0c807be5c2170c9481c1a04d4c11972135d7dc5)
Signed-off-by: Frazer Clews <frazerleslieclews@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding back gtk objects import.
Fix bug introduce when adding validation on gtk import.
(Bitbake rev: 765be5ef60668f8a1cfbcba248f4995725807196)
Signed-off-by: David Khouya <dakhouya@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running bitbake -g -u taskexp without having gi python module or
and invalid gtk version, bitbake fails with a stack trace.
In case of import or version error, bitbake should exit with an error
message instead of a stack trace.
(Bitbake rev: 2a2c507f239b047f34765312df4168030e38b90d)
Signed-off-by: David Khouya <dakhouya@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The logging module provides a shutdown() function that does the same
thing in a much better way
(Bitbake rev: 970cd2fc4f0bbc93069dee5a15a608dd76081c67)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The BitBake.verbconsole needs to be treated like a console output logger
(meaning that the TerminalFilter attaches an InteractConsoleLogFilter to
it), even if it's not directly attached to the root 'BitBake' logger.
First, assign a special "is_console" property to the relevant handlers,
then look for the property in the handlers from the configuration object
return by bb.msg.setLoggingConfig(). Finally, pass the list of all
handlers to the TerminalFilter object; it doesn't care about the
difference between console and errconsole, so pass all the relevant
handlers as a list.
This fixes cases where the console output was corrupted when messages
were sent to the 'BitBake.verbconsole' handler.
(Bitbake rev: 2010be588c74a99256df7b565a309c84c2973546)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updates hash equivalence logging so that the interesting VERBOSE
messages are always logged to the consolelog file so that issues in
individual user builds can be diagnosed. The autobuilder logging config
then updates this so that they also are shown on stdout, since the
consolelog file is not capture there.
In order to facilitate this, 2 new logging handlers were added,
"BitBake.verbconsole" and "BitBake.verbconsolelog". Neither of these
handlers are attached to anything by default, but they will log any
messages that wouldn't otherwise be logged by the normal console or
consolelog handlers. Users can attach whatever loggers the desire to
this handler to get them to appear on the console or in the consolelog,
as demonstrated by the autobuilderlog.json file.
(Bitbake rev: 766587c0baaaeb5cda3e9454395edbb70e33f756)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds code to close all loggers when bitbake exits. This prevents
unclosed file ResourceWarnings. A form of this closing existed
previously, but was removed in the new logging code.
(Bitbake rev: b3f3779adf63c0d970462a558a6205da1d30c0ed)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sets up logging in knotty to use python's structured logging config and
the bb.msg.setLoggingConfig() helper to setup logging. This allows the
user to specify additional logging mechanism in a config file with
BB_LOGCONFIG
(Bitbake rev: 646a68a49364b50a42168b4b16308f7217eec0dc)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Passing around the log formatter variable was unnecessary since the log
levels of interest can be accesses as class members of
bb.msg.BBLogFormatter. Switching to do this will make using the
structured python logging much easier, since it can be difficult to
extract out the formatter for a specific handler.
(Bitbake rev: c1c867df24b4ef204027d485acac7c75c63f2bc0)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a comment section that can be easily uncommented to enable dumping
the logging tree. This module is extremely useful for debugging issued
with logging configuration
(Bitbake rev: 30461310915f911b80f92e03df694af7c1eb1f46)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Handles the log messages from the bitbake server with the specific
logger that the event originated from. This allows hierarchical logging
configurations to work as expected.
(Bitbake rev: 9624d42133e024fd044d0d089c7017ed53eed874)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This matches the other bb.command.Command* events and without it,
running `bitbake --revisions-changed` will hang indefinitely if there
are changed revisions.
(Bitbake rev: 40520d229c8ea51ee9784184ab5d13a82dd1eb61)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
amend the code to handle singleton comparisons properly so it only checks
if they only refer to the same object or not, and not bother
comparing the values.
(Bitbake rev: b809a6812aa15a8a9af97bc382cc4b19571e6bfc)
Signed-off-by: Frazer Clews <frazer.clews@codethink.co.uk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
removed unused imports which made the code harder to read, and slightly
but less efficient
(Bitbake rev: 4367692a932ac135c5aa4f9f2a4e4f0150f76697)
Signed-off-by: Frazer Clews <frazer.clews@codethink.co.uk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When creating a new progress bar (using BBProgress), a colon was
appended to the supplied message. However, when updating the message,
no colon was appended.
Change this so that the colon is instead part of the widgets that make
up the progress bar so that it does not matter when and how the
message is updated, it always displays the same.
(Bitbake rev: 08f35c04f6e1ce4c4ca5c2bef4cd8a192e12e682)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the introduction of the hash equivalence server, the progress bar
for "Checking sstate mirror object availability" is shown repeatedly
while the tasks are being executed. If the footer is not hidden then,
it will be moved up one line every time, creating a messy interface.
(Bitbake rev: 56b5ec4c2b3e658e73ca6c3a12feeb96df0977fb)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've seen cases where a task can execute with a given pid, complete
and a new task can start using the same pid before the UI handler has
had time to adapt.
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/qemux86-alt/build/bitbake/lib/bb/ui/knotty.py", line 484, in main
helper.eventHandler(event)
File "/home/pokybuild/yocto-worker/qemux86-alt/build/bitbake/lib/bb/ui/uihelper.py", line 30, in eventHandler
del self.running_tasks[event.pid]
KeyError: 13490
This means using pids to match up events on the UI side is a bad
idea. Change the code to use task ids instead. There is a small
amount of fuzzy matching for the progress information since there
is no task information there and we don't want the overhead of a task
ID in every event, however since pid reuse is unlikely, we can live
with a progress bar not quite working properly in a corner case like
this.
[YOCTO #13667]
(Bitbake rev: e427eafa1bb04008d12100ccc5c862122bba53e0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of the objects that bitbake reports to Toaster as dependencies to packages
are known objects that are not packages, for example library files and kernel
modules. In the Toaster logs, mark these as "Info" instead of "Warning".
[YOCTO #13386]
(Bitbake rev: 0d66f644d647900e8f5afa526a6d9cee687c41cc)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
$ rm -fr tmp-glibc/cache/default-glibc/qemux86/x86_64/bb_cache.dat* ; bitbake -p
Press the first Ctrl-C when the parsing process is at about 50%:
Keyboard Interrupt, closing down...
Then presss the second Ctrl-C:
File "/path/to/bitbake/bitbake/lib/bb/ui/knotty.py", line 619, in main
event = eventHandler.waitEvent(0.25)
File "/path/to/bitbake/lib/bb/server/process.py", line 591, in waitEvent
self.eventQueueNotify.wait(delay)
File "/usr/lib/python3.5/threading.py", line 549, in wait
signaled = self._cond.wait(timeout)
File "/usr/lib/python3.5/threading.py", line 297, in wait
gotit = waiter.acquire(True, timeout)
KeyboardInterrupt
Capture the second KeyboardInterrupt during stateShutdown is running can fix
the problem. There may be still tracebacks for the third KeyboardInterrupt, but
I'm leaning to not fix it since we aimed for supporting 2 KeyboardInterrupts
only.
(Bitbake rev: 8c26b451f22193ef1c544e2017cc84515566c1b8)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the merge of the scenequeue with real tasks, this now confuses the
statistics. The real tasks are the definitive progress so monitor only
those.
(Bitbake rev: 20956b508a082224139c8f56b68299edff6e0443)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whilst this class has existed for years, it doesn't have any
users and has a questionable interface. Drop it to allow for further
simplification and changes.
(Bitbake rev: 3ab51764f7965d696bb2c5a872bf161473df4289)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are much better ways to handle this and most editors shouldn't need this
in modern times, drop the noise from the files. Its not consitently applied
anyway.
(Bitbake rev: 5e43070e3087d09aea2f459b033d035c5ef747d0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the introduction of SPDX-License-Identifier headers, we don't need a ton
of header boilerplate in every file. Simplify the files and rely on the top
level for the full licence text.
(Bitbake rev: 695d84397b68cc003186e22f395caa378b06bc75)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds the SPDX-License-Identifier license headers to the majority of
our source files to make it clearer exactly which license files are under.
The bulk of the files are under GPL v2.0 with one found to be under V2.0
or later, some under MIT and some have dual license. There are some files
which are potentially harder to classify where we've imported upstream code
and those can be handled specifically in later commits.
The COPYING file is replaced with LICENSE.X files which contain the full
license texts.
(Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
CI systems like jenkins and buildbot will timeout applications which haven't had console output
in some period of time. Add 'keepalive' output to knotty which gives output every 5000s if not
other output was made and tasks are still running. This reduces some problems encountered
with our CI testing.
(Bitbake rev: aa4f31e5741dd98acec73f16f6028e52f4c22d6f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A task's runtime is currently printed in seconds. Change it to
include minutes and hours for easier reading.
(Bitbake rev: c593ae5ec9fecd4bde823948024e4d56314a60ce)
Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Searching in the task explorer requires one to focus the task list. A
readily visible and focused search bar makes searching intuitive.
(Bitbake rev: 43f8a23d56995f552f98a666e86b6cc124e235a4)
Signed-off-by: Tobias Olausson <tol@hms.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The method 'store_log_event' in 'buildinfohelper.py' always puts log
messages from CLI builds into the backlog but never takes them out.
The "close" method now forces all backlogged CLI events to be registered.
[YOCTO #12813]
(Bitbake rev: 6458cc4234337f551dfe189b6f8800d8da886c24)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow for "SinglePackageInfo" events that do not include package data,
for example OPKGN equal 'lib32-*' or 'lib64-*'.
[YOCTO #12204]
(Bitbake rev: 567f072ff260614cde3da220a40a95d5a8b9ab92)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a task fails during build_targets(), we need to print out the log
lines as knotty does or the user will be missing information about the
failure.
(This should get some deeper refactoring, but now isn't the time for
that.)
(Bitbake rev: 24879df071d4803db3d39ae1d5cad852daa92f28)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If for example you run:
bitbake -r somefile.inc
rm somefile.inc
bitbake -e
bitbake will crash with an error about not being able to find somefile.inc. This
is because it tries to reparse the base config for the early getVariable requests
before it sees the updated missing -r option.
Send the updateConfig command earlier to avoid this.
(Bitbake rev: a38164620ebdc770690c5f39ff9ed69d3f82719e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The underlying model is already sorted for use in the other view,
add a sorting model for the 'Dependent Tasks' view.
(Bitbake rev: 27ca94c33234f0ef9753f8285213dde2871a3fcf)
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This isn't useful for knotty itself, but for use from tinfoil
in case we can't get access to either the console or errconsole, allow
either to be unspecified (None).
(Bitbake rev: 7544de437fc66b81502ecdb5db859182c45827cb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is useful for the caller to know whether the uihelper has handled the
event passed so that it can skip other event handling code if so.
(Bitbake rev: be498abfbbb19bdd31e5b53713a74049007e3737)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We really ought to have just one place where the string representation
of these events is produced. This doesn't take any real control away
from the UI - if an alternative representation is desired, that can
still be made.
(Bitbake rev: cb15db2a799be6d8eab9a2a43a9a573f89229cff)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To quote Paul:
Not that long ago we added a prefix to logged messages to allow us to see
where the message has been generated (recipe / task). This is undoubtedly
useful for errors and warnings, however, I'm not sure it's really appropriate
for bb.plain(). As an example, see the output for -c listtasks now:
...
NOTE: Executing RunQueue Tasks
nodejs-native-4.5.0-r0 do_listtasks: do_addto_recipe_sysroot
nodejs-native-4.5.0-r0 do_listtasks: do_build Default task for a recipe - depends on all other normal tasks required to 'build' a recipe
nodejs-native-4.5.0-r0 do_listtasks: do_checklicense
nodejs-native-4.5.0-r0 do_listtasks: do_checklicenseall
nodejs-native-4.5.0-r0 do_listtasks: do_checkpkg
...
This patch excludes PLAIN messages from this prefixing making the log output
neater.
[YOCTO #11457]
(Bitbake rev: 4a14b44b3e4fad3a3e5e53461aa8ba9929a515b8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are some cases where the metadata needs to be aware a multiconfig build
is happening and have access to the multiconfig data stores to merge data into
the common build. This adds such an event allowing access to these datastores.
(Bitbake rev: 160e47f5df90850e64dcb857c81a5039abc9235f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The MockEvent needs to not only stand in for Toaster and Bitbake quick events,
it also needs to stand in for LogRecord, and for that it needs to provide
the new getMessage method.
[YOCTO #11440]
(Bitbake rev: d1ac359d460b1abe9815f323b3fd4cd0231cde6c)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For layers directories with sub-layers (for example meta-intel),
Toaster breaks when trying to exact match a dependency event for
that sub-layer against the top level layer directory paths.
Given that top the level layer directory paths are unique, adding
a test to see if the dependency path is a subset of a top level
path is also unique.
Also, since a warning was issued the processing should not fail
on the assert but instead should continue and gather the recipes
and tasks.
[YOCTO #11149]
(Bitbake rev: 6f1a42af19220a6a8d75790893da995e84ecd7aa)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
tinfoil sets up its own logger by default, but if and when we initialise
the UI (by default knotty) will also set one up, leading to duplicated
messages specifically from tasks. To avoid this, rather than adding some
kind of parameter, just check if there is already a logger outputting to
stdout/stderr and if so, skip adding our own.
Part of the fix for [YOCTO #11275].
(Bitbake rev: 66d866745f35468d1540a793d07e3a401298b84b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
