The whitelist checks for BB_SETSCENE_ENFORCE were running for every call
to execute(). Since the task list doesn't change for each call into
execute, the checks only need to be ran once.
[YOCTO #10369]
(Bitbake rev: f65e631ab6705dfd9188f19ee423eca33bca7d7d)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this the UI just sits there doing nothing. Showing an
infobar in-UI would be nicer but not much more useful since currently
user couldn't do anything in-UI to fix the situation. Implementation
is based on the one in knotty.
Fixes [YOCTO #9288]
(Bitbake rev: eee9231a543f1d0b9ef3cd8377fc46fd23afb97b)
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Either using the memres script or the bitbake call with --server-only
if the port is a string instead of a number then the process hangs
indefinitely causing a loop that never ends.
Add a check at the beginning for the port being a number otherwise
show an error message and exit cleanly.
[YOCTO #10397]
(Bitbake rev: 35927a98daeeb854ef5782e900206af6cd96b3d7)
Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some task id manipulations were suboptimal:
* taskfn_fromtid and fn_from_tid were effectively the same function
* many calls to split_tid(), then taskfn_fromtid()
* taskfn_fromtid() called split_tid() internally
This patch adds split_tid_mcfn() to replace split_tid() and returns the
"taskfn" variant being used in many places. We update all core calls
to the new function and ignore the return values we don't need since the
function call overhead of the split_tid wrapper is higher than ignoring
a return value.
The one remaining standalone use of taskfn_fromtid is replaced with
fn_from_tid. I couldn't see any external usage so it was dropped.
There is external usage of split_tid so a wrapper remains for it.
Combined together these changes should improve some of the runqueue task
manipulation performance.
(Bitbake rev: 1bf2ef874fbe47f1320007efa0bdeef8d630b8a1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
`if w in self.rq.worker` when w *is* self.rq.worker doesn't make a great deal
of sense, and results in this error:
File ".../poky/bitbake/lib/bb/runqueue.py", line 2372, in runQueuePipe.read():
name = None
> if w in self.rq.worker:
name = "Worker"
TypeError: unhashable type: 'dict'
Most likely this was meant to be 'is' rather than 'in', but rather than
checking after the fact, just include the name in the iteration, instead.
While we're here, also clean up and fix the broken error message.
(Bitbake rev: 267e025cad44c8bd0fb157f1f7a2e08df117ba84)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the ability to pass default signal handler for SIGWINCH in BBProgress
because with multiple instace of BBProgress the original signal handler
set by TerminalFilter (sigwinch_handle) is lost.
This is a fix for stack trace due to multiple async calls of ProgressBar
_handle_resize (ioctl to terminal fd), see:
NOTE: Executing SetScene Tasks
Fatal Python error: Cannot recover from stack overflow.
Current thread 0x00007f70a4793700 (most recent call first):
File
"/home/alimonb/repos/poky/bitbake/lib/progressbar/progressbar.py", line
183 in _handle_resize
File "/home/alimonb/repos/poky/bitbake/lib/bb/ui/knotty.py", line 58
in _handle_resize
File "/home/alimonb/repos/poky/bitbake/lib/bb/ui/knotty.py", line 60
in _handle_resize
...
File "/home/alimonb/repos/poky/bitbake/lib/bb/ui/knotty.py", line 60
in _handle_resize
...
Aborted
(Bitbake rev: 812bd49cb569379ee90d5be28a4b6e60645f1e54)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the correct method to get a message value from the LogMessage object
rather than constructing it ourselves which is not recommended. This
causes an exception when the msg contains a '%' such as when there are
wildcards in file names (something2.%.bbappends)
(Bitbake rev: 11b3b6a7087554d14a2812a9ae463dce740b879e)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't always want a traceback when an exception is raised by the
TaskStarted event handler. Silently return if we get a SystemExit or
HandledException, and print the error and return for FuncFailed.
This is done via a separate try/catch block, to avoid firing TaskFailed if all
the TaskStarted event handlers didn't complete, otherwise the bitbake UIs get
unhappy.
(Bitbake rev: ca5b788280ad4303cc08a376e847cbbeda31970c)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use logger.addHandler(), rather than assigning an array of Handlers
to the loggers handlers property directly, to avoid a warning from
Python 3 about unclosed files:
$ bitbake
Nothing to do. Use 'bitbake world' to build everything, or run 'bitbake --help' for usage information.
WARNING: /home/joshuagl/Projects/poky/bitbake/lib/bb/event.py:143: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/joshuagl/Projects/poky/build/tmp/log/cooker/qemux86/20161004094928.log' mode='a' encoding='UTF-8'>
logger.handlers = [stdout]
(Bitbake rev: 1e23b1f1a80066223b98e18b163840051ac74944)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In some cases there is a need to fire bb events into multiple
python threads so locking is needed (writing to a fd/socket).
Adding a helper functions for disable/enable by request to avoid
overhead.
[YOCTO #10330]
(Bitbake rev: a583dc0b296415ec904c081c4de96ceef46732a8)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The value used for exclusion was always being expanded. This is actually
a bad idea since in most cases you'd want to exclude an unexpanded
value and makes it impossible to use the variable as intended.
This adjusts things so the value is not expanded and we can correctly
remove things from checksums much more easily.
(Bitbake rev: 81bc8201c475d2b6bef0168573915ad0140f6dad)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Error message
ERROR: Unprocessed MetadataEvent <bb.event.MetadataEvent object at 0x7f750e671a58>
doesn't give a lot of information about the event. It just prints
event object, which is always bb.event.MetadataEvent.
Including event type into the error message should make it more
informative:
ERROR: Unprocessed MetadataEvent TaskArtifacts
(Bitbake rev: 603c7c13536d3fa1786270e863688c1d2e511196)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New MetadataEvent 'TaskArtifacts' causes this error.
Processing of this event will hopefully be implemented in future.
For now it should be enough to just skip it.
(Bitbake rev: 114a3fe3f23ef09782c5aa18f425d0d0dbdfdd35)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to the bug in processing EnvironmentError exception,
toasterui ignores it. As EnvironmentError is a base for OSError
and IOError this means that all OSError and IOError exceptions
were silently ignored.
(Bitbake rev: c8f4ca008bf9396b0ed45d44bfe2220c82a614a9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Buildinfohelper assumes that all files mentioned in
manifest exist in deploy/ directory, which is not always
the case. Toaster crashes with OSError trying to
call os.stat on non-existing file.
Checking if file exists before processing it should
fix this.
[YOCTO #10185]
(Bitbake rev: 54565e7ca84d2722a2454e7fa52cda564b28b527)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the worker fails to launch, ensure the task is shown as failed rather
than a confusing "all succeeded" message.
Patch from Juro Bystricky
[YOCTO #10335]
(Bitbake rev: 0e9a2ff96d138641501874a1cd7aa6cc7e94d727)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When trying to obtain the current directory from a directory
that does not exist anymore, an exception is raised.
This patch handles such exception.
[YOCTO #10331]
(Bitbake rev: 4bcf77589312d9936340d8c308006c2fc9baf67c)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When calculating how many lines we'd printed we weren't properly taking
the progress bars into account, with the result that sometimes if the
last line printed on the terminal wrapped to the next line (which is
possible) we backed up less lines than we should have.
Additionally, we should always print a newline after updating the
progress bar - there's no need to check if there wasn't output (there
always will be courtesy of our overridden _need_update()) and we now
allow the line to wrap so we don't need to check the other condition
either.
Hopefully this will fix [YOCTO #10046].
(Bitbake rev: 326d18d96faf02675ba34ad3c3a20cd424b39b91)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its often useful to know how long a task has been running for. This patch
adds that information to the task display, updating every 5s if there
were no other updates so the user can see how long tasks have been running
for.
[YOCTO #9737]
(Bitbake rev: 6c42025e5dd7761213be3f82f3252a7892d2239d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A "bitbake world -c unpack" currently breaks as not all tasks have an
unpack task. This change allows addition of world targets only if the
specified task exists which makes certain commands possible when otherwise
you just get errors which can't easily be avoided.
(Bitbake rev: ca4f5e6d01b5c8cf315f59bc86194d63c0d3d042)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if you use the subdir parameter in a SRC_URI and pass an absolute path
then it gets appended to the unpack directory instead of being used directly.
This is inconvenient as it may be useful to use ${S} when you want to unpack a
file into the source tree.
Change this behaviour so that absolute paths are used directly instead of being
appended to the root directory. To ensure that recipes cannot write files to an
arbitrary location enforce that the subdir starts with the unpack root.
(Bitbake rev: c3873346c6fa1021a1d63bddd9b898a77c618432)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't want a traceback for this exception, we need to catch it, fire
TaskFailed, and return failure.
(Bitbake rev: 63966ada459d44d3dc7817ad2a026a22e8f6700f)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you "bitbake glibc-locale" then delete the libpcre-native sstate
and "bitbake glibc-locale -C package_write_rpm", it will fail with
rpmbuild missing the libprce library.
The reason is that libpcre-native fails to install from sstate (since
it isn't present) but doesn't get built and hence rpm-native tries to
run without its dependencies.
The simplest fix is not to add "covered" tasks which have failed to
install sstate. I can't help feeling there is more to this issue but
this does fix the current problem and shouldn't have adverse affects.
It is an unusual situation to have missing dependencies in sstate since
they're usually all present or not at all.
I've taken the opportunity to remove some old cruft from when we had
numeric task ids, the code can be simpler now.
(Bitbake rev: ba566b46d530b495f12f3a74f76434717b22a020)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the layer is local source don't try and work out the location of
the layer by using the git information (getGitCloneDirectory)
[YOCTO #10199]
(Bitbake rev: 3dfea5214d4bd006e26630e5024774ecb84ea527)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no need to lock build environment before changing
build status as this operation is very fast. However, there
is a need to unlock it after changing build status.
Explicitly unlocked BuildEnvironment after build reaches
final status SUCCEEDED, FAILED or CANCELLED. This should
allow runbuilds process to pickup next build faster.
(Bitbake rev: faa88272d656640c039572c5c8f3e6c56535b6f7)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Called signal_runbuilds API when build is scheduled, cancelled or
finished to notify runbuilds process about builds status change.
[YOCTO #8918]
(Bitbake rev: fe08f0fa4b328908e73695ebbceca87bc86a49f9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The event handling 'Exception' was catching and triggering a backtrace. This
trace was obscuring any errors from an event handler that had raised the
BBHandledException, which should indicate do not print additional information.
(Bitbake rev: 51ca5193a5674b27d816140b0254f485912177a2)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add recommended layers to collection_depends[] so that dynamic
priority assignment will work for both depends and recommends.
Recommended layers do not cause an error or warning
if they are not in the collection list, but debug messages
are output for level 3 and above.
explode_dep_versions2 returns a dictionary, so we
change the variable deplist to depDict. The dictionary
values are lists which are either empty or contain only one
version specification.
(Bitbake rev: 20cdc3d609f8aea992f97c3db336574d3a549973)
Signed-off-by: Joe Slater <jslater@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Task name was incorrectly added to the targets that already
contained :task suffix and fired with BuildInit event. This
caused Toaster to create incorrect Target objects and show
them in UI.
[YOCTO #10221]
(Bitbake rev: b7faf1af3bd3110fba347fbe6e432fc4ee66590a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Users are surprised when dirs/cleandirs aren't acted upon for
empty functions. This reorders the code slightly so that those
flags are acted upon for empty functions as there are cases where
this is expected.
[YOCTO #10256]
(Bitbake rev: 5bf874673d75b5f4ff2b34f0ab8502558ee84d00)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The implementation of SRCREV_FORMAT has at least two issues:
1. Given two names "foo" and "foobar" and SRCREV_FORMAT = "foo_foobar",
"foo" might currently get substituted twice, and "foobar" not at
all.
2. If the revision substitued for some name happens to contain another
name as a substring, then that substring might incorrectly get
replaced.
Fix both issues by sorting the names with the longest ones first and
replacing all names at once with a regular expression. This was inspired
by
http://stackoverflow.com/questions/6116978/python-replace-multiple-strings.
(Bitbake rev: 8e6a893cb7f13ea14051fc40c6c9baf41aa47fee)
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we've already fetched a particular URL then we do not need to do so
again within in the same operation. Maintain an internal list of fetched
URLs to avoid doing that.
(Bitbake rev: b4705c80add1f618c11a9223cdd9578d763b50ec)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
All logging messages are printed on stdout when processing
UI event queue. This makes it impossible to distinguish between
errors and normal bitbake output. Output to stderror or stdout
depending on log level should fix this.
(Bitbake rev: 56ac0d4c7a5f47aeb707b15a0c305d9f73aae945)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake collects all events in special event queue when called with
-w option. However, it starts to write events to the eventlog only
after BuildStarted event is received. In some cases this event is
not received at all, e.g. when bitbake is run with --parse-only
command line option.
It makes sense to write all collected events when CookerExit event
received to make sure all events are written into the eventlog even
if BuildStarted event is not fired.
[YOCTO #10145]
(Bitbake rev: 57912de63fa83550c0ae658eb99b76e9cc91a8d1)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When converting variable history file names to relative
paths, keep the layer directory's name so that the user
can distinguish between conf files with the same name.
[YOCTO #8188]
(Bitbake rev: 59561d652af91c2099b735084f0e44275d68e637)
Signed-off-by: David Reyna <david.reyna@windriver.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
No functional changes, just make a couple of minor tweaks to the
comments for edit_metadata():
* There are four elements to be returned by the callback function
* Add an example return statement for when you don't want to modify the
value
(Bitbake rev: 99675c19375c96140bc8ae8f9fc3a1945a77cebb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The correct name of the parameter is "version" not "ver" so ensure we
aren't misleading the user by giving the latter in an example.
(Bitbake rev: 14c045c6a20993d389b91ae2459d811a1430a7b2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow using a top-level shrinkwrap file with one or more npm://
dependencies, i.e. if the module isn't found at the top level then look
one level down.
Part of the fix for [YOCTO #9537].
(Bitbake rev: f7de3f8b5f628dee043fe783148812914ab20813)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"npmpkg" can be a default, but it should respect the subdir parameter as
with other FetchMethods. This allows us to have more than one npm://
entry in SRC_URI without nasty hacks.
Fix required in order to support [YOCTO #9537].
(Bitbake rev: e6a94d2091ec5d42f25102334a8492a731b8dec3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
You cannot set a URL-specific value in an object-level variable on
the FetchMethod in urldata_init() or the result is the value specific to
the last URL will be the one that gets set. This prevented fetching more
than one npm:// URL correctly - the other tarballs would not download to
the correct location and do_unpack failed to find them as a result.
Fix required in order to support [YOCTO #9537].
(Bitbake rev: 1435b49ea7d0f9d4cc4a665fb2aa83d1eea7900f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were downloading into the current directory here, which is fine if
that current directory can be expected to be the right place - but
that's not true when called from recipetool within OE. We should
explicitly specify the directory to run the command in and then there
won't be a problem.
(Bitbake rev: 0ddaf725e5a0675b252b7f80b1706370e478175b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ud.pkgdir argument was being passed as the 'quiet' argument to
runfetchcmd, not the 'workdir' argument, resulting in fetching the svn module
into the root of DL_DIR, not where it belongs.
Cc: Matt Madison <matt@madison.systems>
(Bitbake rev: dc756510a95f88b192352be6fcd1d5d77852c348)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, there is a check to remove the TOSTOP attribute from
a tty to avoid hangs. It assumes that sys.stdout will have a
file descriptor and this is not always true, some IO classes
will throw exceptions when trying to get its file descriptor.
This will add a check for such cases and avoid throwing an
exception.
[YOCTO #10162]
(Bitbake rev: cb4f8f6efa28ef2b13bc738a0118b876baa15b3e)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Because some image_license.manifest files contain multiple
FILES lines, and because those lines can sometimes not contain
a list of files (i.e. they look like "FILES:\n"), we were
resetting the list of kernel artifacts when we hit the second
"empty" line.
Fix by ignoring any FILES line which doesn't list files, and by
appending any files found in a valid FILES line, rather than
overwriting the existing list.
[YOCTO #10107]
(Bitbake rev: 927ec3524625ac731326b3c1c1361c2a4d2bd9e1)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
