The keep alive timeout is excessively long at 83 minutes (5000 seconds),
reduce this to 10 minutes: this should be long enough that it rarely
triggers in normal builds, but when it does it has useful information.
(Bitbake rev: 2e47346b95b09d7ab8f0603e2d62cfb549dc1f5c)
(Bitbake rev: dcf52157d3635925491783be656c6b76d1efe1a4)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In interactive bitbake sessions it is obvious what tasks are running
when one of them hangs or otherwise takes a long time. However, in
non-interactive sessions (such as automated builds) bitbake just prints
a message saying that it is "still alive" with no clues as to what tasks
are active still.
By simply listing the active tasks when printing the keep alive message,
we don't need to parse the bitbake log to identify which of the tasks
is still active and has presumably hung.
(Bitbake rev: f9f57fb7d2c8a13df1eb9d5b9766f15e229dcf97)
(Bitbake rev: 30f6c3f175617beea8e8bb75dcf255611e3fc2fd)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a Dockerfile for building the PR service in a container.
Based on the hash equivalence server container [hashserv].
Use the following environment variables to configure the PR service
container:
- DBFILE: database filename. Default:
/var/lib/bbprserv/prserv.sqlite3.
- LOGFILE: log filename. Default: /var/lib/bbprserv/prserv.log.
- LOGLEVEL: logging level. Default: DEBUG.
- HOST: ip address to bind. Default: 0.0.0.0.
- PORT: port number. Default: 8585.
- DBMODE: database mode. Default: Empty (RW).
Note: DBMODE in RO: "--read-only".
[hashserv]: contrib/hashserv/Dockerfile
(Bitbake rev: a5497428b539e8598263924f63a9df8fe1ea70f9)
(Bitbake rev: 1d05abd92da56e284fcd904cf32bd12485903f10)
Signed-off-by: Daniel Gomez <daniel@qtec.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Force a sync point for end-of build event handler force
the build's outcome status commit, to resolve a race
condition with the build completion takedown.
[YOCTO #14765]
(Bitbake rev: f76c13536e19d2401365c0fc240ef8dbcb3ee7e8)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a polling check on tracebacks in a build's log. This
can for example indicate that bitbake crashed, which would
stop the event stream that Toaster normally uses to detect
build errors.
[YOCTO #14085]
(Bitbake rev: 32b1c0b3477e359d2e2a61a23a294e317e417f95)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Creating the new log instances triggers a lot of python logging overhead
in a commonly called function (about 600,000 for parsing OE-Core).
We only need the log functionality if we're parsing, not if we just hit
from the cache. Therefore defer the log setup overhead until we know it
is a cache miss.
Whilst this complicates the code slightly, the performance gain is worth
it as for parsing OE-Core we drop 60 million funciton calls (from 225
overall).
(Bitbake rev: ac868167ad854f9bb32dcb2e63528870547805a7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can call getVarFlags() instead of the multiple getVarFlag calls
which is a little more efficient. This reduces the number of overall
function calls so is good but probably isn't much faster (or slower).
(Bitbake rev: 505a4dd34e822bdf902d9b348dbcdf5b2c94e784)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code tries to expand missing entities when they're encountered in
python expressions. Looking at traces, these are often things which
would not be in the data store such as "bb".
Optimise to skip the data store queries for things we know will never
be there. The expansion cache usually covers these but skipping
entirely cuts a few million function calls parsing OE-Core.
(Bitbake rev: 1ae712635a2eef3ecbf541e1f7cc2eeb2f3799c9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the upcoming LTS, it is time we changed the bitbake version so
move to 2.0.
(Bitbake rev: 9a13bf8e20b1841ec99281d45be4c4fc1118438c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By alphabetical order, to get the same order as in the HTML
output, sorted thanks to the ":sorted:" directive.
(Bitbake rev: 8b97af89316a4407dd07649e8aadd495edd86ee4)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This template is not referenced anywhere and therefore not used.
It also refers to the obsolete "build mode" in the old docs.
(Bitbake rev: f89a35155e3b8d12ac609a165c63ed206751b8da)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
connection.features.autocommits_when_autocommit_is_off
was deprecated in 3.0 and is no longer present in 3.2
(Bitbake rev: cc0f526fb4298349d0eea44c8f35d3dd226cc8d6)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Follow the default guidelines from Django 3.2 setting DEFAULT_AUTO_FIELD
to django.db.models.BigAutoField which leads to these migrations.
(Bitbake rev: ccfdcf5ed6b5d2024c268bace30e53753d1f4da4)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New in Django 3.2.
Silence warnings by adding:
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'
in toastermain/settings.py
(Bitbake rev: cbc5a8e339a8b5eb4cdd89ad898af432f05cadfb)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update docs links from www.yoctoproject.org to the docs.yoctoproject.org
equivalents.
(Bitbake rev: 74194962eb294c9bd1192ddd7935d57258712ee2)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Django 2.2.x goes EOL April 2022.
Switch the to current LTS Django 3.2.x supported until April 2024.
(Bitbake rev: 23a15c5703d5ef4190921f9bb0273e43b1de489c)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix a bad patch merge and move the file to the correct path.
(Bitbake rev: d173f45a88cc9460734e7cc0b87952c0ea0ff270)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After the override syntax change, toaster isn't working correctly. This
is because it uses IMAGE_INSTALL_append instead of IMAGE_INSTALL:append.
This tweaks the code accordingly to fix this.
I have a suspicion that exiting toaster databases may struggle with
the change and there are some migration steps missing for the whole
overrides syntax change step.
(Bitbake rev: befc93c3e03a9b1a2b630cd5badcda756fe6cd28)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake -e is not showing variable history correctly for all variables.
The issue was triggered by the change to knotty to default to enabling
variable tracking in the base datastore but that exposed another issue.
The real problem is that calling reset() inside cooker reverts to the
data tracking enabled by the UI for the base datastore, then turns
off tracking. In the case of the environment printing code, it needs
it to be left on.
Tweak the code to ensure data store tracking really is enabled.
The code here is clearly a bit of a mess but this at least fixes a
clear regression until more invasive improvements can be made.
(Bitbake rev: 9596bffc9903f023d1cc97ce531ddb1cf010c4d7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Use max() to clamp progress to >= 0.
* Be consistent when evaluating self.quiet (treat it as a boolean).
(Bitbake rev: 160f71372ff93894d9314619e9d3b547c1f3cda3)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In commit 8055ec36 (knotty: Improve setscene task display) the setscene
tasks got their own line in the task output. However, the progress bar
code does not handle newlines in its widgets, so the length of the
setscene line was included when calculating how much space is available
for the progress bar of the running tasks, making it much too short.
Instead of trying to teach the progress bar code to handle newlines,
separate the output of the setscene tasks from the progress bar for the
real tasks.
(Bitbake rev: a41f7792f17acdba8c7ea83b79e413ae6a49da68)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use pluralise() to correct the grammar, and drop the colon at the end if
runnning in quiet mode.
(Bitbake rev: 57396289f935892390c11afe95f3eede28fd80e7)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "Parsing halted due to errors" is shown when user has old-style
variables requiring renaming. Make it clear that it isn't an additional
error, but a result of earlier errors.
(Bitbake rev: e81b6ae61dd8b5d0f77146263d79dfe66c66b060)
Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids backtraces when starting toaster or using bitbake in
remote mode.
(Bitbake rev: bf723f2cb5d288ca730e4f029110b36380420a01)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add links corresponding to recent releases
Align the title of sections with yocto-docs' releases.rst
(Bitbake rev: 5700317975e115aa31186c8cd3eddca16ec387fb)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is general agreement the current display of the total setscene and
executed full tasks is confusing. Fixing it is tricky to be clear to new
users whilst not wasting screen real estate for experienced users and not
compromisng features like the progress bar.
As a compromise, move the setscene total to the previous line making
the status display a two line summary for interactive terminals. This
makes it clear what two of the numbers represent (setscene task) whilst
separating this out from the other items.
Also tweak the non-interactive output to use a comma as a separator,
the use of the forward slash was a poor choice.
(Bitbake rev: 8055ec360507e6a678ee5c4018ec1ab7f5a9cce5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the footer code doesn't quite handle multiline items correct.
Fix this to do so.
(Bitbake rev: 8eeccf73185d986c6abd6426b1d1e72da3a982df)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This implements support for sstate mirrors using ssh as transport
protocol.
(Bitbake rev: 0a3b5b3de7bcb1c5c3748cba42d394cc484e966b)
Signed-off-by: Daniel Wagenknecht <dwagenknecht@emlix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix absolute paths and paths containing the ":" character. Both is
necessary for supporting sstate mirrors via ssh (not implemented yet).
(Bitbake rev: df5505a1ba15524c3a185360d687854300aef342)
Signed-off-by: Daniel Wagenknecht <dwagenknecht@emlix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Support URLs like ssh://HOST/PATH. They were previously not recognized
due to a missing @ sign.
(Bitbake rev: a2aa18bd27dac8902e52b466cb7118f71367d3dc)
Signed-off-by: Daniel Wagenknecht <dwagenknecht@emlix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hashlib.md5() is not permitted on a FIPS enabled host system. This is due
to md5 not being an approved hash algorithm.
Instead use:
hashlib.new('MD5', usedforsecurity=False)
This is allowed, as it's clear the hash is used for a non-security purpose.
Note: utils.py version should never be used to verify file integrity, but
instead be used to identify if the file may have changed. sha256 should be
used for integrity purposes.
(Bitbake rev: af866dd077867cba0129757bfcc689551445e9d7)
Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
File "scripts/lib/checklayer/__init__.py", line 49, in _get_layer_collections
ldata.expandVarref('LAYERDIR')
File "build/bitbake/lib/bb/data_smart.py", line 1007, in expandVarref
if referrervalue and ref in referrervalue:
TypeError: argument of type 'bool' is not iterable
We inject True values as an internal sentinel to the datastore, fix
this codepath to handle it.
(Bitbake rev: 3b88562d87ac94725c1a683c859c2a6a3287d173)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Until now, if a renamed variable was found, bitbake exited
immediately if it was in a class, but continued after an error
message if the variable was in a recipe. This was caused by
cookerdata.py CookerDataBuilder::parseBaseConfiguration checking
a different DataSmart instance than the variable was set in.
To solve the issue, add a special variable and set it when we
find a renamed variable. Check for it in ast.py and bail out
if needed.
(Bitbake rev: d12400600e30549c88dc9e7883dc3d63b1dc1117)
Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In line with the inclusive language migration defined at:
https://wiki.yoctoproject.org/wiki/Inclusive_language
replace the remaining usage of "abort" in documentation, error
messages, and comments with halt/fail/exit as appropriate.
A couple of external Javascript API calls in Toaster remain, as
they cannot currently be changed.
(Bitbake rev: bc27762bf3ffb4a20b58eace5302438c4a526626)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In line with the inclusive language migration defined at:
https://wiki.yoctoproject.org/wiki/Inclusive_language
replace the "ABORT" action in BB_DISKMON_DIRS entries with "HALT".
In order to ease migration, code has been added to warn users to
update their configurations if the old name is used, as opposed to
to throwing an error.
(Bitbake rev: 11dc65dc077398ff9818060769c99c0090291186)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
