New systemd has changed the phrasing when skipping things,
with unfortunate use of 'failed':
[ 1.623667] systemd[1]: Journal Audit Socket was skipped because of a failed condition check (ConditionSecurity=audit).
[ 1.688258] systemd[1]: Load Kernel Modules was skipped because all trigger condition checks failed.
(From OE-Core rev: 743d09665a4ef743b1fa9ac382a713556dfce1a1)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The issue is reported upstream:
https://bugs.ruby-lang.org/issues/18456
Otherwise I do not feel that further investigation of rdoc's parser to find
out the source of non-determinism is worth the time.
(From OE-Core rev: 208021f7212a8a790c350ccca720695c5bcbb1ca)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffoscope can end up running for a very long time if there are a lot of
changes. To put a limit on how long it can run, cap the maximum report
size at 250 MB by default.
(From OE-Core rev: 52d5f76f54eac384f9480dffe96df089d9ee8f33)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As a centos 8 spinoff, it lacks the same vgem kernel module.
(From OE-Core rev: a48bfc9c0938c51c91dd925bd7f5360bc01252ae)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With this changeset, go becomes reproducible \0/
(From OE-Core rev: 03ada343ecbbbe3822a8fd74e678b01be6b9511a)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since honister host manifest stopped to generate, i.e.
manifest file is empty but all ipks/files into sdk is
ok.
(From OE-Core rev: 79a2392f5d2a4cb6509a83afb40bca01bac59914)
Signed-off-by: Oleksiy Obitotskyy <oobitots@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The gnu sources server has been known to disappear. Use the YP sources
mirror instead. If that breaks, the autobuilder is broken anyway. This
should reduce test failures from upstream network issues.
(From OE-Core rev: a5459e42f1a6be9c08f303653cc1f73514eca9ef)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Separate licenses with & operator since it should be satisfied most use
cases and it is a reasonable assumption that all the licenses apply.
Furthermore flat, split and sort the licenses to minimize license string
changes.
Separate package licenses with & operator:
-LICENSE:${PN} = "MIT ISC"
+LICENSE:${PN} = "ISC & MIT"
Respect | and brackets in LICENSE:
-LICENSE = "BSD-3-Clause & (ISC & | & MIT)"
+LICENSE = "BSD-3-Clause & (ISC | MIT)"
Sort licenses:
-LICENSE = "MIT & BSD-3-Clause & ISC"
+LICENSE = "BSD-3-Clause & ISC & MIT"
Remove duplicates:
-LICENSE = "MIT & ISC & MIT"
+LICENSE = "ISC & MIT"
(From OE-Core rev: 60a84ecc53d20118c5e7f86dd3e3cafbfed1cf0a)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds a smoke check for whether the rust toolchain actually
produces working executables across a range of architectures.
(From OE-Core rev: 14bb638b0df7acfa6aa89abf4625357f8cde886b)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Doing that should not change the task signatures.
(From OE-Core rev: a06ceaeaf11aa344e152f3bb6629aa6e000482d3)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move helper functions out of class scope so they can be used in other tests
(From OE-Core rev: 439a5a575b8ca27910431c71f01bf310122ff7ab)
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recipetool support two ways to pass a branch and fallback to master
if no branch is defined. Add tests for default branch, branch parameter
and srcbranch option.
(From OE-Core rev: f232cffc09e917458d6e850b4aca4f87b53b43e4)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The selftest recipetool base class reuse the selftest devtool base
class. Thereby the selftest devtool base class setup its own devtool
sstate and the selftest recipetool classes trigger the build of recipes.
This leads to the problem that the build artifacts doesn't reach the
persistent sstate cache and rebuild on every selftest run.
Move the common selftest devtool functions into its own class and use
the sstate cache in the recipetool tests.
(From OE-Core rev: c0bd0686092181b6a5316c373b5b125d78a24e9f)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Split tests into separate test classes to speed up individual test runs
by reducing the test setup to a minimum.
The pkgdata generation is only needed for the append tests and slow down
the other tests.
(From OE-Core rev: 2aef869bcca3e0de4afb9ef00492de3ee72b035c)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If PATCHTOOL="git" has been specified but workdir is not git repo
bitbake fails to apply the patches with error message:
Command Error: 'git rev-parse --show-toplevel' exited with 0 Output:
fatal: not a git repository (or any of the parent directories): .git
Fix this by initializing the repo before patching.
This allows binary git patches to be applied.
(From OE-Core rev: 6184b56a7a0fc6f5d19fdfb81e7453667f7da940)
Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds a smoke check for whether the Go toolchain actually
produces working executables across a range of architectures.
(From OE-Core rev: 2819bb2cf22c6cfcaeaee79f0280097ec9cb9327)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
distutils is going away and we have functionality in bitbake which can
handle these comparisions so switch to the bb.utils function.
(From OE-Core rev: fe624b520e6c75e16a8f394785ab0216341402f9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Larger systems may have large numbers of cores, but beyond a certain
point they can't all be used for compiling: whilst purely
compute-intensive jobs can be parallelised to hundreds of cores,
operations such as compressing (needs lots of RAM) or compiling (lots of
I/O) don't scale linearly.
For example, the Marvel ThunderX2 has 32 cores, each capable of
executing four threads, and can be configured with two sockets, making
256 CPUs according to Linux. Zstd using 256 threads has been seen to
fail to allocate memory during even small recipes such as iso-codes.
Add a default cap of 64 CPUs to the cpu_count() method so that extreme
parallisation is limited. 64 is high enough that meaningful gains
beyond it are unlikely, but high enough that most systems won't be
effected.
(From OE-Core rev: 765d0f25ce48636b1838a5968e2dc15de2127428)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When looking at logs involving thread pools it is useful if the threads
can be named.
(From OE-Core rev: 18342945b021608794d83ecf567afd43f4379b24)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since commit 044fb04d in bitbake (fetch2: Allow whitespace only mirror
entries) there is no need to separate the entries in PREMIRRORS and
SSTATE_MIRRORS with "\n".
(From OE-Core rev: 3db607deea3b2982edb333f99056119490fcf5ea)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Test that overlayfs.bbclass generates one systemd unit, that
applications can set dependencies on, and that this unit mounts all
required overlays
(From OE-Core rev: b0e335268de78cfe163f3cfdc0209ae26309fc78)
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Application can depend on several overlayfs mount points. Provide a
systemd unit application can depend on to make sure all overlays are
mounted before it is started to avoid any race conditions
(From OE-Core rev: b38e194db0c6825f28c56123cf88af94d3f52beb)
Signed-off-by: Bruno Knittel <Bruno.Knittel@bruker.com>
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should default to xz or zstd instead of gz, pick xz for now as it is more tested.
(From OE-Core rev: 658f274c161ec67b91f133ee9b5c3767c2925787)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Following the scripted conversion adding branches to git://
SRC_URI entries, add the remaining references, mainly in the selftests
and recipetool.
(From OE-Core rev: 5340c0d688036c1be6c938f05d8a8c1e3b49ec38)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is uncertainty about the default branch name in git going forward.
To try and cover the different possible outcomes, add branch names to all
git:// and gitsm:// SRC_URI entries.
This update was made with the script added to contrib in this patch which
aims to help others convert other layers.
(From OE-Core rev: b51c405faf6f8c0365f7533bfaf470d79152a463)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changed the following line from:
[drm] Cannot find any crtc or sizes - going 1024x768 > [drm] Cannot find any crtc or sizes
This will expand the coverage of the failure to also cover the case when fallback size is not set.
(From OE-Core rev: 0569fa735458512d6e15aa3315218ecbdf8510a3)
Signed-off-by: Teoh Jay Shen <jay.shen.teoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having annotations on relationship can provide additional information
about the relationship such as how it was derived.
(From OE-Core rev: d98585aa89e1d3819f8139a07fb7376ef89b37f8)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows catching failures on new build hosts, instead of quietly skipping
the test due to incomplete host setup.
(From OE-Core rev: c93f95f2e48a2bba76b231692f2c15bb32005132)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The controller module loading code needs to be told what directories
to search for modules via the target_modules_path keyword argument, which
is set to BBPATH.
However, as the actual module loading is done via importlib this relies
on the paths being on sys.path, which it is as base.bbclass puts each
layer's lib/ in sys.path.
Simplify the code by removing this indirection, and simply search
sys.path directly.
(From OE-Core rev: 570a19581f582f77e04d6892adb647cd649a6943)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Even if initramfs_bundle_path was used, a wrong compression was reflected
in output its template file. Use linux.bin as universal kernel image.
The linux.bin file covers both cases because it's beying created from
vmlinux.
We know, that vmlinux is created inside compressed directory already,
so no external compression will be used.
(From OE-Core rev: 27fbbeaf972bb7e2535c1b23375cfa9d66b69db6)
Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Walter Schweizer <walter.schweizer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the function a ThreadedWorker is executing raises an exception, don't
use print() as that mostly disappears. Instead, output it to the logger.
This is done using bb.mainlogger.debug directly instead of bb.debug() as
this allows us to pass the exception instance directly, which is then
incorporated into the log stream.
(From OE-Core rev: 2f1ea25c222b344dd8b784b2bc73a6540ab30274)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extract package_qa_write_error, package_qa_handle_error and
package_qa_add_message functions from insane.bbclass to lib/oe/qa.py and
drop the package_qa_ prefixes.
Update various bbclasses to use the new functions. No import is required
since base.bbclass puts oe.qa in OE_IMPORTS.
Stop requiring callers to manually track whether a fatal error has been
encountered via a "sane" flag. Instead replace the QA_SANE variable with
QA_ERRORS_FOUND and call oe.qa.exit_if_errors or
oe.qa.exit_with_message_if_errors at the end of each task.
Inspired by discussion resulting from
https://lists.openembedded.org/g/openembedded-core/message/156793 and
https://lists.openembedded.org/g/openembedded-core/message/156900
(From OE-Core rev: f0ad152ef4cc15c042bc9eeefb6af096d054b220)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We sometimes see exceptions from code seeing the hashserv DB files
being removed at directory cleanup time. Add a check to ensure the
hashserv has written the data base journal (and hence likely exited)
before cleaning up.
This will hopefully avoid errors like:
Traceback (most recent call last):
File "[...]/meta/lib/oeqa/sdk/buildtools-cases/build.py", line 30, in test_libc
delay = delay - 1
File "/usr/lib/python3.6/tempfile.py", line 948, in __exit__
self.cleanup()
File "/usr/lib/python3.6/tempfile.py", line 952, in cleanup
_rmtree(self.name)
File "/usr/lib/python3.6/shutil.py", line 486, in rmtree
_rmtree_safe_fd(fd, path, onerror)
File "/usr/lib/python3.6/shutil.py", line 424, in _rmtree_safe_fd
_rmtree_safe_fd(dirfd, fullname, onerror)
File "/usr/lib/python3.6/shutil.py", line 444, in _rmtree_safe_fd
onerror(os.unlink, fullname, sys.exc_info())
File "/usr/lib/python3.6/shutil.py", line 442, in _rmtree_safe_fd
os.unlink(name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'hashserv.db-wal'
(From OE-Core rev: 0b07d9add687d78495176cda0f3011c10ffa4d4b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want things to be reproduicble and the variable doesn't really change
much any more. Drop the remaining uses and make those code paths always
active.
(From OE-Core rev: d15fb02c7ee7da50e322d74bc6a545234e20c7f3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reproducibility is here to stay and needs to be part of our default workflow.
Move the remaining code to base.bbclass so it is always a first class citizen
and it is clear people need to be mindful of it.
(From OE-Core rev: abb0671d2cebfd7e8df94796404bbe9c7f961058)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To try and avoid parse/memory overhead of functions within bitbake,
move the bulk of the reproducibility functions to the function library.
(From OE-Core rev: f2fd1c9d75e774c8a5271cdc1ec6f65c4492f941)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow a user to validate sstate objects against a list of keys, instead
of just any known key in the user's keychain.
(From OE-Core rev: 52ba0c5e6e2e3d5d01dc3f01404f0ab1bb29b3b5)
Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch switches the compressor from Gzip to ZStandard for ssate cache
files.
Zstandard compression provides a significant improvement in
decompression speed as well as improvement in compression speed and disk
usage over the 'tgz' format in use. Furthermore, its configurable
compression level offers a trade-off between time spent compressing
sstate cache files and disk space used by those files. The reduced disk
usage also contributes to saving network traffic for those sharing their
sstate cache with others.
Zstandard should therefore be a good choice when:
* disk space is at a premium
* network speed / resources are limited
* the CI server can sstate packages can be created at high compression
* less CPU on the build server should be used for sstate decompression
(From OE-Core rev: 0710e98f40ec7394567ede1f88f7c7bae306b925)
Signed-off-by: Henry Kleynhans <hkleynhans@fb.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
