systemd boot has completed when system is in "running" or "degraded"
(some services failed) state. Check for that in the systemd settle()
function instead of listing all services and checking their activation
state since some services are in activation state even when whole
system is already in "running" state. Examples of services which can be
in activation state are rootfs auto mounting related generated services.
Without this patch systemd test_systemd_list (systemd.SystemdBasicTests) times
out on an image with dm-verity /usr partition and systemd generated
rootfs:
NOTE: ... FAIL
Traceback (most recent call last):
File "/home/builder/src/base/build/../poky/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
File "/home/builder/src/base/poky/meta/lib/oeqa/runtime/cases/systemd.py", line 97, in test_systemd_failed
self.assertTrue(settled, msg=msg)
AssertionError: False is not true : Timed out waiting for systemd to settle:
UNIT LOAD ACTIVE SUB
DESCRIPTION
dev-disk-by\x2did-dm\x2dname\x2droot.device loaded activating tentativ
e /dev/disk/by-id/dm-name-root
dev-disk-by\x2did-dm\x2dname\x2dusr.device loaded activating tentativ
e /dev/disk/by-id/dm-name-usr
dev-disk-by\x2did-dm\x2duuid\x2dCRYPT\x2dLUKS2\x2df2b944f394174eb5918cb6af2c6b4cb2\x2droot.device loaded activating tentativ
e /dev/disk/by-id/dm-uuid-CRYPT-LUKS2-f2b944f394174eb5918cb6af2c6b4cb2-root
dev-disk-by\x2did-dm\x2duuid\x2dCRYPT\x2dVERITY\x2d3dd703c88f1946658697a6d57617473b\x2dusr.device loaded activating tentativ
e /dev/disk/by-id/dm-uuid-CRYPT-VERITY-3dd703c88f1946658697a6d57617473b-usr
dev-disk-by\x2duuid-bfbf856e\x2d3c65\x2d4eb2\x2d9ffb\x2d8e0b11641d85.device loaded activating tentativ
e /dev/disk/by-uuid/bfbf856e-3c65-4eb2-9ffb-8e0b11641d85
dev-dm\x2d0.device loaded activating tentativ
e /dev/dm-0
dev-dm\x2d1.device loaded activating tentativ
e /dev/dm-1
...
Fix is to check for the systemd global "running" or "degraded" state.
Note that it would be possible to use a blocking call
"systemctl is-system-running --wait" to exit after system enters "running"
or "degraded" state but using the existing loop for a 2 minute timeout.
(From OE-Core rev: 3b013ae441d117adeda0d9950e02e9f7d0deba2f)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we're changing the return values of the function, add a version so
bitbake can ensure it is using a compatible function.
(From OE-Core rev: 738bf3717002d124958a21a6925b0eca14fef131)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
find_siginfo() returns two different data structures depending
on whether its third argument (list of hashes to find) is empty or
not:
- a dict of timestamps keyed by path
- a dict of paths keyed by hash
This is not a good API design; it's much better to return
a dict of dicts that include both timestamp and path, keyed by
hash. Then the API consumer can decide how they want to use these
fields, particularly for additional diagnostics or informational
output.
I also took the opportunity to add a binary field that
tells if the match came from sstate or local stamps dir, which
will help prioritize local stamps when looking up most
recent task signatures.
(From OE-Core rev: 8721c52041e910bd4d8a9235b52f274f4f02c8a3)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a comment explaining why we need the jitterentropy ignore.
(From OE-Core rev: ed4b202b11fd18e18502fe4e90f421dce4db8b44)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
upstream commit 003e0694fcd3d5 [ACPI: processor: Refine messages
in acpi_early_processor_control_setup()] changes logging during
boot such that some of the informational messages are caught as
QA issues.
Adding the string to the ingore files so that 6.6+ will pass QA.
(From OE-Core rev: 463d40f2d4bf30becce431bfc1cbdf3b673e8e60)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a recipes was modified recommand the use of `CVE_STATUS` instead if
`CVE_CHECK_IGNORE` is used. This is a depreacted variable and will
result in a warning from the cve-check.class and should hence not be
used anymore. [YOCTO #15311]
(From OE-Core rev: 5c264063f6363e5ff88146125217b6089eb22f12)
Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recipe being tested is in `testrecipe`, use that rather than the
literal `zvariant`.
(From OE-Core rev: f14ce354890024a3a0a3d4c7efa53eab5db7a6b1)
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
test_devtool_modify_git_crates_subpath expects 2 or more git URIs,
change the test from Greater to GreateEqual.
(From OE-Core rev: 4a8d03db55e6a1b07a8585cbf5fbf735ec51f4a7)
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now we've switched to the python script, update the tests to match
that by removing the now unneeded parameter.
(From OE-Core rev: 93989f9b4895506e6ad66a78088a5c2801e2a10d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
FIXES [YOCTO #12342]
When testing a Multilib image, the package manifest list contains
the fully qualified package name which includes the Multilib Prefix.
This patch adds the MLPREFIX to the package names that are passed
into the @OEHasPackage() decorator to ensure the set isdisjoint()
matches correctly.
(From OE-Core rev: ab87e4f92305b2a664cc473869e1615cf56e0936)
Signed-off-by: Saul Wold <sgw@bigsur.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The tests were not cleaning up after themselves and making assumptions about
memory resident bitbake being stopped by the scripts.
Add cleanup logic to ensure the tests don't break other things and
clean up created files.
(From OE-Core rev: 692dd762a0c817797c28381c6169205fbaeb2705)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is possible we could crash holding a lock whilst parsing in this code.
Switch to use utils.fileslocked() in the with expression to avoid this.
This may be causing some of our strange intermittent failures in PRServ
tests.
(From OE-Core rev: 4e59db15e5df2cc3d0ae042454812a2d54cef77b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds an API that makes it easier to iterate over the package data for a
all providers of a runtime dependency.
(From OE-Core rev: 68bdc219a4a819e83217f5b54c463624af8d3b9e)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a package had a provider with a '/' in it (e.g. "/bin/sh",
"/bin/bash", etc.), the generated symlinks were broken due to being at a
hard coded depth. Use oe.path.relsymlink() instead to make a correct
relative symbolic link
(From OE-Core rev: 8b1482a4e2adb7cf358d638265cf116b34078b84)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds API to make a relative symbolic link between two directories. The
arguments are the same as oe.path.symlink()
(From OE-Core rev: 3eeec7f3412e881e51763ef947c82772d3858f09)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Converts IPK package generation to use zstd instead of xz. zstd has a
much larger compression/speed tradeoff range allowing users to choose
what suits them best, and fast decompression speeds. It also continues
to support parallel compression as xz did.
A new variable called ZSTD_DEFAULTS is provided to set the defaults for
places that want to use zstd for compression; the zst image conversion
command is also modified to use this.
Finally, in order for this to function properly, opkg must include zstd
support, so it is enabled all the time with no PACKAGECONFIG to turn it
off.
(From OE-Core rev: 1bc3e9bbaa670b6128c74c76b4b5264e60ce3463)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Similar to the sdk test case, build the "guessing-game" example from
https://maturin.rs/tutorial
This test case:
* creates a python3 venv
* echoes "nameserver 8.8.8.8" to /etc/resolv.conf as we need to have
functional DNS to fetch the crates on target
* fetches crates, builds guessing-game crate and wheel
Put the following in your local.conf:
EXTRA_IMAGE_FEATURES += "tools-sdk"
SDK_INCLUDE_TOOLCHAIN = '1'
SDK_TOOLCHAIN_LANGS += 'rust'
IMAGE_INSTALL:append = " python3-maturin"
IMAGE_CLASSES += "testimage"
TEST_QEMUPARAMS ?= "-m 8192 -smp 4"
IMAGE_ROOTFS_EXTRA_SPACE = "10000000"
NOHDD="1"
NOISO="1"
TEST_SUITES = "ping ssh python maturin"
Test with:
bitbake core-image-full-cmdline
bitbake -c testimage core-image-full-cmdline
(From OE-Core rev: ca7e78c8be6aaa2780702eab54715a74fc0dac5e)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we are only tracking stdout and are not using self.readsock we end
up throwing an exception blocking further action from the thread. Fix
this by checking self.readsock is not None first.
While we are at it split even into fd, event to make things clearer
and handle the fail path of stringify_event by echoing the hex value
of the unknown flag.
(From OE-Core rev: 5e58737c66090fe009ec49296f3e7d687eb05766)
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Mikko Rapeli <mikko.rapeli@linaro.org>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The PGO (Profile-guided Optimization) collect data about the typical execution of a program
and then use this data to inform optimizations such as inlining, machine-code layout,
register allocation, etc.
This optimization is by default disabled in rust sources but enabled in Yocto and causing
the reproducibility issue in rustdoc binary. To fix the issue this optimization is set to
it's default 'false'.
More about the optimization: https://doc.rust-lang.org/rustc/profile-guided-optimization.html
With the reproducibility issue fixed, we can enable the reproducibility tests again.
(From OE-Core rev: 189c266378c8c4a918cb205b3888577c7ce76856)
Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The target_dumper code is basically broken. It has been reading binary files
over the text base serial communication and runs at every command failure which
makes no sense. Each run might overwrite files from the previous run and the
output appears corrupted due to confusion from the binary data.
For now, remove the commands and the target dumper code as the command
and execution point are problematic. Also remove the same pieces of the monitor
code but leave the command list since in theory this can be moved to a more
useful place in the code.
(From OE-Core rev: a24d787987dccc95fdd95b7e85bf525a1c55b285)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whilst debugging an autobuilder failure, I wondered why it was rebuilding qemu-system-native
instead of reusing from sstate. The reason was it was overwriting DISTRO_FEATURES,
in this case removing opengl which caused much to rebuild.
The test doesn't need that so don't do it.
(From OE-Core rev: fdcc011608fd9558a081d0ace3eaf7192d9fcaef)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Similar to stdout in the previous commit, we need to ensure serial output
if written is read and put somewhere, else qemu might block on writes to
the serial port leading to hangs in the kernel. Use our existing logging
thread to log data when run_serial is not in use.
(From OE-Core rev: 05761282ba31e4ba3594f7321e2162d01fe12a5f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to ensure we read from and log the output from qemu stdout
as otherwise the buffers can fill and block, leading qemu to hang.
Use our existing logging thread to do this.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
(From OE-Core rev: a9c46ee014ef1e6436b39fdd4fd15d15388ea795)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'maturin develop' first checks that a virtual environment
has been created, which is a good test for our python3 SDK
environment ;)
Source for guessing-game lifted from https://www.maturin.rs/tutorial
The test case is expected to fetch any necessary crates, build a
development version of the crate and package it as a wheel
Needs at a minimum the following in e.g. local.conf:
TOOLCHAIN_HOST_TASK:append = " nativesdk-python3-maturin"
SDK_INCLUDE_TOOLCHAIN = '1'
SDK_TOOLCHAIN_LANGS += 'rust'
The output of 'maturin develop' should be something like:
...
🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.8
🐍 Not using a specific python interpreter
📡 Using build options features from pyproject.toml
...
Compiling guessing-game v0.1.0 (/path/to/guessing-game)
Finished dev [unoptimized + debuginfo] target(s) in 7.14s
📦 Built wheel for abi3 Python ≥ 3.8 to /path/to/tmpdir/guessing_game-0.1.0-cp38-abi3-linux_x86_64.whl
🛠 Installed guessing-game-0.1.0
(From OE-Core rev: 5265dd0b102cd7f3c6bb2ae1b18e9f625b834b39)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We expect 'maturin' will be used in SDKs, so it makes sense to also
test it in the testsdk environment.
To run this test case, you can add the following to local.conf:
TOOLCHAIN_HOST_TASK:append = " nativesdk-python3-maturin"
And then build and test the SDK:
bitbake -c populate_sdk core-image-full-cmdline
bitbake -c testsdk core-image-full-cmdline
You can substitute a different image recipe for "core-image-full-cmdline"
(From OE-Core rev: 7ceff48625d01a0e60eb761a9a668d0c942cda89)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Basic smoke test for maturin to test the 'maturin list-python' case.
(From OE-Core rev: 47c948c3cf6e582abd12021ceeff2c20a3e81fb5)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the new python_maturin PEP-517 backend
Add selftest for 'pydantic-core' pypi package.
(From OE-Core rev: 69b679380616a94a631681caa05d9bf7610f9372)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the rework of printdiff, it is not longer useful for checking
absence of sstate objects in a remote http cache, as it would only
report the top level missing signatures, and leave the recursive
investigation to diffsigs (which relies on ability to list cache
files - not available over http).
The CDN check can be performed by simply running 'bitbake -DD -n'
which is very verbose, but neverthless reports the amount
of missing sstate objects and what they are in a way that can
be programmatically extracted and checked (as suggested by RP).
This also adds local sstate tests, as they can be useful to
determine whether the missing cdn objects were never created or
erroneously cleaned up, or if they were created but didn't propagate
to cdn.
[YOCTO #15303]
(From OE-Core rev: 2a7c653a2eee85e5791a8fdc15857367f0ed0bd9)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are some issues with the printdiff code this has identified, disable the
test for now until we have patches to resolve them.
(From OE-Core rev: 436766983568a8bddc4b9ffa28dc656bf4bf67c1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add new file for C and C++ build tools. The initial implemmentation
contains a class for CMake and one for Meson. At least these first
tests for the qemu-usermode share most of the code. That's why there
is only one c_ccp.py file and not for example a cmake.py and a
meson.py file.
(From OE-Core rev: 41390f5202a6ee7472cb82d12c7c32f89d6e52ff)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #15314]
test_recipetool_appendsrcfile_update_recipe_basic is using base-files as
test recipe but modifies it directly which can corrupt metadata for other
tests relying on this recipe.
So use mtd-utils-selftest as test recipe from meta-selftest to avoid
this kind of issues
(From OE-Core rev: bf5e6c1b6ceca5a2eda30359d5e5e330278a97e1)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
recipetool pypi plugin was originally clobbering SRC_URI checksums.
Now it doesn't do this anymore:
78ef0313ee - recipetool: pypi: do not clobber SRC_URI checksums
so add back the checksum checks on pypi tests.
Also this commit restrict the checksums:
45d2f8d4bc - recipetool: create: Only include the expected SRC_URI checksums
so add only the needed ones.
(From OE-Core rev: 86164f770032bb66d4497c4e3e7591b7246ac2d9)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a task is adde which has a dependency on the do_populate_sysroot task of
the recipe, it will cause it to be installed into the sysroot (similar to
do_addto_recipe_sysroot). This fails since the postinst script is an overlapping
file:
Exception: FileExistsError: [Errno 17] File exists:
'tmp/sysroots-components/all/useraddbadtask/usr/bin/postinst-useradd-useraddbadtask'
->
'tmp/work/all-poky-linux/useraddbadtask/1.0/recipe-sysroot/usr/bin/postinst-useradd-useraddbadtask'
The copy written out at do_prepare_recipe_sysroot time is just for debug so
rename it, meaning there are no longer overlapping files and the installation
can be successful, removing the error.
[YCOTO #14961]
With the bug fixed, enable the test.
(From OE-Core rev: 564339afb73fc52a66c1a08437587cad1c4d46e7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is already done for local stamps just above, and will allow enabling
the full selftest that compares gcc-source signatures via printdiff
(that is, both local stamp and sstate variants).
(From OE-Core rev: 29775b5ecfc8d811293962f050fcfc3b3ad7efde)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the tests fail, these contain useful artefacts, and so should
be kept. If the test succeeds the whole build-st/ is deleted.
Also, give them unique names, as otherwise the tests would
step on each other.
(From OE-Core rev: 92e33a19fbcc6c59199fcd8b17ad8ca29ebcd4fd)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Assert*() functions from python unittest would join the multiline output with \n, making it
almost unreadable.
(From OE-Core rev: 1b01a71e77f70af77887c27be21265ac61f2c9a7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a disabled a test for 14961 - addtask between do_populate_sysroot and do_package breaks useradd class.
A fix is still needed for this.
(From OE-Core rev: b6af5788f7f8fb1e9d8ad14bd12168ff9d6baa21)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If recipe A requires the useradd actions of recipe B we need to
ensure that recipe B is part of the recipe A dependancy chain. In
order to do that, we introduce USERADD_DEPENDS. This makes sure
that the do_populate_sysroot_setscene of recipe B exists for
recipe A in case of a missing TMPDIR. This requires changes made in
runqueue.py by RP.
This commit along with the runqueue fixes effects:
Bug 13419 - recipes that add users to groups cannot rely on other recipes creating those groups (when population from sstate happens)
Bug 13904 - do_prepare_recipe_sysroot: postinst-useradd-* does not run in order of dependency and sometimes fails
Bug 13279 - Make sure users/groups exist for package_write_* tasks
Bug 15084 - For some reason using of same user in two recipes does not work properly
I've included the start of self-testing for useradd by adding tests for
13419 (which ends up testing 13904, 13279, 15084 by virtue of them all
having the same root cause)
(From OE-Core rev: b47f2352376bd16b7e7087b4dab143403e67e094)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When enabled in DISTRO_FEATURES the test may run on a system without systemd.
Fix this.
(From OE-Core rev: c2b473390dec0f5132d5b4bff6d3c35214eb898b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
