Additional test cases for debug symlink generation both binaries
and static libraries.
This also has the side effect of testing for race conditions in the
hardlink debug generation and stripping.
(From OE-Core rev: 7171f41c07a39a7543bb64f075d38b8e74563089)
Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When PACKAGE_STRIP_STATIC is enabled the system did not pay attention to
hardlinks. This could trigger a race condition during stripping of static
libraries where multiple strips (through hardlinks) could run at the same
time triggering a truncated or modified file error.
The hardlink breaking code is based on the existing code for elf files, but
due to the nature of the symlinks needed to be done in a separate block of
code.
Add support for static-library debugfs hardlinking through the existing
inode processing code.
Print a note to the logs if the link target can't be found. This isn't
strictly an error, but may be useful for debugging an issue where a file
isn't present.
(From OE-Core rev: ff371d69f60a1529ed456acb7d8e9305242e74bd)
Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will allow 'lockstep upgrades' of such recipes, improving success
rates in automated version updating process.
devtool check-upgrade-status now prints:
These recipes need to be upgraded together {
glib-2.0 2.80.2 2.80.4 Anuj Mittal <anuj.mittal@intel.com>
glib-2.0-initial 2.80.2 2.80.4 Anuj Mittal <anuj.mittal@intel.com>
}
These recipes need to be upgraded together {
util-linux 2.39.3 2.40.2 Chen Qi <Qi.Chen@windriver.com>
util-linux-libuuid 2.39.3 2.40.2 Chen Qi <Qi.Chen@windriver.com>
}
These recipes need to be upgraded together {
cmake 3.29.3 3.30.0 Unassigned <unassigned@yoctoproject.org>
cmake-native 3.29.3 3.30.0 Unassigned <unassigned@yoctoproject.org>
}
etc.
(From OE-Core rev: 7874aea5c62be3e8dbd19e04fce5389c5ed7aab6)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This functionality is needed for 'lockstep version upgrades' where several
recipes need to be upgraded at the same time to produce a buildable
outcome.
The function itself obtains BBINCLUDED for each recipe and then massages
the data until it takes the form of a list of sets:
[{'cmake','cmake-native'},
{'qemu','qemu-native','qemu-system-native'},
... ]
There's also a selftest that checks for the above.
Unfortunately this won't detect mutually exclusive recipes like mesa and mesa-gl
as they're chosen with PREFERRED_PROVIDER and can't be enabled in the same build
at the same time. ('devtool upgrade' will also accept just one of them but not the other)
(From OE-Core rev: 2400920f8b84cca9d6c1f6a2e850630554fe00fa)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Putting various things in a tuple is an anti-pattern of sorts, as the consumers
have to unpack it into local variables for readability, or access items directly
with indexes, which makes code pretty much unreadable.
(From OE-Core rev: e86aa26d209eb9809198f6dd40cd058366318e3d)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Improve 'unknown status' error message by including the CVE id and by
quoting status field. Otherwise it might be troublesome to understand
messages like 'do_create_spdx: Unknown CVE status ' (the status was
empty).
(From OE-Core rev: 45f64e69a63b544d560e2e5eb05a6e1f058c4da1)
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We do see a few reproducible issues are depending on the path length of the build directory.
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15554
The current implementation of reproducible tests having different names for directories but with same length.
The build directory names are changed to have different length.
(From OE-Core rev: 2466ed34a8e5afd8b6f83173316821b78edf954b)
Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Combining the test result collection and artefact collection hasn't worked out
well as the data has different life cycles, the artefacts can be large and
we need to be able to clean them up on a different timescale.
Separate them out to be controlled by a separate variable, OEQA_ARTEFACT_DIR.
Also rework the code to inject a directory with a date/time and random component
to allow builds to run in parallel. Pass function arguments to avoid re-reading
variables.
(From OE-Core rev: e1cf7e94c3fcbe7dbc29e4286f0e1014b95964a9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Validate that out-of-tree kernel module building using kernel-devsrc
works as expected.
This test uses cryptodev-linux as a idiomatic out of tree module. As the
latest release doesn't actually build with kernel 6.7+, use the same
commit that our recipe uses.
(From OE-Core rev: 8a1c1054815ecc0302c62134f293b8e1f959798a)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the bulk of the python code in the SPDX 3.0 classes into a library
file
(From OE-Core rev: aed6f8c1c2e291bde4d7172742790fa535b2fc7d)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moves the bulk of the code in the spdx-common bbclass into library code
(From OE-Core rev: 3f9b7c7f6b15493b6890031190ca8d1a10f2f384)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds test cases for SPDX 3.0. Reworks the SPDX 2.2 test setup so it can
also be run even if the default is SPDX 3.0
(From OE-Core rev: b1d2309b3ab0fd8b0d8c4dfa59f50c85074bbd3b)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a class to generate SPDX 3.0 output and an image class that is used
when generating images
(From OE-Core rev: b63f6f50458fc6898e4deda5d6739e7bf3639c15)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
SPDX 3.0 introduces a bunch of new SPDX tasks. Instead of explicitly
enumerating them all, modify the regex to match 'create_.*spdx' which
will cover all of the SPDX 2.0 and SPDX 3.0 tasks
(From OE-Core rev: 1d007eaf0155b5ac1c90a15634f7b1473743a4bd)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent ERROR_QA changes adding build_deps exposed what were warnings in thi
test as it restricts the locales, then builds recipes with locale requirements
that were not met (in bash, gawk and python3).
Increase the list of generated locales to ensure this patches the recipes
being built and hence allowing the build to succeed.
(From OE-Core rev: ebe2df8ab6e9d4087f8ddbffbc84eea6c4405738)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Recent changes to ERROR_QA meant that the build was failing in ways the test
wasn't expecting. Tweak the tests to ensure those failures don't impact the test.
(From OE-Core rev: c206a87f2e7128308e38a046b1dcd8cc43c91125)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It looks like most of the assertions of this tests were by-passed
because of combinations of invalid regexes and inverted assert
statement. Try to fix.
Use the variables from the kernel recipe instead of potentially invalid
values from get_bb_vars with undefined recipe. Use one get_bb_vars
statement instead of many seaparate calls mainly to improve the duration
of this test case.
Make the test working for an its file with an absolute path to the
kernel binary. This will be needed with one of the following commits and
does not harm in general.
(From OE-Core rev: 741f0cfa4f0025a2823581cab09e0bf14382e54d)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use more specific assert statements to make the code more readable and
the error messages more verbose.
Cleanup the asserts of the test case test_initramfs_bundle.
(From OE-Core rev: c7901004138d215612816072f20d367d638d2669)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than writing hints into log files and verify the hints can be
found, the tests should verify that the artifacts in the deploy folder
are correctly signed. This is a much better test.
u-boot-tools provide a utility fit_check_sign which can verify the
signatures in fit images. Lets use it.
grepping in temp/run. or temp/log. files also does not work if the tasks
runs from sstate and the corresponding run file is not even generated.
(From OE-Core rev: 86e504b4f792eeadd67ea57dd71a62bcb4f16f02)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Eliminate code duplication with a new function.
Use get_bb_var instead of bitbake -e | grep.
(From OE-Core rev: 0c7b7c5c98217b5f139058533159f61d1989d46e)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It does not always make sense to collect artifacts and data from the
target on failure, e.g. if testing firmware or if the target is not
running an SSH server.
Allow this by setting TESTIMAGE_FAILED_QA_ARTIFACTS to an empty
value.
(From OE-Core rev: 7036f814baa6550e0b1ac2db37fcd51960ee9e48)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO 15536]
The postactions retrieval actions currently rely on scp executed
individually on any file or directory expanded from
TESTIMAGE_FAILED_QA_ARTIFACTS. Unfortunately, symlinks are not preserved
with this mechanism, which lead to big storage space consumption. Things
may go even worse if those symlinks create some circular chains. This
mechanism then needs to be updated to preserve symlinks instead of
following them during copy. There are multiple ways to do it:
- create a local archive on the target and execute scp on this file
- use rsync instead of scp for all files
- create an archive and pipe it to ssh instead of storing it onto the
target
The first solution may create pressure on targets storage space, while the
second assumes that rsync is installed on the target, which may not be
true. So the third one is a compromise: tar is very likely present, at
least through busybox, and no disk space is used on the target.
Replace the current per-file scp call by a single call to tar run on the
target. Retrieve the generated compressed archive directly from SSH output,
and feed it to another tar process but on host, to uncompress and extract
it at the same place as before.
(From OE-Core rev: 4aeb10aa38efc6768928fbb74985e36e972b8e46)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ssh target is currently well tailored to easily retrieve textual output
from a command run on a remote target. It could also be used to retrieve
raw data from a command run onto a remote target (for example, to feed this
data directly to another program), but it currently suffers two minor
issues preventing such use case:
- stderr is piped to stdout, so any error log will be mixed in the program
output
- the final output is decoded as utf-8 and stripped
Allow to return the raw, unmodified output by adding an optional "raw"
parameter. Keep it to False by default to preserve the current behavior.
When enabled, do not return a string but the raw output as bytes.
(From OE-Core rev: 8d05dc6e2284b7ed7c32a8215b9c8bf6f7dabf00)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the retry for the -SIGTERM, it is possible to still see that error
after the 5th attempt and mark the run a success. Check for any
non-zero status in the final check and error out to close the gap.
While there, make the error print match the one above and be a little
more verbose. Also, I'm seeing it take roughly 6 attempts on my local
(very slow) system to pass. So, increasing the number of attempts to
10.
(From OE-Core rev: 3c3ebe591eef6e0479d623ec2237cfea16db5c80)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're seeing reproducibility failures where some packages don't appear
in the "from sstate" builds. The common factor is these are all recipes
with PROVIDES = "virtual/XXX".
In a full build from scratch, these are build but in a build from sstate,
there are situations where they aren't. For now, to try and keep builds
working, work around the problem until we can better look into the problem.
It is likely recent taskhash imrovements have caused this to occur more
regularly.
(From OE-Core rev: 68086a333acc54390e4e589ef928dc90da3edb48)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure we show the failing exit code in case of failures.
We're seeing autobuilder failures with -15 (SIGTERM) which is probably from
slow boot/init. Retry in these cases for now.
(From OE-Core rev: 127d3bd8d5509ae17e359c1365859fd362ffc74f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On exceptionally slow systems, the ssh test can intermittently fail due
to a race between when ping works and the networking applications being
brought up. To work around this issue, add some retry logic when ssh
fails to connect. According to the man page of ssh, "ssh exits
with the exit status of the remote command or with 255 if an error
occurred." So, only retry if the return code is 255, and limit the
number of retries to prevent it looping forever.
(From OE-Core rev: f0fe0b490d309cdf1c97754f85a61b5b948b7f28)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
During testing we're finding some kernel oops messages and other key kernel
errors such as irq issues are not triggering our log parsing.
Add those keywords to the list of things to scan for, making such
failures much more visible.
(From OE-Core rev: ef4623be60226e8caaf7813705aa4941ff354eac)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the process_pkgconfig() only handles the "Requires" key
field from the .pc file and ignores the "Requires.private" field while
generating the dev dependency chain. This results in a broken dependency
list and requires the installation of recommended packages to resolve
the build dependencies when using the dev IPKs. This increases the
hard disk space usage and download time and installs many unnecessary
packages as part of the recommendations.
This patch ensures that the "Requires.private" field is also considered
when creating the dependency list for the dev IPKs. With this, the dev
IPK will have the proper dependencies listed, eliminating the need to
install recommended packages to resolve the build time dependencies.
Example: usr/lib/pkgconfig/libical.pc
----------
Libs: -L${libdir} -lical -licalss -licalvcal
Libs.private: -lpthread
Requires.private: icu-i18n
----------
Depends field generated for libical-dev
Depends: glib-2.0-dev, libical (= 3.0.7-r0)
------------
When trying to resolve the build time dependency with libical
package using “-dev” ipk generated, it will throw the below error.
-----------
Package icu-i18n was not found in the pkg-config search path.
Perhaps you should add the directory containing `icu-i18n.pc'
to the PKG_CONFIG_PATH environment variable
Package 'icu-i18n', required by 'libical', not found
-----------
This patch will fix the broken dependency list.
-------
libical-dev depends field generated with this patch
Depends: glib-2.0-dev, icu-dev, libical (= 3.0.7-r0)
-------
Other examples of packages generated with broken dev dependency.
libflac-dev : https://packages.debian.org/sid/libflac-dev
Without patch:
Depends: flac (= 1.3.3-r0), libflac, libflac++
with patch:
Depends: flac (= 1.3.3-r0), libflac, libflac++, libogg-dev
libglib2.0-dev : https://packages.debian.org/buster/libglib2.0-dev
without patch:
Depends: libffi-dev, libglib-2.0-0 (= 1:2.62.6-r0), libpcre-dev
with patch:
Depends: libffi-dev, libglib-2.0-0 (= 1:2.62.6-r0), libpcre-dev,
util-linux-dev, zlib-dev
(From OE-Core rev: 4b5c8b7006aae2162614ba810ecf4418ca3f36b4)
Signed-off-by: Sreejith Ravi <sreejith.ravi087@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This recipe needs to be renamed because the "-cross-" substring in the
name triggers the cross architecture detection in sstate, but this
recipe is not actually a cross recipe.
(From OE-Core rev: 812c114a8a872ad59b19c7ffb8c1f230fc64c823)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix this error where 'coredumpctl info' warns that the coredump is still being
processed:
```
AssertionError: 1 != 0 : MiniDebugInfo Test failed: No match found.
-- Notice: 1 systemd-coredump@.service unit is running, output may be incomplete.
```
(From OE-Core rev: ad1ce64f5c1f22a7b10025d8cba20dc74354ac81)
(From OE-Core rev: ed562345d5a5f2edb649028553199f3f7966e19e)
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Where we're using DL_DIR in sdk archive to try and cache testing artefacts,
copy into the cache so that it gets populated and this doesn't have to be done
manually. Currently we're making a lot of repeat requests to github as this
wasn't being populated.
(From OE-Core rev: a3284958a2cc6c90a5fac26976bddc23f821c972)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add test_devtool_upgrade_recipe_update_extra_tasks test case
to test upgrade of python3-guessing-game from v0.1.0 to v0.2.0
which will exercise the update_crates task during the upgrade.
Add python3-guessing-game_git.bb.upgraded and
python3-guessing-game-crates.inc.upgraded which are the 0.2.0
variants.
Check that the new recipe file has the expected differences.
Check that the new -crates.inc file has the expected differences,
which should be reproducible because of Cargo.lock.
(From OE-Core rev: d14368bc775cbf5142c1312dfc2076e328381aef)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the test for the addition of SPDX_VERSION to the deploy path.
(From OE-Core rev: 055e8c21908127722abad9e992d6408d8697a119)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
test_systemd_coredump_minidebuginfo was getting skipped in CI, because the feature isn't enabled per default in any image at the moment.
Add this selftest so that test_systemd_coredump_minidebuginfo gets executed in CI.
(From OE-Core rev: 377603886f0d975ec23f32ee462693d4e3370aaf)
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Usage and arguments are not necessarily self-explaining, thus add a bit
of documentation here.
(From OE-Core rev: 62343333d50b75ad04a4220bcf0990e0a89996a0)
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test was failing with core-image-full-cmdline which uses sleep from coreutils instead of sleep from busybox.
(From OE-Core rev: 8497edf0c56da34ea7b7509b8fcd46fcba2fd0e3)
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Recent changes to the selftest-hello recipe caused the patchtest
selftest patches to not apply cleanly, which in turn meant several tests
weren't being fully executed or were reporting unexpected PASS/SKIP.
Update all of the test patch files to use a newer baseline and apply
cleanly again.
(From OE-Core rev: d73373d8a590d68e448af2e9ac600adc85777604)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The io, cap_bounds and filecaps groups were removed in latest version
of ltp. Tweak the test class to match.
(From OE-Core rev: d5460f15cbc56e52cbbc09785a777278208da7f7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Copying in the bb_unihashes cache file was at best a hack and creates a number of
challenges. One is staying in sync with bitbake since it may not have saved the
most recent version of the file. A second is a determinism problem since there may
be more entries in the file than the SDK should have had access to.
To improve the situation, add code to write the data into the locked-sigs.inc file
such that even when locked-sigs aren't used, the right hash mappings are injected
by the get_cached_unihash call.
The functions in copy_buildsystem need to be updated to preserve data they're not
editting.
(From OE-Core rev: 11373def3171e75b3b74ef694da213dd21f3064c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The purpose of this translation API is to make sure that the file name
can be used in a bitbake variable without adding special meaning,
therefore the ":" should also be replaced since it is the override
separator
(From OE-Core rev: b87eb329f20e40c1658460af7ff649aa099f0f36)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds API to reverse the removal of special characters from file names so
it can be correctly done in multiple places without open-coding it.
Replace the translation done in the package_rpm.bbclass with the new API
(From OE-Core rev: 4cb7e93c624987d146aaf626ce8e99568e938a70)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The builtin command "sleep" was getting executed instead of the busybox binary.
Hence "pidof sleep" was empty and the test was failing.
(From OE-Core rev: 3188f3f0718b8a59bc9e3cae4cdf74e82bdb86c3)
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drops support for this variable since bitbake no longer supports
parallel queries to the hash equivalence server
(From OE-Core rev: b0769794ca858baa6e814e81b14cf542c33d3a09)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
