Add status information for each CVE under analysis.
Previously the information passed between different function of the
cve-check class included only tables of patched, unpatched, ignored
vulnerabilities and the general status of the recipe.
The VEX work requires more information, and we need to pass them
between different functions, so that it can be enriched as the
analysis progresses. Instead of multiple tables, use a single one
with annotations for each CVE encountered. For example, a patched
CVE will have:
{"abbrev-status": "Patched", "status": "version-not-in-range"}
abbrev-status contains the general status (Patched, Unpatched,
Ignored and Unknown that will be added in the VEX code)
status contains more detailed information that can come from
CVE_STATUS and the analysis.
Additional fields of the annotation include for example the name
of the patch file fixing a given CVE.
We also use the annotation in CVE_STATUS to filter out entries
that do not apply to the given recipe
(From OE-Core rev: 452e605b55ad61c08f4af7089a5a9c576ca28f7d)
Signed-off-by: Marta Rybczynska <marta.rybczynska@syslinbit.com>
Signed-off-by: Samantha Jalabert <samantha.jalabert@syslinbit.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
CVE_STATUS contains assesment of a given CVE, but until now it didn't have
include the affected vendor/product. In the case of a global system include,
that CVE_STATUS was visible in all recipes.
This patch allows encoding of affected product/vendor to each CVE_STATUS
assessment, also for groups. We can then filter them later and use only
CVEs that correspond to the recipe.
This is going to be used in meta/conf/distro/include/cve-extra-exclusions.inc
and similar places.
(From OE-Core rev: abca80a716e92fc18d3085aba1a15f4bac72379c)
Signed-off-by: Marta Rybczynska <marta.rybczynska@syslinbit.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add some of the upsupported/failing tests to the exclude list
and ignore the failing unit tests.
(From OE-Core rev: 81bbd6db47f45628ff4be400e1f2fa5b09ccd0bb)
Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current postaction module executes a remote tar command, pipe it in a SSH
connection, and uncompress the raw stream with another tar command. With
this command, the whole artifacts tree is directly available on the host
executing the test, but it is not very convenient if we want to download
the whole retrieved ptests directory.
Stop uncompressing the retrieved ptests archive onto host, just save the
archive as it is. The new output then looks like the following:
tmp/log/oeqa-artefacts
└── oeqa-target-artefacts-20240812-juzqdb80
├── host_disk_usage.txt
├── target_disk_usage.txt
└── tests_artifacts.tar.gz
Suggested-By: Alexandre Belloni <alexandre.belloni@bootlin.com>
(From OE-Core rev: f90894d996c8a8f980e46c87b7968b176793b3fe)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current exception handler in list_and_fetch_failed_tests_artifacts
expects a non-exisiting variable and then fail to display the original
exception message since it raises a new one. The issue has been introduced
with commit 6e80b2ab66 ("oeqa/utils/postactions: transfer whole archive
over ssh instead of doing individual copies"). Now that tests artifacts are
now handled individually, there's no point of trying to print individual
names in the exception.
(From OE-Core rev: a50e72bb64fb8b0d14c23164eaeeabd9c271ac19)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
while runnig oe-selftest for gcc, testcases that need to be run on qemu are not running due to below failures.
- Executing on ssh: mkdir -p /tmp/runtest.3549641 (timeout = 300)
spawn [open ...]
Host key verification failed.
ERROR: Couldn't create remote directory /tmp/runtest.3549641 on ssh
Host key verification failure is happening when ssh board config file name is
defined as "ssh.exp" and there are multiple ssh.exp files generated during the build
and a wrong ssh config was taken. To resolve this changed the board config file name
to "linux-ssh.exp" which ensures correct ssh settings are used.
(From OE-Core rev: d32ec0b4d242879ab8eaf96e1cb407e8f0bb9f3d)
Signed-off-by: Harish Sadineni <Harish.Sadineni@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Drop "--doc" option for rust oe-selftest since
it is not supported on bootstrap builds for
cross-targets.
* Drop the following backported patches which are merged
with rust v1.76 upgrade.
- custom-target-cfg.patch
- rustc-bootstrap.patch
- rv32-missing-syscalls.patch
- target-build-value.patch
https://blog.rust-lang.org/2024/02/08/Rust-1.76.0.html
* Drop 'rust-rustdoc' and 'rust-dbg' from 'exclude_packages' list
to check for rust reproducibility.
(From OE-Core rev: 71d17ed3c7be029fc68e9dd3f5d6c4aa72ef861a)
Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the failing tests in rust v1.76 to the exclude list
and add "ignore" tags to ignore failing unit test cases.
(From OE-Core rev: 75399802515ac423503e637281a4585dd00d7c75)
Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a command to bitbake-layers to list the machines available in the
current configuration.
(From OE-Core rev: 837d32dafc125d58bb11da990ac251bd5aad027e)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Loading a load of json files into a memory structure and stashing in a bitbake
variable is relatively anti-social making bitbake -e output hard to read for
example as well as other potential performance issues.
Defer loading of that data until it is actually needed/used in a funciton
where it is now passed as a parameter.
(From OE-Core rev: 6f21cc9598178288784ff451ab3c40b174c0ef3e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upstream has shuffled the sysctl registration via the commit:
commit d4ae80ffa64f87b9c355692b680b603add084e96
Author: Zhen Ni <nizhen@uniontech.com>
Date: Tue Feb 15 19:46:03 2022 +0800
sched: Move cfs_bandwidth_slice sysctls to fair.c
move cfs_bandwidth_slice sysctls to fair.c and use the
new register_sysctl_init() to register the sysctl interface.
Signed-off-by: Zhen Ni <nizhen@uniontech.com>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
The way that we have to configure our mips qemu platforms
results in an empty sysctl table registration and the following
harmless warnings:
"failed when register_sysctl_sz sched_fair_sysctls to kernel"
"failed when register_sysctl_sz sched_core_sysctls to kernel"
Adding them to our list of acceptable dmesg warnings.
(From OE-Core rev: 4cf678858ef6f2c3310ad8f26cac3e7e133d4f0a)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The format of the pci BAR warnings we get on qemu boots has
changed in 6.10+ via the following kernel commit:
commit dc4e6f21c3f844ebc1c52b6920b8ec5dfc73f4e8
Author: Puranjay Mohan <puranjay@kernel.org>
Date: Sat Nov 6 16:56:06 2021 +0530
PCI: Use resource names in PCI log messages
Use the pci_resource_name() to get the name of the resource and use it
while printing log messages.
[bhelgaas: rename to match struct resource * names, also use names in other BAR messages]
Link: https://lore.kernel.org/r/20211106112606.192563-3-puranjay12@gmail.com
Signed-off-by: Puranjay Mohan <puranjay12@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Since it doesn't appear that we can do regex's in parselogs
and the bar number is now in the middle of the message, we
go with a slightly wider format of the message to ignore.
(From OE-Core rev: 0a7126604b6536868600d43aff000a426384995c)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove some minor chunks of code in patch.py that serve no purpose.
(From OE-Core rev: e2146fe0cb8b3c22d2df1cda5a36f458b7313280)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'UPDATE' as a name is somewhat unfortunate as the variable is intended only for
the 'devtool upgrade' operation and devtool also has an 'update-recipe' operation.
(From OE-Core rev: 4467aa0661e233f44c4ce029428c67d88fccfc07)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you are running your builds inside an environment where you don't
have access to the build tree (e.g. an autobuilder where you can only
download final artifacts such as images), then debugging build failures
can be difficult - you can't examine log files, the source tree or
output files. When enabled, by default this class will retain the work
directory for any recipe that has a task failure in the form of a
tarball, and can also be configured to save other directories on failure
or always.
It puts these tarballs in a configurable location (${TMPDIR}/retained by
default), where they can be picked up by a separate process and made
available as downloadable artifacts.
(From OE-Core rev: e2030c0d747eb990b9ad10098c6b74d6f8f4e74e)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Newlib generally requires additional components to function. Skip the
cases where newlib is known to not work.
(From OE-Core rev: b9934755554e40d9980b90c3d541f4c702203561)
Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a create-spdx-* classes is processing documents, it needs to
find the document in a path that is related to the SSTATE_ARCH
when a packge is generated. The SSTATE_ARCH can be affected by
multilib configurations, resulting is something like armv8a-mlib.
When the image (or SDK) is being generated and the components are
collected, the system has no knowledge of the multilib arch and
will fail to find it, such as:
ERROR: meta-toolchain-1.0-r0 do_populate_sdk: No SPDX file found
for package libilp32-libgcc-dbg,
False sstate:libilp32-libgcc:armv8a-ilp32-mllibilp32-elf:14.1.0:r0:armv8a-ilp32:12:
sstate:libilp32-libgcc::14.1.0:r0::12:
Adding in the new SPDX_MULTILIB_SSTATE_ARCHS will provide a full
set of SSTATE_ARCHS including ones that contain the multilib
extension which will allow create-spdx-* to correctly find the
document it is looking for. This would also be valuable to any
other function doing a similar search through SSTATE_ARCH that may
have been extended with multilib configurations.
(From OE-Core rev: f1499c36c1054fc90f7b7268cc95285f2eca72f7)
Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Same change as previously made to the create-spdx-2.2.bbclass,
while debugsrc is almost always a file (or link), there are apparently
cases where a directory could be returned from the dwarfsrcfiles
processing. When this happens, the hashing fails and an error results
when building the SPDX documents.
(From OE-Core rev: 5262f9bbf86c4e529ff988d465ddad2d1db0b073)
Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enables usage of TCLIBC=picolibc extending OE functionality to build and use
picolibc based toolchains to build baremetal applications.
Picolibc is a set of standard C libraries, both libc and libm, designed for
smaller embedded systems with limited ROM and RAM. Picolibc includes code
from Newlib and AVR Libc, but adresses some of newlibs concerns, it retains
newlibs directory structure, math, string and locale implementations, but
removed the GPL bits used to build the library, swiches old C style code for
C18 and replaces autotools with meson.
This patch adds a picolibc recipe for the C library, a picolibc-helloworld
recipe that contains an example application and a testcase that builds it.
Picolibc can be built for ARM and RISCV architectures, its been tested both
for 32 and 64 bits, the provided example recipe produces the following output:
hello, world
Runqemu does not automatically show any output since it hides QEMU stderr which
is where the QEMU monitors output is directed to when using semihosting, but,
manually running the same QEMU command does work properly.
(From OE-Core rev: c7535ecaccb72ef21a61f9aec5c68e61fb4f6fb6)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch ensures that pkg_postinst_ontarget task is executed for read only rootfs when
read-only-rootfs-delayed-postinsts is set as IMAGE_FEATURES. The issue was that run-postinsts
could be uninstalled at the end of rootfs construction and that shouldn't happen for
the delayed usecase.
In addition to the fix, a test in meta/lib/oeqa/selftest/cases/overlayfs.py testing
the fix has been implemented.
(From OE-Core rev: 60f587475dda99eaa07848880058b69286b8900e)
Signed-off-by: Gassner, Tobias.ext <tobias.gassner.ext@karlstorz.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The SCP protocol was deprecated in favor of the SFTP.
For the legacy SCP protocol scp should be run with "-O".
Instead of adding "-O" on the scp_options ssh oeqa we can
require the openssh-sftp-server to be instaled on the target.
This way the test will work more deterministic regardless of
the host machine client used.
For the old fashion clients still using legacy SCP protocol
the openssh-sshd server will be used, for the new ones using
the SFTP the openssh-sftp-server will be picked.
(From OE-Core rev: 2f43da91ba20d18bc419bca7651bb383a51f20af)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Additional test cases for debug symlink generation both binaries
and static libraries.
This also has the side effect of testing for race conditions in the
hardlink debug generation and stripping.
(From OE-Core rev: 7171f41c07a39a7543bb64f075d38b8e74563089)
Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When PACKAGE_STRIP_STATIC is enabled the system did not pay attention to
hardlinks. This could trigger a race condition during stripping of static
libraries where multiple strips (through hardlinks) could run at the same
time triggering a truncated or modified file error.
The hardlink breaking code is based on the existing code for elf files, but
due to the nature of the symlinks needed to be done in a separate block of
code.
Add support for static-library debugfs hardlinking through the existing
inode processing code.
Print a note to the logs if the link target can't be found. This isn't
strictly an error, but may be useful for debugging an issue where a file
isn't present.
(From OE-Core rev: ff371d69f60a1529ed456acb7d8e9305242e74bd)
Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will allow 'lockstep upgrades' of such recipes, improving success
rates in automated version updating process.
devtool check-upgrade-status now prints:
These recipes need to be upgraded together {
glib-2.0 2.80.2 2.80.4 Anuj Mittal <anuj.mittal@intel.com>
glib-2.0-initial 2.80.2 2.80.4 Anuj Mittal <anuj.mittal@intel.com>
}
These recipes need to be upgraded together {
util-linux 2.39.3 2.40.2 Chen Qi <Qi.Chen@windriver.com>
util-linux-libuuid 2.39.3 2.40.2 Chen Qi <Qi.Chen@windriver.com>
}
These recipes need to be upgraded together {
cmake 3.29.3 3.30.0 Unassigned <unassigned@yoctoproject.org>
cmake-native 3.29.3 3.30.0 Unassigned <unassigned@yoctoproject.org>
}
etc.
(From OE-Core rev: 7874aea5c62be3e8dbd19e04fce5389c5ed7aab6)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This functionality is needed for 'lockstep version upgrades' where several
recipes need to be upgraded at the same time to produce a buildable
outcome.
The function itself obtains BBINCLUDED for each recipe and then massages
the data until it takes the form of a list of sets:
[{'cmake','cmake-native'},
{'qemu','qemu-native','qemu-system-native'},
... ]
There's also a selftest that checks for the above.
Unfortunately this won't detect mutually exclusive recipes like mesa and mesa-gl
as they're chosen with PREFERRED_PROVIDER and can't be enabled in the same build
at the same time. ('devtool upgrade' will also accept just one of them but not the other)
(From OE-Core rev: 2400920f8b84cca9d6c1f6a2e850630554fe00fa)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Putting various things in a tuple is an anti-pattern of sorts, as the consumers
have to unpack it into local variables for readability, or access items directly
with indexes, which makes code pretty much unreadable.
(From OE-Core rev: e86aa26d209eb9809198f6dd40cd058366318e3d)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Improve 'unknown status' error message by including the CVE id and by
quoting status field. Otherwise it might be troublesome to understand
messages like 'do_create_spdx: Unknown CVE status ' (the status was
empty).
(From OE-Core rev: 45f64e69a63b544d560e2e5eb05a6e1f058c4da1)
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We do see a few reproducible issues are depending on the path length of the build directory.
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15554
The current implementation of reproducible tests having different names for directories but with same length.
The build directory names are changed to have different length.
(From OE-Core rev: 2466ed34a8e5afd8b6f83173316821b78edf954b)
Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Combining the test result collection and artefact collection hasn't worked out
well as the data has different life cycles, the artefacts can be large and
we need to be able to clean them up on a different timescale.
Separate them out to be controlled by a separate variable, OEQA_ARTEFACT_DIR.
Also rework the code to inject a directory with a date/time and random component
to allow builds to run in parallel. Pass function arguments to avoid re-reading
variables.
(From OE-Core rev: e1cf7e94c3fcbe7dbc29e4286f0e1014b95964a9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Validate that out-of-tree kernel module building using kernel-devsrc
works as expected.
This test uses cryptodev-linux as a idiomatic out of tree module. As the
latest release doesn't actually build with kernel 6.7+, use the same
commit that our recipe uses.
(From OE-Core rev: 8a1c1054815ecc0302c62134f293b8e1f959798a)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the bulk of the python code in the SPDX 3.0 classes into a library
file
(From OE-Core rev: aed6f8c1c2e291bde4d7172742790fa535b2fc7d)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moves the bulk of the code in the spdx-common bbclass into library code
(From OE-Core rev: 3f9b7c7f6b15493b6890031190ca8d1a10f2f384)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds test cases for SPDX 3.0. Reworks the SPDX 2.2 test setup so it can
also be run even if the default is SPDX 3.0
(From OE-Core rev: b1d2309b3ab0fd8b0d8c4dfa59f50c85074bbd3b)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a class to generate SPDX 3.0 output and an image class that is used
when generating images
(From OE-Core rev: b63f6f50458fc6898e4deda5d6739e7bf3639c15)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
SPDX 3.0 introduces a bunch of new SPDX tasks. Instead of explicitly
enumerating them all, modify the regex to match 'create_.*spdx' which
will cover all of the SPDX 2.0 and SPDX 3.0 tasks
(From OE-Core rev: 1d007eaf0155b5ac1c90a15634f7b1473743a4bd)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent ERROR_QA changes adding build_deps exposed what were warnings in thi
test as it restricts the locales, then builds recipes with locale requirements
that were not met (in bash, gawk and python3).
Increase the list of generated locales to ensure this patches the recipes
being built and hence allowing the build to succeed.
(From OE-Core rev: ebe2df8ab6e9d4087f8ddbffbc84eea6c4405738)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Recent changes to ERROR_QA meant that the build was failing in ways the test
wasn't expecting. Tweak the tests to ensure those failures don't impact the test.
(From OE-Core rev: c206a87f2e7128308e38a046b1dcd8cc43c91125)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It looks like most of the assertions of this tests were by-passed
because of combinations of invalid regexes and inverted assert
statement. Try to fix.
Use the variables from the kernel recipe instead of potentially invalid
values from get_bb_vars with undefined recipe. Use one get_bb_vars
statement instead of many seaparate calls mainly to improve the duration
of this test case.
Make the test working for an its file with an absolute path to the
kernel binary. This will be needed with one of the following commits and
does not harm in general.
(From OE-Core rev: 741f0cfa4f0025a2823581cab09e0bf14382e54d)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use more specific assert statements to make the code more readable and
the error messages more verbose.
Cleanup the asserts of the test case test_initramfs_bundle.
(From OE-Core rev: c7901004138d215612816072f20d367d638d2669)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than writing hints into log files and verify the hints can be
found, the tests should verify that the artifacts in the deploy folder
are correctly signed. This is a much better test.
u-boot-tools provide a utility fit_check_sign which can verify the
signatures in fit images. Lets use it.
grepping in temp/run. or temp/log. files also does not work if the tasks
runs from sstate and the corresponding run file is not even generated.
(From OE-Core rev: 86e504b4f792eeadd67ea57dd71a62bcb4f16f02)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Eliminate code duplication with a new function.
Use get_bb_var instead of bitbake -e | grep.
(From OE-Core rev: 0c7b7c5c98217b5f139058533159f61d1989d46e)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It does not always make sense to collect artifacts and data from the
target on failure, e.g. if testing firmware or if the target is not
running an SSH server.
Allow this by setting TESTIMAGE_FAILED_QA_ARTIFACTS to an empty
value.
(From OE-Core rev: 7036f814baa6550e0b1ac2db37fcd51960ee9e48)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO 15536]
The postactions retrieval actions currently rely on scp executed
individually on any file or directory expanded from
TESTIMAGE_FAILED_QA_ARTIFACTS. Unfortunately, symlinks are not preserved
with this mechanism, which lead to big storage space consumption. Things
may go even worse if those symlinks create some circular chains. This
mechanism then needs to be updated to preserve symlinks instead of
following them during copy. There are multiple ways to do it:
- create a local archive on the target and execute scp on this file
- use rsync instead of scp for all files
- create an archive and pipe it to ssh instead of storing it onto the
target
The first solution may create pressure on targets storage space, while the
second assumes that rsync is installed on the target, which may not be
true. So the third one is a compromise: tar is very likely present, at
least through busybox, and no disk space is used on the target.
Replace the current per-file scp call by a single call to tar run on the
target. Retrieve the generated compressed archive directly from SSH output,
and feed it to another tar process but on host, to uncompress and extract
it at the same place as before.
(From OE-Core rev: 4aeb10aa38efc6768928fbb74985e36e972b8e46)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ssh target is currently well tailored to easily retrieve textual output
from a command run on a remote target. It could also be used to retrieve
raw data from a command run onto a remote target (for example, to feed this
data directly to another program), but it currently suffers two minor
issues preventing such use case:
- stderr is piped to stdout, so any error log will be mixed in the program
output
- the final output is decoded as utf-8 and stripped
Allow to return the raw, unmodified output by adding an optional "raw"
parameter. Keep it to False by default to preserve the current behavior.
When enabled, do not return a string but the raw output as bytes.
(From OE-Core rev: 8d05dc6e2284b7ed7c32a8215b9c8bf6f7dabf00)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the retry for the -SIGTERM, it is possible to still see that error
after the 5th attempt and mark the run a success. Check for any
non-zero status in the final check and error out to close the gap.
While there, make the error print match the one above and be a little
more verbose. Also, I'm seeing it take roughly 6 attempts on my local
(very slow) system to pass. So, increasing the number of attempts to
10.
(From OE-Core rev: 3c3ebe591eef6e0479d623ec2237cfea16db5c80)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're seeing reproducibility failures where some packages don't appear
in the "from sstate" builds. The common factor is these are all recipes
with PROVIDES = "virtual/XXX".
In a full build from scratch, these are build but in a build from sstate,
there are situations where they aren't. For now, to try and keep builds
working, work around the problem until we can better look into the problem.
It is likely recent taskhash imrovements have caused this to occur more
regularly.
(From OE-Core rev: 68086a333acc54390e4e589ef928dc90da3edb48)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
