Adds a test for several of the extra options provided by the SPDX
classes. In particular, these are the options that can produce
non-reproducible results, so are not enabled by default in OE core. This
test takes care to configure the build so that the tests do run in a
reproducible manner so that pre-built test objects can be pulled from
sstate
(From OE-Core rev: 14f534f15f7fe6362723d7f064d39783c5bd758f)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tweak the gitarchive exclude handling not to error if excluded files
don't match.
Also return the tagname created so that other code can then use it.
(From OE-Core rev: 1adba3430faffdf6217b6a00533a3b48a9388abc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using 4 space indentation in resulted in hundreds of megabytes of extra file size
in general use. Reduce this to make filesizes more managable and reduce the processing
cost. Some level of indentation and spacing does make the files more readable and allows
use of git diff so we need to retain some of it.
(From OE-Core rev: a274cdcaf852cca9497f0358f44dda99c06aacbe)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are several problems with these paths. Firstly they contain full
system paths which depend upon where the test was run. These are pretty
pointless and just take up a lot of space making the results files large.
Secondly, they contain the same path twice. The reference and target path
will always be the same thing in two different locations.
Strip off the prefix and remove the duplication. This does change the output
data but that can't really be avoided. It does shrink the results data and makes
it more readable.
(From OE-Core rev: 81a44de36e864b08687451fd85aeba7c529fd7f7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "rawlogs" data consists of a long string of results data which is
already in a structured data format. I can't see this is adding much
value in duplciating the data but it does create a huge string with a
lot of long problematic pathnames and inflates the results data size.
I suggest we drop this data as obsolete and not necessary.
(From OE-Core rev: 5b2c70fab2ffa409b861d83f048b65d458d03a90)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've seen cases where a ptest (strace) has a 43GB sparse file in the test
directory. busybox tar doesn't work well with this. The resulting 1.4GB archive
takes hours to extract too.
Ensure tar is added to our full images and use the sparse option to collect
files for debugging. This stops crazy build hangs.
Since tar is GPLv3, we have to exclude it from that test code. We don't boot
any of those images so the debug collection code is safe there, at least for now.
(From OE-Core rev: fefeb919696b6ac76f0997acfb0f612203ef7f1b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current code is not fit for purpose when handling large files via ssh. In the strace
ptest case, we can end up with a 1.4GB archive being transferred for which every
byte is printed into the task logfile twice over. This is then sent over bitbake IPC
which compounds the problems.
Make the following improvements:
* when the output is large (over 64kb), don't print it
* use a bytearray for better concat performance since strings are slow for this
* when there is no ssh output, say that
* print periodic size status output rather than the data itself since this could be binary and/or large
* fix the killed process message logic which appeared broken
(From OE-Core rev: e7dd009a17dc902852983a82bce41bf78bb1e242)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If SLIRP is being used instead of TAP for networking to the guest then
the target IP will be localhost. There's no point in pinging localhost
to see if the target is up but whilst you'd think it is harmless, in
some containers ping doesn't actually have enough rights to work:
ping: socktype: SOCK_RAW
ping: socket: Operation not permitted
ping: => missing cap_net_raw+p capability or setuid?
Look at the target address and if it's localhost or 127.0.0.* return
immediately.
(From OE-Core rev: a06ef43d2a50e16c32bd6edbdc7b32c3528687d5)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We had previously excluded this test case due to failure in earlier versions.
However, with the latest version of Rust, this test case is passing.
As a result, we have removed it from the exclude list and added back into
the test suite for execution.
The rust-1.79 has been successfully tested with the current test inclusion.
(From OE-Core rev: a33edee3efe67f81229496e03a098454dc7df051)
Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If Unified Kernel Image was enabled via IMAGE_CLASSES, then
target should also boot the same uki at runtime.
(From OE-Core rev: 2ae651c869a19fd24b5a915451c1f99e1d7b32f0)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Assigning IMAGE_FEATURES="" in local.conf doesn't actually do anything
useful, as bitbake.conf has IMAGE_FEATURES += EXTRA_IMAGE_FEATURES after
the include of local.conf.
In this test case, this results in IMAGE_FEATURES still using
EXTRA_IMAGE_FEATURES, so the image contains the files installed by the
post-install-logging feature.
As the intention here is that container-test-image is as minimal as
possible, move the IMAGE_FEATURES assignment into the image itself so
it actually works, and update the expected file list to remove the
post-install-logging files.
(From OE-Core rev: c65349e1238ddc6634dfa4759c57e6168a355200)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since [0], the reproducibility test tries to save logs of failed builds.
Using the memory persistent bitbake prevent this by concatenating logs in the
single persistent instance (See [YOCTO #15641]).
So, force a BitBake server restart for each build to ensure a new log
file is created.
This fixes this error (seen on AB [1]):
2024-11-02 08:16:00,816 - oe-selftest - ERROR - reproducibleA build failed. Trying to compute built packages differences but the test will fail.
2024-11-02 08:16:00,819 - oe-selftest - INFO - ... ERROR
2024-11-02 08:16:00,819 - oe-selftest - INFO - Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/meta/lib/oeqa/selftest/cases/reproducible.py", line 321, in test_reproducible_builds
self.copy_file(variables["BB_CONSOLELOG"], os.path.join(save_dir, "bitbake-%s.log" % name))
File "/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/meta/lib/oeqa/selftest/cases/reproducible.py", line 216, in copy_file
shutil.copyfile(source, dest)
File "/usr/lib/python3.11/shutil.py", line 256, in copyfile
with open(src, 'rb') as fsrc:
^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/build/build-st-meta-python/build-st/tmp/log/cooker/qemux86-64/20241102071141.log'
[0]: https://git.openembedded.org/openembedded-core/commit/?id=c78cc753843d4199443052e8deb0c9c3b7e4b580
[1]: https://valkyrie.yoctoproject.org/#/builders/87/builds/17/steps/40/logs/stdio
(From OE-Core rev: e89bbc00ba16574d719b199c01ffbf37646f4f54)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runCmd by default merges stderr into stdout, and only
needs to print stdout when errors occur. When stderr
is requested as a separate stream, and an error occurs,
stderr is discarded, obscuring useful error messages.
This changes the output to include both streams.
(From OE-Core rev: 72c747b37ccdd486ddae06e3d0a99fb2b93643ba)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add wic-tools to the PATH to avoid failures when running the
wic_sector_size test case on a build host that doesn’t have parted.
(From OE-Core rev: 013dcdf75669421bc38d699263cb1e8d5b95d398)
Signed-off-by: Vince Chang <vince_chang@aspeedtech.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The first no-fail check was an attempt to work around the old
CDN's instability (and it didn't really help); it should not be necessary
with the new CDN, and only delays a-full completion.
(From OE-Core rev: 7f75c42b7fcf60a9ca58d3ded9047df675d76dc2)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current reproducibility test stops all build tasks when a single
task fails (default BitBake behavior). This means that a single build
failure prevents the reproducibility of other packages from being
tested, which is not ideal.
To address this, we now use the --continue option of BitBake during test
builds, allowing the build process to proceed even when some tasks fail.
The failure cases are handled as gracefully as possible.
In the event of a build failure:
- The entire reproducibility test will be considered a failure.
- The complete BitBake log will be saved in the "saved output" directory
to facilitate debugging. On the autobuilder, this log should be
available at https://autobuilder.yocto.io/pub/repro-fail/.
- The last 20 lines of the log, which should show the failing recipes,
will be displayed in the oe-selftest console.
- If BitBake fails to create the deployment directory, it will be
manually created to allow the comparison process to proceed.
Here is what the output looks like when testing reproducibility of bash,
a failing recipe (hello-fail) and a non-reproducible recipe
(hello-norepro):
<snip>
2024-10-01 23:09:33,977 - oe-selftest - INFO - test_reproducible_builds (reproducible.ReproducibleTests.test_reproducible_builds)
2024-10-01 23:10:39,093 - oe-selftest - INFO - Non-reproducible packages will be copied to <snip>/yocto/repro_output/oe-reproducible-20241001-ng43_hzd
2024-10-01 23:10:39,094 - oe-selftest - INFO - Building reproducibleA (sstate allowed)...
2024-10-01 23:18:12,378 - oe-selftest - ERROR - Bitbake failed! but keep going... Log:
2024-10-01 23:18:12,379 - oe-selftest - INFO - Command 'bitbake --continue bash hello-norepro hello-fail' returned non-zero exit status 1:
2024-10-01 23:18:12,379 - oe-selftest - INFO -
2024-10-01 23:18:12,379 - oe-selftest - INFO - ... (last 20 lines of output)
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running task 976 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_package_qa)
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_populate_sysroot: Started
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Started
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_populate_sysroot: Succeeded
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running setscene task 342 of 343 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_create_package_spdx_setscene)
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx_setscene: Started
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx_setscene: Succeeded
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running setscene task 343 of 343 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_create_spdx_setscene)
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_spdx_setscene: Started
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_spdx_setscene: Succeeded
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Succeeded
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running noexec task 979 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_build)
2024-10-01 23:18:12,380 - oe-selftest - INFO - NOTE: Tasks Summary: Attempted 979 tasks of which 841 didn't need to be rerun and 1 failed.
2024-10-01 23:18:12,380 - oe-selftest - INFO - NOTE: Generating JSON CVE summary
2024-10-01 23:18:12,380 - oe-selftest - INFO - Complete CVE JSON report summary created at: <snip>/yocto/poky/build-master-st/reproducibleA/tmp/log/cve/cve-summary.json
2024-10-01 23:18:12,380 - oe-selftest - INFO -
2024-10-01 23:18:12,380 - oe-selftest - INFO - Summary: 1 task failed:
2024-10-01 23:18:12,380 - oe-selftest - INFO - <snip>/yocto/poky/meta/recipes-core/hello-single/hello-fail_1.0.bb:do_compile
2024-10-01 23:18:12,380 - oe-selftest - INFO - log: <snip>/yocto/poky/build-master-st/reproducibleA/tmp/work/core2-64-poky-linux/hello-fail/1.0/temp/log.do_compile.3221257
2024-10-01 23:18:12,380 - oe-selftest - INFO - Summary: There was 1 ERROR message, returning a non-zero exit code.
2024-10-01 23:18:12,380 - oe-selftest - ERROR - reproducibleA build failed. Trying to compute built packages differences but the test will fail.
2024-10-01 23:18:12,382 - oe-selftest - INFO - Building reproducibleB-extended (sstate NOT allowed)...
2024-10-01 23:46:58,451 - oe-selftest - ERROR - Bitbake failed! but keep going... Log:
2024-10-01 23:46:58,463 - oe-selftest - INFO - Command 'bitbake --continue bash hello-norepro hello-fail' returned non-zero exit status 1:
2024-10-01 23:46:58,463 - oe-selftest - INFO -
2024-10-01 23:46:58,463 - oe-selftest - INFO - ... (last 20 lines of output)
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_ipk: Started
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Started
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_spdx: Succeeded
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: Running task 978 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_create_package_spdx)
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx: Started
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx: Succeeded
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Succeeded
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_deb: Succeeded
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_ipk: Succeeded
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_rpm: Succeeded
2024-10-01 23:46:58,464 - oe-selftest - INFO - NOTE: Running noexec task 979 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_build)
2024-10-01 23:46:58,464 - oe-selftest - INFO - NOTE: Tasks Summary: Attempted 979 tasks of which 2 didn't need to be rerun and 1 failed.
2024-10-01 23:46:58,464 - oe-selftest - INFO - NOTE: Generating JSON CVE summary
2024-10-01 23:46:58,464 - oe-selftest - INFO - Complete CVE JSON report summary created at: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/log/cve/cve-summary.json
2024-10-01 23:46:58,464 - oe-selftest - INFO -
2024-10-01 23:46:58,464 - oe-selftest - INFO - Summary: 1 task failed:
2024-10-01 23:46:58,464 - oe-selftest - INFO - <snip>/yocto/poky/meta/recipes-core/hello-single/hello-fail_1.0.bb:do_compile
2024-10-01 23:46:58,464 - oe-selftest - INFO - log: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/work/core2-64-poky-linux/hello-fail/1.0/temp/log.do_compile.4136075
2024-10-01 23:46:58,464 - oe-selftest - INFO - Summary: There were 5 WARNING messages.
2024-10-01 23:46:58,464 - oe-selftest - INFO - Summary: There was 1 ERROR message, returning a non-zero exit code.
2024-10-01 23:46:58,467 - oe-selftest - ERROR - reproducibleB-extended build failed. Trying to compute built packages differences but the test will fail.
2024-10-01 23:46:58,481 - oe-selftest - INFO - Checking deb packages for differences...
2024-10-01 23:46:58,586 - oe-selftest - INFO - Reproducibility summary for deb: same=52 different=1 different_excluded=0 missing=0 total=53
unused_exclusions=[]
2024-10-01 23:46:58,588 - oe-selftest - INFO - Checking ipk packages for differences...
2024-10-01 23:46:58,658 - oe-selftest - INFO - Reproducibility summary for ipk: same=52 different=1 different_excluded=0 missing=0 total=53
unused_exclusions=[]
2024-10-01 23:46:58,659 - oe-selftest - INFO - Checking rpm packages for differences...
2024-10-01 23:46:58,691 - oe-selftest - INFO - Reproducibility summary for rpm: same=52 different=1 different_excluded=0 missing=0 total=53
unused_exclusions=[]
2024-10-01 23:46:58,692 - oe-selftest - INFO - Running diffoscope
2024-10-01 23:46:59,765 - oe-selftest - INFO - ... FAIL
2024-10-01 23:46:59,766 - oe-selftest - INFO - Traceback (most recent call last):
File "<snip>/yocto/poky/meta/lib/oeqa/selftest/cases/reproducible.py", line 380, in test_reproducible_builds
self.fail('\n'.join(fails))
AssertionError: Bitbake reproducibleA failure
Bitbake reproducibleB-extended failure
The following deb packages are different and not in exclusion list:
<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb
The following ipk packages are different and not in exclusion list:
<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/ipk/./core2-64/hello-norepro_1.0-r0_core2-64.ipk
The following rpm packages are different and not in exclusion list:
<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/rpm/./core2_64/hello-norepro-1.0-r0.core2_64.rpm
2024-10-01 23:46:59,769 - oe-selftest - INFO - ======================================================================
2024-10-01 23:46:59,770 - oe-selftest - INFO - FAIL: test_reproducible_builds (reproducible.ReproducibleTests.test_reproducible_builds)
2024-10-01 23:46:59,770 - oe-selftest - INFO - ----------------------------------------------------------------------
2024-10-01 23:46:59,770 - oe-selftest - INFO - Traceback (most recent call last):
File "<snip>/yocto/poky/meta/lib/oeqa/selftest/cases/reproducible.py", line 380, in test_reproducible_builds
self.fail('\n'.join(fails))
AssertionError: Bitbake reproducibleA failure
Bitbake reproducibleB-extended failure
The following deb packages are different and not in exclusion list:
<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb
The following ipk packages are different and not in exclusion list:
<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/ipk/./core2-64/hello-norepro_1.0-r0_core2-64.ipk
The following rpm packages are different and not in exclusion list:
<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/rpm/./core2_64/hello-norepro-1.0-r0.core2_64.rpm
2024-10-01 23:46:59,770 - oe-selftest - INFO - ----------------------------------------------------------------------
2024-10-01 23:46:59,770 - oe-selftest - INFO - Ran 1 test in 2246.039s
2024-10-01 23:46:59,770 - oe-selftest - INFO - FAILED
2024-10-01 23:46:59,770 - oe-selftest - INFO - (failures=1)
2024-10-01 23:47:03,200 - oe-selftest - INFO - RESULTS:
2024-10-01 23:47:03,200 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: FAILED (2245.79s)
2024-10-01 23:47:03,203 - oe-selftest - INFO - SUMMARY:
2024-10-01 23:47:03,203 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 2246.040s
2024-10-01 23:47:03,203 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed (successes=0, skipped=0, failures=1, errors=0)
=> Test failed but hello-norepro is displayed as non-reproducible.
The testresult.json contains:
{
"oeselftest_debian-12_qemux86-64_20240930000424": {
"configuration": { <snip> },
"result": {
"reproducible": {
"files": {
"package_deb": {
"different": [
{
"reference": "<snip>/yocto/poky/build-master-st/reproducibleA/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb",
"test": "<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb"
}
],
"different_excluded": [],
"missing": [],
"same": [
<snip>
{
"reference": "<snip>/yocto/poky/build-master-st/reproducibleA/tmp/deploy/deb/./core2-64/bash_5.2.32-r0_amd64.deb",
"test": "<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/bash_5.2.32-r0_amd64.deb"
},
<snip>
]
},
"package_ipk": { <same as deb> },
"package_rpm": { <same as deb> }
}
},
"reproducible.ReproducibleTests.test_reproducible_builds": {
"duration": 2146.5671875476837,
"log": "Traceback (most recent call last):\n File \"<snip>/yocto/poky/meta/lib/oeqa/selftest/cases/reproducible.py\", line 380, in test_reproducible_builds\n self.fail('\\n'.join(fails))\nAssertionError: Bitbake reproducibleA failure\nBitbake reproducibleB-extended failure\nThe following deb packages are different and not in exclusion list:\n<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb\nThe following ipk packages are different and not in exclusion list:\n<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/ipk/./core2-64/hello-norepro_1.0-r0_core2-64.ipk\nThe following rpm packages are different and not in exclusion list:\n<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/rpm/./core2_64/hello-norepro-1.0-r0.core2_64.rpm\n",
"status": "FAILED"
},
"reproducible.rawlogs": {
"log": "DIFFERENT: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb\nSAME: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/bash_5.2.32-r0_amd64.deb\n<snip>"
}
}
}
}
=> "reproducible.ReproducibleTests.test_reproducible_builds".status is
correctly "FAILED" but the reproducibility of bash and hello-norepro is
tested.
(From OE-Core rev: c78cc753843d4199443052e8deb0c9c3b7e4b580)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There were some rust tests which were FAILing only for mips architectures.
Hence, they were added to the exclusion list to avoid the failures when
running rust oe-selftest for mips architectures.The current yocto framework
disabled the rust oe-selftest for mips architectures.Hence, these tests have
been added back to the test framework.
The rust-1.79 has been successfully tested on X86, X86_64, ARM, AARCH64 and
PPC Architectures with the current test inclusions.
(From OE-Core rev: e1ec1ad75feb1321228bf3b52251e1aecfdb8746)
Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Build configuration is applied to bitbake build command
but removed before calling runqemu. Thus disabling KVM
support on aarc64 host was not effective. Note that this
pattern is used in a lot of tests. KVM gets enabled
via CI scripts.
(From OE-Core rev: 7a9fbf509fec2e149fd5702552bc4d819969ab1f)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
test_rawcopy_plugin fails if machine does not build
ext4 images by default. Make the ext4 image build
explicit in the test. Fixes test on genericarm64 machine
which defaults to wic image only.
(From OE-Core rev: 0344b9e7949ef854b09deb0874a45cde2861f55b)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use label to detect rootfs since UKI with kernel command
line is generated before rootfs is generated by wic.
Adapt wic tests to build and boot uki.bbclass generated
UKIs.
Keeping one UKI test in wic.py and rest of the UKI features
are tested with dedicated uki.py test. Add plain non-UKI
systemd-boot tests to wic suite for aarch64 and x86.
(From OE-Core rev: 3f94256b977637d4276f82db7c20b8b5e57b9d86)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To support "slirp" networking on shared build machines instead
of tun/tap devices. Users can set
TEST_RUNQEMUPARAMS = "slirp"
in their build/conf/local.conf to run selftests using "slirp"
networking. The same works for testimage.bbclass and oeqa runtime
tests.
(From OE-Core rev: 4974ec71367492ce314da63c359ccf99acfca882)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test is actually using grub-efi not systemd-boot so
remove it completely. systemd-boot will be tested via uki.py
tests.
(From OE-Core rev: 542ea853bb194629d23e1db9c05e25181bfce145)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
TEST_RUNQEMUPARAMS variable is used to add runqemu parameters like
"slirp" networking. Support this also in selftests so that "slirp"
networking can be used instead of the tun/tap devices setup which
is easier to work with on shared build machines.
(From OE-Core rev: 74e44e63378e1c08bb547a0a04428d88753b5040)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tests builds and boots qemu into uki binary with systemd and sysvinit.
Due to depedency to x86 specific ovmf UEFI firmware, tests
are specific to x86 curently. UEFI firmware for ARM can be generated
via qemuarm64-secureboot machine in meta-arm and similar tests
on qemu will pass.
(From OE-Core rev: 8a3cb17876dbcaf07696a4bcd454e2f9a444fb1b)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is helpful to see all the issues in the source mirror test so
use the continue option.
(From OE-Core rev: f52e7ddc6ca5ce03cbcdf3cc15fc04f3ff31cd5c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove unused imports, or disabling code which has been commented out.
(From OE-Core rev: e0ff4813b1cf4df0d851c857d57fb88d7db51bdd)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As per "The rustc book"[1], mips*-unknown-linux-* are "tier 3" targets:
Tier 3 targets are those which the Rust codebase has support for, but
which the Rust project does not build or test automatically, so they
may or may not work.
We already skip qemumips in this selftest, but we're now also seeing
failures with qemumips64 so refactor the test to skip all machines where
the architecture is mips or mips64.
[1] https://doc.rust-lang.org/nightly/rustc/platform-support.html
(From OE-Core rev: 177e268811c04260923ac4b16fa047315304add0)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The data layout for x86-64 target was different in rust from llvm.
It resulted in the following data layout error:
--------------------------------------------------------------------------------------
error: data-layout for target `x86_64-linux`, `e-m:e-i64:64-f80:128-n8:16:32:64-S128`,
differs from LLVM target's `x86_64-unknown-linux-gnu` default layout,
`e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128`
--------------------------------------------------------------------------------------
The issue was reported in meta-rust layer as:
https://github.com/meta-rust/meta-rust/pull/442
Hence, the data layout for x86-64 is modified as per the llvm sources which fixes the issue.
After running the testsuite with the modified data layout; it was observed that the
"tests/codegen/i128-x86-align.rs" passed. The modified data layout fixed the scalar pair which
resolved the testcase failure. Hence, the testcase has been removed from the exclude list.
(From OE-Core rev: 96f11d299b32a32086eb3d890c88eb7ce42057b0)
Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With qemu 9.1.0 we see parselogs failures with mips and mips64. We've long
since ignored these errors so update to continue to do so with the new form.
(From OE-Core rev: 4f3c299e7a282e953d9adb7549c30015df317171)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I'd meant to change the command timeout in the previous change, fix
the correct one.
(From OE-Core rev: bb991988cb23be2c8947171726ada321f27e6eed)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Avoid "RuntimeError: release unlocked lock" since the lock shouldn't
be locked even in the error path. Add a try/finally path to ensure
this.
(From OE-Core rev: b0732ee009ca47580d1d2ad75334f4aa50e6efd5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds basic tests for qemuarm and qemuarm64.
So far, barebox fails to run properly under KVM for the same reasons
u-boot fails to run.
A patch series to address the problem was submitted by Ahmad Fatoum
after debugging the oe-selftest failures for this series:
https://lore.kernel.org/barebox/20241009060511.4121157-1-a.fatoum@pengutronix.de/
For now, simply disable KVM for these tests.
(From OE-Core rev: 9284ceb4d32a51c77792d9009bba400d0b17d731)
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds basic tests for qemuarm and qemuarm64.
So far, U-Boot fails to run properly under KVM since this requires some
special care with instructions used for MMIO accesses.
Reported upstream by Ahmad Fatoum after debugging the oe-selftest
failures for this series:
https://lore.kernel.org/u-boot/2211f9f0-cd7d-4b55-860d-a34c04877e7b@pengutronix.de/
For now, simply disable KVM for these tests.
(From OE-Core rev: cadbd937e6358f9811f3ba7cf20cc50f8edcd844)
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows defining non-standard patterns from QEMU tests just as they
are already supported by the testimage.bbclass.
Will allow testing non-Linux shells in QEMU, too (e.g. a U-Boot shell or
another bootloader shell).
(From OE-Core rev: f9e6c3deee5a4159a09396b625a7327f6c204df5)
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we talk to terminals that like colors, we need to ignore the vt100
escape sequences when matching strings.
An unprocessed barebox console prompt would e.g. look like:
ESC[1;32mbarebox@ESC[1;36mARM QEMU virt64:/ESC[0m
where we cannot match for something like "barebox@ARM QEMU virt64:/".
The same applies to colored Linux terminal output of course.
The "\x1b\[" from the regex catches the standard start of ANSI escape
sequence while the rest catches the actual command code executed.
(From OE-Core rev: 33bbe4cb040f890121681865fbcf28bc8213a170)
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds the support for the barebox bootloader to oe-core. The recipe
is inspired by meta-ptx [1] but is a major rework of the one found there.
Barebox comes with a wide range of supported architectures and follows
the concepts of Linux in various aspects like the driver model, the
shell, or the virtual file system.
This not only eases porting Linux drivers but also makes barebox a
developer-friendly and feature-rich bootloader alternative [2].
For barebox (like for the kernel or other bootloaders) it is quite
likely that people will not just build the original recipe but need to
adapt it, point to custom repositories, apply patch stacks,
COMPATIBLE_MACHINE etc. They may also choose to have different recipe
names for different variants.
Having only a single .bb file and requiring to copy or .bbappend it is
inconvenient and results in unnecessary code duplication. Therefore, the
base support for building barebox is encapsulated in barebox.bbclass
(like kernel.bbclass for the kernel).
Adds barebox to maintainers.inc but excludes it from the maintainers
check since with the current check mechanism barebox would be skipped
(and making the check fail) due to not being the PREFERRED_PROVIDER for
virtual/bootloader.
[1] https://github.com/pengutronix/meta-ptx/tree/master/recipes-bsp/barebox
[2] https://www.barebox.org/demo/?graphic=0
(From OE-Core rev: 5c69f5626278a6e9756188a5771b18075380f52d)
Signed-off-by: Marco Felsch <m.felsch@pengutronix.de>
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After the changes to improve this test, we keep seeing image testing ssh
failures, particularly on mips.
It looks like part of the problem is that on a loaded system, 5s is too short
for mips to reliably establish an ssh connection. I've seen logs where it keeps
timing out and fails to work, then the debug code successfully uses ssh later after
everything else fails.
Change the timings/retries to give slow platforms enough time to respond.
(From OE-Core rev: ba64ccf3ad6e40461219b72d60eb0fe5cb38fddd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
while runnig oe-selftest for gcc, some of the testcases that need to be run on qemu
are not running due to below failures:
- kex_exchange_identification: read: Connection reset by peer^M
Connection reset by 192.168.7.2 port 22^M
ERROR: Couldn't create remote directory /tmp/runtest.3549814 on ssh
To resolve kex exchange identification error increased the MaxStartups.
(From OE-Core rev: df64d5ab6eb37dcdc2046f449ec539a3f4b985c8)
Signed-off-by: Harish Sadineni <Harish.Sadineni@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Dropping support for S = WORKDIR allows to drop this ugly workaround.
With S = WORKDIR it was possible to refer to a file via oe-local-files
symlink or via direct file path. Ensuring the pseudo database is
consistent for both paths was extra complicated and required this bad
function. Really nice to drop it now!
(From OE-Core rev: 2b799fdf267f44c26797593984d9828c4fd0fd31)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We had previously excluded certain test cases due to failures in earlier versions.
However, with the latest version of Rust, many of these test cases are passing.
As a result, we have removed them from the exclude list and added them back into the test suite for execution.
The rust-1.79 has been successfully tested with the current test inclusions.
(From OE-Core rev: e8dae0ee5ea958bb84af33d9b9a29ab357d96e31)
Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When building the new selftest builddir, paths in environment variables
are rewritten to point to the new buildir, but users can have
environment variables that point outside of the build dir using
relative paths from builddir. We must not rewrite those.
Check this by verifying that the absolute path still contains the
builddir.
Fixes [YOCTO #15241]
(From OE-Core rev: c5e70500caffcd0518899cc6eba23a38bc3be108)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
meta-ide-support:do_write_test_data dumps the bitbake data dictionary to
a file using export2json(). As this obviously includes the value of
MACHINE, and other MACHINE-specific variables, the recipe needs to be
marked as MACHINE-specific.
RP: Note that this patch does change the name of the environment script
since it is no longer package arch specific but machine arch specific.
[RP: Fix selftest to reference new environment file]
(From OE-Core rev: 3be2bc8a9b0c9d6a178329c8b451a6bedf255d6c)
Signed-off-by: Paul Barker <paul.barker.ct@bp.renesas.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've been able to run musl and glibc builds in the same TMPDIR for
many years and a separate directory is not required. Most distros disable
this value for that reason.
Drop support for the variable to make it clear and easy for distros
to be able to set multiconfigs which behave consistently for distros
which do and don't clear it by dropping it entirely.
(From OE-Core rev: ebcd355a32e2711263e22d9b45b502696ecbb4d2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
