Commit Graph

4068 Commits

Author SHA1 Message Date
Richard Purdie
61bcaf74ac oeqa/selftest/rust: Add exclusion to fix riscv64 builds
There was one failure occurring in rust testing for qemuriscv64. Exclude
that test so we can enable in automated testing.

(From OE-Core rev: a7f6ea5b20aa91e4a1b00dbea7a6447effb9220d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-15 10:55:26 +01:00
Richard Purdie
6a2ad60ecc bitbake.conf/pseudo: Switch from exclusion list to inclusion list
Currently, pseudo tracks all files referenced within its presence unless
they're listed in an exclusion list. The exclusion list has grown to be
fairly unwieldy.

This patch swaps PSEUDO_IGNORE_PATHS for PSEUDO_INCLUDE_PATHS which in
theory should be easier and more explicit to maintain.

This change does drop many directories from pseudo coverage including
/home and /tmp. There may be adapatations needed for recipes/classes
using pseudo in specific ways.

(From OE-Core rev: 2502da81709f25de499277b28d33c915638c45f6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:56 +01:00
Ross Burton
7157ffa551 oeqa/sdk/autotools: verify that configure is using the correct host
Search config.log for the correct host_alias assignment to verify that
configure has correctly identified the host system.

(From OE-Core rev: ade66073a1c89918f849eb2932c05342e8f3ab4f)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:55 +01:00
Ross Burton
0fc5716c11 oeqa/sdk/maturin: no need for a setupClass, merge code into test
There's no need to copy files inside setUpClass() when there's only one
test function that uses it. Just do all of the test inside the test.

(From OE-Core rev: 5474677b21f2f4069f355abdc600483c42d0b0b3)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:55 +01:00
Ross Burton
04e4586c66 oeqa/sdk/maturin: be less picky in the list_python test
The test assumed that maturin would only find a single Python binary, in
/usr/bin/python3*.

However in eSDKs with buildtools a Python is shipped with the SDK, so
the test failed.

Generalise the test so that it runs python3 and obtains its path and
version, and then verifies that path and and version are found by
Maturin. This means we're not assuming a single Python, or the paths, or
that the Python is CPython.

(From OE-Core rev: ae9b5dae77ef140422fcf71d239ca028c9208447)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:55 +01:00
Ross Burton
b8009efc19 oeqa/sdk/meson: add check that meson has detected the target correctly
Use 'meson introspect' to dump JSON describing the build configuration
and validate that the target architectures and cross-compiler is
correctly set.

(From OE-Core rev: 1c84361829921e91d782b189e2bde818a2d1491c)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:55 +01:00
Ross Burton
a916ac3844 oeqa/sdk/gtk3: build libhandy instead of galculator
galculator hasn't been touched for a decade now[1] and fails to compile
under GCC 15.

Switch to building libhandy, which is the GTK+3 precursor to libadwaita
in the Gnome stack. Whilst this is in low-maintainence mode, will be
updated if it breaks.

[1] https://github.com/galculator/galculator/

(From OE-Core rev: ff6fa71eb0511d8594c4416a37d75a85470ff9c6)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:55 +01:00
Ross Burton
fa6d138447 oeqa/sdk/meson: generalise test case
Refactor this test case so the generic "build a meson project" code is
separated out and can be reused.

Also currently meson inside eSDKs only works with fully populated eSDKs,
but our testing uses minimal eSDKS, so skip the test if the eSDK is a
minimal build.  A bug has been filed to resolve this.

(From OE-Core rev: 575e0bf52db0467d88af4b5fe467b682f10ca62a)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:55 +01:00
Ross Burton
ff06a1b425 oeqa/sdk/kmod: skip test in eSDKs
At the moment we can't run this test inside an eSDK as it needs the
kernel-devsrc recipe to be present.  Skip the test until this has been
resolved.

(From OE-Core rev: f83beee6e63d25ef2b17618a85f9ad6ca0898600)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:55 +01:00
Ross Burton
393691faf1 oeqa/sdk: use ensure_*_package helpers
Clean up lots of dependency checking code by using the new helpers.

This means that a lot of tests that were previously skipped inside the
eSDK testing on the autobuilder are now executed, and fail.

(From OE-Core rev: 11277efd057685558a744e98082b5709e849dd2a)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:55 +01:00
Ross Burton
eeacb5d906 oeqa/sdk: add helpers to check for and install packages
The existing tests simply look at the manifest to determine if a test
should be ran or not based on dependencies. Whilst this works for
Traditional SDKs, it fails for Extensible SDKs if they've been built in
minimal mode, where the manifest will be empty.  However, minimal eSDKs
might well have available sstate to install the missing dependencies.

Add a pair of helper functions to ensure that a package is available, or
skip the test.  This handles nativesdk- vs -native (SDK vs eSDK) and
will try to sdk-install missing dependencies into an eSDK if they're not
already installed.

(From OE-Core rev: d0e8b83d05957b1f22d08582e364afa4b522801e)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:55 +01:00
Ross Burton
d20341f93a oeqa/selftest/sdk: add test to verify the manifests are generated correctly
Add a test that builds a SDK with specific packages in the host and
target sections, and verifies that they're listed in the manifest.

(From OE-Core rev: a1556c801feb56b79243ba2947b74b84f674072b)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:55 +01:00
Ross Burton
788ea3f7a1 oeqa/sdk: add simple test that the manifests are not empty
Simple test to sanity check that the generated SDK manifest was parsed
correctly and isn't empty.

This test is complicated by the fact that minimal eSDKs without a
toolchain do in fact have an empty manifest, so also check for that.

(From OE-Core rev: 43288b19e93f0c07b347d6e5d6f7f10e96219f96)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:55 +01:00
Ross Burton
4ce977f378 oeqa/sdkext/context: align hasHostPackage with parent class
This subclass overrides hasHostPackage() but back in 2018[1] the parent
class's method gained a regex argument.

[1] oe-core 595e9922cdb ("oeqa/sdk: fixes related to hasPackage semantics")

(From OE-Core rev: cfd1e0a8c8d294510fca1a800ab27e4f1e2292bf)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:55 +01:00
Ross Burton
16ca89f0bb oeqa/sdk/context: fix multilib handling bug in hasTargetPackage
hasTargetPackage has some logic to automatically multilibify package
names if needed (for example, so that gtk+3 becomes lib32-gtk+3).

Due to a logic bug if multilib was True but there were no multilibs
configured then this prepended "-" to the package name, which won't
exist.  This resulted in tests being skipped as the dependent packages
are not installed.

Solve this by only prepending to the package name if requested and if a
multilib environment has been detected.

(From OE-Core rev: 2abb146dd81f677176923ebb3188f5b7c034ed68)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:55 +01:00
Ross Burton
79b6409ef7 oe/sdk: fix empty SDK manifests
The SDK manifests are generated by listing the sstate was that used, but
it hardcodes that the sstate data filenames end in .tgz.

This has not been the case since sstate switched to Zstd[1] in 2021,
which meant that all of the tests which checked for packages existing
were being skipped as the manifests were empty.  For example, see a
representative core-image-sato eSDK test run[2]:

RESULTS - cmake.CMakeTest.test_assimp: SKIPPED (0.00s)
RESULTS - gtk3.GTK3Test.test_galculator: SKIPPED (0.00s)
RESULTS - kmod.KernelModuleTest.test_cryptodev: SKIPPED (0.00s)
RESULTS - maturin.MaturinDevelopTest.test_maturin_develop: SKIPPED (0.00s)
RESULTS - maturin.MaturinTest.test_maturin_list_python: SKIPPED (0.00s)
RESULTS - meson.MesonTest.test_epoxy: SKIPPED (0.00s)
RESULTS - perl.PerlTest.test_perl: SKIPPED (0.00s)
RESULTS - python.Python3Test.test_python3: SKIPPED (0.00s)

All of those tests should have been ran.

Solve this by generalising the filename check so that it doesn't care
what specfic compression algorithm is used.

[1] oe-core 0710e98f40e ("sstate: Switch to ZStandard compressor support")
[2] https://autobuilder.yoctoproject.org/valkyrie/#/builders/16/builds/1517/steps/15/logs/stdio

(From OE-Core rev: b293c44f87b6a52e4239ce14066514e87d9b08d0)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:55 +01:00
Ross Burton
cabcadd9be testsdk: allow user to specify which tests to run
Following the usage of TEST_SUITES in testimage, add TESTSDK_SUITES to
specify the list of tests to execute. By default the variable is empty,
which means to run all discovered tests.

This makes it easier to work on a single test without having to run all
of the tests.

(From OE-Core rev: 28d437c52c77889b2ede0fc2f2d6777c5b0a553d)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 22:01:55 +01:00
Mikko Rapeli
36dcd01bad oeqa/selftest/pokybleeding: change from mtd-utils-selftest to pseudo
Test recipe git repo URL is checked for newer commits and
mtd-utils-selftest check is sometimes failing due to server
errors. Use pseudo which uses Yocto Project in git servers
which should be reliable.

Fixes: [YOCTO #15855]

build/build-st-17692/tmp/hosttools"; export HOME="/srv/pokybuild"; git -c gc.autoDetach=false -c core.pager=cat -c safe.bareRepository=all -c clone.defaultRemoteName=origin ls-remote git://git.infradead.org/mtd-utils.git  failed with exit code 128, output:
fatal: read error: Connection reset by peer

(From meta-yocto rev: a82be5486058c2e5a1e286dba17c33444889c982)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-12 17:06:07 +01:00
Hongxu Jia
2207150bc7 spdx30: Provide software_packageUrl field in SPDX 3.0 SBOM
Define var-SPDX_PACKAGE_URL to provide software_packageUrl field [1][2]
in SPDX 3.0 SBOM, support to override with package name
SPDX_PACKAGE_URL:<pkgname>

Currently, the format of purl is not defined in Yocto, set empty for now
until we have a comprehensive plan for what Yocto purls look like.
But users could customize their own purl by setting var-SPDX_PACKAGE_URL

[1] https://spdx.github.io/spdx-spec/v3.0.1/model/Software/Properties/packageUrl/
[2] https://spdx.github.io/spdx-spec/v3.0.1/annexes/pkg-url-specification/

(From OE-Core rev: c8e6953a0b6f59ffca994c440069db39e60b12d2)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-08 10:28:18 +01:00
Mikko Rapeli
e7c70cfca8 oeqa/selftest/wic: Add ext4 to IMAGE_FSTYPES in test_exclude_path_with_extra_space
All qemu machines have ext4 in IMAGE_FSTYPES which makes
native mkfs.ext4 available at wic image build time.
genericarm64 defaults to only wic in IMAGE_FSTYPES
and this test was failing:

| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing python function set_image_size
| DEBUG: 148548.400000 = 114268 * 1.300000
| DEBUG: 652644.400000 = max(148548.400000, 8192)[148548.400000] + 504096
| DEBUG: 652645.000000 = int(652644.400000)
| DEBUG: 652645 = aligned(652645)
| DEBUG: returning 652645
| DEBUG: Python function set_image_size finished
| DEBUG: Executing shell function do_image_wic
| INFO: Creating image(s)...
|
| ERROR: A native program mkfs.ext4 required to build the image was not found (see details above).
|
| Please make sure wic-tools have e2fsprogs-native in its DEPENDS, build it with 'bitbake wic-tools' and try again.
|
| WARNING: exit code 1 from a shell command.
NOTE: recipe core-image-minimal-1.0-r0: task do_image_wic: Failed

(From OE-Core rev: 59b5ec3d53058237dc6f3e85385851d1a8a97f97)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-01 14:22:54 +01:00
Mikko Rapeli
87c5907bc8 oeqa/selftest/wic: Increase rootfs size in test_offset
This test is failing on genericarm64 which includes all kernel
modules on rootfs by default. Recently added kernel modules
triggered the size limits to be exceeded. Fixes:

ERROR: Actual rootfs size (112884 kB) is larger than allowed size 102400 kB

Doubling the size to avoid issues like this on machines which
may include a lot of packages on default images. Size of the
rootfs does not matter for the test, only that the offsets are
correct in various usecases.

(From OE-Core rev: 1ff542318ea66514ef550d4817ba9b2ef688134c)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-01 14:22:54 +01:00
Trevor Woerner
e97d88578c oe-selftest.wic: add native sysroot handling
The wic test_exclude_path_with_extra_space does not succeed on debian-based
AB workers since they, by default, do not install the 'parted' utility. This
test installs the 'wic-tools' package, which includes the 'parted' utility,
but it is not being found/used for some unknown reason.

In the previous patch it was believed that doing some extra PATH handling
(as performed by some other tests with a python try...finally block) would
solve the issue. That turned out to not be the case. This patch starts by
reverting that change, since it has no benefit.

In order to use the native tools from the 'wic-tools' package, wic's
'--native-sysroot' can be used (as demonstrated in other tests). In fact
the wic help message explaining the purpose of this flag states that it is
for passing "...the path to the native sysroot containing the tools(parted
and mtools) to use"[sic].

Removing the host's 'parted' utility better simulates the situation found
on the debian workers for testing. This patch is seen to make this test pass
with no host 'parted' utility in place.

(From OE-Core rev: 306e4831f07d4a8db6ab880487813a6abd0ffa08)

Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-01 14:22:54 +01:00
Chen Qi
0d5afd1778 lib/classes/conf: refactor qemu.bbclass functions into library functions
Move the functions in qemu.bbclass to meta/lib/oe/qemu.py as they are
generally useful.

The qemu.bbclass is still kept, and recipes can continue to use functions
from it, though they have become wrapper functions on qemu.py functions.

Note that the QEMU_OPTIONS settings are still kept in qemu.bbclass.
This sets a clear barrier for people to use qemu user mode.

(From OE-Core rev: 7b3563b3b3901c96c3e498799a83ab8cabcf84b4)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-01 14:22:54 +01:00
Richard Purdie
cf9c5ac2c1 selftest-ed: Upgrade to version 1.21/1.20.2
Older versions do not compile with GCC 15. We had different versions to test
GPLv3 exclusion.

1.21 compiles with gcc 15 and 1.20.2 has a CFLAGS tweak to allow it to work
with gcc 15 too.

1.21 is licensed under GPL-2.0 so we need to rework some of the tests.

Tweak the gplv3 test by adding a special override of LICENSE to then test
the license exclusion code.

Modify the archiver selftests to use the new version.

Based on a patch from Khem Raj <raj.khem@gmail.com>

(From OE-Core rev: 3b17355ad1ecad17d12c5eb0e6403a59ef11f7d7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-01 14:22:53 +01:00
Khem Raj
d2a2c35c7f sstatesig: Handle special case of llvm-project-source shared-workdir
bitbake-dumpsig or bitbake-diffsig tools do not work on any of tasks
exposed by llvm-project-source recipe. This is due to it being a
shared-workdir recipe.

Fixes

bitbake-diffsigs -t llvm-project-source-20.1.2 do_preconfigure
NOTE: Starting bitbake server...
ERROR: No sigdata files found matching llvm-project-source-20.1.2 do_preconfigure

(From OE-Core rev: a6d46935939a94b8ea2b83c024aa86f05efbd7ce)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-05-01 14:22:53 +01:00
Trevor Woerner
69c5f72278 oe-selftest.wic: add PATH handling
The wic test_exclude_path_with_extra_space test succeeds on non-debian AB
workers. Add PATH handling so parted from the wic-tools can be found on
debian-based AB workers.

Fixes [YOCTO #15838]

(From OE-Core rev: 3994e727f10c5a0143d52bdd6e1d9ca037296d59)

Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-29 09:59:01 +01:00
Peter Marko
9fd08fcd94 cve-check: extract extending CVE_STATUS to library function
The same code for extending CVE_STATUS by CVE_CHECK_IGNORE and
CVE_STATUS_GROUPS is used on multiple places.
Create a library funtion to have the code on single place and ready for
reuse by additional classes.

(From OE-Core rev: 45e18f4270d084d81c21b1e5a4a601ce975d8a77)

Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-24 11:27:06 +01:00
Olga Denisova
a0c83d9069 buildstats.py: Add tracking of network I/O per interface
This patch extends SystemStats to collect and store data from /proc/net/dev.
It extracts per-interface received and transmitted bytes, calculates deltas
between samples, and stores them for further analysis.

Useful for identifying network bottlenecks during long-running builds.

(From OE-Core rev: 09cbe17e43783fc6b8e3a341d564956452a04c0a)

Signed-off-by: denisova-ok <denisova.olga.k@yandex.ru>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-24 11:27:06 +01:00
denisova-ok
5b74c76964 buildstats.py: extend diskstats support for NVMe and flexible token count
Added support for NVMe devices in the diskstats regex pattern to ensure stats are properly collected from devices like nvme0n1.

Relaxed the check for the number of fields in /proc/diskstats from an exact match (14) to a minimum check (at least 14), to handle kernel variations and additional fields gracefully.

(From OE-Core rev: 87a31bc4ca3661aae94cf43f3f579b02f4fb4923)

Signed-off-by: denisova-ok <denisova.olga.k@yandex.ru>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-24 11:27:06 +01:00
Martin Jansa
f4d8e84fd9 buildcfg: get_metadata_git_*: catch also bb.process.NotFoundError
* bb.process.NotFoundError is triggered when e.g. oe.buildcfg.get_metadata_git_branch
  is called on non-existent directory

(From OE-Core rev: 34c1f66c4c689b26a4c3129eb62f4ff9b6ec14be)

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-24 11:27:06 +01:00
Khem Raj
ea00dbbcf3 bitbake.conf: Switch prefix mapping to use -ffile-prefix-map
-ffile-prefix map is more comprehensive when it comes to reproducible
builds and its superset of all prefix-mapping options in compilers

This makes is cleaner and workable across gcc and clang, clang does not
support -fcanon-prefix-map and it has to be explicitly omitted when using
clang.

There are lambdas generated in templates by clang which still get the
absolute paths despite -fdebug-prefix-map, this helps with that as well.

nasm is an outlier and we have fixed it by adding -fdebug-prefix-map option
luckily we do not pass DEBUG_PREFIX_MAP to nasm, in all recipes which use
nasm either pass -fdebug-prefix-map explicitly to nasm or they rewrite it
to use nasm flags syntax.

We have discussed this in past [1]

[1] https://patchwork.yoctoproject.org/project/oe-core/patch/20230428032030.2047920-1-raj.khem@gmail.com/#10281

(From OE-Core rev: ff73fa7ef7666a6dbe34f15515bc3ab6e574c5b0)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Cc: Jacob Kroon <jacob.kroon@gmail.com>
Cc: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-23 09:47:42 +01:00
Richard Purdie
37bb053f34 oeqa/selftest/fetch: Fix whitespace
Update the test whitespace to avoid bitbake warnings.

(From OE-Core rev: 8a324aa2111872138715bde7fbc86bcfc2d8b20e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-20 10:11:16 +01:00
Trevor Woerner
35807e8f63 wic: do not ignore ROOTFS_SIZE if the rootfs is modified
If the *.wks file contains a "--source rootfs" then
lib/wic/plugins/source/rootfs.py will be invoked to generate (what is assumed
to be) the rootfs partition. If the rootfs partition needs to be tweaked or
modified, the "rootfs.py" plugin will make a copy of the filesystem and then
perform the changes on that copy. In other words, if the "--source rootfs"
line of the *.wks file also contains any of:

	--exclude-path
	--include-path
	--change-directory
	--use-label (i.e. modify etc/fstab)

then the rootfs will be copied first, then the copy is modified.

If, for example, the unmodified IMAGE_ROOTFS is:

	.../tmp/work/qemuarm64_secureboot-oe-linux/core-image-base/1.0/rootfs

then the copy would be made at:

	.../tmp/work/qemuarm64_secureboot-oe-linux/core-image-base/1.0/tmp-wic/rootfs${LINENO}

where ${LINENO} is the line number where this "--source rootfs" line appears
in the *wks file.

When it comes time to make an actual partition of a specific filesystem type,
lib/wic/partition.py::prepare_rootfs() is called. It is in this function that
wic figures out if any extra size needs to be added. The bitbake variable used
to specify the ultimate rootfs size is ROOTFS_SIZE, and since this variable is
only valid for the rootfs (and not any other partitions), the code also
verifies that the partition being created is ${IMAGE_ROOTFS}:

	rsize_bb = get_bitbake_var('ROOTFS_SIZE')
	rdir = get_bitbake_var('IMAGE_ROOTFS')
	if rsize_bb and rdir == rootfs_dir:
		<use rsize_bb>
	else:
		<calculate the partition size using "du -ks $p">

As noted above, if lib/wic/plugins/source/rootfs.py has made a copy, then the
"rdir == rootfs_dir" clause will fail and the code will assume this partition
is not a rootfs since the strings do not compare equal.

Therefore, in order to determine if this is a rootfs, retain the existing
"rdir == rootfs_dir" comparison, but also add another one to check whether or
not this is a wic-generated copy of the rootfs.

STEPS TO REPRODUCE:
	- start with the following *wks file:
		bootloader --ptable gpt
		part /boot --size=100M --active --fstype=ext4 --label boot
		part /     --source rootfs      --fstype=ext4 --label root
	- and the following extra variable in conf/local.conf:
		IMAGE_ROOTFS_EXTRA_SPACE = "500000"
	- build an image
	- run it in qemu
		$ runqemu slirp nographic serial
	- verify the root partition has extra space:
		root@qemuarm64-secureboot:~# df -h
		Filesystem                Size      Used Available Use% Mounted on
		/dev/root               721.5M     67.4M    600.6M  10% /
		devtmpfs                477.7M         0    477.7M   0% /dev
		tmpfs                    40.0K         0     40.0K   0% /mnt
		tmpfs                   489.3M     92.0K    489.2M   0% /run
		tmpfs                   489.3M     68.0K    489.2M   0% /var/volatile
		/dev/vda1               120.4M     19.9M     91.4M  18% /boot
	- modify the "/" line of the *wks file to be:
		part /     --source rootfs      --fstype=ext4 --label root --exclude-path boot/
	- build image

	when it fails:
		root@qemuarm64-secureboot:~# df -h
		Filesystem                Size      Used Available Use% Mounted on
		/dev/root                73.4M     41.9M     25.8M  62% /
		devtmpfs                477.7M         0    477.7M   0% /dev
		tmpfs                    40.0K         0     40.0K   0% /mnt
		tmpfs                   489.3M     92.0K    489.2M   0% /run
		tmpfs                   489.3M     68.0K    489.2M   0% /var/volatile
		/dev/vda1               120.4M     19.9M     91.4M  18% /boot

	after this fix:
		root@qemuarm64-secureboot:~# df -h
		Filesystem                Size      Used Available Use% Mounted on
		/dev/root               721.5M     47.4M    620.6M   7% /
		devtmpfs                477.7M         0    477.7M   0% /dev
		tmpfs                    40.0K         0     40.0K   0% /mnt
		tmpfs                   489.3M     92.0K    489.2M   0% /run
		tmpfs                   489.3M     68.0K    489.2M   0% /var/volatile
		/dev/vda1               120.4M     19.9M     91.4M  18% /boot

Doing the math we see that the /boot partition is ~20MB and in the first image
the / partition contains this ~20MB in addition to the rest of the rootfs.
This ~20MB is completely wasted since it is used in the / partition, but then
the /boot partition is mounted on top of it, making the /boot directory of /
inaccessible. After the fix the / partition has an additional ~20MB since the
/boot portion is excluded.

Fixes [YOCTO #15555]

(From OE-Core rev: 1c690aa046ebca13d7b29de50d42b5d8a4a8486c)

Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-17 11:03:22 +01:00
Joshua Watt
0ed51cf434 lib/oe/cve_check: Mark variable flag dependencies
Marks CVE check functions which depend on non-constant variable flags as
depending on the variables. This allows changes in the flags to
correctly trigger a rebuild

(From OE-Core rev: 2cc43c72ff28aa39a417dd8d57cd7c8741c0e541)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-16 17:41:30 +01:00
Mikko Rapeli
3369269a1e selftest pokybleeding.py: add test for poky-bleeding.bbclass
The class sets SRCREV to AUTOINC for recipes with single
or multiple repositories. Test those cases.

(From meta-yocto rev: b7185b27b8acb43eaf9c27787903173ea34a0738)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-10 11:05:59 +01:00
Joshua Watt
920efbf519 lib: oe: Add cve_check to BBIMPORTS
Adds cve_check.py to BBIMPORTS so the functions it exposes will be
correctly scanned for dependencies in the dependency scanner

(From OE-Core rev: 52ead33c6b6e2532c57b7b28b862ba38b575f9e3)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-10 11:05:34 +01:00
Alexander Kanavin
e08ca20695 mdadm: fetch from git
Upstream has released a new version (4.4) but not the tarball for it.

Adjust one of the devtool selftests, as it requires that the recipe
under test is using a tarball. Another selftest also needs to be
tweaked to correctly clean up its modifications to that same recipe on
test completion.

(From OE-Core rev: de635a9bc0392689ff36b50e7f91572d3fbaac09)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-10 11:05:34 +01:00
Khem Raj
e2c0b0ceeb oeqa: Update lzip version in buildzip testcase
(From OE-Core rev: 5200dcb39834790542c429d70cf54db07ff7a378)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-08 15:49:17 +01:00
Yi Zhao
5040d252c6 oeqa/selftest/wic: add test for excluding symlinks
Add test to check if --exclude-path option can exclude symlinks. This
test validates commit[1].

Test result:
$ oe-selftest -r wic.Wic.test_exclude_path
2025-04-03 15:11:25,211 - oe-selftest - INFO - meta-selftest layer not found in BBLAYERS, adding it
2025-04-03 15:11:30,016 - oe-selftest - INFO - Adding layer libraries:
2025-04-03 15:11:30,017 - oe-selftest - INFO -  /buildarea/poky/meta/lib
2025-04-03 15:11:30,017 - oe-selftest - INFO - /buildarea/poky/meta-yocto-bsp/lib
2025-04-03 15:11:30,017 - oe-selftest - INFO - /buildarea/poky/meta-selftest/lib
2025-04-03 15:11:30,019 - oe-selftest - INFO - Checking base configuration is valid/parsable
NOTE: Starting bitbake server...
2025-04-03 15:11:31,652 - oe-selftest - INFO - Adding: "include selftest.inc" in /buildarea/poky/build-st/conf/local.conf
2025-04-03 15:11:31,653 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
2025-04-03 15:11:31,653 - oe-selftest - INFO - test_exclude_path (wic.Wic)
2025-04-03 15:43:11,341 - oe-selftest - INFO -  ... ok
2025-04-03 15:43:11,341 - oe-selftest - INFO - ----------------------------------------------------------------------
2025-04-03 15:43:11,342 - oe-selftest - INFO - Ran 1 test in 1899.900s
2025-04-03 15:43:11,342 - oe-selftest - INFO - OK
2025-04-03 15:43:14,834 - oe-selftest - INFO - RESULTS:
2025-04-03 15:43:14,835 - oe-selftest - INFO - RESULTS - wic.Wic.test_exclude_path: PASSED (1899.69s)
2025-04-03 15:43:14,836 - oe-selftest - INFO - SUMMARY:
2025-04-03 15:43:14,836 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 1899.900s
2025-04-03 15:43:14,836 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0)

[1] https://git.openembedded.org/openembedded-core/commit/?id=42e829ac1e9d74646b6dfb327b18b15f6b0df60b

(From OE-Core rev: 0dd455bed9b52c0cf237ea2f8bd1a8f7890078e9)

Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-08 15:49:17 +01:00
Peter Marko
057049c1b6 spdx30: handle Unknown CVE_STATUS
CVE_STATUS can be also "Unknown" since oe-core commit
d25f1817752bc8a84c40dcbef75f7559801ce15e

When this status type is used, build fails with e.g.
ERROR: openssl-3.4.1-r0 do_create_spdx: Unknown CVE-2025-0001 status 'Unknown'

Since this is now a valid status, it needs to be handled.
It cannot be mapped to any VEX status (see below), so just skip it.
Possible VEX statuses are: NOT AFFECTED, AFFECTED, FIXED, and UNDER INVESTIGATION.

(From OE-Core rev: 2d3081ef63c8a54df62a2a08bd36008c20eed65a)

Signed-off-by: Peter Marko <peter.marko@siemens.com>
cc: Marta Rybczynska <rybczynska@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-08 15:49:17 +01:00
Ross Burton
cb7e73b358 lib/oe: remove redundant __name__ == "__main__" checks
There's no point in checking if __name__ == "__main__" (i.e., is this
module being invoked) and then doing nothing.

(From OE-Core rev: 020b6b1411c9fd3adb208808c0d56623190873f8)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-03 11:06:20 +01:00
Changqing Li
4f9a3e751c patch.py: set commituser and commitemail for addNote
When PATCHTOOL is set to 'git', and user don't setup
user.name and user.email for git, do_patch fail with
the following error, fix by passing -c options.
CmdError("git notes --ref refs/notes/devtool append -m 'original patch: 0001-PATCH-increase-to-cpp17-version.patch' HEAD", 0, 'stdout:
stderr: Author identity unknown
*** Please tell me who you are.
Run
  git config --global user.email "you@example.com"
  git config --global user.name "Your Name"

(From OE-Core rev: a3c6706d31ae1345b571ca10b290a4e1f5a9384b)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-03 11:06:20 +01:00
Hongxu Jia
0834a9cdf6 spdx3: support to override the version of a package in SBOM 3
By default, still use ${PV} as the the version of a package in SBOM 3
$ bitbake acl
$ jq . tmp/deploy/spdx/3.0.1/core2-64/packages/package-acl.spdx.json
...
    {
      "type": "software_Package",
       ...
      "name": "acl",
      "software_packageVersion": "2.3.2"
    },
...

Support to override it by setting SPDX_PACKAGE_VERSION, such as
set SPDX_PACKAGE_VERSION = "${EXTENDPKGV}" in local.conf to append
PR to software_packageVersion in SBOM 3
$ echo 'SPDX_PACKAGE_VERSION = "${EXTENDPKGV}"' >> conf/local.conf
$ bitbake acl
$ jq . tmp/deploy/spdx/3.0.1/core2-64/packages/package-acl.spdx.json
...
    {
      "type": "software_Package",
       ...
      "name": "acl",
      "software_packageVersion": "2.3.2-r0"
    },
...

(From OE-Core rev: e6ff5f4d870624795bd36572f5c2bfeec90d83ce)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-24 17:29:16 +00:00
Joshua Watt
3b585c5e11 oeqa: spdx: Add tar test for SPDX 2.2
The base-files test for SPDX 2.2 did not give good coverage, since
base-files doesn't have any dependencies. Add building tar as another
test which more fully exercises the code

(From OE-Core rev: d678e25419c89e09c1c438363bf3a940ce903d43)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-24 17:29:16 +00:00
Khem Raj
b76c04bdcb oeqa: Build cpio with C17 std
cpio is not yet buildable with C23 standard which is default with
GCC 15, therefore ensure to apply needed bandage to keep it compiling
in C17 mode even with GCC 15

(From OE-Core rev: 0c637099887f1be421c8e1203f99631a1e040150)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-24 17:29:16 +00:00
Peter Marko
769a4479e1 spdx30: handle links to inaccessible locations
This is the same as e105befbe4ee0d85e94c2048a744f0373e2dbcdf on
additional place in the code.

When a link is pointing to location inaccessible to build user (e.g. "/root/something"),
filepath.is_file() throws "PermissionError: [Errno 13] Permission denied".
Fix this by first checking if it is a link.

(From OE-Core rev: 26f35f866cf7888431963cf4fc5d2019cd28de74)

Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-20 17:49:16 +00:00
Richard Purdie
110b2c124b spdx: Update for bitbake changes
Bitbake is dropping the need for fetcher name iteration and multiple revisions
per url. Update the code to match (removal of the for loop).

(From OE-Core rev: 4859cdf97fd9a260036e148e25f0b78eb393df1e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-20 13:50:40 +00:00
Peter Kjellerstedt
c0f3bc79b3 devtool: reset: Escape command line input used in regular expression
Running, e.g., `devtool reset sdbus-c++` would result in the following
error:

  re.error: multiple repeat at position 35

This was due to the ++ in the recipe name, which would be treated as an
incorrect regular expression in _reset().

Use re.escape() to make sure all characters in the recipe name are
treated literally.

(From OE-Core rev: 6e73bd9b3e6d529752db93879f2c0ed53873dd1a)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-20 11:29:04 +00:00
hongxu
d029e4e033 spdx30: test the existence of directory before walking
Due to commit [spdx30: Improve os.walk() handling][1] applied,
it reported an error if walk directory failed

While SPDX_INCLUDE_SOURCES = "1", if recipe does not provide sysroots,
the walk in function add_package_files is broken

$ echo 'SPDX_INCLUDE_SOURCES = "1"' >> conf/local.conf
$ bitbake packagegroup-core-boot
|DEBUG: Adding sysroot files to SPDX
|ERROR: packagegroup-core-boot-1.0-r0 do_create_spdx: ERROR walking tmp/sysroots-components/intel_x86_64/packagegroup-core-boot: [Errno 2]
| No such file or directory: 'tmp/sysroots-components/intel_x86_64/packagegroup-core-boot'

Test the existence of directory before walking

[1] https://git.openembedded.org/openembedded-core/commit/?id=86b581e80637cd8136ce7a7e95db94d9553d2f60

(From OE-Core rev: cb1792e4950d5075be9bbe4c5337a5215db9669e)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-19 09:48:14 +00:00
Alexander Kanavin
05d23378c2 meta/lib/oe/recipeutils.py: handle fetcher errors when checking for new commits
Recent freedesktop instabilities are causing 'devtool check-upgrade-status' to
fail with:

bb.fetch2.FetchError: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; git -c gc.autoDetach=false -c core.pager=cat -c safe.bareRepository=all -c clone.defaultRemoteName=origin ls-remote https://gitlab.>
fatal: unable to access 'https://gitlab.freedesktop.org/mesa/kmscube/': The requested URL returned error: 502

and not print any results for this one or any unrelated recipes included in the check.

This change handles the error, so that if some upstream server isn't working
properly, latest upstream revision for that is marked as unknown, a
warning is printed and upstream version check for other recipes
isn't thwarted:

WARNING: Unable to obtain latest revision: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; git -c gc.autoDetach=false -c core.pager=cat -c safe.bareRepository=all -c clone.defaultRemoteName=origin ls-remote https://gitlab.freedesktop.org/mesa/piglit.git  failed with exit code 128, output:
remote: GitLab is not responding
fatal: unable to access 'https://gitlab.freedesktop.org/mesa/piglit.git/': The requested URL returned error: 502

piglit                    1.0             UNKNOWN_BROKEN  Ross Burton <ross.burton@arm.com>

(From OE-Core rev: c1056293f7cb32ee2bdf31441cc0b59d9ccfe556)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-19 09:48:14 +00:00