Commit Graph

1288 Commits

Author SHA1 Message Date
Benjamin Robin (Schneider Electric)
2d57a09792 meta: fix generation of kernel CONFIG_ in SPDX3
With the current solution, using a separate task
(do_create_kernel_config_spdx) there is a dependency issue. Sometimes
the final rootfs SBOM does not contain the CONFIG_ values.

do_create_kernel_config_spdx is executed after do_create_spdx which
deploys the SPDX file. do_create_kernel_config_spdx calls
oe.sbom30.find_root_obj_in_jsonld to read from the deploy directory,
which is OK, but the do_create_kernel_config_spdx ends up writing to
this deployed file (updating it).

do_create_rootfs_spdx has an explicit dependency to all do_create_spdx
tasks, but there is nothing that prevents executing
do_create_kernel_config_spdx after do_create_rootfs_spdx.

To fix it, instead, now read from the workdir, and write to the
workdir, and do the processing from the do_create_spdx task:
we append to the do_create_spdx task.
Furthermore, update oeqa selftest to execute do_create_spdx instead
of removed function.

Also only execute this task if create-spdx-3.0 was inherited,
previously this code could be executed if create-spdx-2.2 is
inherited.

(cherry picked from commit 8417f4a186e78a9d309541f5d0e711178bb80488)

Fixes: 1fff29a04287 ("kernel.bbclass: Add task to export kernel configuration to SPDX")
(From OE-Core rev: 22e8bc2bcfe762c83c00b73a33384e63548e82c0)

Signed-off-by: Benjamin Robin (Schneider Electric) <benjamin.robin@bootlin.com>
Reviewed-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Paul Barker <paul@pbarker.dev>
2026-06-19 12:49:07 +01:00
Sunil Dora
cb7499e8fe license.py: Drop visit_Str from SeenVisitor in selftest
ast.Str was deprecated in Python 3.8 and removed in Python 3.14.
In [1], visit_Str was already removed from the LicenseVisitor
subclasses (FlattenVisitor, ListVisitor) in oe/license.py since
bitbake now requires Python 3.8+.

However, the test-only SeenVisitor class in the selftest was
missed at that time and still uses visit_Str/node.s. On Python
3.14, ast.Str is fully removed so visit_Str is never called,
causing test_single_licenses to return [] instead of the
expected license list.

Replace visit_Str/node.s with visit_Constant/node.value in
SeenVisitor.

[1] https://git.openembedded.org/openembedded-core/commit/meta/lib/oe/license.py?id=6d3da37adbcaf5a7a3dade08f9d052571b195249

Fixes [YOCTO #16220]

(From OE-Core rev: f8ac1b567d7c09ee45cb0d23fc9dd25987a76129)

Signed-off-by: Sunil Dora <sunilkumar.dora@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8a5b019eec72676893507d018e7609745d2e3f49)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Paul Barker <paul@pbarker.dev>
2026-05-04 13:57:33 +01:00
Alexander Kanavin
71edfb1de8 selftest/gdbserver: replace shutil.unpack_archive with tarfile extract
This is a followup to
https://lists.openembedded.org/g/openembedded-core/message/233609
as the same issue happened in a different selftest.

[YOCTO #16195]

(From OE-Core rev: 948028ca3466fb7071ee49000e847d6e886e349c)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c7468e70c238b056acbe06ef722b62b02626db8f)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Paul Barker <paul@pbarker.dev>
2026-05-04 13:57:33 +01:00
Alexander Kanavin
e13525ed22 selftest/minidebuginfo: extract files from tar archive using tarfile module
Python 3.14 added security checks around archive extraction, and by
default will refuse to handle symlinks with absolute paths. It's possible
to handle this using 'filter' argument, but it is not always available
in older Python versions on various host distributions we need to support,
so let's extract only the needed files directly using tarfile module.

busybox is itself a symlink to busybox.nosuid, so both are extracted.

[YOCTO #16195]

(From OE-Core rev: 799e2124cdcb27bd924598619da61cbd11730b56)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d52d00a3bb4a1ba93e88f1d24d8bb99d6aa321eb)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Paul Barker <paul@pbarker.dev>
2026-05-04 13:57:33 +01:00
Richard Purdie
0549c498bd selftest/scripts: Update old git protocol references
git protocol accesses to our infrastructure are currently struggling and this
has highlighted a number of places we're making those obsolete access forms.

Update them to use https instead of the git protocol since it is preferred
and more reliable.

The devtool test needed quoting to handle the ';' in the url. The -f option
to devtool also shows a deprecation warning so remove that.

There were internal references to git protocol urls inside the nested git
submodules test report, which means those repos need updating to use
new git revisions.

(From OE-Core rev: cbb3e323b74d4351c772a9bcd553008c31a220f0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1ceba42623c5187d2f5a100d6a523abcdc75d34e)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Paul Barker <paul@pbarker.dev>
2026-05-04 13:57:32 +01:00
Yoann Congal
0e511229ef oeqa/selftest/git-submodule-test: Default to https git protocol for YP/OE repos
Following up on commit 139102a73d41 ("recipes: Default to https git protocol where possible"),
> The recommendation from server maintainers is that the https protocol
> is both faster and more reliable than the dedicated git protocol at this point.
> Switch to it where possible.

Use ";protocol=https" for the parent git submodule and, also, update the
SRCREV to point to a commit where submodules are reference through a
https:// URL instead of a git:// one.

Update the expected output of the archiver test.

(cherry picked from commit 5c78db972f1b97ce67c8c461a2d389b9248793db)
(From OE-Core rev: d839f4908ae6fb54c85bea5b658c23bfa9dc34da)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Paul Barker <paul@pbarker.dev>
2026-05-04 13:57:32 +01:00
Dragomir, Daniel
77f60c1165 oeqa/selftest/wic: test recursive dir copy on ext partitions
Extend the wic selftests to cover recursive directory copying
into ext partitions.

Previously, copying a directory into an ext partition could
appear to succeed, but attempting to access the directory
contents would fail with:

  -l: Ext2 inode is not a directory

This was fixed in commit 4fc3b42774 ("wic/engine: fix copying
directories into wic image with ext* partition").

This test now verifies that directories copied with "wic cp"
into an ext4 partition:
  - are created with correct inode types
  - can be listed recursively with "wic ls"
  - preserve files and subdirectories
  - can be copied back out of the image without data loss

A simple directory structure is used in this test:

wic-test-cp-ext-dir/
├── topfile.txt
└── subdir/
    └── subfile.txt

(From OE-Core rev: 1e051c02bc05367d6b911ac9486403029f6f7cd8)

Signed-off-by: Daniel Dragomir <daniel.dragomir@windriver.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6de3d2602f4f4a8192d6a6040e89e814187dcf93)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Paul Barker <paul@pbarker.dev>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2026-02-27 17:45:06 +00:00
Kamel Bouhara (Schneider Electric)
12a7475659 oeqa/selftest: oe-selftest: Add SPDX tests for kernel config and PACKAGECONFIG
Add test_kernel_config_spdx and test_packageconfig_spdx to verify
SPDX document generation includes kernel configuration and package
feature metadata when enabled.

(From OE-Core rev: a172a0e8d543796ee78bb66650726168352f1cdf)

Signed-off-by: Kamel Bouhara (Schneider Electric) <kamel.bouhara@bootlin.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2f0ab110d7521510c60e0493ef3cb021130758cd)
Signed-off-by: Kamel Bouhara <kamel.bouhara@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-12-31 07:49:31 -08:00
Joshua Watt
6328f52695 oeqa: spdx: Add tar test for SPDX 2.2
The base-files test for SPDX 2.2 did not give good coverage, since
base-files doesn't have any dependencies. Add building tar as another
test which more fully exercises the code

(From OE-Core rev: 2c299c17ef1a97505fd7de8d3ebc9de25fb838fc)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-11-14 06:45:30 -08:00
Joshua Watt
71aca87ca7 spdx 3.0: Rework how SPDX aliases are linked
The SPDX code needs to be able to look up an Element by its SPDX ID,
locating the file that (should) contain the SPDX ID and opening it for
parsing. Previously, the code would do this be hashing each Element
SPDX ID and Alias, and the creating a symbolic link to the file that
contains the element with a name of the hash.

This worked well as it was possible to look up any arbitrary SPDX ID or
alias by simply hashing it and following the symbolic link to get the
file. However, the down side of this approach is that it creates a lot
of symbolic links, since it will make one or two per Element in the
document. This can be a problem when using SPDX_INCLUDE_SOURCES, for
example.

This change reworks this strategy so that the only Element that gets a
symbolic link based on the hash is the singular SpdxDocument that is
create for each file. All other Elements are assigned an alias with a
special prefix that encodes the hash of SpdxDocument alias. Thus, when
attempting to look up an arbitrary alias, the code sees the special
prefix, extract the hash, opens the file based on the symlink with that
hash name, then finds the matching Element in the file. This drastically
reduces the number of symbolic links by making only one per file.

This also means that the custom link extension can be removed since it
is now superfluous.

(From OE-Core rev: 551433c7a1eddf5090c87a243ea104bf091992b0)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 838d64c09657ac53175737fc4e7fd6f01f3dcf47)
Signed-off-by: Kamel Bouhara (Schneider Electric) <kamel.bouhara@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-11-14 06:45:30 -08:00
Joshua Watt
1919d76f68 lib: oeqa: spdx: Add tests for extra options
Adds a test for several of the extra options provided by the SPDX
classes. In particular, these are the options that can produce
non-reproducible results, so are not enabled by default in OE core. This
test takes care to configure the build so that the tests do run in a
reproducible manner so that pre-built test objects can be pulled from
sstate

(From OE-Core rev: 72ee311d4f74499674a29223fb02d4e774097a54)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 14f534f15f7fe6362723d7f064d39783c5bd758f)
Signed-off-by: Kamel Bouhara (Schneider Electric) <kamel.bouhara@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-11-14 06:45:30 -08:00
Hongxu Jia
568c1afab4 oeqa/selftest: Add SPDX 3.0 include source case for work-share
Build gcc and check gcc-14.2.0/README in objset is available

$ oe-selftest -r spdx.SPDX30Check.test_gcc_include_source
...
2024-10-26 01:24:57,063 - oe-selftest - INFO - test_gcc_include_source (spdx.SPDX30Check.test_gcc_include_source)
2024-10-26 01:28:24,204 - oe-selftest - INFO - The spdxId of gcc-14.2.0/README in gcc.spdx.json is http://spdx.org/spdxdocs/gcc-f2eaeb0d-b54b-53ba-899a-8c36c21139bf/88d5068ffd41e5ea6b4e0dd390b23bf499bb2b6674a41e09eaf2a887eced16c8/sourcefile/42
2024-10-26 01:28:26,369 - oe-selftest - INFO -  ... ok
2024-10-26 01:28:33,315 - oe-selftest - INFO - ----------------------------------------------------------------------
2024-10-26 01:28:33,316 - oe-selftest - INFO - Ran 1 test in 216.457s
2024-10-26 01:28:33,316 - oe-selftest - INFO - OK
2024-10-26 01:28:45,254 - oe-selftest - INFO - RESULTS:
2024-10-26 01:28:45,254 - oe-selftest - INFO - RESULTS - spdx.SPDX30Check.test_gcc_include_source: PASSED (209.31s)
2024-10-26 01:28:45,260 - oe-selftest - INFO - SUMMARY:
2024-10-26 01:28:45,260 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 216.457s
2024-10-26 01:28:45,260 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0)

(From OE-Core rev: 45f3039a4f0fcea208e317ed7134458f7aa7a4cc)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Reviewed-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ccd6dde301dc8c45c8f901ebd4676b488d638b08)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-11-14 06:45:30 -08:00
Joshua Watt
f5a9af087b selftest: spdx: Add SPDX 3.0 test cases
Adds test cases for SPDX 3.0. Reworks the SPDX 2.2 test setup so it can
also be run even if the default is SPDX 3.0

(From OE-Core rev: e182f76a866d4d750d2baf7b56ffebead5264de2)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b1d2309b3ab0fd8b0d8c4dfa59f50c85074bbd3b)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-11-14 06:45:30 -08:00
Richard Purdie
75922d42b0 selftest/spdx: Fix for SPDX_VERSION addition
Update the test for the addition of SPDX_VERSION to the deploy path.

(From OE-Core rev: d75bfbaf69292f80cacc5b8d6cbff03418a34ebc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 055e8c21908127722abad9e992d6408d8697a119)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-11-14 06:45:30 -08:00
Richard Purdie
7ff4ef9855 oeqa/selftest/devtool: Update after upstream repo changes
The upstream changed the branch name, moving to archive/ so we need to
update too. Take the opportunity to match the new location too to avoid
the redirect.

We could use a different branch but upstream would probably eventually
rename that too so this may last longer.

(From OE-Core rev: 1e0a64a1890a62e130595f46d93c8d08af9170f5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 478a645bad150f04dee1b0085c4542c2eefe7007)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-11-14 06:45:29 -08:00
Steve Sakoman
6e74167ad4 selftest/cases/meta_ide.py: use use gnu mirror instead of main server
ftp.gnu.org is the main server of the GNU project, however download speed
can vary greatly based on one's location.

Using ftpmirror.gnu.org should redirect the request to the closest up-to-date mirror,
which should result sometimes in significantly faster download speed, depending
on one's location. This should also distribute the traffic more across the mirrors.

This information was sourced from https://www.gnu.org/prep/ftp.html

(From OE-Core rev: aa7ff5a115f55c092f8ca5badad63734c8f4f5b7)

Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-10-13 12:42:58 -07:00
Richard Purdie
67c7b78bd0 sstatetests: Switch to new CDN
The project is switching the way handle our CDN provision of sstate objects,
update the URL accordingly.

(From OE-Core rev: d1b64f190c1686f081f5ba2c4f2b320048f6a514)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit fea96974f1ee6ae6dceb39e3ca8157797d81586c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-06-05 08:41:15 -07:00
Alexander Kanavin
fda81f7c20 selftest/rust: correctly form the PATH environment variable
There were two issues here:

- the path to native python was not included, so the test
suite was running with host python, causing failures due to
host python being given native libraries:
https://autobuilder.yoctoproject.org/typhoon/#/builders/148/builds/1711/steps/12/logs/stdio
(rust recipe uses native python throughout)

- tmp/hosttools was listead ahead of native-sysroot/usr/bin

(From OE-Core rev: 72156282059aa5a013a386eb95f89dc38726326e)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit aec96b5aec8d6b4a5ae8a719ca05bb727a6197cb)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-02-12 06:25:37 -08:00
Ross Burton
48ac27c8dc oeqa/selftest/rust: skip on all MIPS platforms
As per "The rustc book"[1], mips*-unknown-linux-* are "tier 3" targets:

  Tier 3 targets are those which the Rust codebase has support for, but
  which the Rust project does not build or test automatically, so they
  may or may not work.

We already skip qemumips in this selftest, but we're now also seeing
failures with qemumips64 so refactor the test to skip all machines where
the architecture is mips or mips64.

[1] https://doc.rust-lang.org/nightly/rustc/platform-support.html

(From OE-Core rev: 681f5ea8d6a5050aebfb1f656da9ba679433e366)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 177e268811c04260923ac4b16fa047315304add0)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-02-12 06:25:37 -08:00
Adrian Freihofer
37b09dcab5 oe-selftest: devtool ide-sdk use modify debug-build
(From OE-Core rev: 1528d6aa060811402eaecb1a9fe16bfec0417c98)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-02-03 06:13:13 -08:00
Alexander Kanavin
e4de7259e4 rust: build the default set of tools
Setting it explicitly replaces rust's default choice which is rustdoc
(needed for example in selftests and otherwise expected to be present
in typical rust installations):

https://github.com/rust-lang/rust/blob/master/config.example.toml#L320

This addresses some of the rust selftest failures but not all. Help
is appreciate to restore the selftest.

Unfortunately, this also breaks rust reproducibility (or rather exposes
that it was never properly fixed, as explained here:
https://lists.openembedded.org/g/openembedded-core/message/199288
)

(From OE-Core rev: 4d739fe248d1023eb2c3c040fc4d33273dd16bc1)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 58eaf2ee6c0809bf0a0d3c1d177e62bda7241651)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-01-26 14:04:46 -08:00
Richard Purdie
e1c3e84dd5 selftest/reproducible: Clean up pathnames
There are several problems with these paths. Firstly they contain full
system paths which depend upon where the test was run. These are pretty
pointless and just take up a lot of space making the results files large.

Secondly, they contain the same path twice. The reference and target path
will always be the same thing in two different locations.

Strip off the prefix and remove the duplication. This does change the output
data but that can't really be avoided. It does shrink the results data and makes
it more readable.

(From OE-Core rev: 44ee8f69db5e008b5ac5e0dbf390bb4fcf704f07)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 81a44de36e864b08687451fd85aeba7c529fd7f7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-12-09 06:54:14 -08:00
Richard Purdie
96900f953f selftest/reproducible: Drop rawlogs
The "rawlogs" data consists of a long string of results data which is
already in a structured data format. I can't see this is adding much
value in duplciating the data but it does create a huge string with a
lot of long problematic pathnames and inflates the results data size.

I suggest we drop this data as obsolete and not necessary.

(From OE-Core rev: 296f64cbf83c40759b368526335502a6b130aa14)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5b2c70fab2ffa409b861d83f048b65d458d03a90)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-12-09 06:54:14 -08:00
Gassner, Tobias.ext
aad028a455 rootfs: Ensure run-postinsts is not uninstalled for read-only-rootfs-delayed-postinsts
This patch ensures that pkg_postinst_ontarget task is executed for read only rootfs when
read-only-rootfs-delayed-postinsts is set as IMAGE_FEATURES. The issue was that run-postinsts
could be uninstalled at the end of rootfs construction and that shouldn't happen for
the delayed usecase.

In addition to the fix, a test in meta/lib/oeqa/selftest/cases/overlayfs.py testing
the fix has been implemented.

(From OE-Core rev: e0c4f2f69723f64a29ba35c4de5fb1a7c79ef718)

Signed-off-by: Gassner, Tobias.ext <tobias.gassner.ext@karlstorz.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 60f587475dda99eaa07848880058b69286b8900e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-12-06 05:50:25 -08:00
Paul Barker
8156bb675c meta-ide-support: Mark recipe as MACHINE-specific
meta-ide-support:do_write_test_data dumps the bitbake data dictionary to
a file using export2json(). As this obviously includes the value of
MACHINE, and other MACHINE-specific variables, the recipe needs to be
marked as MACHINE-specific.

RP: Note that this patch does change the name of the environment script
since it is no longer package arch specific but machine arch specific.

[RP: Fix selftest to reference new environment file]
(From OE-Core rev: bec6b535c9a817a5ed51453bcb9ecae0b81a8590)

Signed-off-by: Paul Barker <paul.barker.ct@bp.renesas.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3be2bc8a9b0c9d6a178329c8b451a6bedf255d6c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-10-18 06:04:41 -07:00
Richard Purdie
3b6c55fdba oeqa/selftest/reproducibile: Explicitly list virtual targets
We're seeing reproducibility failures where some packages don't appear
in the "from sstate" builds. The common factor is these are all recipes
with PROVIDES = "virtual/XXX".

In a full build from scratch, these are build but in a build from sstate,
there are situations where they aren't. For now, to try and keep builds
working, work around the problem until we can better look into the problem.
It is likely recent taskhash imrovements have caused this to occur more
regularly.

(From OE-Core rev: 9f0eba6aba5e9b37975f86556c1234b145a0859e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 68086a333acc54390e4e589ef928dc90da3edb48)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-08-26 05:18:44 -07:00
Mark Hatle
c80d82054e selftest-hardlink: Add additional test cases
Additional test cases for debug symlink generation both binaries
and static libraries.

This also has the side effect of testing for race conditions in the
hardlink debug generation and stripping.

(From OE-Core rev: 39823d3211411e661320e1164ba4c50370804425)

Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 7171f41c07a39a7543bb64f075d38b8e74563089)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-08-06 19:11:18 -07:00
Richard Purdie
aec6a4f009 selftest/cases/runtime_test: Exclude centos-9 from virgl tests
Similarly to centos 8, centos 9 doesn't support the render device we need
for this test.

(From OE-Core rev: 37ddcbd51ec39e5df94b307085cec2a045fea4a1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c2be3afabf84f287c90b61ae2509728a6634fb8f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-07-17 05:36:14 -07:00
Richard Purdie
91363d054b oeqa/selftest/devtool: Fix for usrmerge in DISTRO_FEATURES
If usrmerge is in DISTRO_FEATURES, assumptions in one of the tests would
fail. Improve the test so it works in both cases.

(From OE-Core rev: 4dc35667143f19a369bdea9ce2f011a04132509c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit afa211746a2aa1993a54cc5a5e1937679341da8e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-07-09 06:02:56 -07:00
Richard Purdie
5568d33e68 oeqa/selftest/recipetool: Fix for usrmerge in DISTRO_FEATURES
If usrmerge is in DISTRO_FEATURES, assumptions in one of the tests would
fail. Improve the test so it works in both cases.

(From OE-Core rev: 01ccccb949bd82e89dae4679ff5b30ada6672d9a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 68a27d307a7042e242c49cf3d069469f40e09902)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-07-09 06:02:56 -07:00
Alexandre Truong
554e8de6a4 Revert "oeqa/selftest/devtool: fix test_devtool_add_git_style2"
This reverts commit ab6d3e3d64
from poky repository.

The previous reverted commit was a workaround. The fix
"oeqa/selftest/devtool: fix _test_devtool_add_git_url"
tackle the issue. So, the workaround is not needed anymore.

(From OE-Core rev: fd30cc2ab99fe6a8e4beb29d9cb46a772db8b8d8)

Signed-off-by: Alexandre Truong <alexandre.truong@smile.fr>
Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 731f47ecfd8ad6558aac629806810789c623986b)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-06-19 08:34:58 -07:00
Alexandre Truong
afba27a984 oeqa/selftest/devtool: fix _test_devtool_add_git_url
This patch is a follow-up to the bug#15466.

As a reminder, the bug was about devtool’s submodule detection
checking for HEAD when a version is being passed.

As Vincent Kriek pointed out:
the --version that is being passed to devtool is only used for
setting the PV value in the recipe. To take into account the tag,
we add --srcrev to the command:
devtool add --srcrev v3.1.0 --version v3.1.0 mbedtls git://git@github.com/ARMmbed/mbedtls.git;protocol=https

Changes to _test_devtool_add_git_url have been made to take
into account the srcrev. srcrev will be passed as an optional
parameter because the test_devtool_add_git_style1
does not need the srcrev contrary to test_devtool_add_git_style2

Fixes [YOCTO #15466]

(From OE-Core rev: c297b78454da2a668e62dec95d7b6eb6e5429480)

Signed-off-by: Alexandre Truong <alexandre.truong@smile.fr>
Reported-by: Alexandre Truong <alexandre.truong@smile.fr>
Suggested-by: Vincent Kriek <vincent@coelebs.dev>
Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a8686f3641e4407dee3d807898ffd620e2732b78)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-06-19 08:34:58 -07:00
Julien Stephan
a4cd5dc1ee oeqa: selftest: context: run tests serially if testtools/subunit modules are not found
If testtools and/or subunit modules are not found we get the following backtrace
(example for testtools):

  NOTE: Starting bitbake server...
  Traceback (most recent call last):
    File "<..>/poky/scripts/oe-selftest", line 60, in
  <module>
      ret = main()
    File "<..>/poky/scripts/oe-selftest", line 47, in main
      results = args.func(logger, args)
    File "<..>/poky/meta/lib/oeqa/selftest/context.py",
  line 391, in run
      rc = self._internal_run(logger, args)
    File "<..>/poky/meta/lib/oeqa/selftest/context.py",
  line 377, in _internal_run
      rc = self.tc.runTests(**self.tc_kwargs['run'])
    File "<..>/poky/meta/lib/oeqa/selftest/context.py",
  line 161, in runTests
      return super(OESelftestTestContext, self).runTests(processes, skips)
    File "<..>/poky/meta/lib/oeqa/core/context.py", line
  91, in runTests
      result = self.runner.run(self.prepareSuite(self.suites, processes))
    File "<..>/poky/meta/lib/oeqa/selftest/context.py",
  line 154, in prepareSuite
      from oeqa.core.utils.concurrencytest import ConcurrentTestSuite
    File
  "<..>/poky/meta/lib/oeqa/core/utils/concurrencytest.py",
  line 22, in <module>
      import testtools
  ModuleNotFoundError: No module named 'testtools'

Fix this by adding a custom callback on -j/--num-processes parameter to
check testtools and subunit modules. Fallback to serial testing if
missing. This strategy is already used in sdk/context.py

(From OE-Core rev: ffd1db7d7813f6da22c0d9ef5fde6738058f1eb2)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 35284404473b2c2d9f69594582868ed66ef3525e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-06-19 08:34:58 -07:00
Ola x Nilsson
5ff13f1b01 oeqa/selftest/devtool: add test for modifying recipes using go.bbclass
go.bbclass uses a special do_unpack function that causes the git root
to be different from S.  Verify that it unpacks as expected.

[ YOCTO #15483 ]

(From OE-Core rev: 71adf7c18215faf73454be79d7b51f2a0c338c0f)

Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit fab0c737b95b8d0c0bbf58336bc308776c956406)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-06-19 08:34:58 -07:00
Julien Stephan
a905386e10 oeqa/selftest/devtool: add test for updating local files into another layer
We don't have a test to check if we can correctly devtool update-recipe/finish
into another layer. So update the existing test_devtool_update_recipe_local_files
to also check the updates into another layer.

(From OE-Core rev: bd44c895d36e246a25c7a6e40bf9f4089dc7a297)

(From OE-Core rev: 0532a6292edbe68303b6d85017ebcdb36a60886f)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Jeff Harris <jefftharris@gmail.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-06-05 05:57:12 -07:00
Ross Burton
68f9a4b73d oeqa/selftest/debuginfod: use localpkgfeed to speed server startup
Sometimes the debuginfod selftest fails due to a timeout, because it
spends too long scanning a huge deploy directory that due to what tests
were ran previously can contain 30K packages.

The test only needs a subset of the feed, so use the new localpkgfeed
class to construct a minimal feed before running the test.

[ YOCTO #14937 ]

(From OE-Core rev: 0795169be206f1d4d140fe378e2476a44d0ce02b)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 855376f518b28248ccd82ef5b2e89e6a8c970542)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-05-23 08:55:01 -07:00
Alexandre Truong
ab6d3e3d64 oeqa/selftest/devtool: fix test_devtool_add_git_style2
The problem is the following:
AssertionError: 'gitsm://git@github.com/ARMmbed/mbedtls.git;protocol=https;branch=master' != 'git://git@github.com/ARMmbed/mbedtls.git;protocol=https;branch=master'

Mbedlts made changes to their repository, adding a sub-module, thus the
assert triggers an error with the url:
git://git@github.com/ARMmbed/mbedtls.git;protocol=https;branch=master

456a54da8e
was the upstream change.

To fix the issue, the url has been changed to:
gitsm://git@github.com/ARMmbed/mbedtls.git;protocol=https;branch=master

(From OE-Core rev: 9ac737fbe05c85ec8333b396ce2f89de6654916f)

Signed-off-by: Alexandre Truong <alexandre.truong@smile.fr>
Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-04 21:50:24 +01:00
Baruch Siach
0c9d83138e oeqa/selftest/overlayfs: test read-only rootfs
Use the read-only squashfs filesystem to test the read-only case.

(From OE-Core rev: 1a61da196f014dc34a998d3e46f03abc79ebc84d)

Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-04 14:05:03 +01:00
Richard Purdie
30d88a2043 oeqa/sstatetests: Fix race issue
Under some load conditions, the result event can come back before the
command complete event. If that happens, the code would hang
indefinitely.

Rework the code to wait for both events and avoid the hang.

(From OE-Core rev: 8cfc94a4404c54bc73eab9f98d9da1f84c2135ad)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-31 06:57:01 +01:00
Alexander Kanavin
d1622b87f8 selftest/sstatetests: run CDN check twice, ignoring errors the first time
The current CDN isn't able to serve all objects on first request, and it
was suggested to work around that by trying again:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15335#c16

Once CDN moves to a better location this can probably be reverted.

[YOCTO #15335]

(From OE-Core rev: 0db3466303f56736d2fa9a105435eaaadb385082)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-30 22:22:19 +00:00
Eilís 'pidge' Ní Fhlannagáin
2b5903f511 sstatetests.py: Add testing for correct sstate permissions
This patch adds to run_test_sstate_creation so that it also tests
that sstate directories don't accidentally pickup umask permissions
from the user upon creation.

[RP: Python style tweaking]
(From OE-Core rev: 7d6eb828e97ad3f27d94efdccd920fb2aef36743)

Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-07 17:27:51 +00:00
Alexander Kanavin
d339047b83 minicom: upgrade 2.8 -> 2.9
Adjust selftest to git-add the directory with newly added patches,
as the new minicom recipe has no default patches, and thus no directory
with them (and the selftest assumed it does).

(From OE-Core rev: 1fb2aa3f242ef20f8edfb518164b629258a04dd4)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-07 17:25:02 +00:00
Jose Quaresma
f92e32957c oeqa/gotoolchain: set GOPROXY
Since go-1.21 GOPROXY needs to be set explicitly, otherwise it fails with:
- GOPROXY list is not the empty string, but contains no entries

This fixes the selftest

(From OE-Core rev: c491d967858c01fead21495f44f1a9f8cdf8e833)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-03 16:28:21 +00:00
Adrian Freihofer
ea5a74cc67 oe-selftest devtool: ide-sdk tests
Improve the GDB related tests. Verify GDB finds the correct source
files.

(From OE-Core rev: 67eed460c0bf18d23f2c9180f195417895acfd55)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-27 11:38:46 +00:00
Eilís 'pidge' Ní Fhlannagáin
396bc832b7 selftest-users: Convoluted selftest for USERADD_DEPENDS
This adds a test for 13904's fix by creating a convoluted set of recipes
with USERADD_DEPENDS in non-alpha order.

(From OE-Core rev: bfff81195cb9ba2493e366022470b2e0051d8071)

Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-24 16:10:22 +00:00
Richard Purdie
4dba026177 recipetool: Fix errors with meta-poky bbappend
When a new base-files bbappend was added to meta-poky, it causes selftest
failures. Whilst this isn't ideal, workaround that issue for now since
the append is being added for security visibility and changing the tests
to support this more generically looks invasive.

(From OE-Core rev: 7cf85204f0943bf741ffce5c4105340197c714df)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-23 14:31:30 +00:00
Richard Purdie
c59d570024 oeqa/selftest/rust: Simplify the rust testsuite output gathering/processing
The rust testsuite was redirecting command output to a file, which made it
hard to debug failure cases since the logs were not available to print to
the console.

Rework the code so it uses the existing popen logging and hence allows us
to improve the error logging situation and make debugging failures easier.

(From OE-Core rev: ac82dc43b8151ed34c4ad51e9ab7f4a612990486)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-23 14:29:03 +00:00
Tim Orling
9f7475b4e5 recipetool; add support for python_mesonpy class
* Add support to detect the "mesonpy" build-backend for recipetool create.
* Add oe-selftest case for creating a recipe for "siphash24" from pypi.
  https://pypi.org/project/siphash24/

This is by far the simplest recipe using the mesonpy build backend.

Upstream does not provide LICENSE file(s) and we do not detect the
LICENSE so don't check for that result in the test. Likewise, upstream
does not define HOMEPAGE, so skip that result.

(From OE-Core rev: 256749322671d2f4ea994db671d73c4de10e1723)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-20 11:39:45 +00:00
Richard Purdie
0b33104a97 python: Drop ${PYTHON_PN}
python 2 is gone and we don't need the abstraction now, drop the remaining usage
of this variable.

The definition in python3-dir.bbclass is left for now for other layers.

(From OE-Core rev: b566b1e32c7993d1ab7795562f648e52ce186a70)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-20 11:39:45 +00:00
Peter Kjellerstedt
4cfd0f7e4e lib/oe/patch: Use git notes to store the filenames for the patches
The old way of keeping track of the filenames for the patches that
correspond to the commits was to add a special comment line to the end
of the commit message, e.g., "%% original patch: <filename>", using a
temporary git hook. This method had some drawbacks, e.g.:

* It caused problems if one wanted to push the commits upstream as the
  comment line had to be manually removed.
* The comment line would end up in patches if someone used git
  format-path rather than devtool finish to generate the patches.
* The comment line could interfere with global Git hooks used to
  validate the format of the Git commit message.
* When regenerating patches with `devtool finish --force-patch-refresh`,
  the process typically resulted in adding empty lines to the end of the
  commit messages in the updated patches.

A better way of keeping track of the patch filenames is to use Git
notes. This way the commit messages remain unaffected, but the
information is still shown when, e.g., doing `git log`. A special Git
notes space, refs/notes/devtool, is used to not intefere with the
default Git notes. It is configured to be shown in, e.g., `git log` and
to survive rewrites (i.e., `git commit --amend` and `git rebase`).

Since there is no longer any need for a temporary Git hook, the code
that manipulated the .git/hooks directory has also been removed. To
avoid potential problems due to global Git hooks, --no-verify was added
to the `git commit` command.

To not cause troubles for those who have done `devtool modify` for a
recipe with the old solution and then do `devtool finish` with the new
solution, the code will fall back to look for the old strings in the
commit message if no Git note can be found.

While not technically motivated like above, the way to keep track of
ignored commits is also changed to use Git notes to avoid having
different methods to store similar information.

(From OE-Core rev: f5e6183b9557477bef74024a587de0bfcc2b7c0d)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-19 16:03:22 +00:00