Commit Graph

2259 Commits

Author SHA1 Message Date
Fabien Mahot
617d2303c0 oeqa/selftest/bbtests: add non-existent prefile/postfile tests
Fixes [YOCTO #10725]

(From OE-Core rev: b0c33655fad5b2e7d96a45b6210527dfb766797b)

Signed-off-by: Fabien Mahot <fabien.mahot@smile.fr>
Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-15 08:00:41 +01:00
Richard Purdie
00f3d58064 selftest/reproducible: Allow native/cross reuse in test
We don't compare reproducibility of the native/cross components, only the
target ones. With the long build times of rust-native, the test now takes
crazy lengths of time so this tweak should allow us to reuse native/cross
artefacts from sstate whilst still testing the target output is
reproducible.

(From OE-Core rev: b494d83c639a877cefeb7cbab6d37195e492f059)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-07 23:41:20 +01:00
Chen Qi
5eec70a456 qemurunner.py: fix error message about qmp
The error message is a little misleading as the qmp module is a
directory with __init__.py file, not qmp.py file. Also, put the
path where we try to import it from in the error message to make
the message more indicative.

(From OE-Core rev: 08bacbf797f6a50ae8abe8fc3455b3a15a0a94b3)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-06 07:41:43 +01:00
Richard Purdie
511d007248 oeqa/selftest/wic: Fix host contamination issue
If wic.Wic2.test_gpt_partition_name is run on a system without sfdisk, the test
will currently fail. As done in another test, it needs to use sfdisk from the
wic-tools sysroot. This patch fixes that host contamination issue.

(From OE-Core rev: 6f9200cc30d50888b9b63103824880abaf8b5eea)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-06 07:41:42 +01:00
Richard Purdie
c47ba43785 oeqa/selftest/sstatetests: Add easier debug option
In order to debug these selftest failures you end up having to comment out
the file cleanup. Make this an option at the top of the file to make it
a bit easier, I've had to do this too many times now.

(From OE-Core rev: ae69d48b0e5b58b18553a6b3cd03ffbd3f369aa3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-02 15:59:07 +01:00
Richard Purdie
e2a71c7d25 selftest/incompatible_lic: Ensure create_sdpx isn't used with the tests
The SPDX class doens't get on well with non-standard licenses. Disable it for
the purposes of this test to avoid errors.

Add a new helper function to the core test code to allow this to be done easily.

(From OE-Core rev: b9fb4c68f2ab5becb0a10418884e09dee93cd247)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-02 15:59:07 +01:00
Martin Jansa
ca12fac112 selftest: wic.py respect IMAGE_LINK_NAME
* use IMAGE_LINK_NAME instead of hardcoding
  core-image-minimal-${MACHINE} assumption

[YOCTO #12937]

(From OE-Core rev: e34d7617b00ec9423221c9c7b96be6f9e5358e6a)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-29 13:42:56 +01:00
Thomas Roos
5f08f9e0fc oeqa/selftest/cases/devtool.py: skip all tests require folder a git repo
Devtool selftests require poky dir a git repo, when downloading poky as a tar,
this is not the case. Those tests will now skipped.

[YOCTO #12389]

(From OE-Core rev: 95a5bc130dc51ea9de95c64dbf0e9c7892415d50)

Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-25 10:29:09 +01:00
Mikko Rapeli
7fbf5fdd02 qemurunner: add boot logs to do_testimage task logs in real time
Complex boot sequences take a lot of time to reach serial console login.
In our case boot involves multiple bootloaders like u-boot and grub,
then hypervisors like xen, then initramfs images which do things like
validating secure boot and encrypting the main rootfs. All these happen
before main rootfs serial console login prompt is reached and thus many
things can go wrong, including race conditions, hangs and deadlocks, and
thus logging the details is important for development.

This change adds the qemu serial console logs to do_testimage task logs while
they are being read out. The logs are not added to bitbake output. This
enables debugging and tracing the do_testimage via task log file while the boot
and tests are running.

(From OE-Core rev: cb62fae6ea3e411643395b12e56c363ce4d73e5d)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-25 10:29:08 +01:00
Thomas Roos
6c8fe83bb5 oeqa: adding selftest-hello and use it to speed up tests
Adding a selftest version of hello world to run it in tests where no download is necessary.
Also using this in several tests to speed them up.
Using the -native version wherever possible will also speed up tests a lot.

[YOCTO #11142]

(From OE-Core rev: c3f26b63934888df0e3cd563c1c2804eb78a368e)

Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-22 10:53:49 +01:00
Mikko Rapeli
c7a68ff3cc qemurunner: avoid leaking server_socket
close() the server_socket before qemusock replaces
it.

(From OE-Core rev: be7bbbeeb44158b181d1abc5a4d7871a9045fdd6)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-22 10:53:47 +01:00
Mikko Rapeli
830f4cb0a6 qemurunner: support serial console login via qemu stdout
runqemu script works with qemu machines which provide login
and serial console to the qemu process stdout. Add the same support
to qemurunner so that testing with testimage.bbclass is possible.

Default qemu machines provide serial console boot logs and login via
socket to qemu process but I don't see a reason why qemu process stdout
should not be supported too since they work with runqemu as well.

(From OE-Core rev: a3d4b80c5b4cce933c759d023c75b8671c56fe12)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-22 10:53:47 +01:00
Alexander Kanavin
850b5d5eed oeqa/sdk/assimp: run only when zlib is in the SDK
Otherwise assimp will silently fall back to a vendored copy of zlib
which will fail with -D_TIME_BITS=64 due to https://github.com/madler/zlib/pull/764

This was exposed by multilib mips core-image-minimal SDKs, where the default
64 bit sysroot has zlib, but 32 bit sysroot does not.

(From OE-Core rev: c0fb603c9e26e91388320c02842b42cc7b091d6c)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-05 11:07:26 +01:00
Frederic Martinsons
e7e7999b23 zvariant: add ptest feature for zvariant test suite
Below is the output of run-ptest script under qemu

root@qemux86-64:~# /usr/lib/zvariant/ptest/run-ptest

running 37 tests
test framing_offset_size::tests::framing_offset_size_bump ... ok$<2>
test owned_value::tests::from_value ... ok$<2>
test object_path::unit::owned_from_reader ... ok$<2>
test str::tests::from_string ... ok$<2>
test signature::tests::signature_slicing ... ok$<2>
test str::tests::test_ordering ... ok$<2>
test owned_value::tests::map_conversion ... ok$<2>
test owned_value::tests::serde ... ok$<2>
test tests::enums ... ok$<2>
test tests::derive ... ok$<2>
test tests::f64_value ... ok$<2>
test tests::i16_value ... ok$<2>
test tests::fd_value ... ok$<2>
test tests::i32_value ... ok$<2>
test tests::i8_value ... ok$<2>
test tests::i64_value ... ok$<2>
test tests::ip_addr ... ok$<2>
test tests::issue_59 ... ok$<2>
test tests::issue_99 ... ok$<2>
test tests::array_value ... ok$<2>
test tests::object_path_value ... ok$<2>
test tests::dict_value ... ok$<2>
test tests::signature_value ... ok$<2>
test tests::serialized_size ... ok$<2>
test tests::struct_byte_array ... ok$<2>
test tests::struct_ref ... ok$<2>
test tests::str_value ... ok$<2>
test tests::option_value ... ok$<2>
test tests::struct_value ... ok$<2>
test tests::struct_with_hashmap ... ok$<2>
test tests::u16_value ... ok$<2>
test tests::u32_value ... ok$<2>
test tests::unit ... ok$<2>
test tests::u8_value ... ok$<2>
test tests::unit_fds ... ok$<2>
test tests::value_value ... ok$<2>
test tests::recursion_limits ... ok$<2>

test result: ok$<2>. 37 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.21s

root@qemux86-64:~#

(From OE-Core rev: 912bbec9fe44f22ab70c3553af6cb699543b8411)

Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-05 11:07:26 +01:00
Alexander Kanavin
1f21419fc8 selftest/distrodata: clean up exception lists in recipe maintainers test
Specifically:
- add missing maintainer.inc entries for initramfs-module-*, systemd-machine-units and
target-sdk-provides-dummy and drop them from exception list.

- remove rust from exception list for unbuildable-by-default recipes as it is now buildable.

- add missing maintainer.inc entry for libx11-compose-data and cve-update-nvd2-native;
as they are also unbuildable by default, they needs to be in exception list as well.

(From OE-Core rev: e9158b191c1cfc16f97abed6c05891aa84fe9463)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-05 11:07:25 +01:00
Richard Purdie
1285f78ce8 recipes: Default to https git protocol where possible
The recommendation from server maintainers is that the https protocol
is both faster and more reliable than the dedicated git protocol at this point.
Switch to it where possible.

(From OE-Core rev: 139102a73d4151f4748b4a861bd4ab28dda7dab7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-05 11:07:25 +01:00
Richard Purdie
5caf9e375f oeqa/runtime/ptest: Make returning no test results a failure
Ensure that even if a ptests results section is empty, the log parser adds that
empty section. Then ensure that empty sections trigger warnings.

This means if a ptest suddently stops returning any results, we notice and see
warnings about it. This has gone unnoticed on the autobuilder far too many times
so is very much worth highlighting as a regression. We shouldn't have empty ptests.

(From OE-Core rev: 5ad0cf57b41ec7f44647a03bc568d0b24906cc8d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-03 07:33:18 +01:00
Thomas Roos
150464e2ff oeqa/utils/metadata.py: Fix running oe-selftest running with no distro set
This will use default values when no distribution is set.

[YOCTO #15086]

(From OE-Core rev: 888fe63b46efceeff08dbe8c4f66fec33d06cb7a)

Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-20 11:56:05 +01:00
Thomas Roos
67e22b9fc9 oeqa/selftest: Use SSTATE_DIR of parent build dir
This will configure the build-st/conf/local.conf with the SSTATE_DIR
of the parent build dir to speed up the build, test, dev process.

(From OE-Core rev: 71dca60ddcf4370ff7e91de6ddf8a7d4cf76db50)

Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-20 11:56:05 +01:00
Dit Kozmaj
a60925d00a selftest: wic: Add test for --part-name argument
Add test for wic --part-name argument in .wks file.
Test three different cases:
- Set only the --part-name argument.
- Set only the --label argument. In this case the GPT partition label
  is set to the value of --label for compatibility reasons.
- Set both. In this case the code has been changed to set the GPT
  partition label to the --part-name value.

The test uses a test_gpt_partition_name.wks file created for this
purpose.

(From OE-Core rev: 06af6795cf6f079362303bddb39a14459e228e72)

Signed-off-by: Dit Kozmaj <dit.kozmaj@kynetics.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-13 11:56:07 +01:00
Mikko Rapeli
3083993c71 oeqa ping.py: fail test if target IP address has not been set
It is possible to call exported tests with --target-ip set to ":22"
where IP address is not set at all. Detect this case and fail the test
instead of calling ping without an IP address.

(From OE-Core rev: 17c995c53775b8cee279ca4ced916092067e1195)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-06 14:31:42 +01:00
Mikko Rapeli
5ecafc3fec oeqa ping.py: avoid busylooping failing ping command
Use a sleep on error path before trying again. For example
when oeqa runtime tests are executed without setting target
IP address correctly, the tests are drowning logs with messages:

2023-04-04 07:19:24,985 - runtime - INFO - test_ping (ping.PingTest.test_ping)
ping: usage error: Destination address required
ping: usage error: Destination address required
ping: usage error: Destination address required
ping: usage error: Destination address required
ping: usage error: Destination address required
ping: usage error: Destination address required
ping: usage error: Destination address required
ping: usage error: Destination address required
ping: usage error: Destination address required
ping: usage error: Destination address required
ping: usage error: Destination address required
ping: usage error: Destination address required
...
2023-04-04 07:19:55,002 - runtime - INFO -  ... FAIL
2023-04-04 07:19:55,002 - runtime - INFO - Traceback (most recent call
last):
  File \"/lava-62618/3/tests/3_oeqa-runtime-tests/image/lib/oeqa/runtime/cases/ping.py\", line 23, in test_ping
    output += proc.communicate()[0].decode('utf-8')
              ^^^^^^^^^^^^^^^^^^
  File \"/usr/lib/python3.11/subprocess.py\", line 1194, in communicate
    stdout = self.stdout.read()
             ^^^^^^^^^^^^^^^^^^
  File \"/lava-62618/3/tests/3_oeqa-runtime-tests/image/lib/oeqa/core/decorator/oetimeout.py\", line 18, in _timeoutHandler
    raise OEQATimeoutError(\"Timed out after %s \"
oeqa.core.exception.OEQATimeoutError: Timed out after 30 seconds of execution

(From OE-Core rev: 87ec75710b5cd7b3f35d886003844d62d3182b54)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-06 14:31:42 +01:00
Martin Jansa
f135a26aa3 selftest: efibootpartition.py: fix QEMU_USE_KVM usage
* if you don't have QEMU_USE_KVM in local.conf it fails with:

  2023-03-12 18:06:29,591 - oe-selftest - DEBUG - Checking if qemux86-64 is not this MACHINE
  2023-03-12 18:06:29,594 - oe-selftest - INFO -  ... ERROR
  2023-03-12 18:06:29,594 - oe-selftest - INFO - Traceback (most recent call last):
    File "/OE/build/poky/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
      return func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
    File "/OE/build/poky/meta/lib/oeqa/selftest/cases/efibootpartition.py", line 18, in test_boot_efi
      if oe.types.qemu_use_kvm(self.td['QEMU_USE_KVM'], self.td["TARGET_ARCH"]):
                               ~~~~~~~^^^^^^^^^^^^^^^^
  KeyError: 'QEMU_USE_KVM'

[YOCTO #12937]

(From OE-Core rev: 7c32ca2f91beb98769c89470b37f06cb4b99aebb)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-06 14:31:42 +01:00
Mark Asselstine
53a1ad0c88 oeqa/selftest/bblogging: uncomment python stdout checks
Since bitbake commit 81a58647b2f4fc0a2589b2978fc9d81b2bfe6aec
[bitbake: build: Make python output print to stdout when running with
-v (verbose)] we no longer need to comment out the python stdout
checks.

(From OE-Core rev: 67886a8473c511c8ab3db2e4587cc5a070979d11)

Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-05 17:26:53 +01:00
Ross Burton
1c3c56ca19 oeqa/runtime: clean up deprecated backslash expansion
(From OE-Core rev: 77085a05240c3f8226b9f2199c977f2555807789)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-03 11:39:59 +01:00
Martin Jansa
36f736bd39 selftest: runqemu: use better error message when asserts fail
* It was showing whole log and that the runqemu command failed, but not
  where the log file is, nor why it thinks the runqemu failed

[YOCTO #12937]

(From OE-Core rev: ad4b72e6524114a3bdee10cab71f194ea143cd24)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-03 11:39:59 +01:00
Martin Jansa
34aa323f18 selftest: runqemu: better check for ROOTFS: in the log
* it was searching for line like this:
  ROOTFS: [/OE/build/poky/build/build-st-2023-03-20-esdk-runqemu-patch1/runqemu.RunqemuTests.test_boot_machine_ext4/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64-20230320081121.rootfs.ext4]
  but with IMAGE_NAME_SUFFIX changed to something else than default ".rootfs"
  or with my pending changes the line looks like this:
  ROOTFS: [/OE/build/poky/build/build-st-2023-03-20-esdk-runqemu-patch2/runqemu.RunqemuTests.test_boot_machine_ext4/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs-20230320085744.ext4]
  and test was failing.

* Check for whole line starting with ROOTFS: and ending just with .ext4

[YOCTO #12937]

(From OE-Core rev: 7a0f622f23aff2c4eeca0606e7682931eb53287a)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-03 11:39:59 +01:00
Martin Jansa
edd4fe8cb3 selftest: imagefeatures.py: don't mix tabs and spaces for indentation
* introduced in:
  https://git.openembedded.org/openembedded-core/commit/?id=96d4486df6d870ef19e2055b026729e66bc118f3

(From OE-Core rev: 143f624116802579e7a6804c8d4b562f931880c8)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-03 11:39:59 +01:00
Frederic Martinsons
ad460bb6aa meta-selftest: provide a recipe for zvariant
This recipe is for showing a "real world" example of
a crate that depends on some git repositories.

Usually, this kind of crate is built within a global
workspace (here it is the zbus project) and so
doesn't need a Cargo.lock on its own.

For the sake of the demonstration, I had to tweak things
a little to be able to compile zvariant in standalone
(no relative path in dependency, no symlink to LICENSE
provide a Cargo.lock)

The use case where the crate had some git repository
in dependency is very common for "private" crate that
are not aimed to be published on crates.io.
When the project grow bigger, it is common to have
a bin and multiple lib developped in parallel, and these
libs are surely on a git repostitory.

A test case have been also added to check for:
  - the previous patch about git subpath parameter and devtool
  - the correctness of overriding dependencies (first patch of the
series)

(From OE-Core rev: 409e045f96f69877de6f36ed14c5c19a9cb74eaf)

Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-01 11:36:26 +01:00
Enrico Jörns
d59756912c oeqa: whitespace and indentation cleanups
(From OE-Core rev: 4922221d1259e2f78233f17bb901cdac5b9aa520)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-31 23:30:36 +01:00
Enrico Jörns
4651c42cab oeqa/utils/qemurunner: fix undefined TimeoutExpired
(From OE-Core rev: 19ae3f4440b09d4cfe4bc589d859d476168541bd)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-31 23:30:36 +01:00
Enrico Jörns
6ef891415f oeqa/utils/qemurunner: limit precision of timing debugging output
* There is no need to be that precise. It just irritates in the logs.
* There is also no point in printing plain time.time() value a single
  time while only using formatted printout everywhere else, thus remove
  it.
* Use %d for printing integer times

(From OE-Core rev: f0988cb8cf8d03708490cca4eba345492ef78d52)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-31 23:30:36 +01:00
Enrico Jörns
ff7b3d6810 oeqa/utils/qemurunner: replace hard-coded user 'root' in debug output
Since login user is configurable with boot_patterns['send_login_user'],
unconditionally using 'root' in the debug message can be confusing.

Also fix the debug message to say 'Logged in' instead of 'Logged'.

(From OE-Core rev: 900e3d42b918b5a33d8d952b3a8078fbe72ba98f)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-31 23:30:36 +01:00
Enrico Jörns
a5f848f0e0 oeqa/utils/commands: remove unused imports
(From OE-Core rev: 0c00b5cdd57c3d9c47d4780d6627c74221911b72)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-31 23:30:36 +01:00
Enrico Jörns
c27a505f58 oeqa/utils/commands: fix usage of undefined EPIPE
(From OE-Core rev: 1cb17e37e275794e26debed2d171c2394d8bb945)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-31 23:30:36 +01:00
Enrico Jörns
fbd4843544 oeqa/targetcontrol: remove unused imports
(From OE-Core rev: 46b42edbf814d15be3a18283018ede2d4f817bc6)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-31 23:30:36 +01:00
Enrico Jörns
6ae5bd896a oeqa/targetcontrol: do not set dump_host_cmds redundantly
This is already set above, thus simply use the set value from there.

(From OE-Core rev: 2f6571ffb23b1a0b5bd6a1212fc108e536f04539)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-31 23:30:36 +01:00
Enrico Jörns
ed363d0ec9 oeqa/targetcontrol: fix misspelled RuntimeError
(From OE-Core rev: 92ffc35052768c753a89b4839c70db87072437a2)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-31 23:30:36 +01:00
Enrico Jörns
4d2c6baba5 oeqa/selftest/cases/runqemu: update imports
'tempfile' and 'oeqa.utils.command.runCmd' are unused while 'os' is
actually used.

(From OE-Core rev: 50f766233e83528eebeca92877277374c7193530)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-31 23:30:36 +01:00
Richard Purdie
f67258fcbf oeqa/loader: Ensure module names don't contain uppercase characters
Python modules aren't supposed to have uppercase characters in their names
according to python conventions. We have regexs in the code which work
on that assumption too. Rather than showing errors under some filtering
situations, make it clear and error if a problematic name is seen.

(From OE-Core rev: 7964a6ee54e881a6128707a7f2a42eec2ed63881)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-31 23:30:36 +01:00
Romuald Jeanne
6815f71692 oeqa/selftest/imagefeatures: set a test for mutliubi in test_image_fstypes
Allow 'multiubi' fstype image.
Set 'MULTIUBI_BUILD' with two entries to allow configuration for two
different '.ubifs'.
Configure 'MKUBIFS_ARGS_*' and 'UBINIZE_ARGS_*' for both configurations.

(From OE-Core rev: 96d4486df6d870ef19e2055b026729e66bc118f3)

Signed-off-by: Romuald Jeanne <romuald.jeanne@st.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-30 23:26:04 +01:00
Martin Jansa
953007b68b selftest: eSDK rename to esdk
* I was hit by oe-selftest -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers
  running all tests except only this selected one:

  poky $ oe-selftest -v -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers -K -B /OE/build/poky/build-eSDK
  2023-03-13 14:00:52,955 - oe-selftest - DEBUG - Selected tests with -r: ['eSDK.oeSDKExtSelfTest.test_install_libraries_headers']
  2023-03-13 14:00:55,531 - oe-selftest - INFO - Changing cwd to /OE/build/poky/build
  ..
  2023-03-13 14:00:58,128 - oe-selftest - INFO - test_archiver_allows_to_filter_on_recipe_name (archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name)

  this is caused by _built_modules_dict(modules) function
  which filters out eSDK.oeSDKExtSelfTest.test_install_libraries_headers
  based on the regexp and then it runs all loaded tests, because
  modules are empty

  the initial regexp and comment from 2017:
     https://git.openembedded.org/openembedded-core/commit/?id=80db3d999ae26d298d9d5418a32b11a4f27af9d5

     # Assumption: package and module names do not contain upper case
     # characters, whereas class names do
     m = re.match(r'^([^A-Z]+)(?:\.([A-Z][^.]*)(?:\.([^.]+))?)?$', module)

  might still be valid, but it was loosened in 2018 to accept upper case in module:

     https://git.openembedded.org/openembedded-core/commit/?id=1ecf48fd286a77078451b67879a44f9c9dc7a894
     Some test cases (eSDK.oeSDK*, runtime_test/*) does not match with current regex, fix it accept all.

  Then skipping the not matching modules was added later in 2018:

     https://git.openembedded.org/openembedded-core/commit/?id=f2042bf3638ed4edfb167e7f7d4be6da60997ead

  and regexp was updated again in 2020 not to accept upper case in modules:

     https://git.openembedded.org/openembedded-core/commit/?id=ad81ea90a815389e45ff302a85151724c71f71c3
     oeqa/core/loader: refine regex to find module
     test case in format <module name>.<class name>.<test case name>
     this is clear when test cases is only 3 item deep.

     but confused when it is 4 item deep, eg,
     oelib.types.TestList.test_list_nosep

  I'm afraid that changing this regexp again to accept eSDK will break
  someone's favorite test case, renaming eSDK looks much safer option

  There is only 1 such case in poky:
  $ oe-selftest --list-modules | grep INFO.- | sed 's/^.*INFO - //g' | grep -v '^[a-z_\.]*$'
  Listing all available test modules:
  eSDK

  Most modules are just a-z (52x), then oelib.<foo> (6x) and 7 modules with underscore '_'.

(From OE-Core rev: 173ef4397b5be19f72385b1a0d892a1fa8979d53)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-30 23:26:04 +01:00
Fawzi KHABER
3ae3669468 oeqa/selftest/cases/package.py: adding unittest for package rename conflicts
This Unittest tries to rename a package, using an already used name and
fails on do_package.

Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
(From OE-Core rev: c3fe173d5196506d89aa464ba56aabcf581a60db)

Signed-off-by: Fawzi KHABER <fawzi.khaber@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-30 12:32:30 +01:00
Geoffrey GIRY
81740facf4 cve-check: Fix false negative version issue
NVD DB store version and update in the same value, separated by '_'.
The proposed patch check if the version from NVD DB contains a "_",
ie 9.2.0_p1 is convert to 9.2.0p1 before version comparison.

[YOCTO #14127]

Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
(From OE-Core rev: 7d00f6ec578084a0a0e5caf36241d53036d996c4)

Signed-off-by: Geoffrey GIRY <geoffrey.giry@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-30 12:30:38 +01:00
Jose Quaresma
2ba13d9447 oeqa/selftest/reproducible: Split different packages from missing packages output
When we provide sstate_targets recipes they are reported as missing
which is not quite correct because they are proveided to cut build/debugging time.

(From OE-Core rev: fd70a564d6946fa460638dd04ce2daecf4566cf3)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-28 22:28:45 +01:00
Jose Quaresma
f5f465ff57 oeqa/selftest: OESelftestTestContext: convert relative to full path when newbuilddir is provided
Relative paths in BBLAYERS only works when the new build dir are on the same
ascending directory node:
 .
 ├── build
 ├── build-st

It works because they share the same ascending relative directory node.

So use the full path when the argument newbuilddir is provided
to make the oe-selftest work everywere regardless of the location chosen.

(From OE-Core rev: 2e022c1977bc1006c00a87e08a2dca5b69db4801)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-28 22:28:45 +01:00
Jose Quaresma
4147b7e76d oeqs/selftest: OESelftestTestContext: replace the os.environ after subprocess.check_output
No function change but is needed to simplify the next patch.

(From OE-Core rev: 193d8f0d6383f54c61f90d58eeb8f41580c2360a)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-28 22:28:45 +01:00
Martin Jansa
7eef95d723 oeqa: loader.py: show warning when skipping selected module and abort if all are skipped
* skipped modules were triggering an ERROR before:

  poky $ oe-selftest -v -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers imagefeatures.ImageFeatures.test_image_gen_debugfs -K -B /OE/build/poky/build-eSDK
  2023-03-13 15:07:53,430 - oe-selftest - ERROR - Not found eSDK.oeSDKExtSelfTest.test_install_libraries_headers in loaded test cases

* but didn't show the reason why it wasn't loaded and more importantly -r
  was ignored when all selected modules were silently skipped

* add a warning when skipping some module and abort if some modules were
  selected, but all ended being skipped:

  poky $ oe-selftest -v -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers -K -B /OE/build/poky/build-eSDK
  2023-03-13 15:11:51,028 - oe-selftest - WARNING - module 'eSDK.oeSDKExtSelfTest.test_install_libraries_headers' was skipped from selected modules, because it doesn't match with module name assumptions: package and module names do not contain upper case characters, whereas class names do
  2023-03-13 15:11:51,028 - oe-selftest - ERROR - All selected modules were skipped, this would trigger selftest with all tests and -r ignored.

* I was hit by this in oe-selftest -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers
  which is skipped due to upper case characters in module name and selftest started to run
  all tests (archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name as first from 529)

  poky $ oe-selftest -v -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers -K -B /OE/build/poky/build-eSDK
  2023-03-13 14:00:52,955 - oe-selftest - DEBUG - Selected tests with -r: ['eSDK.oeSDKExtSelfTest.test_install_libraries_headers']
  2023-03-13 14:00:55,531 - oe-selftest - INFO - Changing cwd to /OE/build/poky/build
  ..
  2023-03-13 14:00:58,128 - oe-selftest - INFO - test_archiver_allows_to_filter_on_recipe_name (archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name)

  I'll rename eSDK to esdk in next commit to avoid this.

* also fix small typo in context I've noticed when debugging this

(From OE-Core rev: 15229c9abaf4cc398c31d97d171b9fac57141873)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
248f6f01cf selftest: imagefeatures.py: respect IMAGE_LINK_NAME for debugfs and manifest as well
* these cases were correctly respecting IMAGE_LINK_NAME in most tests

* the only exception was relatively wide glob for manifest:
  "test-empty-image-*.manifest"
* and even wider glob for -dbg:
  "*-dbg.rootfs.tar.bz2"

* replace them with the exact filename we expect for given image

* be aware that gzip won't accept the symlink in IMAGE_LINK_NAME causing:
  2023-03-13 08:58:23,845 - oe-selftest - INFO -  ... FAIL
  2023-03-13 08:58:23,845 - oe-selftest - INFO - Traceback (most recent call last):
    File "/OE/build/poky/meta/lib/oeqa/selftest/cases/imagefeatures.py", line 124, in test_bmap
      self.assertTrue(runCmd('gzip -t %s' % gzip_path))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/OE/build/poky/meta/lib/oeqa/utils/commands.py", line 214, in runCmd
      raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
  AssertionError: Command 'gzip -t /OE/build/poky/build/build-st-2023-03-12-todo-patch2/imagefeatures.ImageFeatures.test_bmap/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.ext4.bmap.gz' returned non-zero exit status 1:
  gzip: skipping: /OE/build/poky/build/build-st-2023-03-12-todo-patch2/imagefeatures.ImageFeatures.test_bmap/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.ext4.bmap.gz is a symbolic link

* and "*-dbg.rootfs.tar.bz2" doesn't work if IMAGE_NAME_SUFFIX
  is changed to anything else than ".rootfs" or moved into
  IMAGE_LINK_NAME (like I plan in future changes where this will
  need to be updated again).

  Luckily we can use the symlink which currently doesn't have
  IMAGE_NAME_SUFFIX nor the DATETIME, so we don't need to search
  for it with glob, e.g. currently:

  core-image-minimal-qemux86-64-dbg-20230313112546-dbg.rootfs.tar.bz2
  core-image-minimal-qemux86-64-dbg.tar.bz2 -> core-image-minimal-qemux86-64-dbg-20230313112546-dbg.rootfs.tar.bz2

[YOCTO #12937]

(From OE-Core rev: 39285e981343930e41afe4eb8f2db675a85d54c2)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
752d8ab3ed selftest: minidebuginfo.py respect IMAGE_LINK_NAME
* use IMAGE_LINK_NAME instead of hardcoding
  core-image-minimal-${MACHINE} assumption

[YOCTO #12937]

(From OE-Core rev: 9f8ffe22d9bb7cd93b9bc9ece917a48b27ab22d3)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00