Commit Graph

2690 Commits

Author SHA1 Message Date
Ross Burton
41ffa16912 oeqa/runtime/parselogs: load ignores from disk
Instead of hardcoding the list of ignored errors/warnings in the test
itself, read them plain text files on disk.

This uses importlib to try to open a file called
oeqa.runtime.cases.parselogs-ignores-[candidate].txt, where the
candidate will be:

- "common"
- The TARGET_ARCH
- Each of the MACHINEOVERRDES

This allows the common and tune-specific ignores to be retained in
oe-core, and machine-specific ignores added to the layer where the
machine is defined.

[ YOCTO #14604 ]

(From OE-Core rev: 7a04063f7cff243fe2bee09664ad7979612110cb)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Lukas Funke
2da0213eee selftest: wic: add test for zerorize option of empty plugin
Add test for empty plugin which tests whether the plugin creates
partitions with actual data which is 'zero'.

(From OE-Core rev: 6c6b236b34b35d0e2c020e0f9c447ac35adf8faf)

Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
a02279663f oeqa/selftest/recipetool/devtool: add test for pypi class
recipetool now supports the pypi class and python recipes can by created
using the new following syntax:

* recipetool create https://pypi.org/project/<package>
* recipetool create https://pypi.org/project/<package>/<version>
* recipetool create https://pypi.org/project/<package> --version <version>

or the old syntax:
* recipetool create https://files.pythonhosted.org/packages/<...>

So add tests for the new syntax and modify old tests

(From OE-Core rev: 50779b7d45a492e9564005274f1858234a871e10)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
9f4df13f64 oeqa/selftest/recipetool: remove spaces on empty lines
(From OE-Core rev: 365c3fd9def03e07af025a949e27c4f324fd6094)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Kareem Zarka
95d8a12b31 oeqa/selftest/wic: Add tests for kernel image installation
- test_skip_kernel_install: This test verifies that the kernel is not
installed in the boot partition when the
'install-kernel-into-boot-dir' parameter is set to false.
- test_kernel_install: This test verifies that the kernel is installed
in the boot partition when the 'install-kernel-into-boot-dir'
parameter is set to true.

Both tests use a WKS (Kickstart) file to specify the desired
configuration, build a disk image using WIC, and extract the disk
image to a temporary directory to verify the results.

(From OE-Core rev: a99bc5ed8bf67f171be24c0e2220aae6cccf230e)

Signed-off-by: Kareem Zarka <kareem.zarka@huawei.com>
Signed-off-by: Christian Taedcke <christian.taedcke@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-04 11:23:37 +00:00
Julien Stephan
49f549a142 oeqa/selftest/devtool: add test for git submodules
Add a test for gitsm recipes.
This tests that we can do changes on submodules, commit them and
properly extract the patches

(From OE-Core rev: 2fb69161fe9d25691b75a043ec5566ffe4a25b37)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-01 11:48:25 +00:00
Sundeep KOKKONDA
3e50e45917 rust: Split rustdoc into a separate package
The 'rustdoc' is moved to a separate 'rust-rustdoc' package. This is a
workaround to test if the main rust binary is reproducible even if
rustdoc isn't.

(From OE-Core rev: dfed7e63e199009ecca8e2e34f6a823c6e1677e0)

Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-27 22:54:59 +00:00
Lee Chee Yang
dfedec4b2d wic: add test for partition hidden attributes
Add test for the --hidden argument introduced in Oe-Core
rev 7a111ff58d7390b79e2e63c8059f6c25f40f8977.

(From OE-Core rev: 819286cdce6471ca3dd048d3950b943b28e6d222)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-27 22:54:59 +00:00
Peter Kjellerstedt
541868630f oeqa/selftest/tinfoil: Add tests that parse virtual recipes
Running `devtool search gcc` would result in errors for virtual recipes.
The reason was the newly introduced :layer-<layername> override.

When tinfoil parsed the recipes using the parseRecipeFile() function, it
failed to properly identify the layername for virtual recipes, which
resulted in the errors.

Add tests that exercise these code paths through parseRecipeFile().

(From OE-Core rev: cf7f34f93f9f90279412ea57d6d5930854d411ba)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-23 12:46:42 +00:00
Richard Purdie
938abfa570 oeqa/selftest/sstatetests: Re-enable CDN tests
I believe other fixes and cleanup means the CDN tests are now working
correctly, so enable it.

(From OE-Core rev: baabe012e158e387a692b234920eeab977c27440)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-23 12:12:00 +00:00
Ross Burton
c15a717aa0 oeqa/selftest/debuginfod: improve selftest
This test was occasionally failing for no obvious reason, so refactor
and improve:

- While waiting for the daemon, check that it is still running and
  explicitly timeout after 10s when making the HTTP call.

- While waiting for the daemon to be ready, log the current state of the
  daemon so we can tell if we're timing out as it is still scanning.

- This was in fact the cause of the intermittant failures, because the
  TMPDIR is reused between tests and may contain a large number of
  packages. Do the tests in an isolated TMPDIR to hopefully mitigate this
  issue and increase the timeout to two minutes.

- Decorate the test using runqemu as such so that can be skipped in
  environments without runqemu

- Add a second test that doesn't use runqemu or images, which is faster
  but less realistic.

(From OE-Core rev: 88b660aaae2527736b6eccec4c952eee969e20a2)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-22 14:08:48 +00:00
Julien Stephan
62ad6660e0 oeqa/selftest/devtool: fix test_devtool_modify_overrides test
This test fails for machines qemuarm and qemux86 because when doing
devtool modify, the default devtool branch contains the patch that match
the current configuration, so for both qemuarm and qemux86  machines the
corresponding override patch is applied and we get the following error
(for qemuarm machine):

  AssertionError: 'This is a test for qemuarm\n' != 'This is a test for something\n'
  - This is a test for qemuarm
  ?                    ^ ^^^^^
  + This is a test for something
  ?                    ^^^ ^^^^^

Fix the test by looking at the correct value depending on the current
machine configuration

(From OE-Core rev: fe03789d9555c025316325b559bbde40d5e770a8)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-20 15:30:52 +00:00
Richard Purdie
abeea5c797 selftest/reproducible: Allow packages exclusion via config
OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES can be used to prevent known
not-reproducible packages to make the reproducible test fail.

For example, in local.conf:
  OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES = "example-not-reproducible"

To quote the original exclusion commit (4864764667)
> selftest/reproducible: add an exclusion list for items that are not yet reproducible
> [...]
> Non-reproducible excluded packages are not given to diffoscope and do not cause a
> failure, but still saved side-by-side with non-reproducible failing ones to make
> investigation easier.

NB: Patch was written by Richard with small fixes and a commit message
from Yoann.
(From OE-Core rev: bea0e3f5b8b383ae8590b874a5878f67832d6929)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-15 15:42:44 +00:00
Yoann Congal
cc2980ab7f selftest/reproducible: Split a long line
No other change.
Just cosmetic to avoid tripping the maximum line length of patchtest
when the next element will be  added to the list.

(From OE-Core rev: 7c401fe421c2085f2e7fc14589ca44ec1791db03)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-15 15:42:44 +00:00
luca fancellu
0f25c8aa77 oeqa/ssh: Handle SSHCall timeout error code
The current code in ssh.py is terminating the ssh process that
does not finish its computation in a given timeout (when timeout
is passed), the SSHCall function is returning the process error
code.

The Openssl ssh before version 8.6_p1 is returning 0 when it is
terminated, from commit 8a9520836e71830f4fccca066dba73fea3d16bda
onwards (version >= 8.6_p1) ssh is returning 255 instead.

So for version of ssh older than 8.6_p1 when the SSHCall time out,
the return code will be 0, meaning success, which is wrong.

Fix this issue checking if the process has timeout (hence it's been
terminated) and checking if the returned code is 0, in that case
set it to 255 to advertise that an error occurred.

Add a test case excercising the timeout in the SSHTest, test_ssh
test function.

(From OE-Core rev: 948fecca1db4c7a30fcca5fcf5eef95cd12efb00)

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-10 17:44:27 +00:00
Richard Purdie
c6276d8e57 oeqa/selftest/sstatetests: Fix intermitttent errors and improve performance
You could reproduce an error in this test with:

bitbake core-image-minimal
bitbake tzcode-native -c cleansstate
oe-selftest -r sstatetests.SStatePrintdiff.test_image_minimal_vs_base_do_configure

since tzcode-native isn't needed once tzdata is available and isn't rebuilt
by "bitbake core-image-minimal" if it is missig. tzdata is allarch so if tzdata is
built on an x86 host, a aarch64 build machine would never build tzcode-native with
this set of calls.

Add a --runall option to the initial bitbake invocation to force these things to be
created if they're missing.

This explains why some failures were occurring on the infrastructure. With that issue
fixed, drop the hash mode change since I believe this fixes that issue. That
restriction was hurting performance, this should allow sstate reuse for the test
and improve the speed of it.

(From OE-Core rev: 8f03ac39e7fe21f3d6eca35b12b203a73a15285d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-09 11:45:37 +00:00
Lukas Funke
b2c07c2501 selftest: recipetool: Add test for go recipe handler
This commit adds a test for the go recipetool handler. The choosen go
project to test the created recipe was picked randomly. The SRC_URIs and
the LIC_FILES_CHKSUMs are checked against there reference values.

(From OE-Core rev: 3337b9ec069a21b87676a90d0f7819e6a8dc856a)

Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-06 16:41:11 +00:00
Richard Purdie
e598223e0b oeqa/selftest: Drop machines support
The machines option to oe-selftest isn't used in our CI and is never likely to
be, we focus and execute testing explictly. The YOCTO #15247 is about how
this code doesn't interact well with build directory cleanup and at this point
I think we should just remove the option/code.

(From OE-Core rev: 815d04a2007e1154b69f1a027c8677ea86935354)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-05 10:57:56 +00:00
Alexander Kanavin
3a7b408178 selftest/sstatetests: add a test for CDN sstate cache
Specifically, the test checks that everything needed for building
standard oe-core images for x86_64 and arm64 is available from
the cache (with minor exceptions). Going forward, a complete
world check could be enabled and additional configurations,
but that requires improvements to performance of hash equivalence
server in particular.

RP: I've disabled the tests by default so we can merge them. We will
make them live once we get to the bottom of the failures.

(From OE-Core rev: 5f3aeadb65d3b7216db783b2c500ac241b03deb8)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-31 11:12:33 +00:00
Tim Orling
9ffac3fc3d recipetool: add python_hatchling support
One of the newer PEP-517 backends to be added was python_hatchling.bbclass
but it was not included in the recent improvements.

Add selftest for 'jsonschema' pypi package.

(From OE-Core rev: d99b4883b4fee82bc588fd235ba90fedf1550cb8)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-30 08:50:46 +00:00
Alexander Kanavin
2cf13f2513 lib/oe/sstatesig.py: dump locked.sigs.inc only when explicitly asked via -S lockedsigs
This was writing out locked-sigs.inc into cwd with every
'bitbake -S' invocation. When the intent is only to to get task
stamps (-S none), or print the difference between them (-S printdiff),
the file is unnecessary clutter.

A couple of selftests/scripts were however relying on this, so they're
adjusted to explicitly request the file.

eSDK code calls dump_lockedsigs() separately via
oe.copy_buildsystem.generate_locked_sigs() and so isn't affected.

(From OE-Core rev: ad57c3cac2a8d3e60222e3cca0685f582dcea135)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 10:53:43 +01:00
Alexander Kanavin
2187e823ad selftest/sstatetests: add tests for 'bitbake -S printdiff'
'bitbake -S printdiff' is a useful diagnostic facility for finding out
why sstate is not being reused, but until now it had no tests that would
ensure it works. This commit adds three basic scenarios:

1. make a change in a really basic, common recipe that is at the very root
of dependency trees (quilt-native), and ensure that change is correctly discovered when
building an image.

2. make a change in gcc-source recipe, which is somewhat special
(operates in work-shared), and ensure that gcc-runtime builds track
that down as well.

3. make a change in base_do_configure() definition from base.bbclass,
which is not recipe-specific, but affects many basic recipes, and ensure that
is correctly reported as well.

The test itself actually runs twice:
- first against a fully populated build directory, where
the printdiff code is guaranteed to find the correct previous
stamp that can be compared with in a predictable manner.

- then in an empty build directory where the printdiff code
goes to look in the sstate cache, and so the existence of the
previous signature can be tested, but not the difference with it
(what the exact difference would be is unpredictable as the
sstate cache is indeed shared between many builds).

(From OE-Core rev: 7a7d76aa8a8d590ebc99156f9f4b9535cdf868c7)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 10:53:43 +01:00
Julien Stephan
9f306937ba oeqa/selftest/recipetool: add selftest for PEP-517 recipe creation
Add 3 tests to check the creation of PEP-517 project using the 3
 backends supported by bitbake:
  - setuptools.build_meta
  - poetry.core.masonry.api
  - flit_core.buildapi

Theses tests requires the tomllib python module, so skip theses tests
if module is not present. tomllib module is part of python starting from 3.11

(From OE-Core rev: 54356c6f1290d0d4170ed52f7bb358bb9efc1aec)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Julien Stephan
be129bd0bc recipetool/create_buildsys_python: prefix created recipes with python3-
By convention, all python recipes start with "python3-" so update
create_buildsys_python to do this

This rule doesn't apply for packages already starting with "python"

Update recipetool's selftest accordingly

(From OE-Core rev: b0d87440e610b80f763d09784d4a90a148bb3e7b)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Julien Stephan
df011d7498 oeqa/selftest/devtool: remove spaces on empty line
(From OE-Core rev: a37430f682bca9787d939b0722dd5d0d810c12c6)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Julien Stephan
8ed88a98d5 oeqa/selftest/devtool: abort if a local workspace already exist
if user run devtool selftests with a local workspacelayer
the tests fail with various error such as:

- devtool.DevtoolAddTests.test_devtool_add just hangs
- devtool.DevtoolModifyTests.* fail with the following error:

 ERROR: Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it.
 Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it.

Check if a workspacelayer exists, warn the user and abort the tests

(From OE-Core rev: a74962cfb0485f6f2b9e2b751c33c8eafca8705a)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Michael Opdenacker
7ac5639812 oeqa/runtime/cases/parselogs: remove "edgerouter" case
The "edgerouter" machine has been removed since
https://git.yoctoproject.org/poky/commit/?id=0c64d0e4317e3749f7f7ed9ecd5d08bbb0cedc9e

(From OE-Core rev: b82514c821add181e141d12b0c1723760b445fea)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-26 21:11:28 +01:00
Alexander Kanavin
d180e9f1c2 selftest/buildoptions: tag the download mirror test with 'yocto-mirrors'
This will allow bundling all yocto mirror tests together, both for
the purposes of running only them specifically,
and excluding them from 'general' oe-selftest runs.

There is an upcoming test for sstate cache served over content
delivery network which will use the same tag, so it can be run
together with this.

(From OE-Core rev: 2a3f69b83ed10622d6a39774528dbfaebb3e5ded)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-20 13:54:23 +01:00
Sean Nyekjaer
96310a5867 oeqa/sdk/rust: Add build and run test of rust binary with SDK host
Add a QA test to the SDK to test that a basic cargo build works for the
SDK host.

(From OE-Core rev: 7f05760debd3aeb69c3294f3ceb92d4f1aceec1f)

Signed-off-by: Sean Nyekjaer <sean@geanix.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-19 13:38:56 +01:00
Richard Purdie
ddd38a3b15 oeqa/qemurunner: Drop newlines serial workaround
Drop the newlines serial workaround, it doesn't seem to fix things
enough of the time to be useful.

(From OE-Core rev: 6cd57b2d148c5de7839d52d8cec359e4a10e0cd6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-15 09:11:47 +01:00
Richard Purdie
0e351df042 reproducible: Exclude rust for now again
Unfortunately there is still an issue with rustdoc not being reproducible
so part of the problem has been fixed but not everything.

Add the exclusion back until this has been addressed to avoid autobuilder
failures and long diffoscope analysis time.

(From OE-Core rev: ab03b3416dda65e39a756c02d87492e1fe5e0fb0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-13 11:53:06 +01:00
Ross Burton
d44333b92c oeqa/runtime/_qemutiny: rewrite test to be functional
The _qemutiny is a small test case that was explicitly designed to do a
minimal level of testing for poky-tiny images.  These typically don't
have SSH servers so we need to assume that qemu is being used and access
the serial console directly.

(From OE-Core rev: 2245b2754d6f4798127ce85a2ab7cb48f458c1f7)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-11 09:43:45 +01:00
Ross Burton
947f7d7b8c oeqa/selftest: don't skip test_read_only_image on qemuarm64
There's no need to skip this test anymore, as skipping on qemuarm64 was
a proxy for "uses SERIAL_CONSOLES_CHECK" which no longer exists.

[ YOCTO #14962 ]

(From OE-Core rev: 1b8b1cb74c8f5f387a14d86f03f37f68c07dd9aa)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-11 09:43:45 +01:00
Alexis Lothoré
3c53168682 oeqa/utils/gitarchive: ensure tag matches regex before getting its fields
Whenever we ask gitarchive to retrieve test results for specific revisions,
we first do a "large" search in get_tags, which uses glob patterns with git
ls-remote, and then we filter received tags with a regex to parse the tags
fields.
Currently gitarchive assumes that all tags returned by get_tags will match
the regex. This assumption is wrong (for example searching "master-next" in
get_tags may return some tags like "abelloni/master-next), and leads then
to exception when we try to retrieve tags fields:
Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 78, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 72, in main
    ret = args.func(args, logger)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pokybuild/yocto-worker/a-full/build/scripts/lib/resulttool/regression.py", line 315, in regression_git
    revs2 = gitarchive.get_test_revs(logger, repo, tag_name, branch=args.branch2)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pokybuild/yocto-worker/a-full/build/meta/lib/oeqa/utils/gitarchive.py", line 246, in get_test_revs
    fields, runs = get_test_runs(log, repo, tag_name, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pokybuild/yocto-worker/a-full/build/meta/lib/oeqa/utils/gitarchive.py", line 238, in get_test_runs
    groups = m.groupdict()

Fix this exception by merely skipping those additionals tags which won't
match the regex

(From OE-Core rev: 8b5ace47372e958db9e4abb23378947fb02f6fc2)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-11 09:43:45 +01:00
Alexis Lothoré
800665612a oeqa/utils/gitarchive: fix tag pattern searching
Whenever we ask gitarchive to search for tags, we can provide it with a
pattern (containing glob patterns). However, when searching for example for
tags matching branch master-next, it can find more tags which does not
correspond exactly to branch master-next (e.g. abelloni/master-next tags
will match).

Prevent those additional tags from being fetched by gitarchive by using a
more specific pattern: prefix user-provided pattern with "refs/tags"

(From OE-Core rev: c24b7ea28021da48aa8f3498a9b899f595efde56)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-11 09:43:45 +01:00
Richard Purdie
4411460016 oeqa/qemurunner: Ensure we retry after BrokenPipeError
If the BrokenPipeError occurs when writing to the serial port to wake it
up, defer the write and try again (which will happen on the 5s timeout
of the select call). Why it should return ESHUTDOWN and then work later
I'm not sure but it does appear to make it work. For now we need 'working'
QA tests whilst the issue is debugged.

(From OE-Core rev: ea9e6ba0ab31a0b20012c283aa768496a50b527a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-11 09:39:46 +01:00
Sundeep KOKKONDA
b36cbd3f2b rust: reproducibility issue fix
The '--remap-path-prefix' option removes all references to build directory
structure in the debug information within the compiled output for Cargo
dependencies and the project's binary.

However, some references to build directories remains in the final binary
in .rustc section in the form of compressed metadata and this makes the
build output dependent on the folder structure of the computer it's compiled on.

So, for reproducible builds, use the configuration option
'remap-debuginfo = true' along with the '--remap-path-prefix'.

[YOCTO# 14875]

(From OE-Core rev: 6ae62259afbbe861ed74211dab18a27b8c8d8b7a)

Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-11 09:36:53 +01:00
Richard Purdie
c0af426af8 oeqa/qemurunner: Add extra logging when console doesn't appear
If the console doesn't appear, breifly sleep and try an extra read. This
is a useful debugging trick which we may as well preserve in the code. We're
already failing at this point so extra data is useful.

This means if we perturb things on the other port, we may gain useful logging
insight.

(From OE-Core rev: be3724854947ded160789d0353b7efb8c593040f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-10 09:53:32 +01:00
Richard Purdie
44b4c74cad runqemu/qemurunner: Use nodelay with tcp serial connections
This disables Nagle's algorithm for our tcp serial connections which may
be causing data transfer issues.

(From OE-Core rev: f8eff4c427881a98333fdf7c42f66ed6603e4f03)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-10 09:53:32 +01:00
Richard Purdie
ef7e7b446a oeqa/qemurunner: Add newlines serial workaround
We're struggling with the 6.5 kernel as the serial port getty doesn't appears sometimes
leading to failures in CI. Add a workaround of sending some newlines as a way of
unblocking the kernel/release issues whilst we try and work out how to get to the bottom
of the issue.

(From OE-Core rev: 0a65f0d272895ba13c8c133ee71f3605d765a8a7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-10 09:53:32 +01:00
Julien Stephan
4f2f3c12a9 oeqa/selftest/bblock: add self test for bblock tool
it implements various combination of locking single/multiple recipe(s)/task(s)

it also tests that locked sig are architecture dependant

(From OE-Core rev: 7047a067548acd592a834d4315579fb606b4b9d7)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-09 15:58:47 +01:00
Mikko Rapeli
23a50c8cca qemurunner.py: detect login prompt without without utf-8 conversion
In case utf-8 conversion of the serial console data is flaky
due to bad characters in the stream.

(From OE-Core rev: 3f4921ced1e2aae546ff7cef232eec3e214c28be)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-08 13:05:25 +01:00
Richard Purdie
c9caf79cab oeqa/concurrencytest: Remove invalid buffering option
Fix warnings from oe-selftest -j:

/usr/lib/python3.10/os.py:1030: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  return io.open(fd, mode, buffering, encoding, *args, **kwargs)

Remove the option since it clearly doesn't do much.

(From OE-Core rev: 6b872ee72942951fd464c4c6cb9eadcb9b4749c1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-05 21:09:10 +01:00
Richard Purdie
ce804a6797 qemurunner: Show both the login console log and all logging upon failure
It is unclear when things fail which output was on which serial port. Improve the output
to show the last lines of both data to improve debugging.

(From OE-Core rev: 605938b3bb19dcf6c7218648b4d16df9eaa675fc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-04 23:05:54 +01:00
Richard Purdie
aaaa314807 qemurunner: Log the second serial console as well as the first
To aid debugging, always log the second serial console as well as the first
to a seperate log file. This should make it clearer what happened when we
see test failures.

(From OE-Core rev: 372cd58fe672726900af80f3e65a108984da2750)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-04 23:05:54 +01:00
Richard Purdie
4abef7a9d9 qemurunner: Use backslashreplace with utf8 to make invalid characters clear
Switch to the backslashreplace error handling when decoding strings so that
invalid characters are clear in the stream to improve debugging.

(From OE-Core rev: 8ec86d46f736ec3e625b741e97545377c79414ee)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-04 23:05:54 +01:00
Richard Purdie
2305928a00 qemurunner: Skip conversion from/to binary data for logfile
There is no point in decoding binary data only to encode it again risking
conversion issues. Write the raw data to the log file as binary and skip
the conversion.

Also always update self.msg even if a logfile isn't specified to improve
logging/debug.

(From OE-Core rev: b5d67471dddd198723c4f711747783a33e8e5987)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-04 23:05:54 +01:00
Richard Purdie
fc5a15cc64 oeqa/utils/gitarchive: Handle broken commit counts in results repo
The test results repository contains tags like:

master/64501-g65c94ca3196e5ef3344a469fea8e30444f2e967a/0
master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/3
master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/2
master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/1
master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/0

where the commit count is correct in one case and not in the others. This causes
assertion errors in the current code.

Add in some code to work around these historical issues where the commit counts are low.

(From OE-Core rev: d51fc5c8c469730885af7bbde7122032de411d89)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-28 12:37:47 +01:00
Samantha Jalabert
aa8b9d601a qa: Add selftest for python3-spdx-tools
(From OE-Core rev: e766fe7a9391cc9348d92ab704ec58c4806e38c6)

Signed-off-by: Marta Rybczynska <mrybczynska@syslinbit.com>
Signed-off-by: Samantha Jalabert <samantha.jalabert@syslinbit.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-27 11:47:42 +01:00
Ross Burton
f15f380650 oeqa/runtime/parselogs: parse the logs with Python, not grep
Instead of constructing huge grep statements, we can simply open the logs
in Python and do the relevant string operations directly.

The trick is to remember to casefold() all of the strings, so that the
"in" operator can be used.

Just one of the ignores needs to be adjusted because it uses a regular
expression and the new logic doesn't support that.  This is handled
by simply reducing the size of the ignore match.

(From OE-Core rev: 78ae254c4a78a025a712281ce9de373cdccf5472)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-26 10:25:42 +01:00