Commit Graph

4068 Commits

Author SHA1 Message Date
Kareem Zarka
95d8a12b31 oeqa/selftest/wic: Add tests for kernel image installation
- test_skip_kernel_install: This test verifies that the kernel is not
installed in the boot partition when the
'install-kernel-into-boot-dir' parameter is set to false.
- test_kernel_install: This test verifies that the kernel is installed
in the boot partition when the 'install-kernel-into-boot-dir'
parameter is set to true.

Both tests use a WKS (Kickstart) file to specify the desired
configuration, build a disk image using WIC, and extract the disk
image to a temporary directory to verify the results.

(From OE-Core rev: a99bc5ed8bf67f171be24c0e2220aae6cccf230e)

Signed-off-by: Kareem Zarka <kareem.zarka@huawei.com>
Signed-off-by: Christian Taedcke <christian.taedcke@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-04 11:23:37 +00:00
Trevor Gamblin
5c1070e258 patchtest: shorten patch signed-off-by test output
The patch Signed-off-by test's output line is excessively long, and can
trigger a failsafe in the patchtest automated service's email content
generation. Shorten the output by reducing redundant phrasing and using
os.path.basename to get only the failing patch's name, not the entire
path, as the submitter should have a good idea of where it is located
regardless.

(From OE-Core rev: cc7546ded87dd44a988d7a23f1d7645094b5cdd4)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-04 11:23:37 +00:00
Julien Stephan
49f549a142 oeqa/selftest/devtool: add test for git submodules
Add a test for gitsm recipes.
This tests that we can do changes on submodules, commit them and
properly extract the patches

(From OE-Core rev: 2fb69161fe9d25691b75a043ec5566ffe4a25b37)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-01 11:48:25 +00:00
Julien Stephan
89f1662484 devtool: add support for git submodules
Adding the support of submodules required a lot of changes on the
internal data structures:
* initial_rev/startcommit used as a starting point for looking at new
  / updated commits was replaced by a dictionary where the keys are the
  submodule name ("." for main repo) and the values are the
  initial_rev/startcommit

* the extractPatches function now extracts patch for the main repo and
  for all submodules and stores them in a hierarchical way describing the
    submodule path

* store initial_rev/commit also for all submodules inside the recipe
  bbappend file

* _export_patches now returns dictionaries that contains the 'patchdir'
  parameter (if any). This parameter is used to add the correct
  'patchdir=' parameter on the recipe

Also, recipe can extract a secondary git tree inside the workdir.

By default, at the end of the do_patch function, there is a hook in
devtool that commits everything that was modified to have a clean
repository. It uses the command: "git add .; git commit ..."

The issue here is that, it adds the secondary git tree as a submodule
but in a wrong way. Doing "git add <git dir>" declares a submodule but do
not adds a url associated to it, and all following "git submodule foreach"
commands will fail.

So detect that a git tree was extracted inside S and correctly add it
using "git submodule add <url> <path>", so that it will be considered as a
regular git submodule

(From OE-Core rev: 900129cbdf25297a42ab5dbd02d1adbea405c935)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-01 11:48:25 +00:00
Julien Stephan
c7ed37b5ff lib/oe/recipeutils.py: remove trailing white-spaces
Remove useless trailing white-spaces

(From OE-Core rev: 4aa28d5ea74016087979afd1faea478fda253cc3)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-30 08:43:04 +00:00
Sundeep KOKKONDA
3e50e45917 rust: Split rustdoc into a separate package
The 'rustdoc' is moved to a separate 'rust-rustdoc' package. This is a
workaround to test if the main rust binary is reproducible even if
rustdoc isn't.

(From OE-Core rev: dfed7e63e199009ecca8e2e34f6a823c6e1677e0)

Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-27 22:54:59 +00:00
Lee Chee Yang
dfedec4b2d wic: add test for partition hidden attributes
Add test for the --hidden argument introduced in Oe-Core
rev 7a111ff58d7390b79e2e63c8059f6c25f40f8977.

(From OE-Core rev: 819286cdce6471ca3dd048d3950b943b28e6d222)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-27 22:54:59 +00:00
Peter Kjellerstedt
541868630f oeqa/selftest/tinfoil: Add tests that parse virtual recipes
Running `devtool search gcc` would result in errors for virtual recipes.
The reason was the newly introduced :layer-<layername> override.

When tinfoil parsed the recipes using the parseRecipeFile() function, it
failed to properly identify the layername for virtual recipes, which
resulted in the errors.

Add tests that exercise these code paths through parseRecipeFile().

(From OE-Core rev: cf7f34f93f9f90279412ea57d6d5930854d411ba)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-23 12:46:42 +00:00
Richard Purdie
938abfa570 oeqa/selftest/sstatetests: Re-enable CDN tests
I believe other fixes and cleanup means the CDN tests are now working
correctly, so enable it.

(From OE-Core rev: baabe012e158e387a692b234920eeab977c27440)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-23 12:12:00 +00:00
Ross Burton
c15a717aa0 oeqa/selftest/debuginfod: improve selftest
This test was occasionally failing for no obvious reason, so refactor
and improve:

- While waiting for the daemon, check that it is still running and
  explicitly timeout after 10s when making the HTTP call.

- While waiting for the daemon to be ready, log the current state of the
  daemon so we can tell if we're timing out as it is still scanning.

- This was in fact the cause of the intermittant failures, because the
  TMPDIR is reused between tests and may contain a large number of
  packages. Do the tests in an isolated TMPDIR to hopefully mitigate this
  issue and increase the timeout to two minutes.

- Decorate the test using runqemu as such so that can be skipped in
  environments without runqemu

- Add a second test that doesn't use runqemu or images, which is faster
  but less realistic.

(From OE-Core rev: 88b660aaae2527736b6eccec4c952eee969e20a2)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-22 14:08:48 +00:00
Ross Burton
f841d7918d lib/oe/patch: ensure os.chdir restoring always happens
If we chdir(), do the chdir back to the original directory in a finally
block so they always run.

(From OE-Core rev: cdc40292818683b6df1c814498c7589450a163fa)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-22 14:08:48 +00:00
Richard Purdie
0769e8ef6b bitbake.conf: Drop oldincludedir
Autoconf defines this as:

"The directory for installing C header files for non-GCC compilers."

Whilst this is something autoconf does allow changing, I find it hard
to believe it has much use in the wild now and that headers don't get
split like this in reality, it would probably only be useful on really
old unixes.. The values are the same in our configuration anyway.

Drop the value and just use includedir everywhere.

(From OE-Core rev: 506c91cbc6a604a84e37e53ccff430436369802e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-20 15:30:52 +00:00
Julien Stephan
62ad6660e0 oeqa/selftest/devtool: fix test_devtool_modify_overrides test
This test fails for machines qemuarm and qemux86 because when doing
devtool modify, the default devtool branch contains the patch that match
the current configuration, so for both qemuarm and qemux86  machines the
corresponding override patch is applied and we get the following error
(for qemuarm machine):

  AssertionError: 'This is a test for qemuarm\n' != 'This is a test for something\n'
  - This is a test for qemuarm
  ?                    ^ ^^^^^
  + This is a test for something
  ?                    ^^^ ^^^^^

Fix the test by looking at the correct value depending on the current
machine configuration

(From OE-Core rev: fe03789d9555c025316325b559bbde40d5e770a8)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-20 15:30:52 +00:00
Richard Purdie
abeea5c797 selftest/reproducible: Allow packages exclusion via config
OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES can be used to prevent known
not-reproducible packages to make the reproducible test fail.

For example, in local.conf:
  OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES = "example-not-reproducible"

To quote the original exclusion commit (4864764667)
> selftest/reproducible: add an exclusion list for items that are not yet reproducible
> [...]
> Non-reproducible excluded packages are not given to diffoscope and do not cause a
> failure, but still saved side-by-side with non-reproducible failing ones to make
> investigation easier.

NB: Patch was written by Richard with small fixes and a commit message
from Yoann.
(From OE-Core rev: bea0e3f5b8b383ae8590b874a5878f67832d6929)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-15 15:42:44 +00:00
Yoann Congal
cc2980ab7f selftest/reproducible: Split a long line
No other change.
Just cosmetic to avoid tripping the maximum line length of patchtest
when the next element will be  added to the list.

(From OE-Core rev: 7c401fe421c2085f2e7fc14589ca44ec1791db03)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-15 15:42:44 +00:00
luca fancellu
0f25c8aa77 oeqa/ssh: Handle SSHCall timeout error code
The current code in ssh.py is terminating the ssh process that
does not finish its computation in a given timeout (when timeout
is passed), the SSHCall function is returning the process error
code.

The Openssl ssh before version 8.6_p1 is returning 0 when it is
terminated, from commit 8a9520836e71830f4fccca066dba73fea3d16bda
onwards (version >= 8.6_p1) ssh is returning 255 instead.

So for version of ssh older than 8.6_p1 when the SSHCall time out,
the return code will be 0, meaning success, which is wrong.

Fix this issue checking if the process has timeout (hence it's been
terminated) and checking if the returned code is 0, in that case
set it to 255 to advertise that an error occurred.

Add a test case excercising the timeout in the SSHTest, test_ssh
test function.

(From OE-Core rev: 948fecca1db4c7a30fcca5fcf5eef95cd12efb00)

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-10 17:44:27 +00:00
Jermain Horsman
61e7c1d99c lib/bblayers/setupwriters/oe-setup-layers.py: Fix indentation
The writer always printed the script was written even if this
was not the case when specififying '--json-only'.

(From OE-Core rev: b2efb4c6c8bd938aac5146e14bd0c38f3f632bc4)

Signed-off-by: Jermain Horsman <jermain.horsman@nedap.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-09 16:57:26 +00:00
Richard Purdie
c6276d8e57 oeqa/selftest/sstatetests: Fix intermitttent errors and improve performance
You could reproduce an error in this test with:

bitbake core-image-minimal
bitbake tzcode-native -c cleansstate
oe-selftest -r sstatetests.SStatePrintdiff.test_image_minimal_vs_base_do_configure

since tzcode-native isn't needed once tzdata is available and isn't rebuilt
by "bitbake core-image-minimal" if it is missig. tzdata is allarch so if tzdata is
built on an x86 host, a aarch64 build machine would never build tzcode-native with
this set of calls.

Add a --runall option to the initial bitbake invocation to force these things to be
created if they're missing.

This explains why some failures were occurring on the infrastructure. With that issue
fixed, drop the hash mode change since I believe this fixes that issue. That
restriction was hurting performance, this should allow sstate reuse for the test
and improve the speed of it.

(From OE-Core rev: 8f03ac39e7fe21f3d6eca35b12b203a73a15285d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-09 11:45:37 +00:00
Trevor Gamblin
1c135f1d7a patchtest: rework license checksum tests
Remove the pretest_lic_files_chksum_modified_not_mentioned test entirely
and use pyparsing in test_lic_files_chksum_modified_not_mentioned to
scan the patches for lines starting with either "+LIC_FILES_CHKSUM" or
"-LIC_FILES_CHKSUM".  If either is found but no "License-Update" tag is
present in the commit, fail the test.

(From OE-Core rev: 8e1bda0eb225ada22fdf5990edfec512be1d6629)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-08 11:00:09 +00:00
Richard Purdie
3d35d09ff2 sstate: Ensure sstate searches update file mtime
Commands like "bitbake XXX -S printdiff" search for sstate files but don't download
them. This means that local files aren't touched as the download code would do, meaning
the sstate cleanup scripts can delete them. This can then lead to obtuse build failures.

Have the search code touch local files in the same way as the main code paths would to
avoid these files disappearing.

Move the function to a common touch() function in lib/oe instead of duplicating code.

(From OE-Core rev: a27fc0bd5706ab5b9c68a0271fcf57377a678cdf)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-08 10:56:35 +00:00
Lukas Funke
b2c07c2501 selftest: recipetool: Add test for go recipe handler
This commit adds a test for the go recipetool handler. The choosen go
project to test the created recipe was picked randomly. The SRC_URIs and
the LIC_FILES_CHKSUMs are checked against there reference values.

(From OE-Core rev: 3337b9ec069a21b87676a90d0f7819e6a8dc856a)

Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-06 16:41:11 +00:00
Vyacheslav Yurkov
914a921c33 lib/oe/path: Deploy files can start only with a dot
There might be only hidden files deployed. In that case we don't need a
generic wildcard present in copy command, otherwise it fails with:

Exception: subprocess.CalledProcessError: Command 'cp -afl --preserve=xattr ./.??* ./* <BUILDDIR>/tmp/deploy/images/qemux86-64' returned non-zero exit status 1.

Subprocess output:
cp: cannot stat './*': No such file or directory

(From OE-Core rev: f92c751281609ea6bd6b838307de4bc70bf26ab9)

Signed-off-by: Vyacheslav Yurkov <Vyacheslav.Yurkov@wika.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-05 11:28:35 +00:00
Jermain Horsman
a9063f845b lib/oe/buildcfg.py: Remove unused parameter
Several functions included the 'd' parameter but never used it,
additionally the value passed is always None.

(From OE-Core rev: 9e03ce0426576ebef3739dc1dfec4f7cd73ae094)

Signed-off-by: Jermain Horsman <jermain.horsman@nedap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-05 11:28:33 +00:00
Jermain Horsman
d012de568f lib/oe/buildcfg.py: Include missing import
get_layer_revisions() uses the os.path module but it is not included
in the imports.

(From OE-Core rev: 4339ef5b46c39cce402d1cc3526b39702837b839)

Signed-off-by: Jermain Horsman <jermain.horsman@nedap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-05 11:28:32 +00:00
Richard Purdie
e598223e0b oeqa/selftest: Drop machines support
The machines option to oe-selftest isn't used in our CI and is never likely to
be, we focus and execute testing explictly. The YOCTO #15247 is about how
this code doesn't interact well with build directory cleanup and at this point
I think we should just remove the option/code.

(From OE-Core rev: 815d04a2007e1154b69f1a027c8677ea86935354)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-05 10:57:56 +00:00
Alexander Kanavin
a92d76e4da meta/lib/oe/copy_buildsystem.py: do not derefence symlinks
This was added (I think) for the purpose of supporting layers that refer
to items outside of the layer via relative symlinks:
https://git.yoctoproject.org/poky-contrib/commit/?id=d31d1ad4e566e42d0bbcf1f41ac25e33181fb517

I do not think copying the link target into the layer that references it is the
correct solution: rather the original target should be included
into the SDK with the same relative path.

This change is done for the sake of preserving symlinks
that are referencing things inside the layer as they are;
particularly the content of scripts/esdk-tools/.

(From OE-Core rev: 52a7bbd5c4875c5f61ea65dda38e495a2925a20d)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-05 08:44:06 +00:00
Ross Burton
7ffa4d4044 cve-check: don't warn if a patch is remote
We don't make do_cve_check depend on do_unpack because that would be a
waste of time 99% of the time.  The compromise here is that we can't
scan remote patches for issues, but this isn't a problem so downgrade
the warning to a note.

Also move the check for CVEs in the filename before the local file check
so that even with remote patches, we still check for CVE references in
the name.

(From OE-Core rev: 0251cad677579f5b4dcc25fa2f8552c6040ac2cf)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-03 16:58:40 +00:00
Trevor Gamblin
06a7e7a14d patchtest: remove test for CVE tag in mbox
After patchtest went live it was determined that testing for a CVE tag
in the mbox commit message is unnecessary, since it will already be in
the shortlog and in any carried patches. Remove the test and the
associated selftest files so that its absence isn't flagged in future
test results.

(From OE-Core rev: 54690f18f04a2ab993a85d551ce4f8d0fa56618a)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-01 19:19:40 +00:00
Trevor Gamblin
3ad5aa3f16 patchtest: make pylint tests compatible with 3.x
pylint 3.x has removed epylint, which is now a separate module. To avoid
adding another recipe or using outdated modules, modify the
test_python_pylint tests so that they use the standard pylint API.

(From OE-Core rev: 72be3d6a116febf46130cccbe12afe5ad93779b5)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-01 19:19:40 +00:00
Alexander Kanavin
3a7b408178 selftest/sstatetests: add a test for CDN sstate cache
Specifically, the test checks that everything needed for building
standard oe-core images for x86_64 and arm64 is available from
the cache (with minor exceptions). Going forward, a complete
world check could be enabled and additional configurations,
but that requires improvements to performance of hash equivalence
server in particular.

RP: I've disabled the tests by default so we can merge them. We will
make them live once we get to the bottom of the failures.

(From OE-Core rev: 5f3aeadb65d3b7216db783b2c500ac241b03deb8)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-31 11:12:33 +00:00
Tim Orling
9ffac3fc3d recipetool: add python_hatchling support
One of the newer PEP-517 backends to be added was python_hatchling.bbclass
but it was not included in the recent improvements.

Add selftest for 'jsonschema' pypi package.

(From OE-Core rev: d99b4883b4fee82bc588fd235ba90fedf1550cb8)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-30 08:50:46 +00:00
Trevor Gamblin
9b22a5aa8c patchtest: shorten test result outputs
Some test result lines in TestMbox and TestPatch are still too long to
avoid being flagged by the mailer script. Clean them up by removing
redundant information, so that they are all under the length limit of
220 characters.

(From OE-Core rev: c10d0bb542b23fbdc14d76dfa8e5885aa4d33083)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-29 22:39:17 +00:00
Trevor Gamblin
631d19ed6c patchtest: reduce checksum test output length
The test_lic_files_chksum_modified_not_mentioned test in TestMetadata is
outputting very long lines that fail the maximum length check when
sending email results, preventing the actual errors from being
displayed. Reduce the length of the failure message by rewording and
removing redundant information.

(From OE-Core rev: 2e2625735181160e9760a6f3af4955bda2ea6d4d)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 17:48:11 +01:00
Alexander Kanavin
2cf13f2513 lib/oe/sstatesig.py: dump locked.sigs.inc only when explicitly asked via -S lockedsigs
This was writing out locked-sigs.inc into cwd with every
'bitbake -S' invocation. When the intent is only to to get task
stamps (-S none), or print the difference between them (-S printdiff),
the file is unnecessary clutter.

A couple of selftests/scripts were however relying on this, so they're
adjusted to explicitly request the file.

eSDK code calls dump_lockedsigs() separately via
oe.copy_buildsystem.generate_locked_sigs() and so isn't affected.

(From OE-Core rev: ad57c3cac2a8d3e60222e3cca0685f582dcea135)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 10:53:43 +01:00
Alexander Kanavin
2187e823ad selftest/sstatetests: add tests for 'bitbake -S printdiff'
'bitbake -S printdiff' is a useful diagnostic facility for finding out
why sstate is not being reused, but until now it had no tests that would
ensure it works. This commit adds three basic scenarios:

1. make a change in a really basic, common recipe that is at the very root
of dependency trees (quilt-native), and ensure that change is correctly discovered when
building an image.

2. make a change in gcc-source recipe, which is somewhat special
(operates in work-shared), and ensure that gcc-runtime builds track
that down as well.

3. make a change in base_do_configure() definition from base.bbclass,
which is not recipe-specific, but affects many basic recipes, and ensure that
is correctly reported as well.

The test itself actually runs twice:
- first against a fully populated build directory, where
the printdiff code is guaranteed to find the correct previous
stamp that can be compared with in a predictable manner.

- then in an empty build directory where the printdiff code
goes to look in the sstate cache, and so the existence of the
previous signature can be tested, but not the difference with it
(what the exact difference would be is unpredictable as the
sstate cache is indeed shared between many builds).

(From OE-Core rev: 7a7d76aa8a8d590ebc99156f9f4b9535cdf868c7)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 10:53:43 +01:00
Julien Stephan
9f306937ba oeqa/selftest/recipetool: add selftest for PEP-517 recipe creation
Add 3 tests to check the creation of PEP-517 project using the 3
 backends supported by bitbake:
  - setuptools.build_meta
  - poetry.core.masonry.api
  - flit_core.buildapi

Theses tests requires the tomllib python module, so skip theses tests
if module is not present. tomllib module is part of python starting from 3.11

(From OE-Core rev: 54356c6f1290d0d4170ed52f7bb358bb9efc1aec)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Julien Stephan
be129bd0bc recipetool/create_buildsys_python: prefix created recipes with python3-
By convention, all python recipes start with "python3-" so update
create_buildsys_python to do this

This rule doesn't apply for packages already starting with "python"

Update recipetool's selftest accordingly

(From OE-Core rev: b0d87440e610b80f763d09784d4a90a148bb3e7b)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Julien Stephan
df011d7498 oeqa/selftest/devtool: remove spaces on empty line
(From OE-Core rev: a37430f682bca9787d939b0722dd5d0d810c12c6)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Julien Stephan
8ed88a98d5 oeqa/selftest/devtool: abort if a local workspace already exist
if user run devtool selftests with a local workspacelayer
the tests fail with various error such as:

- devtool.DevtoolAddTests.test_devtool_add just hangs
- devtool.DevtoolModifyTests.* fail with the following error:

 ERROR: Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it.
 Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it.

Check if a workspacelayer exists, warn the user and abort the tests

(From OE-Core rev: a74962cfb0485f6f2b9e2b751c33c8eafca8705a)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Michael Opdenacker
7ac5639812 oeqa/runtime/cases/parselogs: remove "edgerouter" case
The "edgerouter" machine has been removed since
https://git.yoctoproject.org/poky/commit/?id=0c64d0e4317e3749f7f7ed9ecd5d08bbb0cedc9e

(From OE-Core rev: b82514c821add181e141d12b0c1723760b445fea)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-26 21:11:28 +01:00
Trevor Gamblin
207a5a21d3 patchtest: simplify test directory structure
Consolidate the various mbox tests into a new TestMbox class, metadata
tests into TestMetadata, and patch tests into TestPatch. Also update the
selftest filenames to match the changes. The test contents are not
significantly changed (other than to reference the new class names).
While this doesn't improve overall readability, it does result in more
obvious categorization, and more importantly reduces the number of calls
to setup tinfoil in the tests, resulting in a roughly 25% reduction in
runtime.

Before:

[tgamblin@megalith poky]$ time ./meta/lib/patchtest/selftest/selftest
XPASS: PatchSignedOffBy.test_signed_off_by_presence (file: PatchSignedOffBy.test_signed_off_by_presence.pass)
XFAIL: Shortlog.test_shortlog_format (file: Shortlog.test_shortlog_format.fail)
XFAIL: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.1.fail)
XPASS: Shortlog.test_shortlog_length (file: Shortlog.test_shortlog_length.pass)
XFAIL: CommitMessage.test_commit_message_presence (file: CommitMessage.test_commit_message_presence.fail)
XFAIL: SrcUri.test_src_uri_left_files (file: SrcUri.test_src_uri_left_files.fail)
XPASS: Author.test_author_valid (file: Author.test_author_valid.1.pass)
XFAIL: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned (file: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.fail)
XPASS: CVE.test_cve_tag_format (file: CVE.test_cve_tag_format.pass)
XPASS: CVE.test_cve_presence_in_commit_message (file: CVE.test_cve_presence_in_commit_message.pass)
XFAIL: CVE.test_cve_tag_format (file: CVE.test_cve_tag_format.fail)
XFAIL: Author.test_author_valid (file: Author.test_author_valid.1.fail)
XFAIL: LicFilesChkSum.test_lic_files_chksum_presence (file: LicFilesChkSum.test_lic_files_chksum_presence.fail)
XSKIP: Merge.test_series_merge_on_head (file: Merge.test_series_merge_on_head.2.skip)
XPASS: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.pass)
XFAIL: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.1.fail)
XPASS: Shortlog.test_shortlog_format (file: Shortlog.test_shortlog_format.pass)
XFAIL: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.2.fail)
XFAIL: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.2.fail)
XFAIL: Summary.test_summary_presence (file: Summary.test_summary_presence.fail)
XPASS: Author.test_author_valid (file: Author.test_author_valid.2.pass)
XSKIP: Merge.test_series_merge_on_head (file: Merge.test_series_merge_on_head.1.skip)
XPASS: Bugzilla.test_bugzilla_entry_format (file: Bugzilla.test_bugzilla_entry_format.pass)
XFAIL: CVE.test_cve_presence_in_commit_message (file: CVE.test_cve_presence_in_commit_message.fail)
XPASS: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.pass)
XPASS: LicFilesChkSum.test_lic_files_chksum_presence (file: LicFilesChkSum.test_lic_files_chksum_presence.pass)
XPASS: CommitMessage.test_commit_message_presence (file: CommitMessage.test_commit_message_presence.pass)
XPASS: Summary.test_summary_presence (file: Summary.test_summary_presence.pass)
XPASS: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned (file: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.pass)
XFAIL: Shortlog.test_shortlog_length (file: Shortlog.test_shortlog_length.fail)
XFAIL: PatchSignedOffBy.test_signed_off_by_presence (file: PatchSignedOffBy.test_signed_off_by_presence.fail)
XFAIL: Bugzilla.test_bugzilla_entry_format (file: Bugzilla.test_bugzilla_entry_format.fail)
XPASS: SrcUri.test_src_uri_left_files (file: SrcUri.test_src_uri_left_files.pass)
XFAIL: Author.test_author_valid (file: Author.test_author_valid.2.fail)
============================================================================
Testsuite summary for patchtest
============================================================================
============================================================================

real    24m14.386s
user    1m13.599s
sys     0m21.477s

After:

[tgamblin@megalith poky]$ time ./meta/lib/patchtest/selftest/selftest
XFAIL: TestMbox.test_bugzilla_entry_format (file: TestMbox.test_bugzilla_entry_format.fail)
XPASS: TestMetadata.test_summary_presence (file: TestMetadata.test_summary_presence.pass)
XFAIL: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.1.fail)
XFAIL: TestMetadata.test_src_uri_left_files (file: TestMetadata.test_src_uri_left_files.fail)
XSKIP: TestMbox.test_series_merge_on_head (file: TestMbox.test_series_merge_on_head.2.skip)
XPASS: TestMbox.test_commit_message_presence (file: TestMbox.test_commit_message_presence.pass)
XFAIL: TestMbox.test_commit_message_presence (file: TestMbox.test_commit_message_presence.fail)
XPASS: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.pass)
XFAIL: TestPatch.test_cve_tag_format (file: TestPatch.test_cve_tag_format.fail)
XFAIL: TestMbox.test_author_valid (file: TestMbox.test_author_valid.1.fail)
XFAIL: TestMbox.test_shortlog_length (file: TestMbox.test_shortlog_length.fail)
XPASS: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.pass)
XFAIL: TestPatch.test_signed_off_by_presence (file: TestPatch.test_signed_off_by_presence.fail)
XFAIL: TestMbox.test_shortlog_format (file: TestMbox.test_shortlog_format.fail)
XFAIL: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.2.fail)
XPASS: TestPatch.test_cve_tag_format (file: TestPatch.test_cve_tag_format.pass)
XSKIP: TestMbox.test_series_merge_on_head (file: TestMbox.test_series_merge_on_head.1.skip)
XPASS: TestMbox.test_author_valid (file: TestMbox.test_author_valid.2.pass)
XPASS: TestMetadata.test_lic_files_chksum_modified_not_mentioned (file: TestMetadata.test_lic_files_chksum_modified_not_mentioned.pass)
XPASS: TestMbox.test_bugzilla_entry_format (file: TestMbox.test_bugzilla_entry_format.pass)
XPASS: TestMetadata.test_src_uri_left_files (file: TestMetadata.test_src_uri_left_files.pass)
XPASS: TestMetadata.test_lic_files_chksum_presence (file: TestMetadata.test_lic_files_chksum_presence.pass)
XPASS: TestMbox.test_cve_presence_in_commit_message (file: TestMbox.test_cve_presence_in_commit_message.pass)
XFAIL: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.2.fail)
XFAIL: TestMbox.test_author_valid (file: TestMbox.test_author_valid.2.fail)
XFAIL: TestMetadata.test_lic_files_chksum_presence (file: TestMetadata.test_lic_files_chksum_presence.fail)
XPASS: TestMbox.test_shortlog_format (file: TestMbox.test_shortlog_format.pass)
XPASS: TestMbox.test_author_valid (file: TestMbox.test_author_valid.1.pass)
XPASS: TestPatch.test_signed_off_by_presence (file: TestPatch.test_signed_off_by_presence.pass)
XFAIL: TestMetadata.test_lic_files_chksum_modified_not_mentioned (file: TestMetadata.test_lic_files_chksum_modified_not_mentioned.fail)
XPASS: TestMbox.test_shortlog_length (file: TestMbox.test_shortlog_length.pass)
XFAIL: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.1.fail)
XFAIL: TestMbox.test_cve_presence_in_commit_message (file: TestMbox.test_cve_presence_in_commit_message.fail)
XFAIL: TestMetadata.test_summary_presence (file: TestMetadata.test_summary_presence.fail)
============================================================================
Testsuite summary for patchtest
============================================================================
============================================================================
real    18m39.749s
user    0m41.857s
sys     0m14.708s

(From OE-Core rev: f788592da2fd0e21638ce2c3326675a060ba51cf)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-26 21:11:28 +01:00
Ross Burton
c1e7eed4f7 cve-check: slightly more verbose warning when adding the same package twice
Occasionally the cve-check tool will warn that it is adding the same
package twice.  Knowing what this package is might be the first step
towards understanding where this message comes from.

(From OE-Core rev: c1179faec8583a8b7df192cf1cbf221f0e3001fc)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-26 15:29:34 +01:00
Trevor Gamblin
71bf3fcf0f patchtest/selftest: add XSKIP, update test files
Since we are skipping the merge test, two of the selftests now report
SKIP instead of XPASS/XFAIL as expected. Adjust the two files to have
the right endings for XSKIP, and add the category so that it can be used
for more extensive testing in the future.

(From OE-Core rev: 3331f53c0be2575784a042bb2401eeba4f2a5a3e)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-25 23:10:19 +01:00
Trevor Gamblin
440c9f9b77 patchtest: disable merge test
Disable the merge-on-head test until patchtest properly handles merging
of series subsets and accounts for patches that are rapidly merged (i.e.
before patchtest is run).

(From OE-Core rev: e561c614dc72b7f8bf5e09a09bbe6ebc3cf500bb)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-24 21:41:58 +01:00
Trevor Gamblin
95a3f69be5 patchtest: fix lic_files_chksum test regex
the test_lic_files_chksum_modified_not_mentioned test in patchtest
wasn't picking up on 'License-Update:' tags correctly. Use pyparsing's
AtLineStart class to simplify the regex setup and search.

(From OE-Core rev: dc9126e45e74b915faaf296037e7ece41785bf4a)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-20 17:02:13 +01:00
Alexander Kanavin
d180e9f1c2 selftest/buildoptions: tag the download mirror test with 'yocto-mirrors'
This will allow bundling all yocto mirror tests together, both for
the purposes of running only them specifically,
and excluding them from 'general' oe-selftest runs.

There is an upcoming test for sstate cache served over content
delivery network which will use the same tag, so it can be run
together with this.

(From OE-Core rev: 2a3f69b83ed10622d6a39774528dbfaebb3e5ded)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-20 13:54:23 +01:00
Trevor Gamblin
8cea0676b7 patchtest: skip merge test if not targeting master
Avoid testing mergeability of a patch when not targeting master, so that
patches tested via other means (e.g. maintainer branches and AB runs)
don't get unnecessarily reviewed an extra time.

(From OE-Core rev: e6cf23e353f48c57249681bd0b12bd8494d4959a)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-19 22:25:23 +01:00
Trevor Gamblin
9b8b9a9fa3 patchtest: test regardless of mergeability
(From OE-Core rev: dc089073eb120de76c8907e476c341ed3e97c164)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-19 22:25:23 +01:00
Ross Burton
4cea51a9ad patchtest: remove unused imports
(From OE-Core rev: be8429d986335aae65c2426862b97836ba46e42a)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-19 14:53:45 +01:00
Sean Nyekjaer
96310a5867 oeqa/sdk/rust: Add build and run test of rust binary with SDK host
Add a QA test to the SDK to test that a basic cargo build works for the
SDK host.

(From OE-Core rev: 7f05760debd3aeb69c3294f3ceb92d4f1aceec1f)

Signed-off-by: Sean Nyekjaer <sean@geanix.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-19 13:38:56 +01:00