This was added (I think) for the purpose of supporting layers that refer
to items outside of the layer via relative symlinks:
https://git.yoctoproject.org/poky-contrib/commit/?id=d31d1ad4e566e42d0bbcf1f41ac25e33181fb517
I do not think copying the link target into the layer that references it is the
correct solution: rather the original target should be included
into the SDK with the same relative path.
This change is done for the sake of preserving symlinks
that are referencing things inside the layer as they are;
particularly the content of scripts/esdk-tools/.
(From OE-Core rev: 52a7bbd5c4875c5f61ea65dda38e495a2925a20d)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't make do_cve_check depend on do_unpack because that would be a
waste of time 99% of the time. The compromise here is that we can't
scan remote patches for issues, but this isn't a problem so downgrade
the warning to a note.
Also move the check for CVEs in the filename before the local file check
so that even with remote patches, we still check for CVE references in
the name.
(From OE-Core rev: 0251cad677579f5b4dcc25fa2f8552c6040ac2cf)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After patchtest went live it was determined that testing for a CVE tag
in the mbox commit message is unnecessary, since it will already be in
the shortlog and in any carried patches. Remove the test and the
associated selftest files so that its absence isn't flagged in future
test results.
(From OE-Core rev: 54690f18f04a2ab993a85d551ce4f8d0fa56618a)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
pylint 3.x has removed epylint, which is now a separate module. To avoid
adding another recipe or using outdated modules, modify the
test_python_pylint tests so that they use the standard pylint API.
(From OE-Core rev: 72be3d6a116febf46130cccbe12afe5ad93779b5)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Specifically, the test checks that everything needed for building
standard oe-core images for x86_64 and arm64 is available from
the cache (with minor exceptions). Going forward, a complete
world check could be enabled and additional configurations,
but that requires improvements to performance of hash equivalence
server in particular.
RP: I've disabled the tests by default so we can merge them. We will
make them live once we get to the bottom of the failures.
(From OE-Core rev: 5f3aeadb65d3b7216db783b2c500ac241b03deb8)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
One of the newer PEP-517 backends to be added was python_hatchling.bbclass
but it was not included in the recent improvements.
Add selftest for 'jsonschema' pypi package.
(From OE-Core rev: d99b4883b4fee82bc588fd235ba90fedf1550cb8)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some test result lines in TestMbox and TestPatch are still too long to
avoid being flagged by the mailer script. Clean them up by removing
redundant information, so that they are all under the length limit of
220 characters.
(From OE-Core rev: c10d0bb542b23fbdc14d76dfa8e5885aa4d33083)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test_lic_files_chksum_modified_not_mentioned test in TestMetadata is
outputting very long lines that fail the maximum length check when
sending email results, preventing the actual errors from being
displayed. Reduce the length of the failure message by rewording and
removing redundant information.
(From OE-Core rev: 2e2625735181160e9760a6f3af4955bda2ea6d4d)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was writing out locked-sigs.inc into cwd with every
'bitbake -S' invocation. When the intent is only to to get task
stamps (-S none), or print the difference between them (-S printdiff),
the file is unnecessary clutter.
A couple of selftests/scripts were however relying on this, so they're
adjusted to explicitly request the file.
eSDK code calls dump_lockedsigs() separately via
oe.copy_buildsystem.generate_locked_sigs() and so isn't affected.
(From OE-Core rev: ad57c3cac2a8d3e60222e3cca0685f582dcea135)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'bitbake -S printdiff' is a useful diagnostic facility for finding out
why sstate is not being reused, but until now it had no tests that would
ensure it works. This commit adds three basic scenarios:
1. make a change in a really basic, common recipe that is at the very root
of dependency trees (quilt-native), and ensure that change is correctly discovered when
building an image.
2. make a change in gcc-source recipe, which is somewhat special
(operates in work-shared), and ensure that gcc-runtime builds track
that down as well.
3. make a change in base_do_configure() definition from base.bbclass,
which is not recipe-specific, but affects many basic recipes, and ensure that
is correctly reported as well.
The test itself actually runs twice:
- first against a fully populated build directory, where
the printdiff code is guaranteed to find the correct previous
stamp that can be compared with in a predictable manner.
- then in an empty build directory where the printdiff code
goes to look in the sstate cache, and so the existence of the
previous signature can be tested, but not the difference with it
(what the exact difference would be is unpredictable as the
sstate cache is indeed shared between many builds).
(From OE-Core rev: 7a7d76aa8a8d590ebc99156f9f4b9535cdf868c7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add 3 tests to check the creation of PEP-517 project using the 3
backends supported by bitbake:
- setuptools.build_meta
- poetry.core.masonry.api
- flit_core.buildapi
Theses tests requires the tomllib python module, so skip theses tests
if module is not present. tomllib module is part of python starting from 3.11
(From OE-Core rev: 54356c6f1290d0d4170ed52f7bb358bb9efc1aec)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By convention, all python recipes start with "python3-" so update
create_buildsys_python to do this
This rule doesn't apply for packages already starting with "python"
Update recipetool's selftest accordingly
(From OE-Core rev: b0d87440e610b80f763d09784d4a90a148bb3e7b)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
if user run devtool selftests with a local workspacelayer
the tests fail with various error such as:
- devtool.DevtoolAddTests.test_devtool_add just hangs
- devtool.DevtoolModifyTests.* fail with the following error:
ERROR: Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it.
Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it.
Check if a workspacelayer exists, warn the user and abort the tests
(From OE-Core rev: a74962cfb0485f6f2b9e2b751c33c8eafca8705a)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Occasionally the cve-check tool will warn that it is adding the same
package twice. Knowing what this package is might be the first step
towards understanding where this message comes from.
(From OE-Core rev: c1179faec8583a8b7df192cf1cbf221f0e3001fc)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we are skipping the merge test, two of the selftests now report
SKIP instead of XPASS/XFAIL as expected. Adjust the two files to have
the right endings for XSKIP, and add the category so that it can be used
for more extensive testing in the future.
(From OE-Core rev: 3331f53c0be2575784a042bb2401eeba4f2a5a3e)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Disable the merge-on-head test until patchtest properly handles merging
of series subsets and accounts for patches that are rapidly merged (i.e.
before patchtest is run).
(From OE-Core rev: e561c614dc72b7f8bf5e09a09bbe6ebc3cf500bb)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
the test_lic_files_chksum_modified_not_mentioned test in patchtest
wasn't picking up on 'License-Update:' tags correctly. Use pyparsing's
AtLineStart class to simplify the regex setup and search.
(From OE-Core rev: dc9126e45e74b915faaf296037e7ece41785bf4a)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will allow bundling all yocto mirror tests together, both for
the purposes of running only them specifically,
and excluding them from 'general' oe-selftest runs.
There is an upcoming test for sstate cache served over content
delivery network which will use the same tag, so it can be run
together with this.
(From OE-Core rev: 2a3f69b83ed10622d6a39774528dbfaebb3e5ded)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Avoid testing mergeability of a patch when not targeting master, so that
patches tested via other means (e.g. maintainer branches and AB runs)
don't get unnecessarily reviewed an extra time.
(From OE-Core rev: e6cf23e353f48c57249681bd0b12bd8494d4959a)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a QA test to the SDK to test that a basic cargo build works for the
SDK host.
(From OE-Core rev: 7f05760debd3aeb69c3294f3ceb92d4f1aceec1f)
Signed-off-by: Sean Nyekjaer <sean@geanix.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Also remove the warning than doesn't make sense as the code will generate
an exception and bitbake will abort.
Before:
| WARNING: core-image-minimal-initramfs-1.0-r0 do_image_complete: KeyError in .
| Exception: Exception: KeyError: 'getpwuid(): uid not found: xxxxx'
| Path . is owned by uid xxxxx, gid yyy, which doesn't match any user/group on target. This may be due to host contamination.
After:
| Exception: Exception: KeyError: 'getpwuid(): uid not found: xxxxx'
| Path /build/tmp/work/intel_corei7_64-lmp-linux/core-image-minimal-initramfs/1.0-r0/sstate-build-image_complete is owned by uid xxxxx, gid yyy, which doesn't match any user/group on target. This may be due to host contamination.
(From OE-Core rev: 90f84e345950a02bf91f823cc9c6d893e7cd1100)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Other spaces uses the Go architecture definitions as their own (for
example, container arches are defined to be Go arches). To make it
easier for other places to use this mapping, move the code that does the
translation of OpenEmbedded arches to Go arches to a library.
(From OE-Core rev: 3e86f72fc2e1cc2e5ea4b4499722d736941167ce)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changes to patchtest's command-line arguments to work with oe-core by
default do not match the selftest script's argument list. Explicitly use
the --testdir and --repodir flags in selftest so that it is compatible
them.
(From OE-Core rev: 6cd547b24896596d4e0fe57f26f553842c5560b5)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace full license headers with SPDX identifiers and adjust all
patchtest-related code to use GPL-2.0-only.
(From OE-Core rev: 9bea6b39074296bb8d8719a3300636e316f19d1b)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since patchtest is in oe-core, the Python os module's methods can be
used to retrieve the repository path and tests directory by default.
This reduces the number of mandatory arguments for invocation of
patchtest unless the user wants to use a custom test suite or test
patches against a different repo. The REPO and TESTDIR arguments are
likewise adjusted so that they are optional. Also, make it more obvious
what the --startdir flag is meant for on the command line by renaming it
to --testdir, and update the scripts/patchtest.README file to be
consistent with the new usage.
(From OE-Core rev: bae7421ece4806f5148f164293810b9fe75e0756)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add modules that support core patchtest functionality to
meta/lib/patchtest. These include classes and functions for handling
repository and patch objects, parsing the patchtest CLI arguments, and
other utilities.
(From OE-Core rev: 499cdad7a16f6cc256837069c7add294132127a4)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add git-pw and boto3 to the list of requirements for patchtest.
(From OE-Core rev: f88e295cb5034950e9a0899c1dc3ca685a30a176)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop the newlines serial workaround, it doesn't seem to fix things
enough of the time to be useful.
(From OE-Core rev: 6cd57b2d148c5de7839d52d8cec359e4a10e0cd6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately there is still an issue with rustdoc not being reproducible
so part of the problem has been fixed but not everything.
Add the exclusion back until this has been addressed to avoid autobuilder
failures and long diffoscope analysis time.
(From OE-Core rev: ab03b3416dda65e39a756c02d87492e1fe5e0fb0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Various tweaks to make the test suite cleaner and more efficient:
- Replace use of "re" module with "pyparsing" in tests (but not base.py)
- Make test_mbox_cve only check for CVE tags in the commit if the added
patch has them
- Make test_mbox_cve SKIP instead of PASS if there's no CVE tag
- Simplify the bugzilla tag checking test now that pyparsing is used
- Modify the selftest script to correctly parse the new result output
(From OE-Core rev: 7a187c2475aa762e2bc830950f608143f2535a72)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The patchtest tests provide vague feedback to the user, and many of them
also provide redundant 'fix' strings that could easily be incorporated
into the issue messages themselves. Simplify them so that it is more
clear what the errors are and how they can be addressed. No
recommendation is given when the issue string adequately conveys the
issue, e.g. with a missing "LICENSE" entry in a newly-created recipe.
(From OE-Core rev: 0bfb3614244ec7aa79b6424bc63f9f2bccdabe98)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The _qemutiny is a small test case that was explicitly designed to do a
minimal level of testing for poky-tiny images. These typically don't
have SSH servers so we need to assume that qemu is being used and access
the serial console directly.
(From OE-Core rev: 2245b2754d6f4798127ce85a2ab7cb48f458c1f7)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to skip this test anymore, as skipping on qemuarm64 was
a proxy for "uses SERIAL_CONSOLES_CHECK" which no longer exists.
[ YOCTO #14962 ]
(From OE-Core rev: 1b8b1cb74c8f5f387a14d86f03f37f68c07dd9aa)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whenever we ask gitarchive to retrieve test results for specific revisions,
we first do a "large" search in get_tags, which uses glob patterns with git
ls-remote, and then we filter received tags with a regex to parse the tags
fields.
Currently gitarchive assumes that all tags returned by get_tags will match
the regex. This assumption is wrong (for example searching "master-next" in
get_tags may return some tags like "abelloni/master-next), and leads then
to exception when we try to retrieve tags fields:
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 78, in <module>
sys.exit(main())
^^^^^^
File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 72, in main
ret = args.func(args, logger)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pokybuild/yocto-worker/a-full/build/scripts/lib/resulttool/regression.py", line 315, in regression_git
revs2 = gitarchive.get_test_revs(logger, repo, tag_name, branch=args.branch2)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pokybuild/yocto-worker/a-full/build/meta/lib/oeqa/utils/gitarchive.py", line 246, in get_test_revs
fields, runs = get_test_runs(log, repo, tag_name, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pokybuild/yocto-worker/a-full/build/meta/lib/oeqa/utils/gitarchive.py", line 238, in get_test_runs
groups = m.groupdict()
Fix this exception by merely skipping those additionals tags which won't
match the regex
(From OE-Core rev: 8b5ace47372e958db9e4abb23378947fb02f6fc2)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whenever we ask gitarchive to search for tags, we can provide it with a
pattern (containing glob patterns). However, when searching for example for
tags matching branch master-next, it can find more tags which does not
correspond exactly to branch master-next (e.g. abelloni/master-next tags
will match).
Prevent those additional tags from being fetched by gitarchive by using a
more specific pattern: prefix user-provided pattern with "refs/tags"
(From OE-Core rev: c24b7ea28021da48aa8f3498a9b899f595efde56)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the BrokenPipeError occurs when writing to the serial port to wake it
up, defer the write and try again (which will happen on the 5s timeout
of the select call). Why it should return ESHUTDOWN and then work later
I'm not sure but it does appear to make it work. For now we need 'working'
QA tests whilst the issue is debugged.
(From OE-Core rev: ea9e6ba0ab31a0b20012c283aa768496a50b527a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The '--remap-path-prefix' option removes all references to build directory
structure in the debug information within the compiled output for Cargo
dependencies and the project's binary.
However, some references to build directories remains in the final binary
in .rustc section in the form of compressed metadata and this makes the
build output dependent on the folder structure of the computer it's compiled on.
So, for reproducible builds, use the configuration option
'remap-debuginfo = true' along with the '--remap-path-prefix'.
[YOCTO# 14875]
(From OE-Core rev: 6ae62259afbbe861ed74211dab18a27b8c8d8b7a)
Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the console doesn't appear, breifly sleep and try an extra read. This
is a useful debugging trick which we may as well preserve in the code. We're
already failing at this point so extra data is useful.
This means if we perturb things on the other port, we may gain useful logging
insight.
(From OE-Core rev: be3724854947ded160789d0353b7efb8c593040f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This disables Nagle's algorithm for our tcp serial connections which may
be causing data transfer issues.
(From OE-Core rev: f8eff4c427881a98333fdf7c42f66ed6603e4f03)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're struggling with the 6.5 kernel as the serial port getty doesn't appears sometimes
leading to failures in CI. Add a workaround of sending some newlines as a way of
unblocking the kernel/release issues whilst we try and work out how to get to the bottom
of the issue.
(From OE-Core rev: 0a65f0d272895ba13c8c133ee71f3605d765a8a7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
it implements various combination of locking single/multiple recipe(s)/task(s)
it also tests that locked sig are architecture dependant
(From OE-Core rev: 7047a067548acd592a834d4315579fb606b4b9d7)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
as of now, SIGGEN_LOCKEDSIGS_TASKSIG_CHECK can take 2 values: "warn" and
"error", displaying respectively a warning or a fatal error message
only when a task is locked and the task signature is different from
the locked one.
The "info" level is introduced to add a "note" message to remind the
user that a recipe is locked even if the signature is equivalent to the
locked one.
The "warn" and "error" level display the warn/error message for each
task having a mismatch of the signature. Doing this with the "info"
level would result in very verbose output if there are several tasks
locked, so the info level will only print once the list of recipes that
have locked signature.
(From OE-Core rev: 840402181d36ca3f60119984478979afb5bb3bbf)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case utf-8 conversion of the serial console data is flaky
due to bad characters in the stream.
(From OE-Core rev: 3f4921ced1e2aae546ff7cef232eec3e214c28be)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix warnings from oe-selftest -j:
/usr/lib/python3.10/os.py:1030: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
return io.open(fd, mode, buffering, encoding, *args, **kwargs)
Remove the option since it clearly doesn't do much.
(From OE-Core rev: 6b872ee72942951fd464c4c6cb9eadcb9b4749c1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is unclear when things fail which output was on which serial port. Improve the output
to show the last lines of both data to improve debugging.
(From OE-Core rev: 605938b3bb19dcf6c7218648b4d16df9eaa675fc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
