The machines option to oe-selftest isn't used in our CI and is never likely to
be, we focus and execute testing explictly. The YOCTO #15247 is about how
this code doesn't interact well with build directory cleanup and at this point
I think we should just remove the option/code.
(From OE-Core rev: 815d04a2007e1154b69f1a027c8677ea86935354)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Specifically, the test checks that everything needed for building
standard oe-core images for x86_64 and arm64 is available from
the cache (with minor exceptions). Going forward, a complete
world check could be enabled and additional configurations,
but that requires improvements to performance of hash equivalence
server in particular.
RP: I've disabled the tests by default so we can merge them. We will
make them live once we get to the bottom of the failures.
(From OE-Core rev: 5f3aeadb65d3b7216db783b2c500ac241b03deb8)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
One of the newer PEP-517 backends to be added was python_hatchling.bbclass
but it was not included in the recent improvements.
Add selftest for 'jsonschema' pypi package.
(From OE-Core rev: d99b4883b4fee82bc588fd235ba90fedf1550cb8)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was writing out locked-sigs.inc into cwd with every
'bitbake -S' invocation. When the intent is only to to get task
stamps (-S none), or print the difference between them (-S printdiff),
the file is unnecessary clutter.
A couple of selftests/scripts were however relying on this, so they're
adjusted to explicitly request the file.
eSDK code calls dump_lockedsigs() separately via
oe.copy_buildsystem.generate_locked_sigs() and so isn't affected.
(From OE-Core rev: ad57c3cac2a8d3e60222e3cca0685f582dcea135)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'bitbake -S printdiff' is a useful diagnostic facility for finding out
why sstate is not being reused, but until now it had no tests that would
ensure it works. This commit adds three basic scenarios:
1. make a change in a really basic, common recipe that is at the very root
of dependency trees (quilt-native), and ensure that change is correctly discovered when
building an image.
2. make a change in gcc-source recipe, which is somewhat special
(operates in work-shared), and ensure that gcc-runtime builds track
that down as well.
3. make a change in base_do_configure() definition from base.bbclass,
which is not recipe-specific, but affects many basic recipes, and ensure that
is correctly reported as well.
The test itself actually runs twice:
- first against a fully populated build directory, where
the printdiff code is guaranteed to find the correct previous
stamp that can be compared with in a predictable manner.
- then in an empty build directory where the printdiff code
goes to look in the sstate cache, and so the existence of the
previous signature can be tested, but not the difference with it
(what the exact difference would be is unpredictable as the
sstate cache is indeed shared between many builds).
(From OE-Core rev: 7a7d76aa8a8d590ebc99156f9f4b9535cdf868c7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add 3 tests to check the creation of PEP-517 project using the 3
backends supported by bitbake:
- setuptools.build_meta
- poetry.core.masonry.api
- flit_core.buildapi
Theses tests requires the tomllib python module, so skip theses tests
if module is not present. tomllib module is part of python starting from 3.11
(From OE-Core rev: 54356c6f1290d0d4170ed52f7bb358bb9efc1aec)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By convention, all python recipes start with "python3-" so update
create_buildsys_python to do this
This rule doesn't apply for packages already starting with "python"
Update recipetool's selftest accordingly
(From OE-Core rev: b0d87440e610b80f763d09784d4a90a148bb3e7b)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
if user run devtool selftests with a local workspacelayer
the tests fail with various error such as:
- devtool.DevtoolAddTests.test_devtool_add just hangs
- devtool.DevtoolModifyTests.* fail with the following error:
ERROR: Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it.
Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it.
Check if a workspacelayer exists, warn the user and abort the tests
(From OE-Core rev: a74962cfb0485f6f2b9e2b751c33c8eafca8705a)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will allow bundling all yocto mirror tests together, both for
the purposes of running only them specifically,
and excluding them from 'general' oe-selftest runs.
There is an upcoming test for sstate cache served over content
delivery network which will use the same tag, so it can be run
together with this.
(From OE-Core rev: 2a3f69b83ed10622d6a39774528dbfaebb3e5ded)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a QA test to the SDK to test that a basic cargo build works for the
SDK host.
(From OE-Core rev: 7f05760debd3aeb69c3294f3ceb92d4f1aceec1f)
Signed-off-by: Sean Nyekjaer <sean@geanix.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop the newlines serial workaround, it doesn't seem to fix things
enough of the time to be useful.
(From OE-Core rev: 6cd57b2d148c5de7839d52d8cec359e4a10e0cd6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately there is still an issue with rustdoc not being reproducible
so part of the problem has been fixed but not everything.
Add the exclusion back until this has been addressed to avoid autobuilder
failures and long diffoscope analysis time.
(From OE-Core rev: ab03b3416dda65e39a756c02d87492e1fe5e0fb0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The _qemutiny is a small test case that was explicitly designed to do a
minimal level of testing for poky-tiny images. These typically don't
have SSH servers so we need to assume that qemu is being used and access
the serial console directly.
(From OE-Core rev: 2245b2754d6f4798127ce85a2ab7cb48f458c1f7)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to skip this test anymore, as skipping on qemuarm64 was
a proxy for "uses SERIAL_CONSOLES_CHECK" which no longer exists.
[ YOCTO #14962 ]
(From OE-Core rev: 1b8b1cb74c8f5f387a14d86f03f37f68c07dd9aa)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whenever we ask gitarchive to retrieve test results for specific revisions,
we first do a "large" search in get_tags, which uses glob patterns with git
ls-remote, and then we filter received tags with a regex to parse the tags
fields.
Currently gitarchive assumes that all tags returned by get_tags will match
the regex. This assumption is wrong (for example searching "master-next" in
get_tags may return some tags like "abelloni/master-next), and leads then
to exception when we try to retrieve tags fields:
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 78, in <module>
sys.exit(main())
^^^^^^
File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 72, in main
ret = args.func(args, logger)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pokybuild/yocto-worker/a-full/build/scripts/lib/resulttool/regression.py", line 315, in regression_git
revs2 = gitarchive.get_test_revs(logger, repo, tag_name, branch=args.branch2)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pokybuild/yocto-worker/a-full/build/meta/lib/oeqa/utils/gitarchive.py", line 246, in get_test_revs
fields, runs = get_test_runs(log, repo, tag_name, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pokybuild/yocto-worker/a-full/build/meta/lib/oeqa/utils/gitarchive.py", line 238, in get_test_runs
groups = m.groupdict()
Fix this exception by merely skipping those additionals tags which won't
match the regex
(From OE-Core rev: 8b5ace47372e958db9e4abb23378947fb02f6fc2)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whenever we ask gitarchive to search for tags, we can provide it with a
pattern (containing glob patterns). However, when searching for example for
tags matching branch master-next, it can find more tags which does not
correspond exactly to branch master-next (e.g. abelloni/master-next tags
will match).
Prevent those additional tags from being fetched by gitarchive by using a
more specific pattern: prefix user-provided pattern with "refs/tags"
(From OE-Core rev: c24b7ea28021da48aa8f3498a9b899f595efde56)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the BrokenPipeError occurs when writing to the serial port to wake it
up, defer the write and try again (which will happen on the 5s timeout
of the select call). Why it should return ESHUTDOWN and then work later
I'm not sure but it does appear to make it work. For now we need 'working'
QA tests whilst the issue is debugged.
(From OE-Core rev: ea9e6ba0ab31a0b20012c283aa768496a50b527a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The '--remap-path-prefix' option removes all references to build directory
structure in the debug information within the compiled output for Cargo
dependencies and the project's binary.
However, some references to build directories remains in the final binary
in .rustc section in the form of compressed metadata and this makes the
build output dependent on the folder structure of the computer it's compiled on.
So, for reproducible builds, use the configuration option
'remap-debuginfo = true' along with the '--remap-path-prefix'.
[YOCTO# 14875]
(From OE-Core rev: 6ae62259afbbe861ed74211dab18a27b8c8d8b7a)
Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the console doesn't appear, breifly sleep and try an extra read. This
is a useful debugging trick which we may as well preserve in the code. We're
already failing at this point so extra data is useful.
This means if we perturb things on the other port, we may gain useful logging
insight.
(From OE-Core rev: be3724854947ded160789d0353b7efb8c593040f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This disables Nagle's algorithm for our tcp serial connections which may
be causing data transfer issues.
(From OE-Core rev: f8eff4c427881a98333fdf7c42f66ed6603e4f03)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're struggling with the 6.5 kernel as the serial port getty doesn't appears sometimes
leading to failures in CI. Add a workaround of sending some newlines as a way of
unblocking the kernel/release issues whilst we try and work out how to get to the bottom
of the issue.
(From OE-Core rev: 0a65f0d272895ba13c8c133ee71f3605d765a8a7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
it implements various combination of locking single/multiple recipe(s)/task(s)
it also tests that locked sig are architecture dependant
(From OE-Core rev: 7047a067548acd592a834d4315579fb606b4b9d7)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case utf-8 conversion of the serial console data is flaky
due to bad characters in the stream.
(From OE-Core rev: 3f4921ced1e2aae546ff7cef232eec3e214c28be)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix warnings from oe-selftest -j:
/usr/lib/python3.10/os.py:1030: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
return io.open(fd, mode, buffering, encoding, *args, **kwargs)
Remove the option since it clearly doesn't do much.
(From OE-Core rev: 6b872ee72942951fd464c4c6cb9eadcb9b4749c1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is unclear when things fail which output was on which serial port. Improve the output
to show the last lines of both data to improve debugging.
(From OE-Core rev: 605938b3bb19dcf6c7218648b4d16df9eaa675fc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To aid debugging, always log the second serial console as well as the first
to a seperate log file. This should make it clearer what happened when we
see test failures.
(From OE-Core rev: 372cd58fe672726900af80f3e65a108984da2750)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switch to the backslashreplace error handling when decoding strings so that
invalid characters are clear in the stream to improve debugging.
(From OE-Core rev: 8ec86d46f736ec3e625b741e97545377c79414ee)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no point in decoding binary data only to encode it again risking
conversion issues. Write the raw data to the log file as binary and skip
the conversion.
Also always update self.msg even if a logfile isn't specified to improve
logging/debug.
(From OE-Core rev: b5d67471dddd198723c4f711747783a33e8e5987)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test results repository contains tags like:
master/64501-g65c94ca3196e5ef3344a469fea8e30444f2e967a/0
master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/3
master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/2
master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/1
master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/0
where the commit count is correct in one case and not in the others. This causes
assertion errors in the current code.
Add in some code to work around these historical issues where the commit counts are low.
(From OE-Core rev: d51fc5c8c469730885af7bbde7122032de411d89)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of constructing huge grep statements, we can simply open the logs
in Python and do the relevant string operations directly.
The trick is to remember to casefold() all of the strings, so that the
"in" operator can be used.
Just one of the ignores needs to be adjusted because it uses a regular
expression and the new logic doesn't support that. This is handled
by simply reducing the size of the ignore match.
(From OE-Core rev: 78ae254c4a78a025a712281ce9de373cdccf5472)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This has no impact to the execution, but makes the following changes
neater.
(From OE-Core rev: 124dede2b20930d09a5cc319cb9333c5a912fa48)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
errors and log_locations can be trivially set in the class directly,
instead of being defined in the module and then copied into the class.
(From OE-Core rev: dd1416b719a30b18f21916d50fa431a88503918f)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no point in passing around member fields, just access them
directly.
(From OE-Core rev: a24d6eda061363cdcfa95980cd2698a674737d23)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
getLogList() uses remote find invocations to find the logs. Instead of
relying on shell expansion of wildcards and redundant use of -maxdepth
(pointless as the shell expansion means the find is passed the files to
return), invoke find idiomatically by telling it what directory to
search for and escape the glob so find processes it.
Also remove many pointless str() calls.
(From OE-Core rev: 03bb14cebf5879472a8da78d892ecd5c5df5c3cf)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2.13 may not be buildable with latest compilers without patching
(From OE-Core rev: 406a33f896accc35a9cb6ab156f1e0f42dda67d8)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This took a bit of digging but failure messages from testimage are shown to bitbake's
logging through stopTest. In the case of a setUpClass failure stopTest is never
called and the bitbake logging never sees the error. It would still be in the task
logfile. Add some code+comment to ensure logs not shown to the user mid stream are shown
at the end.
(From OE-Core rev: 7342c418bda2cc3c337a190089864ea731ff62e4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using get_bb_var in the class setup leads to slow startup of oe-selftest.
Move the calls into setupClass instead to remove the overhead at the expense
of some code duplication.
(From OE-Core rev: 64786106746fbcc9d8a35eb6cfa82fd5c6bce7f8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"bitbake -e" executions from get_bb_var calls are slow and slow down oe-selftest
startup. Rationalise the code to avoid them and minimise the number of "parsing"
locations we use by caching key variables and passing them around more.
This was particularly problematic with oe-selftest -j usage since it would
have multiple bitbake -e executions per process making parallel usage
particularly slow.
(From OE-Core rev: 3689cadeb07d76e66f97d890e844f899f69666fe)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the bb_get_var calls trigger multiple "bitbake -e" executions
which slow the start of oe-selftest for any test. Rework the code to avoid
these delays.
(From OE-Core rev: c9ea3570f5dbdaf8c61e445ae9de759a0b71e21f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test above this removal correctly looks at symlinks however to
remove a symlink we should call unlink(), not remove(). This avoids
some build failures/tracebacks.
(From OE-Core rev: dbdb6e73b0f52bc5f9429aca47802d51edbbc834)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to have one-liner functions to get the MACHINE or
WORKDIR when they're only called once.
(From OE-Core rev: 9478a665641f55cdc14f12a4409121ef95883d74)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The LSB compliance tests were removed in 2019[1], so this is obsolete.
[1] oe-core fb064356
(From OE-Core rev: a465b61bc3a16152f90032d5d5a041f87bf30108)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This information is pretty useless as we know what the target is from
the build configuration (be it a qemu machine or real hardware).
(From OE-Core rev: 7821c42172191e3c16ab8662a9e4f7e921830db0)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test no longer worked with fixes to avoid overlapping license files
in deploy. Fix the test but also improve the logging messages so we don't
get "False is not True" messages upon failure.
(From OE-Core rev: 2fc7714dab5cce585bce4d6c4e8ffbefce2a709b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
assertTrue is a problematic call use in test cases since when it fails,
you just get an unhelpful "False is not True" message.
Replace some uses with assertIn/assertNotIn which will give more helpful results
and for the rest, add msg entries which given more helpful debugging.
For example, this patch would help debugging of #15176.
(From OE-Core rev: 35d4c39e0df1a304f557471151a03d1e4b0f30c7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix up test case to match what binary prints.
(From OE-Core rev: 4683804b9669c71d31ea6a8a300e6e87e817ee12)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
