The patchtest tests provide vague feedback to the user, and many of them
also provide redundant 'fix' strings that could easily be incorporated
into the issue messages themselves. Simplify them so that it is more
clear what the errors are and how they can be addressed. No
recommendation is given when the issue string adequately conveys the
issue, e.g. with a missing "LICENSE" entry in a newly-created recipe.
(From OE-Core rev: 0bfb3614244ec7aa79b6424bc63f9f2bccdabe98)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The _qemutiny is a small test case that was explicitly designed to do a
minimal level of testing for poky-tiny images. These typically don't
have SSH servers so we need to assume that qemu is being used and access
the serial console directly.
(From OE-Core rev: 2245b2754d6f4798127ce85a2ab7cb48f458c1f7)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to skip this test anymore, as skipping on qemuarm64 was
a proxy for "uses SERIAL_CONSOLES_CHECK" which no longer exists.
[ YOCTO #14962 ]
(From OE-Core rev: 1b8b1cb74c8f5f387a14d86f03f37f68c07dd9aa)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whenever we ask gitarchive to retrieve test results for specific revisions,
we first do a "large" search in get_tags, which uses glob patterns with git
ls-remote, and then we filter received tags with a regex to parse the tags
fields.
Currently gitarchive assumes that all tags returned by get_tags will match
the regex. This assumption is wrong (for example searching "master-next" in
get_tags may return some tags like "abelloni/master-next), and leads then
to exception when we try to retrieve tags fields:
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 78, in <module>
sys.exit(main())
^^^^^^
File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 72, in main
ret = args.func(args, logger)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pokybuild/yocto-worker/a-full/build/scripts/lib/resulttool/regression.py", line 315, in regression_git
revs2 = gitarchive.get_test_revs(logger, repo, tag_name, branch=args.branch2)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pokybuild/yocto-worker/a-full/build/meta/lib/oeqa/utils/gitarchive.py", line 246, in get_test_revs
fields, runs = get_test_runs(log, repo, tag_name, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pokybuild/yocto-worker/a-full/build/meta/lib/oeqa/utils/gitarchive.py", line 238, in get_test_runs
groups = m.groupdict()
Fix this exception by merely skipping those additionals tags which won't
match the regex
(From OE-Core rev: 8b5ace47372e958db9e4abb23378947fb02f6fc2)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whenever we ask gitarchive to search for tags, we can provide it with a
pattern (containing glob patterns). However, when searching for example for
tags matching branch master-next, it can find more tags which does not
correspond exactly to branch master-next (e.g. abelloni/master-next tags
will match).
Prevent those additional tags from being fetched by gitarchive by using a
more specific pattern: prefix user-provided pattern with "refs/tags"
(From OE-Core rev: c24b7ea28021da48aa8f3498a9b899f595efde56)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the BrokenPipeError occurs when writing to the serial port to wake it
up, defer the write and try again (which will happen on the 5s timeout
of the select call). Why it should return ESHUTDOWN and then work later
I'm not sure but it does appear to make it work. For now we need 'working'
QA tests whilst the issue is debugged.
(From OE-Core rev: ea9e6ba0ab31a0b20012c283aa768496a50b527a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The '--remap-path-prefix' option removes all references to build directory
structure in the debug information within the compiled output for Cargo
dependencies and the project's binary.
However, some references to build directories remains in the final binary
in .rustc section in the form of compressed metadata and this makes the
build output dependent on the folder structure of the computer it's compiled on.
So, for reproducible builds, use the configuration option
'remap-debuginfo = true' along with the '--remap-path-prefix'.
[YOCTO# 14875]
(From OE-Core rev: 6ae62259afbbe861ed74211dab18a27b8c8d8b7a)
Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the console doesn't appear, breifly sleep and try an extra read. This
is a useful debugging trick which we may as well preserve in the code. We're
already failing at this point so extra data is useful.
This means if we perturb things on the other port, we may gain useful logging
insight.
(From OE-Core rev: be3724854947ded160789d0353b7efb8c593040f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This disables Nagle's algorithm for our tcp serial connections which may
be causing data transfer issues.
(From OE-Core rev: f8eff4c427881a98333fdf7c42f66ed6603e4f03)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're struggling with the 6.5 kernel as the serial port getty doesn't appears sometimes
leading to failures in CI. Add a workaround of sending some newlines as a way of
unblocking the kernel/release issues whilst we try and work out how to get to the bottom
of the issue.
(From OE-Core rev: 0a65f0d272895ba13c8c133ee71f3605d765a8a7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
it implements various combination of locking single/multiple recipe(s)/task(s)
it also tests that locked sig are architecture dependant
(From OE-Core rev: 7047a067548acd592a834d4315579fb606b4b9d7)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
as of now, SIGGEN_LOCKEDSIGS_TASKSIG_CHECK can take 2 values: "warn" and
"error", displaying respectively a warning or a fatal error message
only when a task is locked and the task signature is different from
the locked one.
The "info" level is introduced to add a "note" message to remind the
user that a recipe is locked even if the signature is equivalent to the
locked one.
The "warn" and "error" level display the warn/error message for each
task having a mismatch of the signature. Doing this with the "info"
level would result in very verbose output if there are several tasks
locked, so the info level will only print once the list of recipes that
have locked signature.
(From OE-Core rev: 840402181d36ca3f60119984478979afb5bb3bbf)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case utf-8 conversion of the serial console data is flaky
due to bad characters in the stream.
(From OE-Core rev: 3f4921ced1e2aae546ff7cef232eec3e214c28be)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix warnings from oe-selftest -j:
/usr/lib/python3.10/os.py:1030: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
return io.open(fd, mode, buffering, encoding, *args, **kwargs)
Remove the option since it clearly doesn't do much.
(From OE-Core rev: 6b872ee72942951fd464c4c6cb9eadcb9b4749c1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is unclear when things fail which output was on which serial port. Improve the output
to show the last lines of both data to improve debugging.
(From OE-Core rev: 605938b3bb19dcf6c7218648b4d16df9eaa675fc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To aid debugging, always log the second serial console as well as the first
to a seperate log file. This should make it clearer what happened when we
see test failures.
(From OE-Core rev: 372cd58fe672726900af80f3e65a108984da2750)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switch to the backslashreplace error handling when decoding strings so that
invalid characters are clear in the stream to improve debugging.
(From OE-Core rev: 8ec86d46f736ec3e625b741e97545377c79414ee)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no point in decoding binary data only to encode it again risking
conversion issues. Write the raw data to the log file as binary and skip
the conversion.
Also always update self.msg even if a logfile isn't specified to improve
logging/debug.
(From OE-Core rev: b5d67471dddd198723c4f711747783a33e8e5987)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test results repository contains tags like:
master/64501-g65c94ca3196e5ef3344a469fea8e30444f2e967a/0
master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/3
master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/2
master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/1
master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/0
where the commit count is correct in one case and not in the others. This causes
assertion errors in the current code.
Add in some code to work around these historical issues where the commit counts are low.
(From OE-Core rev: d51fc5c8c469730885af7bbde7122032de411d89)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of constructing huge grep statements, we can simply open the logs
in Python and do the relevant string operations directly.
The trick is to remember to casefold() all of the strings, so that the
"in" operator can be used.
Just one of the ignores needs to be adjusted because it uses a regular
expression and the new logic doesn't support that. This is handled
by simply reducing the size of the ignore match.
(From OE-Core rev: 78ae254c4a78a025a712281ce9de373cdccf5472)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This has no impact to the execution, but makes the following changes
neater.
(From OE-Core rev: 124dede2b20930d09a5cc319cb9333c5a912fa48)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
errors and log_locations can be trivially set in the class directly,
instead of being defined in the module and then copied into the class.
(From OE-Core rev: dd1416b719a30b18f21916d50fa431a88503918f)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no point in passing around member fields, just access them
directly.
(From OE-Core rev: a24d6eda061363cdcfa95980cd2698a674737d23)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
getLogList() uses remote find invocations to find the logs. Instead of
relying on shell expansion of wildcards and redundant use of -maxdepth
(pointless as the shell expansion means the find is passed the files to
return), invoke find idiomatically by telling it what directory to
search for and escape the glob so find processes it.
Also remove many pointless str() calls.
(From OE-Core rev: 03bb14cebf5879472a8da78d892ecd5c5df5c3cf)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2.13 may not be buildable with latest compilers without patching
(From OE-Core rev: 406a33f896accc35a9cb6ab156f1e0f42dda67d8)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This took a bit of digging but failure messages from testimage are shown to bitbake's
logging through stopTest. In the case of a setUpClass failure stopTest is never
called and the bitbake logging never sees the error. It would still be in the task
logfile. Add some code+comment to ensure logs not shown to the user mid stream are shown
at the end.
(From OE-Core rev: 7342c418bda2cc3c337a190089864ea731ff62e4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Variable dependencies for functions in the oe.packagedata module were
missing as it was not present in BBIMPORTS. Add it as the fact it was
missing is likely just historical oversight from base.bbclass history
and the dependencies are useful.
Add an exclusion to bitbake.conf to ensure BB_NUMBER_THREADS doesn't
change task checksums.
(From OE-Core rev: 7df9178766ee7939d139648e04f5747b0ffe1e13)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using get_bb_var in the class setup leads to slow startup of oe-selftest.
Move the calls into setupClass instead to remove the overhead at the expense
of some code duplication.
(From OE-Core rev: 64786106746fbcc9d8a35eb6cfa82fd5c6bce7f8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"bitbake -e" executions from get_bb_var calls are slow and slow down oe-selftest
startup. Rationalise the code to avoid them and minimise the number of "parsing"
locations we use by caching key variables and passing them around more.
This was particularly problematic with oe-selftest -j usage since it would
have multiple bitbake -e executions per process making parallel usage
particularly slow.
(From OE-Core rev: 3689cadeb07d76e66f97d890e844f899f69666fe)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the bb_get_var calls trigger multiple "bitbake -e" executions
which slow the start of oe-selftest for any test. Rework the code to avoid
these delays.
(From OE-Core rev: c9ea3570f5dbdaf8c61e445ae9de759a0b71e21f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test above this removal correctly looks at symlinks however to
remove a symlink we should call unlink(), not remove(). This avoids
some build failures/tracebacks.
(From OE-Core rev: dbdb6e73b0f52bc5f9429aca47802d51edbbc834)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
selftest assumes that the test result output lines will consist of two
tokens separated by whitespace, which are the actual result and the name
of the test run. As a result, the script fails if any verbosity is added
to the output of patchtest itself (e.g. by including a failure reason).
Make the call to split() only do so once in order to ensure that two
tokens are obtained.
(From OE-Core rev: b7ff1e7363261780c015a3ee3b6d4bc69567f67f)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to have one-liner functions to get the MACHINE or
WORKDIR when they're only called once.
(From OE-Core rev: 9478a665641f55cdc14f12a4409121ef95883d74)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The LSB compliance tests were removed in 2019[1], so this is obsolete.
[1] oe-core fb064356
(From OE-Core rev: a465b61bc3a16152f90032d5d5a041f87bf30108)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This information is pretty useless as we know what the target is from
the build configuration (be it a qemu machine or real hardware).
(From OE-Core rev: 7821c42172191e3c16ab8662a9e4f7e921830db0)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the by-id and by-namespace SPDX files are created without reference
to PACKAGE_ARCH. This means that for two machines using a common package architecture
(e.g. genericx86-64 and qqemux86-64), there would be overlapping files. This means
that the build of one can remove files from the other leading to build failures. An
example would be:
MACHINE=qemux86-64 bitbake core-image-minimal
MACHINE=genericx86-64 bitbake core-image-minimal
MACHINE=qemux86-64 bitbake linux-yocto -c clean
MACHINE=genericx86-64 bitbake core-image-minimal -C rootfs
To fix this, add PACKAGE_ARCH to the path used for the files and use a search
path based upon PACKAGE_ARCHS to access them.
(From OE-Core rev: b2db10e966438071d00d2057b84d5f347613d841)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test no longer worked with fixes to avoid overlapping license files
in deploy. Fix the test but also improve the logging messages so we don't
get "False is not True" messages upon failure.
(From OE-Core rev: 2fc7714dab5cce585bce4d6c4e8ffbefce2a709b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
assertTrue is a problematic call use in test cases since when it fails,
you just get an unhelpful "False is not True" message.
Replace some uses with assertIn/assertNotIn which will give more helpful results
and for the rest, add msg entries which given more helpful debugging.
For example, this patch would help debugging of #15176.
(From OE-Core rev: 35d4c39e0df1a304f557471151a03d1e4b0f30c7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix up test case to match what binary prints.
(From OE-Core rev: 4683804b9669c71d31ea6a8a300e6e87e817ee12)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I'm not sure if this was reported correctly before, but it
currently is not. Test that is stuck is an error in itself.
(From OE-Core rev: 002e27c9932a83e46be0b03a5232594cfba7212c)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #15023]
This patch adds (and removes after function execution) the
nativesdk-intercept/chown|chgrp PATH before target_pm.run_intercepts
calls during populate_sdk builds.
This has been tested with cleanall builds and testsdk and fails on deb
due to an issue where $D${localstatedir}/cache/man/ does not exist for
some reason. I've a work around for that in the next patch in this
series.
(From OE-Core rev: e7afdfe9da150209ab2676d09eae040de2155c6d)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't ignore return values from the git command lines. If something goes
wrong, fail the test right away.
(From OE-Core rev: dfc178a70d6fa60e89d4716f05d68e2c72c6ecd3)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a short README describing how to setup patchtest's selftests for
oe-core.
(From OE-Core rev: afd4f3d9fa22dd0fbb0c30bbfc4a3de37d695c76)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a requirements.txt file with the Python modules needed for Patchtest
to run.
(From OE-Core rev: 2fd953ac5dd86d3045a6af240d61c6e36c62d133)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ability to pass the target (i.e. oe-core) as an argument was a
testing mechanism and isn't needed when the tests are part of the repo,
so remove it and use os.path.dirname to get it instead.
(From OE-Core rev: 87c54eae350e358f32b12ae807719fa845fd54af)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Copy the core components of the patchtest-oe repo into
meta/lib/patchtest in oe-core.
(From OE-Core rev: 257f64f4e4414b78981104aec132b067beb5a92a)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
