The dbus-wait command returns a timeout after 60s but reports "success", detect this.
Unfortunately it does effectively break the test as the signal is nearly never being
correctly detected since it was already sent.
For that reason comment out the code instead too.
Also fix the loop conditional as the logic was incorrect and it was looping
indefinitely when an image match didn't occur.
(From OE-Core rev: 89c930e9e4b38b116edcba59e88621a39f8bda67)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch uses dbus-wait to wait for matchbox to be up. Once that
happens, it sets a timeout of 60 seconds and takes a screenshot
and compares it, every 2 seconds. If diff=0 it passes. If the timeout
ends, it fails.
(From OE-Core rev: 287b4f0a8244f7214f6a1aaa84ef16cc528f8326)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Excluding riscv64 due to mouse rather than a touchscreen which adds a
moving cursor, so the diff ends up > 0. Need to fix the image to use the
touchscreen rather than mouse input.
(From OE-Core rev: 7f7032c7613abd62ce510c98211c75fc7c5e7090)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The mouse panel icon can move when the time changes between 3 and 4 digits.
Ideally we'd fix the alignment of the clock on the panel but to get the
tests working, increase the size of the mask for now.
(From OE-Core rev: 8ec02142ab98741749281bdb29d5973c96b839d7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Allow tools to be found from the host PATH so that imagemagick from a buildtools
tarball/sdk can work
* Reformat the code to have imports at the start of the file and have more standard
formatting and whitespace
* Always save copies of the images, the space imapct is negligle compared to the
debug win
* Write the images to ${T}
* Use bb.utils.mkdirhier() instead of more complex code
* Restrict the tests to images containing matchbox-desktop
(From OE-Core rev: d09989b49517830297654e4d1d150aaa8723c41a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This takes the work rburton did on image screenshot testing and
expands it.
Right now this works with most of the qemu based machines except for
- qemuppc64
- qemuarmv5
- qemuriscv32
- qemuloongarch64
See "Known Issues" further down.
This test takes a screendump of a qemu image, blanks out the clock
and compares it to an image we have on record. If the diff is exact,
the test passes. If not, it stores the image in build/failed-images and
fails out.
In order to enable this test, you will need meta-openembedded/meta-oe in
your bblayers.conf for imagemagick and the following in local.conf:
IMAGE_CLASSES += "testimage"
TEST_SUITES = "login"
TESTIMAGEDEPENDS:append:qemuall = " imagemagick-native:do_populate_sysroot "
Known Issues
------------
The main issue is that I've yet to find a gating factor that would allow
me to tell when the qemu instance is fully up and rendered. I've tried a
few tactics here, (dbus-wait, qmp) but for now a disgusting
time.sleep(30) is there.
You can replicate this by running qemumips. The screen load takes forever,
but you even see it on qemux86 where the Home and Workspace Switch icons
will sometimes take a while to fully load.
Eventually I'm going to have to take multiple screenshots and compare
them, but then you get into the issue where the question is, is the diff
greater than 0 because it hasn't fully loaded or something is actually
incorrect.
There are the issues I know about:
- runqemu qemuppc64 comes up blank.
- qemuarmv5 comes up with multiple heads but sending "head" to screendump.
seems to create a png with a bad header.
- qemuriscv32 and qemuloongarch64 don't work with testimage apparently?
- qemumips64 is missing mouse icon.
- qemumips takes forever to render and is missing mouse icon.
- qemuarm and qemuppc return incorrect width
- All images have home and screen flipper icons not always rendered fully at first.
The sleep seems to help this out some, depending on machine load.
(From OE-Core rev: dc7cefbaccde50df6c4396e66d50659a45e00631)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Co-authored-by: Ross Burton <ross.burton@arm.com>
Co-authored-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This helps in getting it building with newer architectures like riscv32
since it has upgraded gnulib over 2.14 which has the needed fixes.
Drop the -fno-common workaround as it is already applied to cpio
drop --disable-maintainer-mode
Fixes
configure: WARNING: unrecognized options: --disable-maintainer-mode
(From OE-Core rev: 18d303497089d3a7a893ee0eec5b0f0c78cca06d)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Petr Vorel (upstream ltp maintainer) recommended some extra tests we should run.
Enable these for extra test coverage.
(From OE-Core rev: ebae56999dc2ed7abb65062a9abee53827c3bad3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed patch accepted upstream as
8c53cc947 ("testcases/kernel/syscalls/fcntl: define _LARGEFILE64_SOURCE")
Rebase runtest/mm patch.
[RP:
Drop connectors group since dropped upstream:
9b642d89c0
Drop fsx too:
fb2b6a0b3c
]
(From OE-Core rev: b2867cac5ac58862469f0c6c056607abeca0b0d0)
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Discovery of the test file was happening in a class initializer.
That block of code cannot fail (it's not a test), and so it
falls through to completion even if the needed file could not be found.
Then the tests themselves fail later due to class variables not
being set, but all information as to why is already lost at that point.
This converts the discovery to a helper function called from
the tests, so that the function can fail the tests precisely when the
problems occur.
(From OE-Core rev: 5d7a6ede105ea1efc9c324c7029f9d08dadf7255)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
systemd boot has completed when system is in "running" or "degraded"
(some services failed) state. Check for that in the systemd settle()
function instead of listing all services and checking their activation
state since some services are in activation state even when whole
system is already in "running" state. Examples of services which can be
in activation state are rootfs auto mounting related generated services.
Without this patch systemd test_systemd_list (systemd.SystemdBasicTests) times
out on an image with dm-verity /usr partition and systemd generated
rootfs:
NOTE: ... FAIL
Traceback (most recent call last):
File "/home/builder/src/base/build/../poky/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
File "/home/builder/src/base/poky/meta/lib/oeqa/runtime/cases/systemd.py", line 97, in test_systemd_failed
self.assertTrue(settled, msg=msg)
AssertionError: False is not true : Timed out waiting for systemd to settle:
UNIT LOAD ACTIVE SUB
DESCRIPTION
dev-disk-by\x2did-dm\x2dname\x2droot.device loaded activating tentativ
e /dev/disk/by-id/dm-name-root
dev-disk-by\x2did-dm\x2dname\x2dusr.device loaded activating tentativ
e /dev/disk/by-id/dm-name-usr
dev-disk-by\x2did-dm\x2duuid\x2dCRYPT\x2dLUKS2\x2df2b944f394174eb5918cb6af2c6b4cb2\x2droot.device loaded activating tentativ
e /dev/disk/by-id/dm-uuid-CRYPT-LUKS2-f2b944f394174eb5918cb6af2c6b4cb2-root
dev-disk-by\x2did-dm\x2duuid\x2dCRYPT\x2dVERITY\x2d3dd703c88f1946658697a6d57617473b\x2dusr.device loaded activating tentativ
e /dev/disk/by-id/dm-uuid-CRYPT-VERITY-3dd703c88f1946658697a6d57617473b-usr
dev-disk-by\x2duuid-bfbf856e\x2d3c65\x2d4eb2\x2d9ffb\x2d8e0b11641d85.device loaded activating tentativ
e /dev/disk/by-uuid/bfbf856e-3c65-4eb2-9ffb-8e0b11641d85
dev-dm\x2d0.device loaded activating tentativ
e /dev/dm-0
dev-dm\x2d1.device loaded activating tentativ
e /dev/dm-1
...
Fix is to check for the systemd global "running" or "degraded" state.
Note that it would be possible to use a blocking call
"systemctl is-system-running --wait" to exit after system enters "running"
or "degraded" state but using the existing loop for a 2 minute timeout.
(From OE-Core rev: 3b013ae441d117adeda0d9950e02e9f7d0deba2f)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a comment explaining why we need the jitterentropy ignore.
(From OE-Core rev: ed4b202b11fd18e18502fe4e90f421dce4db8b44)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
upstream commit 003e0694fcd3d5 [ACPI: processor: Refine messages
in acpi_early_processor_control_setup()] changes logging during
boot such that some of the informational messages are caught as
QA issues.
Adding the string to the ingore files so that 6.6+ will pass QA.
(From OE-Core rev: 463d40f2d4bf30becce431bfc1cbdf3b673e8e60)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
FIXES [YOCTO #12342]
When testing a Multilib image, the package manifest list contains
the fully qualified package name which includes the Multilib Prefix.
This patch adds the MLPREFIX to the package names that are passed
into the @OEHasPackage() decorator to ensure the set isdisjoint()
matches correctly.
(From OE-Core rev: ab87e4f92305b2a664cc473869e1615cf56e0936)
Signed-off-by: Saul Wold <sgw@bigsur.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Similar to the sdk test case, build the "guessing-game" example from
https://maturin.rs/tutorial
This test case:
* creates a python3 venv
* echoes "nameserver 8.8.8.8" to /etc/resolv.conf as we need to have
functional DNS to fetch the crates on target
* fetches crates, builds guessing-game crate and wheel
Put the following in your local.conf:
EXTRA_IMAGE_FEATURES += "tools-sdk"
SDK_INCLUDE_TOOLCHAIN = '1'
SDK_TOOLCHAIN_LANGS += 'rust'
IMAGE_INSTALL:append = " python3-maturin"
IMAGE_CLASSES += "testimage"
TEST_QEMUPARAMS ?= "-m 8192 -smp 4"
IMAGE_ROOTFS_EXTRA_SPACE = "10000000"
NOHDD="1"
NOISO="1"
TEST_SUITES = "ping ssh python maturin"
Test with:
bitbake core-image-full-cmdline
bitbake -c testimage core-image-full-cmdline
(From OE-Core rev: ca7e78c8be6aaa2780702eab54715a74fc0dac5e)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Basic smoke test for maturin to test the 'maturin list-python' case.
(From OE-Core rev: 47c948c3cf6e582abd12021ceeff2c20a3e81fb5)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When enabled in DISTRO_FEATURES the test may run on a system without systemd.
Fix this.
(From OE-Core rev: c2b473390dec0f5132d5b4bff6d3c35214eb898b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enabling minidebuginfo is not useful if gdb and systemd-coredump
are unable to parse it.
In order to parse it, gdb needs xz support. Systemd needs coredump enabled, as
well as elfutil enabled as well (systemd-coredump loads libdw which is part of elfutils using dlopen).
(From OE-Core rev: 0d2df803bebfd7e832ab7da54c4dacaaeeb424a9)
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the ignores from a huge dict in the parselogs.py module to .txt
files. This is just the common, tune, and qemu machine ignores; the
machine ignores that are not in oe-core will be added to the relevant
layers.
The list of ignores has not been reviewed in any meaningful way, this
should be done soon as I suspect a number of these are redundant.
(From OE-Core rev: bba243e1d18b954578afcdb3c727d8f687187ee8)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of hardcoding the list of ignored errors/warnings in the test
itself, read them plain text files on disk.
This uses importlib to try to open a file called
oeqa.runtime.cases.parselogs-ignores-[candidate].txt, where the
candidate will be:
- "common"
- The TARGET_ARCH
- Each of the MACHINEOVERRDES
This allows the common and tune-specific ignores to be retained in
oe-core, and machine-specific ignores added to the layer where the
machine is defined.
[ YOCTO #14604 ]
(From OE-Core rev: 7a04063f7cff243fe2bee09664ad7979612110cb)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current code in ssh.py is terminating the ssh process that
does not finish its computation in a given timeout (when timeout
is passed), the SSHCall function is returning the process error
code.
The Openssl ssh before version 8.6_p1 is returning 0 when it is
terminated, from commit 8a9520836e71830f4fccca066dba73fea3d16bda
onwards (version >= 8.6_p1) ssh is returning 255 instead.
So for version of ssh older than 8.6_p1 when the SSHCall time out,
the return code will be 0, meaning success, which is wrong.
Fix this issue checking if the process has timeout (hence it's been
terminated) and checking if the returned code is 0, in that case
set it to 255 to advertise that an error occurred.
Add a test case excercising the timeout in the SSHTest, test_ssh
test function.
(From OE-Core rev: 948fecca1db4c7a30fcca5fcf5eef95cd12efb00)
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The _qemutiny is a small test case that was explicitly designed to do a
minimal level of testing for poky-tiny images. These typically don't
have SSH servers so we need to assume that qemu is being used and access
the serial console directly.
(From OE-Core rev: 2245b2754d6f4798127ce85a2ab7cb48f458c1f7)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of constructing huge grep statements, we can simply open the logs
in Python and do the relevant string operations directly.
The trick is to remember to casefold() all of the strings, so that the
"in" operator can be used.
Just one of the ignores needs to be adjusted because it uses a regular
expression and the new logic doesn't support that. This is handled
by simply reducing the size of the ignore match.
(From OE-Core rev: 78ae254c4a78a025a712281ce9de373cdccf5472)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This has no impact to the execution, but makes the following changes
neater.
(From OE-Core rev: 124dede2b20930d09a5cc319cb9333c5a912fa48)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
errors and log_locations can be trivially set in the class directly,
instead of being defined in the module and then copied into the class.
(From OE-Core rev: dd1416b719a30b18f21916d50fa431a88503918f)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no point in passing around member fields, just access them
directly.
(From OE-Core rev: a24d6eda061363cdcfa95980cd2698a674737d23)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
getLogList() uses remote find invocations to find the logs. Instead of
relying on shell expansion of wildcards and redundant use of -maxdepth
(pointless as the shell expansion means the find is passed the files to
return), invoke find idiomatically by telling it what directory to
search for and escape the glob so find processes it.
Also remove many pointless str() calls.
(From OE-Core rev: 03bb14cebf5879472a8da78d892ecd5c5df5c3cf)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2.13 may not be buildable with latest compilers without patching
(From OE-Core rev: 406a33f896accc35a9cb6ab156f1e0f42dda67d8)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to have one-liner functions to get the MACHINE or
WORKDIR when they're only called once.
(From OE-Core rev: 9478a665641f55cdc14f12a4409121ef95883d74)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The LSB compliance tests were removed in 2019[1], so this is obsolete.
[1] oe-core fb064356
(From OE-Core rev: a465b61bc3a16152f90032d5d5a041f87bf30108)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This information is pretty useless as we know what the target is from
the build configuration (be it a qemu machine or real hardware).
(From OE-Core rev: 7821c42172191e3c16ab8662a9e4f7e921830db0)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix up test case to match what binary prints.
(From OE-Core rev: 4683804b9669c71d31ea6a8a300e6e87e817ee12)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I'm not sure if this was reported correctly before, but it
currently is not. Test that is stuck is an error in itself.
(From OE-Core rev: 002e27c9932a83e46be0b03a5232594cfba7212c)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's now a real fix in commits "linux-yocto/6.*: fix IRQ-80 warnings".
This reverts commit 1451df346a0e2433714774421ce8f339a37a844a.
(From OE-Core rev: f5c1b0f2b17814315bc2a09acc3aaa20f2522cff)
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
rust-c-lib-example is a little rust code which provide
a single function to print a formatted date (via the chrono crate)
from an input timestamp in millisecond. It has the necessary FFI
annotation and inherit cargo_c class for the C ABI compatible
library generation.
rust-c-lib-example is meson project for the C code which
will call the print_date function from rust-c-lib-example
if no argument is provided, if any argument is provided
it will print "Hello world in rust from C!"
add a runtime test case to check if all went well.
(From OE-Core rev: bb177c7764b1bc47157d57d7a34930e59a7acef3)
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the new 6.4 kernel we see this preempt-rt error. It is blocking changing
to the new kernel and has sat on mailing lists unresolved for a long time. Ignore
it in testing for now and allow upgrading until we can better understand the
issues.
(From OE-Core rev: 1451df346a0e2433714774421ce8f339a37a844a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
we now have more rust/cargo recipes and tests that cover various use cases
so this is no longer needed.
(From OE-Core rev: 8baaf94d200f5355791ecd980727698b1ab0e539)
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When test command timeouts occur, there was no warning in the logs. Change
this to ignore "exit 1" but report all other exit codes so that timeouts
are clear.
(From OE-Core rev: 37851f948f3e7703560cf6346eb7d348d584dc7e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On our slower arm server, the tests currently timeout leading to inconsistent test
results. Increase the timeout to avoid this and aim to make the test results
consistent.
(From OE-Core rev: 9a8b49208f3c99e184eab426360b137bc773aa31)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The LTP test reporting appears to be a little fragile so I tried to make
it more reliable.
Primarily this is done by not passing -p to runltp, which results in
machine-readable logfiles instead of human-readable. These are easier
to parse and have more context in, so we can also report correctly
skipped tests.
(From OE-Core rev: d585c6062fcf452e7288f6f8fb540fd92cbf5ea2)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
str.format() doesn't use % notation, update the formatting to work.
assertTrue() is a member of self not a global, and assertTrue(True) will
always pass. Change this to just self.fail() as this is the failure case.
(From OE-Core rev: 017f3a0b1265c1a3b69c20bdb56bbf446111977e)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The intent behind these functions was to dump the system state when issues occured
but it has never really worked as we'd planned. Regular monitoring as the build
runs has largely replaced this as that allows a trend to be seen rather than a spot
value which was never really useful. The code is bitrotting and not functioning
correctly so drop it.
[YOCTO #13872]
RP: Reword commit message
(From OE-Core rev: dea37ba49a236029da73d5cfbfc069bffc38b508)
Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that even if a ptests results section is empty, the log parser adds that
empty section. Then ensure that empty sections trigger warnings.
This means if a ptest suddently stops returning any results, we notice and see
warnings about it. This has gone unnoticed on the autobuilder far too many times
so is very much worth highlighting as a regression. We shouldn't have empty ptests.
(From OE-Core rev: 5ad0cf57b41ec7f44647a03bc568d0b24906cc8d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is possible to call exported tests with --target-ip set to ":22"
where IP address is not set at all. Detect this case and fail the test
instead of calling ping without an IP address.
(From OE-Core rev: 17c995c53775b8cee279ca4ced916092067e1195)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
