Where we're using DL_DIR in sdk archive to try and cache testing artefacts,
copy into the cache so that it gets populated and this doesn't have to be done
manually. Currently we're making a lot of repeat requests to github as this
wasn't being populated.
(From OE-Core rev: a3284958a2cc6c90a5fac26976bddc23f821c972)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add test_devtool_upgrade_recipe_update_extra_tasks test case
to test upgrade of python3-guessing-game from v0.1.0 to v0.2.0
which will exercise the update_crates task during the upgrade.
Add python3-guessing-game_git.bb.upgraded and
python3-guessing-game-crates.inc.upgraded which are the 0.2.0
variants.
Check that the new recipe file has the expected differences.
Check that the new -crates.inc file has the expected differences,
which should be reproducible because of Cargo.lock.
(From OE-Core rev: d14368bc775cbf5142c1312dfc2076e328381aef)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the test for the addition of SPDX_VERSION to the deploy path.
(From OE-Core rev: 055e8c21908127722abad9e992d6408d8697a119)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
test_systemd_coredump_minidebuginfo was getting skipped in CI, because the feature isn't enabled per default in any image at the moment.
Add this selftest so that test_systemd_coredump_minidebuginfo gets executed in CI.
(From OE-Core rev: 377603886f0d975ec23f32ee462693d4e3370aaf)
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Usage and arguments are not necessarily self-explaining, thus add a bit
of documentation here.
(From OE-Core rev: 62343333d50b75ad04a4220bcf0990e0a89996a0)
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test was failing with core-image-full-cmdline which uses sleep from coreutils instead of sleep from busybox.
(From OE-Core rev: 8497edf0c56da34ea7b7509b8fcd46fcba2fd0e3)
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The io, cap_bounds and filecaps groups were removed in latest version
of ltp. Tweak the test class to match.
(From OE-Core rev: d5460f15cbc56e52cbbc09785a777278208da7f7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The builtin command "sleep" was getting executed instead of the busybox binary.
Hence "pidof sleep" was empty and the test was failing.
(From OE-Core rev: 3188f3f0718b8a59bc9e3cae4cdf74e82bdb86c3)
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Eclipse plugin is very dead, we don't need to carry the manual test
guide for it.
CROPS has its own automated testing, nobody is doing the testing by hand.
(From OE-Core rev: f74a634f199782ce5fa3023d02079c1534bc4aeb)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
go.bbclass uses a special do_unpack function that causes the git root
to be different from S. Verify that it unpacks as expected.
[ YOCTO #15483 ]
(From OE-Core rev: fab0c737b95b8d0c0bbf58336bc308776c956406)
Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is useful not just in cdn/local cache tests, but also in the build bundle test,
and so should be applied by default.
(From OE-Core rev: 37f0ba71868ef1907883ab751c82ba222fbc87da)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows reusing the check function in tests other than CDN/local mirror
ones, such as sstate bundle testing.
(From OE-Core rev: 8a4ab4c063f307b480d98c6aac1e76cde0b9e99e)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of having a test called eg "assimp", rename it to "cmake" as the
point of the test is to verify that CMake works. This should make it
clearer what the tests are actually exercising.
(From OE-Core rev: ccf7fdc0e5b6df218b319f972cd5ba142c06c243)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To enable this test to work with gcc 14, pass the option to make
warnings non-fatal. Also upgrade to version 5.4.1 from 5.3.1.
(From OE-Core rev: c3df6287ae26dc9d7f11eb7e26fdbcaefe4dfead)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
librdfa hasn't had a commit for a decade now and as such has problems
with modern compilers (specifically gcc 14.1). Switch the recipe
creation test to something much simpler that we also control: dbus-wait.
(From OE-Core rev: d801de1f702d8d0def55011b5b6ad39d85f978f1)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Several recipes have S pointing at a directory that does not exist.
Set S in these cases to somethig valid making the metadata and
recipe behaviour more consistent.
Tweak one of the QA test diff offsets to match the changed recipe.
(From OE-Core rev: 22f1f5849a9a3bf287dbe8933546e52e39ddc86e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The only real reason for oe-local-files was to support S = WORKDIR. With changes to
drop support for that, it makes sense to simplify devtool and to try and make both
the code and the processes/workflows simpler.
This patch drops support for S = WORKDIR, removes oe-local-files and then updates
the test cases to match this new situation.
At the code level, we assume we can always now track code changes using git and
that things committed into git are handled as patches (as before) but delta against
HEAD is saved as specific file level changes to the recipe.
One test is disabled as it is no longer approproate. It is being keped until we can
make WORKDIR != UNPACKDIR at which point it should be revisited.
(From OE-Core rev: ce8190c519052fed10b5233697b69a75868db45a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace references of WORKDIR with UNPACKDIR where it makes sense to do
so in preparation for changing the default value of UNPACKDIR.
(From OE-Core rev: 1f18b9a512800860d5153d89eb82b56388efad6f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Similarly to centos 8, centos 9 doesn't support the render device we need
for this test.
(From OE-Core rev: c2be3afabf84f287c90b61ae2509728a6634fb8f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes the debuginfod selftest fails due to a timeout, because it
spends too long scanning a huge deploy directory that due to what tests
were ran previously can contain 30K packages.
The test only needs a subset of the feed, so use the new localpkgfeed
class to construct a minimal feed before running the test.
[ YOCTO #14937 ]
(From OE-Core rev: 855376f518b28248ccd82ef5b2e89e6a8c970542)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If usrmerge is in DISTRO_FEATURES, assumptions in one of the tests would
fail. Improve the test so it works in both cases.
(From OE-Core rev: afa211746a2aa1993a54cc5a5e1937679341da8e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If usrmerge is in DISTRO_FEATURES, assumptions in one of the tests would
fail. Improve the test so it works in both cases.
(From OE-Core rev: 68a27d307a7042e242c49cf3d069469f40e09902)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit ab6d3e3d64
from poky repository.
The previous reverted commit was a workaround. The fix
"oeqa/selftest/devtool: fix _test_devtool_add_git_url"
tackle the issue. So, the workaround is not needed anymore.
(From OE-Core rev: 731f47ecfd8ad6558aac629806810789c623986b)
Signed-off-by: Alexandre Truong <alexandre.truong@smile.fr>
Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch is a follow-up to the bug#15466.
As a reminder, the bug was about devtool’s submodule detection
checking for HEAD when a version is being passed.
As Vincent Kriek pointed out:
the --version that is being passed to devtool is only used for
setting the PV value in the recipe. To take into account the tag,
we add --srcrev to the command:
devtool add --srcrev v3.1.0 --version v3.1.0 mbedtls git://git@github.com/ARMmbed/mbedtls.git;protocol=https
Changes to _test_devtool_add_git_url have been made to take
into account the srcrev. srcrev will be passed as an optional
parameter because the test_devtool_add_git_style1
does not need the srcrev contrary to test_devtool_add_git_style2
Fixes [YOCTO #15466]
(From OE-Core rev: a8686f3641e4407dee3d807898ffd620e2732b78)
Signed-off-by: Alexandre Truong <alexandre.truong@smile.fr>
Reported-by: Alexandre Truong <alexandre.truong@smile.fr>
Suggested-by: Vincent Kriek <vincent@coelebs.dev>
Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't have a test to check if we can correctly devtool update-recipe/finish
into another layer. So update the existing test_devtool_update_recipe_local_files
to also check the updates into another layer.
(From OE-Core rev: bd44c895d36e246a25c7a6e40bf9f4089dc7a297)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If testtools and/or subunit modules are not found we get the following backtrace
(example for testtools):
NOTE: Starting bitbake server...
Traceback (most recent call last):
File "<..>/poky/scripts/oe-selftest", line 60, in
<module>
ret = main()
File "<..>/poky/scripts/oe-selftest", line 47, in main
results = args.func(logger, args)
File "<..>/poky/meta/lib/oeqa/selftest/context.py",
line 391, in run
rc = self._internal_run(logger, args)
File "<..>/poky/meta/lib/oeqa/selftest/context.py",
line 377, in _internal_run
rc = self.tc.runTests(**self.tc_kwargs['run'])
File "<..>/poky/meta/lib/oeqa/selftest/context.py",
line 161, in runTests
return super(OESelftestTestContext, self).runTests(processes, skips)
File "<..>/poky/meta/lib/oeqa/core/context.py", line
91, in runTests
result = self.runner.run(self.prepareSuite(self.suites, processes))
File "<..>/poky/meta/lib/oeqa/selftest/context.py",
line 154, in prepareSuite
from oeqa.core.utils.concurrencytest import ConcurrentTestSuite
File
"<..>/poky/meta/lib/oeqa/core/utils/concurrencytest.py",
line 22, in <module>
import testtools
ModuleNotFoundError: No module named 'testtools'
Fix this by adding a custom callback on -j/--num-processes parameter to
check testtools and subunit modules. Fallback to serial testing if
missing. This strategy is already used in sdk/context.py
(From OE-Core rev: 35284404473b2c2d9f69594582868ed66ef3525e)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
-l option is specific to df provided by coreutils, if df
applet from busybox is used then it does not work and fails
like below
Fixes
df: invalid option -- 'l'
BusyBox v1.36.1 () multi-call binary.
Usage: df [-PkmhT] [-t TYPE] [FILESYSTEM]...
DEBUG: [Command returned '1' after 0.71 seconds]
DEBUG: Command: df -hl
Status: 1 Output: df: invalid option -- 'l'
it seems worth a compromise to show remote mounted filesystems if any
during ptests and it works with both df implementations
(From OE-Core rev: 75cc1ea4348a2294fdc5ab20530fcff27056ff06)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The problem is the following:
AssertionError: 'gitsm://git@github.com/ARMmbed/mbedtls.git;protocol=https;branch=master' != 'git://git@github.com/ARMmbed/mbedtls.git;protocol=https;branch=master'
Mbedlts made changes to their repository, adding a sub-module, thus the
assert triggers an error with the url:
git://git@github.com/ARMmbed/mbedtls.git;protocol=https;branch=master
456a54da8e
was the upstream change.
To fix the issue, the url has been changed to:
gitsm://git@github.com/ARMmbed/mbedtls.git;protocol=https;branch=master
(From OE-Core rev: 9ac737fbe05c85ec8333b396ce2f89de6654916f)
Signed-off-by: Alexandre Truong <alexandre.truong@smile.fr>
Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Under some load conditions, the result event can come back before the
command complete event. If that happens, the code would hang
indefinitely.
Rework the code to wait for both events and avoid the hang.
(From OE-Core rev: 8cfc94a4404c54bc73eab9f98d9da1f84c2135ad)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds to run_test_sstate_creation so that it also tests
that sstate directories don't accidentally pickup umask permissions
from the user upon creation.
[RP: Python style tweaking]
(From OE-Core rev: 7d6eb828e97ad3f27d94efdccd920fb2aef36743)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The dbus-wait command returns a timeout after 60s but reports "success", detect this.
Unfortunately it does effectively break the test as the signal is nearly never being
correctly detected since it was already sent.
For that reason comment out the code instead too.
Also fix the loop conditional as the logic was incorrect and it was looping
indefinitely when an image match didn't occur.
(From OE-Core rev: 89c930e9e4b38b116edcba59e88621a39f8bda67)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch uses dbus-wait to wait for matchbox to be up. Once that
happens, it sets a timeout of 60 seconds and takes a screenshot
and compares it, every 2 seconds. If diff=0 it passes. If the timeout
ends, it fails.
(From OE-Core rev: 287b4f0a8244f7214f6a1aaa84ef16cc528f8326)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Excluding riscv64 due to mouse rather than a touchscreen which adds a
moving cursor, so the diff ends up > 0. Need to fix the image to use the
touchscreen rather than mouse input.
(From OE-Core rev: 7f7032c7613abd62ce510c98211c75fc7c5e7090)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The mouse panel icon can move when the time changes between 3 and 4 digits.
Ideally we'd fix the alignment of the clock on the panel but to get the
tests working, increase the size of the mask for now.
(From OE-Core rev: 8ec02142ab98741749281bdb29d5973c96b839d7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Allow tools to be found from the host PATH so that imagemagick from a buildtools
tarball/sdk can work
* Reformat the code to have imports at the start of the file and have more standard
formatting and whitespace
* Always save copies of the images, the space imapct is negligle compared to the
debug win
* Write the images to ${T}
* Use bb.utils.mkdirhier() instead of more complex code
* Restrict the tests to images containing matchbox-desktop
(From OE-Core rev: d09989b49517830297654e4d1d150aaa8723c41a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This takes the work rburton did on image screenshot testing and
expands it.
Right now this works with most of the qemu based machines except for
- qemuppc64
- qemuarmv5
- qemuriscv32
- qemuloongarch64
See "Known Issues" further down.
This test takes a screendump of a qemu image, blanks out the clock
and compares it to an image we have on record. If the diff is exact,
the test passes. If not, it stores the image in build/failed-images and
fails out.
In order to enable this test, you will need meta-openembedded/meta-oe in
your bblayers.conf for imagemagick and the following in local.conf:
IMAGE_CLASSES += "testimage"
TEST_SUITES = "login"
TESTIMAGEDEPENDS:append:qemuall = " imagemagick-native:do_populate_sysroot "
Known Issues
------------
The main issue is that I've yet to find a gating factor that would allow
me to tell when the qemu instance is fully up and rendered. I've tried a
few tactics here, (dbus-wait, qmp) but for now a disgusting
time.sleep(30) is there.
You can replicate this by running qemumips. The screen load takes forever,
but you even see it on qemux86 where the Home and Workspace Switch icons
will sometimes take a while to fully load.
Eventually I'm going to have to take multiple screenshots and compare
them, but then you get into the issue where the question is, is the diff
greater than 0 because it hasn't fully loaded or something is actually
incorrect.
There are the issues I know about:
- runqemu qemuppc64 comes up blank.
- qemuarmv5 comes up with multiple heads but sending "head" to screendump.
seems to create a png with a bad header.
- qemuriscv32 and qemuloongarch64 don't work with testimage apparently?
- qemumips64 is missing mouse icon.
- qemumips takes forever to render and is missing mouse icon.
- qemuarm and qemuppc return incorrect width
- All images have home and screen flipper icons not always rendered fully at first.
The sleep seems to help this out some, depending on machine load.
(From OE-Core rev: dc7cefbaccde50df6c4396e66d50659a45e00631)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Co-authored-by: Ross Burton <ross.burton@arm.com>
Co-authored-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A change in QEMUMonitorProtocol.cmd() requires that we either pass in
kwargs instead of an argument dict or move to cmd_raw()
cmd() was renamed to cmd_raw() (and command() was renamed to cmd())
See:
37274707f6684750ab4f
My concern with this patch is that I haven't seen this come up with
utils/dump.py which also uses QemuMonitor's run_monitor. If it is
occuring, this should fix issues there as well
(From OE-Core rev: 9665d38ab60c1c3b27887c2b1a6396f13a1b33ea)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adjust selftest to git-add the directory with newly added patches,
as the new minicom recipe has no default patches, and thus no directory
with them (and the selftest assumed it does).
(From OE-Core rev: 1fb2aa3f242ef20f8edfb518164b629258a04dd4)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since go-1.21 GOPROXY needs to be set explicitly, otherwise it fails with:
- GOPROXY list is not the empty string, but contains no entries
This fixes the selftest
(From OE-Core rev: c491d967858c01fead21495f44f1a9f8cdf8e833)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recently introduced postactions module can raise, on failing ptests,
the following warning:
WARNING: core-image-ptest-glib-2.0-1.0-r0 do_testimage: Can not get host
disk usage: [Errno 2] No such file or directory: '/usr/bin/df'
The issue is likely not happening because of df absence (to be confirmed
amongst the variety of workers) but because of the wrong path. Fix it by
letting subprocess search for df, passing only the binary name. To make it
work, we also have to reset the environment, otherwise the environment
configured before running bitbake will be used, and search will fail.
(From OE-Core rev: da7cc5def2839a0e15d07244f858847479c12caa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Improve the GDB related tests. Verify GDB finds the correct source
files.
(From OE-Core rev: 67eed460c0bf18d23f2c9180f195417895acfd55)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the target under test can be a virtualized guest, when some tests
fail because of disk usage (see [1]), also fetch disk usage statistics from
host to allow checking whether a host disk space saturation could affect
running tests.
[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15220
(From OE-Core rev: 2ab3a0935b1e7a016402f149da1fc01b38d7af55)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
