Sporadic errors have been observed in autobuilder when trying to store new
tests results:
error: failed to push some refs to 'push.yoctoproject.org:yocto-testresults'
hint: Updates were rejected because the tag already exists in the remote.
The new tag name is generated by gitarchive based on known tags from the
repository (learnt with git tag). In autobuilder case, this repository is a
shallow clone, so git tag only returns most recent tags, which mean we
could miss some older tags which exist in remote but not locally. In this
case, gitarchive will likely create a tag which already exists in remote,
and so will fail to push
Fix this tag duplication by using git ls-remote to learn about existing
tags instead of git tag. Two places which wrongly read only local tags has
been identified in gitarchive: expand_tag_strings and get_test_runs
Fixes [YOCTO #15140]
(From OE-Core rev: 5e3962a53657044431f687aaa08a993563f779e7)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5a0a7da85a3acfd4a20a07478eabefdab60f313a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
If you run an arm build followed by an x86 one and then ask for a
full repo to be created, it will include all of the arm and x86 packages.
testexport will then find the arm socat package rather than the x86 one
and try and run arm binaries within an x86 qemu image with no success.
The reproducer for this was:
oe-selftest -r fitimage.FitImageTests.test_initramfs_bundle runtime_test.TestImage.test_testimage_install
This patch only symlinks in the compatible package archictures rather
than all of them which fixes the failure and the resulting autobuilder
intermittent failure too.
[YOCTO #15190]
(From OE-Core rev: 96954ff5ea258e8456c66bfa13a11fb4fe7a3808)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 30b45bcf49bf8207fd96bb45a55d7708661f3359)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
It looks like there were further cases where orphaned processes may be left
behind since the .kill() calls may be unsuccessful if the process terminated
due to the terminate or through normal exit. In that situation .wait()
wouldn't have been called.
Further tweak the exit code paths to ensure .wait() is called to update the
returncode value before returning in all cases.
(From OE-Core rev: f647d1400bfa5b8eab81b2b2e2c86df306970a71)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0a0a1731e38edfa72a141e8fd8f2de52be562e94)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
As spotted by Joshua Watt, the returncode isn't set until .poll() or .wait()
is called so we need to call this after the .kill() call.
This fixes return code reporting so that timeouts for example now return an
exit code when they didn't before.
(From OE-Core rev: 6bd6b7110ea2029fc736a40760536adfaf28eec0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3924e94214b5135369be2551d54fb92097d35e95)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
On our slower arm server, the tests currently timeout leading to inconsistent test
results. Increase the timeout to avoid this and aim to make the test results
consistent.
(From OE-Core rev: a9e2179f3b4a4fe7321539de9e8e2707c9ca350a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9a8b49208f3c99e184eab426360b137bc773aa31)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
We have a suspicion that the read() call may return EAGAIN on the non-blocking
fd and this may truncate test output leading to some of our intermittent failures.
Tweak the code to avoid this potential issue.
(From OE-Core rev: fd9e47ee5653dad1f24d823d32d2012e7f8bb3a6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a8920c105725431e989cceb616bd04eaa52127ec)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
This provides a more reliable test execution when running tests that
write a large buffer/file and significantly reduces the localedata test
failures.
(From OE-Core rev: e138b648a56a0146588dc52edd9b44929ed8a73e)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 97a7612e3959bc9c75116a4e696f47cc31aea75d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Allows setting up NFS over TCP as well.
(From OE-Core rev: a177ce907ec7a044659ec49c679e66f49cd71c43)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e1ff9b9a3b7f7924aea67d2024581bea2e916036)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Some of the tests trigger OOM and fail. Increase the amount of memory
available so we dont run into these issues.
(From OE-Core rev: b4e7158aca35235c86afee2b68236e250488ef98)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4d22dba482cb19ffcff5abee73f24526ea9d1c2a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Rather than copying the entire /etc hierarchy, specify the pieces we
actually need.
(From OE-Core rev: c8fbba7c0bb442282fe19abdcee492407e20b8cb)
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit f0fea55ab02b013484282177a636795a254e7986)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
When copying the package database files for the debugfs, add individual
file copy as well as tree copying. After the debug rootfs has been
created, cleanup the package files.
This then allows us to avoid a problem where (for rpm at least)
extraneous files in the debug rootfs would cause failures during
oe-selftest because some files existed in both regular and debugfs
images.
(From OE-Core rev: 4931839ff97e8746cdabe1d06298a96a47638b2b)
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit ce49ea435ce55eb5b6da442c12e03a806534c38d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Fix the override so we actually pass the correct value to glibc.
(From OE-Core rev: 8ad6000c814631e1147fddd5d2462b39ea44b99e)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 38fd2120f0f48512091ddad6205ce19839eaf589)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
str.format() doesn't use % notation, update the formatting to work.
assertTrue() is a member of self not a global, and assertTrue(True) will
always pass. Change this to just self.fail() as this is the failure case.
(From OE-Core rev: c88b9eedc81529c67f2e628ae5e7325daff2535e)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 017f3a0b1265c1a3b69c20bdb56bbf446111977e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The intent behind these functions was to dump the system state when issues occured
but it has never really worked as we'd planned. Regular monitoring as the build
runs has largely replaced this as that allows a trend to be seen rather than a spot
value which was never really useful. The code is bitrotting and not functioning
correctly so drop it.
[YOCTO #13872]
RP: Reword commit message
(From OE-Core rev: 8d1bc34cffdd9f054e51db4e880747c79bf834fe)
Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit dea37ba49a236029da73d5cfbfc069bffc38b508)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The dnf contents should be moved to <host_sysroot>/etc/dnf/xxx
instead of just <host_sysroot>/etc.
(From OE-Core rev: 313f009e346a5aa10a09d65706c12ceefddaf59d)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 74b78d160a985e98f869c777847ab798e419dd2d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Instead of printing an error message and continuing, we should just
error out when moving file fails.
(From OE-Core rev: 3a195a2da81755c2a030b5b0354ef177d826bdb2)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 12aecd9da94b5f27041982c661e8bab316d365d4)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
NPM registry cache should support caracaters like '(' and ')'
Explanation: NPM packages can contains these caracters like : @(._.)/execute
(From OE-Core rev: 9d51a9f819a0ffdfd273635aa4e2062bf30a1db2)
Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 6110d9e24e43e286781afd1b3634a4ad1a2050d0)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The error message is a little misleading as the qmp module is a
directory with __init__.py file, not qmp.py file. Also, put the
path where we try to import it from in the error message to make
the message more indicative.
(From OE-Core rev: 60466c8ee5c055bdc01582f3809c7b36bb646a1d)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 08bacbf797f6a50ae8abe8fc3455b3a15a0a94b3)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
We don't compare reproducibility of the native/cross components, only the
target ones. With the long build times of rust-native, the test now takes
crazy lengths of time so this tweak should allow us to reuse native/cross
artefacts from sstate whilst still testing the target output is
reproducible.
(From OE-Core rev: ea3d21235633dc5b01c6ce39e768d00d1da5d87a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b494d83c639a877cefeb7cbab6d37195e492f059)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Adding a selftest version of hello world to run it in tests where no download is necessary.
Also using this in several tests to speed them up.
Using the -native version wherever possible will also speed up tests a lot.
[YOCTO #11142]
(From OE-Core rev: 95241ece5fcef53901b97f96ae961cea208ca31a)
Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit c3f26b63934888df0e3cd563c1c2804eb78a368e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Devtool selftests require poky dir a git repo, when downloading poky as a tar,
this is not the case. Those tests will now skipped.
[YOCTO #12389]
(From OE-Core rev: 5b913494b472ef03435cabf984361b43c3d813f4)
Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 95a5bc130dc51ea9de95c64dbf0e9c7892415d50)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
close() the server_socket before qemusock replaces
it.
(From OE-Core rev: fb1de20b22119fd62b338ab69cd2b6a6c4e9dd8f)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit be7bbbeeb44158b181d1abc5a4d7871a9045fdd6)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
cf9df9e8d89f ("ipk: Decode byte data to string in manifest handling")
did a bit of least-effort fix to a string vs byte sequence issue in the
manifest handling. The approach was chosen as it localised the fix,
rather than having to analyse further call sites.
However since then f2167ae80258 ("package_manager/ipk: do not pipe
stderr to stdout") was applied, reworking the output handling from the
subcommand. dummy_bytes() now returns a string, so stop trying to decode
it.
Fixes: f2167ae80258 ("package_manager/ipk: do not pipe stderr to stdout")
Cc: Curtis Meier <cmeier@us.ibm.com>
Cc: Pam Eggler <eggler@us.ibm.com>
(From OE-Core rev: b61739554780d70307d2b6b37d2b3b1c7df93c77)
(From OE-Core rev: 29bd0b6272680769b438f39fa96b47d21c846f37)
Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 07e5a6331be60d5e35d7336a6215a972ced6eb57)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Specifically:
- add missing maintainer.inc entries for initramfs-module-*, systemd-machine-units and
target-sdk-provides-dummy and drop them from exception list.
- remove rust from exception list for unbuildable-by-default recipes as it is now buildable.
- add missing maintainer.inc entry for libx11-compose-data and cve-update-nvd2-native;
as they are also unbuildable by default, they needs to be in exception list as well.
(From OE-Core rev: d1ad59bb7a5360433403733de71b66ff3e1484ef)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e9158b191c1cfc16f97abed6c05891aa84fe9463)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
"sysconfdir" contains "/" by definition and thus using os.path.join()
leads to self.target_rootfs being always ignored (and thus attempting to
generate paths in host's /etc).
Use oe.path.join() instead which was made for this purpose.
(From OE-Core rev: 116e2b1d0c75c6be333339812468550145915774)
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8414c504138f6de663f5130c6b4a6ede5605d88b)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The recommendation from server maintainers is that the https protocol
is both faster and more reliable than the dedicated git protocol at this point.
Switch to it where possible.
(From OE-Core rev: 8f3669f81db8a58f8ed2faef76acab3499f59619)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 139102a73d4151f4748b4a861bd4ab28dda7dab7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
This will use default values when no distribution is set.
[YOCTO #15086]
(From OE-Core rev: 9e576ab7035d52fba961c400d53657608ad5b99d)
Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit 888fe63b46efceeff08dbe8c4f66fec33d06cb7a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
It is possible to call exported tests with --target-ip set to ":22"
where IP address is not set at all. Detect this case and fail the test
instead of calling ping without an IP address.
(From OE-Core rev: 17c995c53775b8cee279ca4ced916092067e1195)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* if you don't have QEMU_USE_KVM in local.conf it fails with:
2023-03-12 18:06:29,591 - oe-selftest - DEBUG - Checking if qemux86-64 is not this MACHINE
2023-03-12 18:06:29,594 - oe-selftest - INFO - ... ERROR
2023-03-12 18:06:29,594 - oe-selftest - INFO - Traceback (most recent call last):
File "/OE/build/poky/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/OE/build/poky/meta/lib/oeqa/selftest/cases/efibootpartition.py", line 18, in test_boot_efi
if oe.types.qemu_use_kvm(self.td['QEMU_USE_KVM'], self.td["TARGET_ARCH"]):
~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'QEMU_USE_KVM'
[YOCTO #12937]
(From OE-Core rev: 7c32ca2f91beb98769c89470b37f06cb4b99aebb)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since bitbake commit 81a58647b2f4fc0a2589b2978fc9d81b2bfe6aec
[bitbake: build: Make python output print to stdout when running with
-v (verbose)] we no longer need to comment out the python stdout
checks.
(From OE-Core rev: 67886a8473c511c8ab3db2e4587cc5a070979d11)
Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* It was showing whole log and that the runqemu command failed, but not
where the log file is, nor why it thinks the runqemu failed
[YOCTO #12937]
(From OE-Core rev: ad4b72e6524114a3bdee10cab71f194ea143cd24)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* it was searching for line like this:
ROOTFS: [/OE/build/poky/build/build-st-2023-03-20-esdk-runqemu-patch1/runqemu.RunqemuTests.test_boot_machine_ext4/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64-20230320081121.rootfs.ext4]
but with IMAGE_NAME_SUFFIX changed to something else than default ".rootfs"
or with my pending changes the line looks like this:
ROOTFS: [/OE/build/poky/build/build-st-2023-03-20-esdk-runqemu-patch2/runqemu.RunqemuTests.test_boot_machine_ext4/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs-20230320085744.ext4]
and test was failing.
* Check for whole line starting with ROOTFS: and ending just with .ext4
[YOCTO #12937]
(From OE-Core rev: 7a0f622f23aff2c4eeca0606e7682931eb53287a)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This recipe is for showing a "real world" example of
a crate that depends on some git repositories.
Usually, this kind of crate is built within a global
workspace (here it is the zbus project) and so
doesn't need a Cargo.lock on its own.
For the sake of the demonstration, I had to tweak things
a little to be able to compile zvariant in standalone
(no relative path in dependency, no symlink to LICENSE
provide a Cargo.lock)
The use case where the crate had some git repository
in dependency is very common for "private" crate that
are not aimed to be published on crates.io.
When the project grow bigger, it is common to have
a bin and multiple lib developped in parallel, and these
libs are surely on a git repostitory.
A test case have been also added to check for:
- the previous patch about git subpath parameter and devtool
- the correctness of overriding dependencies (first patch of the
series)
(From OE-Core rev: 409e045f96f69877de6f36ed14c5c19a9cb74eaf)
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is for a specific case where:
- A recipe use a subpath on a git repo (e.g. git://repo.git/projects;subpath=subproject)
- The recipe contains a patch to apply
- a devtool modify is used on this recipe
With these conditions, the patch cannot be applied at all.
GitApplyTree class is used for handling patch under devtool, but
when subpath is present in SRC_URI, the resulting git tree
is dirty (every files and directories which was not in subpath are suppressed)
and so "git am" refuse to apply patches.
That would not be an issue since the GitApplyTree have a fallback
to PatchTree in case of error, but during this error management,
there is a "git reset --hard HEAD" call which suppress the subpath
operation and finally prevents the patch to be applied even with PatchTree.
When devtool is not involved, only PatchTree class is used and the
above problem is irrelevant.
To support git patching during devtool, the presence of subpath and
the dirtyness of the repo are checked. If both conditions are
met, we directly call PatchTree like it was already done
in case of error during git apply.
(From OE-Core rev: d86cac2759cf7e91f4ff240833385e28e729ab79)
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* There is no need to be that precise. It just irritates in the logs.
* There is also no point in printing plain time.time() value a single
time while only using formatted printout everywhere else, thus remove
it.
* Use %d for printing integer times
(From OE-Core rev: f0988cb8cf8d03708490cca4eba345492ef78d52)
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since login user is configurable with boot_patterns['send_login_user'],
unconditionally using 'root' in the debug message can be confusing.
Also fix the debug message to say 'Logged in' instead of 'Logged'.
(From OE-Core rev: 900e3d42b918b5a33d8d952b3a8078fbe72ba98f)
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is already set above, thus simply use the set value from there.
(From OE-Core rev: 2f6571ffb23b1a0b5bd6a1212fc108e536f04539)
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'tempfile' and 'oeqa.utils.command.runCmd' are unused while 'os' is
actually used.
(From OE-Core rev: 50f766233e83528eebeca92877277374c7193530)
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
