If a task is adde which has a dependency on the do_populate_sysroot task of
the recipe, it will cause it to be installed into the sysroot (similar to
do_addto_recipe_sysroot). This fails since the postinst script is an overlapping
file:
Exception: FileExistsError: [Errno 17] File exists:
'tmp/sysroots-components/all/useraddbadtask/usr/bin/postinst-useradd-useraddbadtask'
->
'tmp/work/all-poky-linux/useraddbadtask/1.0/recipe-sysroot/usr/bin/postinst-useradd-useraddbadtask'
The copy written out at do_prepare_recipe_sysroot time is just for debug so
rename it, meaning there are no longer overlapping files and the installation
can be successful, removing the error.
[YCOTO #14961]
With the bug fixed, enable the test.
(From OE-Core rev: 564339afb73fc52a66c1a08437587cad1c4d46e7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is already done for local stamps just above, and will allow enabling
the full selftest that compares gcc-source signatures via printdiff
(that is, both local stamp and sstate variants).
(From OE-Core rev: 29775b5ecfc8d811293962f050fcfc3b3ad7efde)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the tests fail, these contain useful artefacts, and so should
be kept. If the test succeeds the whole build-st/ is deleted.
Also, give them unique names, as otherwise the tests would
step on each other.
(From OE-Core rev: 92e33a19fbcc6c59199fcd8b17ad8ca29ebcd4fd)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Assert*() functions from python unittest would join the multiline output with \n, making it
almost unreadable.
(From OE-Core rev: 1b01a71e77f70af77887c27be21265ac61f2c9a7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a disabled a test for 14961 - addtask between do_populate_sysroot and do_package breaks useradd class.
A fix is still needed for this.
(From OE-Core rev: b6af5788f7f8fb1e9d8ad14bd12168ff9d6baa21)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If recipe A requires the useradd actions of recipe B we need to
ensure that recipe B is part of the recipe A dependancy chain. In
order to do that, we introduce USERADD_DEPENDS. This makes sure
that the do_populate_sysroot_setscene of recipe B exists for
recipe A in case of a missing TMPDIR. This requires changes made in
runqueue.py by RP.
This commit along with the runqueue fixes effects:
Bug 13419 - recipes that add users to groups cannot rely on other recipes creating those groups (when population from sstate happens)
Bug 13904 - do_prepare_recipe_sysroot: postinst-useradd-* does not run in order of dependency and sometimes fails
Bug 13279 - Make sure users/groups exist for package_write_* tasks
Bug 15084 - For some reason using of same user in two recipes does not work properly
I've included the start of self-testing for useradd by adding tests for
13419 (which ends up testing 13904, 13279, 15084 by virtue of them all
having the same root cause)
(From OE-Core rev: b47f2352376bd16b7e7087b4dab143403e67e094)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When enabled in DISTRO_FEATURES the test may run on a system without systemd.
Fix this.
(From OE-Core rev: c2b473390dec0f5132d5b4bff6d3c35214eb898b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The md5sum is no longer generated by recipetool, stop expecting it.
(From OE-Core rev: d9b5f6a2eefa68fcecfca20b293d593f5cd53b7c)
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In addition to updating the sha256sum and removing the md5sum, update
all other existing checksums. If the only existing checksum is md5sum,
then replace it with the default expected checksums (currently only
sha256sum).
(From OE-Core rev: 8ea8827ee49b7f0443b1c4bd47d1344a689d73a3)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than including all SRC_URI checksums, include the ones that are
expected. These are the same as are output if no checksums are included
when building the recipe.
(From OE-Core rev: c2af83eb5e8573480179b6c0bcce50606b547099)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before, a variable such as SRC_URI[sha512sum] would end up as:
SRC_URI[sha512sum] = "45ff3abce4dab24a8090409e6d7bb26afa7fa7812a51e067 \
28c2aa47d5b4de610d97ba4609cf13d9173087bd909fdf377235eee988a6fdcf52abb7 \
0341c40b5b"
when updated by patch_recipe_lines().
(From OE-Core rev: a67e2feed1420739504d2a59d018dff7e6e17e04)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To avoid potential problems due to global Git hooks, add --no-verify to
a `git commit --amend` command.
(From OE-Core rev: 802359c0ec6db0b3a4103f8ad8bc9bed67884555)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the build environment is setup using `repo`, then poky/.git/object
is a symbolic link rather than a directory. To clone such repositories,
the source path must be prefixed with "file://". This avoids the
following error:
fatal: failed to start iterator over '.../poky/.git/objects': Not a directory
(From OE-Core rev: 8e3d08cb9274832a346ac3dffa8c9d5f6e93c478)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enabling minidebuginfo is not useful if gdb and systemd-coredump
are unable to parse it.
In order to parse it, gdb needs xz support. Systemd needs coredump enabled, as
well as elfutil enabled as well (systemd-coredump loads libdw which is part of elfutils using dlopen).
(From OE-Core rev: 0d2df803bebfd7e832ab7da54c4dacaaeeb424a9)
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new parameter update_original_recipe to allow to patch a recipe
instead of creating/updating a bbappend
(From OE-Core rev: 2f68ab2464bfad1b377df44a7b51203df59d66ce)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
appendsrc function relies on oe.recipeutils.bbappend_recipe to
copy files and add the corresponding entries in SRC_URI.
Currently, appendsrc function build itself the new SRC_URI entry to add the
correct subdir param, and gives it using the extralines parameter.
This has 2 drawbacks:
- oe.recipeutils.bbappend_recipe can already do this if we specify the
correct params, so we have duplicate code
- the duplicated code is not fully functional: for example, it doesn't
take into account the -m/--machine parameter
So fix this by not using extralines but give correctly formatted params.
Also remove the check for already existing entries as
oe.recipeutils.bbappend_recipe already implement it
The new bbappend file now have the SRC_URI entry after the
FILESEXTRAPATHS so fix the selftest.
Update test_recipetool_appendsrcfile_existing_in_src_uri_diff_params
test because recipetool appendsrcfiles used to not add new src_uri entry
if the entry already exist even with different parameters while
oe.recipeutils.bbappend_recipe adds it if parameters are different (and
remove the old entry)
(From OE-Core rev: cd5de8d53849a6f3bb6f82e45fb301e39892c789)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we do not add a new src_ury entry if the entry already exists
AND the parameters are the same.
I believe that when an entry already exist with different parameters,
we should remove it and add the new entry otherwise we end up with two
entries with different parameters
(From OE-Core rev: a4628fffcfecb5cd95dc2558dfd39ebd71121eab)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bbappend_recipe can take a dict of source files to add to SRC_URI where
the key is the full path to the file to be added and the value is a dict
Add a new optionnal entry "newname" to specify the name of the newly added file
(From OE-Core rev: e7bc09e5c9d7a0f4f8f4eba40730b68857b00677)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the case get_bbappend_path returns None (could not find the layer
containing the recipe) the error message tries to print the recipefile,
but it is not defined. Fix it.
(From OE-Core rev: 234111fb67ffbcc5492cb0cd96db25ed8f5acea0)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using devtool to patch CRLF based sources creates patch files which have
mixed end of lines : LF for headers and CRLF for source context and
modified lines.
Python open(..., newline=None) (default for newline arg)does detect
end-of-line in this mixed file but only outputs LF EOL data. This
result in patch files that does not apply on the original sources.
Switching to open(..., newline='') allows to detect end-of-line but keep
the original end-of-line intact. This generate correct patches for CRLF
based sources.
Fixes [YOCTO #15285]
(From OE-Core rev: 58f845499c0277a2b8069eefa235430b5f5f7661)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the ignores from a huge dict in the parselogs.py module to .txt
files. This is just the common, tune, and qemu machine ignores; the
machine ignores that are not in oe-core will be added to the relevant
layers.
The list of ignores has not been reviewed in any meaningful way, this
should be done soon as I suspect a number of these are redundant.
(From OE-Core rev: bba243e1d18b954578afcdb3c727d8f687187ee8)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of hardcoding the list of ignored errors/warnings in the test
itself, read them plain text files on disk.
This uses importlib to try to open a file called
oeqa.runtime.cases.parselogs-ignores-[candidate].txt, where the
candidate will be:
- "common"
- The TARGET_ARCH
- Each of the MACHINEOVERRDES
This allows the common and tune-specific ignores to be retained in
oe-core, and machine-specific ignores added to the layer where the
machine is defined.
[ YOCTO #14604 ]
(From OE-Core rev: 7a04063f7cff243fe2bee09664ad7979612110cb)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- test_skip_kernel_install: This test verifies that the kernel is not
installed in the boot partition when the
'install-kernel-into-boot-dir' parameter is set to false.
- test_kernel_install: This test verifies that the kernel is installed
in the boot partition when the 'install-kernel-into-boot-dir'
parameter is set to true.
Both tests use a WKS (Kickstart) file to specify the desired
configuration, build a disk image using WIC, and extract the disk
image to a temporary directory to verify the results.
(From OE-Core rev: a99bc5ed8bf67f171be24c0e2220aae6cccf230e)
Signed-off-by: Kareem Zarka <kareem.zarka@huawei.com>
Signed-off-by: Christian Taedcke <christian.taedcke@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The patch Signed-off-by test's output line is excessively long, and can
trigger a failsafe in the patchtest automated service's email content
generation. Shorten the output by reducing redundant phrasing and using
os.path.basename to get only the failing patch's name, not the entire
path, as the submitter should have a good idea of where it is located
regardless.
(From OE-Core rev: cc7546ded87dd44a988d7a23f1d7645094b5cdd4)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a test for gitsm recipes.
This tests that we can do changes on submodules, commit them and
properly extract the patches
(From OE-Core rev: 2fb69161fe9d25691b75a043ec5566ffe4a25b37)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding the support of submodules required a lot of changes on the
internal data structures:
* initial_rev/startcommit used as a starting point for looking at new
/ updated commits was replaced by a dictionary where the keys are the
submodule name ("." for main repo) and the values are the
initial_rev/startcommit
* the extractPatches function now extracts patch for the main repo and
for all submodules and stores them in a hierarchical way describing the
submodule path
* store initial_rev/commit also for all submodules inside the recipe
bbappend file
* _export_patches now returns dictionaries that contains the 'patchdir'
parameter (if any). This parameter is used to add the correct
'patchdir=' parameter on the recipe
Also, recipe can extract a secondary git tree inside the workdir.
By default, at the end of the do_patch function, there is a hook in
devtool that commits everything that was modified to have a clean
repository. It uses the command: "git add .; git commit ..."
The issue here is that, it adds the secondary git tree as a submodule
but in a wrong way. Doing "git add <git dir>" declares a submodule but do
not adds a url associated to it, and all following "git submodule foreach"
commands will fail.
So detect that a git tree was extracted inside S and correctly add it
using "git submodule add <url> <path>", so that it will be considered as a
regular git submodule
(From OE-Core rev: 900129cbdf25297a42ab5dbd02d1adbea405c935)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 'rustdoc' is moved to a separate 'rust-rustdoc' package. This is a
workaround to test if the main rust binary is reproducible even if
rustdoc isn't.
(From OE-Core rev: dfed7e63e199009ecca8e2e34f6a823c6e1677e0)
Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add test for the --hidden argument introduced in Oe-Core
rev 7a111ff58d7390b79e2e63c8059f6c25f40f8977.
(From OE-Core rev: 819286cdce6471ca3dd048d3950b943b28e6d222)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Running `devtool search gcc` would result in errors for virtual recipes.
The reason was the newly introduced :layer-<layername> override.
When tinfoil parsed the recipes using the parseRecipeFile() function, it
failed to properly identify the layername for virtual recipes, which
resulted in the errors.
Add tests that exercise these code paths through parseRecipeFile().
(From OE-Core rev: cf7f34f93f9f90279412ea57d6d5930854d411ba)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I believe other fixes and cleanup means the CDN tests are now working
correctly, so enable it.
(From OE-Core rev: baabe012e158e387a692b234920eeab977c27440)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test was occasionally failing for no obvious reason, so refactor
and improve:
- While waiting for the daemon, check that it is still running and
explicitly timeout after 10s when making the HTTP call.
- While waiting for the daemon to be ready, log the current state of the
daemon so we can tell if we're timing out as it is still scanning.
- This was in fact the cause of the intermittant failures, because the
TMPDIR is reused between tests and may contain a large number of
packages. Do the tests in an isolated TMPDIR to hopefully mitigate this
issue and increase the timeout to two minutes.
- Decorate the test using runqemu as such so that can be skipped in
environments without runqemu
- Add a second test that doesn't use runqemu or images, which is faster
but less realistic.
(From OE-Core rev: 88b660aaae2527736b6eccec4c952eee969e20a2)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we chdir(), do the chdir back to the original directory in a finally
block so they always run.
(From OE-Core rev: cdc40292818683b6df1c814498c7589450a163fa)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Autoconf defines this as:
"The directory for installing C header files for non-GCC compilers."
Whilst this is something autoconf does allow changing, I find it hard
to believe it has much use in the wild now and that headers don't get
split like this in reality, it would probably only be useful on really
old unixes.. The values are the same in our configuration anyway.
Drop the value and just use includedir everywhere.
(From OE-Core rev: 506c91cbc6a604a84e37e53ccff430436369802e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test fails for machines qemuarm and qemux86 because when doing
devtool modify, the default devtool branch contains the patch that match
the current configuration, so for both qemuarm and qemux86 machines the
corresponding override patch is applied and we get the following error
(for qemuarm machine):
AssertionError: 'This is a test for qemuarm\n' != 'This is a test for something\n'
- This is a test for qemuarm
? ^ ^^^^^
+ This is a test for something
? ^^^ ^^^^^
Fix the test by looking at the correct value depending on the current
machine configuration
(From OE-Core rev: fe03789d9555c025316325b559bbde40d5e770a8)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES can be used to prevent known
not-reproducible packages to make the reproducible test fail.
For example, in local.conf:
OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES = "example-not-reproducible"
To quote the original exclusion commit (4864764667)
> selftest/reproducible: add an exclusion list for items that are not yet reproducible
> [...]
> Non-reproducible excluded packages are not given to diffoscope and do not cause a
> failure, but still saved side-by-side with non-reproducible failing ones to make
> investigation easier.
NB: Patch was written by Richard with small fixes and a commit message
from Yoann.
(From OE-Core rev: bea0e3f5b8b383ae8590b874a5878f67832d6929)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
No other change.
Just cosmetic to avoid tripping the maximum line length of patchtest
when the next element will be added to the list.
(From OE-Core rev: 7c401fe421c2085f2e7fc14589ca44ec1791db03)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current code in ssh.py is terminating the ssh process that
does not finish its computation in a given timeout (when timeout
is passed), the SSHCall function is returning the process error
code.
The Openssl ssh before version 8.6_p1 is returning 0 when it is
terminated, from commit 8a9520836e71830f4fccca066dba73fea3d16bda
onwards (version >= 8.6_p1) ssh is returning 255 instead.
So for version of ssh older than 8.6_p1 when the SSHCall time out,
the return code will be 0, meaning success, which is wrong.
Fix this issue checking if the process has timeout (hence it's been
terminated) and checking if the returned code is 0, in that case
set it to 255 to advertise that an error occurred.
Add a test case excercising the timeout in the SSHTest, test_ssh
test function.
(From OE-Core rev: 948fecca1db4c7a30fcca5fcf5eef95cd12efb00)
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The writer always printed the script was written even if this
was not the case when specififying '--json-only'.
(From OE-Core rev: b2efb4c6c8bd938aac5146e14bd0c38f3f632bc4)
Signed-off-by: Jermain Horsman <jermain.horsman@nedap.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
You could reproduce an error in this test with:
bitbake core-image-minimal
bitbake tzcode-native -c cleansstate
oe-selftest -r sstatetests.SStatePrintdiff.test_image_minimal_vs_base_do_configure
since tzcode-native isn't needed once tzdata is available and isn't rebuilt
by "bitbake core-image-minimal" if it is missig. tzdata is allarch so if tzdata is
built on an x86 host, a aarch64 build machine would never build tzcode-native with
this set of calls.
Add a --runall option to the initial bitbake invocation to force these things to be
created if they're missing.
This explains why some failures were occurring on the infrastructure. With that issue
fixed, drop the hash mode change since I believe this fixes that issue. That
restriction was hurting performance, this should allow sstate reuse for the test
and improve the speed of it.
(From OE-Core rev: 8f03ac39e7fe21f3d6eca35b12b203a73a15285d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
