If you are running your builds inside an environment where you don't
have access to the build tree (e.g. an autobuilder where you can only
download final artifacts such as images), then debugging build failures
can be difficult - you can't examine log files, the source tree or
output files. When enabled, by default this class will retain the work
directory for any recipe that has a task failure in the form of a
tarball, and can also be configured to save other directories on failure
or always.
It puts these tarballs in a configurable location (${TMPDIR}/retained by
default), where they can be picked up by a separate process and made
available as downloadable artifacts.
(From OE-Core rev: e2030c0d747eb990b9ad10098c6b74d6f8f4e74e)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Those were removed quite some time ago:
- perms: 5da7ad1a483d0840a9a2e3b95fa62a1901be73f2
- split-strip: bcc03ea19e103f6aa93bada2f49fcc5cc7bc0790
- (compile|install)-host-path: a67e9ebfd5b8002fd4a7d8d27ff0d997817f76e1
(From OE-Core rev: 068d3821430734132c3eb70fd95461e0917fd1e8)
Signed-off-by: Michal Sieron <michalwsieron@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable the new pep517-backend warning from setuptools3, initially as a
warning so as not to break builds straight away.
(From OE-Core rev: 27597d986ad7b3a6c2d36150a163951be7c640f1)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids searching through ${S} multiple times if unimplemented-ptest
QA check is disabled (the default case).
(From OE-Core rev: 8ee42430a91d13de2b7a53c2ae04aa54bd76fad0)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
S=WORKDIR is not supported anymore, so the check is now redundant.
This reverts commit 9a2d2f7c2b7236667a6d80355f73db4c27e6582e (in OE-Core).
(From OE-Core rev: 71c4bba0235b4cd45dc88844263e7b3f8ad9f079)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Most of these warnings have been around and enabled for a long time. In particular,
buildpaths has been like this for two years. I'm aware some layers still have not
been able to resolve all the warnings but I believe that regardless, it is still
time to raise the bar. If the warnings don't get fixed, it is probably a sign
that nobody cares about the recipe and it should be dropped.
For anyone coming here to find out what changed and how to disable it, if
you are going to remove from ERROR_QA and add back to WARN_QA (or just ignore
the warnings), please do it with a layer specific override rather than making
it global. We have fixed these issues in core and intend to keep them all fixed.
If you globally disable the errors, it just means we get patches which end up
regressing things.
You can do things like:
ERROR_QA:remove:layer-mylayername = "buildpaths"
not that I'd recommend it.
Also note that the next version of Yocto Project Compatible will only be
available to layers which are not disabling some set of these errors.
(From OE-Core rev: b79b191cc43a45dde2adb61ea349b426cb2461d1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
do_create_spdx is a outlier in that it doesn't need the RSS to be
extended just because it depends on do_populate_sysroot. In fact, it
only depends on do_populate_sysroot so it can see the actual recipes
sysroot, and attempting to extend the sysroot can cause problems for
some recipes (e.g. if a recipe does do_populate_sysroot[noexec] = "1")
As such, explicitly exclude do_create_spdx from extending the sysroot
just because it depends on do_populate_sysroot.
(From OE-Core rev: 8b506f327838b885bfeb2cf144f43c8be68b8591)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent pkgconfig change and subsqeuent fixes have left the hash equivalence
server in a corrupted state with hashes linking the changes beofore and after
the pkgconfig change, breaking reproducibile builds.
Bump the appropriate version numbers to allow us to move on and avoid
build failures and corrupt equivalence data now the underlying issue
was fixed.
(From OE-Core rev: 7b2cdd257132645534642a1461fe14b835eee6e8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
LLVM has changed the ELF header dump format [1], the code in oe_libinstall
relied upon the format and processed the SONAME inside square brackets
e.g.
0x000000000000000e (SONAME) Library soname: libreadline.so.8
with older readelf from ( llvm <19 or GNU binutils objdump ) we get
0x000000000000000e (SONAME) Library soname: [libreadline.so.8]
The check in oe_libinstall will now trip over ELF files read by llvm-readelf
from llvm19+
To make it portable which works across GNU binutils and LLVM tools
switch to using objdump -p to dump the ELF file and modify the regexp
accordingly, as an aside, the post processing expression is simplified
too
[1] https://github.com/llvm/llvm-project/pull/96562
(From OE-Core rev: 11ea8dc57f275057e19db564e6c55d2baea980b0)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This recipe needs to be renamed because the "-cross-" substring in the
name triggers the cross architecture detection in sstate, but this
recipe is not actually a cross recipe.
(From OE-Core rev: 812c114a8a872ad59b19c7ffb8c1f230fc64c823)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A few tests were still manually cleaning their build paths, change them
to use package_qa_clean_path().
(From OE-Core rev: f6550c3ee1bc076015d85db36b3d281e6a7ace9d)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
as this will clear WORKDIR and create race conditions
across various handling tasks
(From OE-Core rev: 1cf99ce3f79b2c96bdef5aa9b69c2b3ead7e46f1)
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
libtool auto detects the sysroot from gcc's parameters or configuration so we
don't need to pass in this configuration separately to libtool.
Whilst the option names do conflict with gcc/binutils, that is an issue for those
projects to resolve, not us. Upstream libtool did reject the patch. We can
drop this patch and simplify our code.
(From OE-Core rev: 7c8553f81bccc3e8c2bb1116ee1e89f5f8af4c9e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code that used these variable and the comment was introduced in
commit b44d32ef41 ("insane.bbclass: Portions of code were not running,
fix this and sync with OE.dev. Also add tests for bad sysroot rpaths in
binaries"). Later, in commit 17dae13fabe2 ("insane.bbclass: Fix ELF
bitsize comparison"), some of that code was removed again, but not the
variables and the comment.
(From OE-Core rev: 730d00b0d1d1d617b62900be12fa034bb41fc48b)
Signed-off-by: Emil Kronborg <emil.kronborg@protonmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit cd25e5544ca3 ("insane: use HOST_ variables, not TARGET_ to
determine the cross system") updated the variables themselves, but not
their names. To prevent confusion, match the Python variable name to the
BitBake variable name.
(From OE-Core rev: f5bebc96580ec74d10bc96b4265357ebc9bcd6ad)
Signed-off-by: Emil Kronborg <emil.kronborg@protonmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After commit cd25e5544ca3 ("insane: use HOST_ variables, not TARGET_ to
determine the cross system"), this check is no longer necessary. The
introduction of HOST_ variables ensures architecture compatibility is
correctly checked.
(From OE-Core rev: 6e1ddeb05dcd5ff77e0f5526a6e56a484daa4864)
Signed-off-by: Emil Kronborg <emil.kronborg@protonmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds API to reverse the removal of special characters from file names so
it can be correctly done in multiple places without open-coding it.
Replace the translation done in the package_rpm.bbclass with the new API
(From OE-Core rev: 4cb7e93c624987d146aaf626ce8e99568e938a70)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In sanity.bbclass the tar version is checked as tar needs to be recent enough
for reproducible builds. Tar could also be provided by other means then gnutar,
but we mean the version of gnutar in the check. Hence we also should ensure
that the installed tar is gnutar.
[YOCTO #14205]
(From OE-Core rev: bdef30bd887cd208d7822dd7853d33e24a6b7a4c)
Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildhistory is showing issues where plaindirs installed files (such as package
listings) are not reliably being handled with installs from sstate.
The reason is that plaindirs is being handled after SSTATEPOSTINSTFUNCS
instead of before it, meaning the files visible in a non-sstate accelerated
code run are different to show from an accelerated run.
This can be observed by the missing files lists for packages in buildhistory, both
in from scratch builds and in builds from sstate. In builds where sstate is installed
over an existing build directory, the files are present though, so there is a
determinism problem.
Fix this by moving the code into sstate_install, this is the only call
site for the funciton.
Since the move needs prepdir, move that as well as it's call site,
being careful to handle the two different definitions of SSTATE_INSTDIR. The
version originally in the function was obsolete and was causing the postinstfuncs
to run in an incorrect directory. The only user is buildhistory and it wasn't
sensitive to cwd however so this happened not to cause a problem. Fix the
code to use the correct location.
(From OE-Core rev: 62ee349cf18532dac8736488752c00e89de78fcd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The only user was siteconfig which has been removed. The API was horrible
and we don't want to encourage this kind of usage.
(From OE-Core rev: cfbfd0b2e89eb71783c55a1be4a7e63e6cd82c2f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now S and B can't be set to WORKDIR, add to PSEUDO_IGNORE_PATHS unconditionally
and simplify the code.
(From OE-Core rev: 26cd2d56261827ad8d07e2145e95f82422accac2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was never a good idea and would have mostly happened from S = WORKDIR
however explictly disallow it and error if anyone tries.
(From OE-Core rev: e3c2c1fac904bb518d85e10a2ac0177c81cbf7e8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Where a recipe uses WORKDIR as S, exit with a fatal error since the
code is no longer safe for this layout.
(From OE-Core rev: 32cba1cc916ad530c5e6630a927e74ca6f06289b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change do_unpack to unpack files to a subdirectory of WORKDIR instead of WORKDIR
itself. There are several good reasons for this but it is mainly about being able
to isolate the output of the unpack task and tell the files apart from other things
which are created in workdir (logs, sysroots, temp dirs and more).
This means that when the do_unpack task reruns, we can clean UNPACKDIR and know
we have a standard point to start builds from.
It also makes code in tools like devtool and recipetool easier.
To reduce the impact to users, if a subdirectory under UNPACKDIR matches
the first subdirectory under WORKDIR of S, that directory is moved into position
inside WORKDIR. This preserves the behaviour of S = "${WORKDIR}/git",
S = "${WORKDIR}/${BPN}" and other commonly used source directory setups.
The directory is moved since sadly many autotools based projects can't cope with
symlinks in their paths.
The patch also updates reproducible and SOURCE_DATE_EPOCH handling to
match the new potential source locations. We can get rid of the horrible
list of hardcoded directories in WORKDIR to ignore from that code.
(From OE-Core rev: b84eec5c4cbf4b39d6712800dd0d2fe5337721cb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If S doesn't exist, do_qa_patch would fail. Fix the code to not fail
in this situation.
(From OE-Core rev: 4041d91b63ff2315657499e22c74ec90adbf9e19)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having the unpack directory hardcoded to WORKDIR makes it really hard to
make any changes to the unpack process to try and allow for cleanup for example.
As a first step toward unraveling the intertwined location usages, add a
variable, UNPACKDIR which is where the fetcher is asked to unpack fetched
sources. It defaults to the existing value of WORKDIR at this point.
(From OE-Core rev: e022d62ba917790af2121da57646271ef17c03fa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using tinfoil to control the build, multiple commands (serially) could
trigger an error such as:
When reparsing ....bb:do_package, the basehash value changed from ... to .... The metadata is not deterministic and this needs to be fixed.
ERROR: The following commands may help:
ERROR: $ bitbake esw-conf -cdo_package -Snone
ERROR: Then:
ERROR: $ bitbake esw-conf -cdo_package -Sprintdiff
However following these commands it was not able to be reproduced. Forcing
bitbake to dump the signatures and then running bitbake-diffsigs showed
that the value of _SSTATE_EXCLUDEDEPS_SYSROOT was being set in one run, but
was blank is a different version.
Upon inspecting the code in sstate.bbclass, one usage (without the _) is
already excludes, the leading _ version is used as a cache, only if set but
is not actually required to be defined. So ignoring the value should work
properly.
(From OE-Core rev: 4ec704ed6a1cfaf0a6c20f2038e7192e361ef590)
Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "libdir" QA check tries to open every file it finds as an ELF. If
it finds a dangling symlink that looks like a library by the filename it
will try to open it and fail with FileNotFoundError error. As this
dangling symlink probably points to a real file, silently absorb the
error.
[ YOCTO #13949 ]
(From OE-Core rev: f044290f98ea66f2cecfbffd7d392dbc3d986da9)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In testing websocket hashequivalence, corrupted sstate was injected into the
autobuilder extensively. With the new release/LTS, being able to clearly
differentiate between old and new sstate is probably desireable anyway
so bump the appropriate versions.
(From OE-Core rev: 7f107c180f592be29f57f580c60a6adbbebd7714)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
meld fallback assumes that user has a gnome desktop
its better to fallback to good old diff tool which
will most distributions on build host will carry
(From OE-Core rev: d027236eac2a136b06903d4a47adbc50ccd6b7c6)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace `${HOST_PREFIX}readelf` with `${READELF}`.
When utilizing llvm for compiling packages, the invocation of GNU readelf
will consistently occur if the oe_soinstall and oe_libinstall functions,
which are defined in utils.bbclass, are called. This behavior is unfriendly
to llvm. So prefer `${READELF}` over `${HOST_PREFIX}readelf`.
(From OE-Core rev: daecdd577213da0c045e45c47e7acbd279956d41)
Signed-off-by: lixiaoyong <lixiaoyong19@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With newer glibc versions, pthread was merged into the main library so there is
no longer any need to link with -lpthread. On newer systems this means that linker
flag is dropped from places like pkgconfig .pc files. If uninative then uses this
newer library on an older system, the flag is missing and linker errors about missing
symbols occur.
Adding the linkage unconditionally to our uninative linking flags avoids that problem
at a potential cost of slight over linking.
(From OE-Core rev: 5ec2bc7ef663db4c04c85eee518297d442556481)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The llvm github repo appears to see a lot of churn on branches we're not
interested in and is in general huge and unreliable to clone.
Switch to shallow clones to improve the user experience.
(From OE-Core rev: 6cf98a4954751977fb02fb2024ef842865352138)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
postinst-useradd-* haven't been running in order of dependency.
This patch is reworked from Piotr Łobacz's patch and fixes:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15084https://bugzilla.yoctoproject.org/show_bug.cgi?id=13904
basepasswd_sysroot_postinst in base-passwd can install postinst-useradd-*
scripts with any order. Sometimes this means, for example a useradd postinst
will attempt to run without the corresponding group postinst causing errors.
This patch ensures that we first run groupadd, then useradd and then
group membership.
[RP: Tweaked to avoid removing previous fixes and for whitespace/style issues
Also ensure the scripts are changed to execute with -e to highlight errors]
(From OE-Core rev: 322ef726132a47d977d2c6ee41de5358f1e85994)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
Signed-off-by: Jan Górski <j.gorski@welotec.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit f673d3d239799fb1ab50f4aa5d44187666aa0cd7 introduced a warning for
virtual/ being used in RPROVIDES and RDEPENDS. Make it possible to
disable the warning by removing "virtual-slash from WARN_QA.
(From OE-Core rev: 968ffdb9fee5017eecce36ce878ea604c869ce95)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes use of the oe.patch.GitApplyTree.commitIgnored() function to
create commits that shall be ignored by `devtool finish`.
(From OE-Core rev: 2393dc35a93546eccee0dd313a6927c7d1512c3b)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The purpose of the change is to never sign a package not created by
the build itself.
sstate_create_package is refactored into Python and re-designed
to handle signing inside the function. Thus, the signing should never apply
to existing sstate packages. The function is therefore renamed into
sstate_create_and_sign_package.
The creation of the archive remains in a separate shellscript function.
Co-authored-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
(From OE-Core rev: ba223f8fff19ea59440d56cf3fe46200f3f71e22)
Signed-off-by: Tobias Hagelborn <tobias.hagelborn@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some packages may contain localized files not located in default path
${datadir}/locale. Add the new variable LOCALE_PATHS to allow a recipe
to define extra paths or even fully override the scanned directories.
LOCALE_PATHS is set at ${datadir}/locale by default to keep the exact
same behavior for the recipes which did not need modification.
(From OE-Core rev: 0ffc7cf01225743789ac30dd325fca05b9203be1)
Signed-off-by: Jonathan GUILLOT <jonathan@joggee.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Older meson versions save temporary scripts in /tmp.
Similarly some recipies also do that (e.g. ccan in sbsigntool).
As this can lead to unexpected build failures with no simple way
to workaround, make such setup a fatal error.
(From OE-Core rev: ee93a8e89322143252040bd5bc99259c5efff831)
Signed-off-by: Michal Sieron <michalwsieron@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should be pointing people at VIRTUAL-RUNTIME, not virtual so tweak
the warning. Try and make it clear the difference between the build
dependencies and the runtime ones.
(From OE-Core rev: 01d815aa2c0bea113fb79b51bf67c0ff90d57dd2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the new infraroot.at mirror to the MIRRORS string for https sources
since infradead.org does not support https and certain proxy configs
cause this fetch to fail.
Mirror info fetched from the following announcement:
https://lists.infradead.org/pipermail/linux-mtd/2023-August/100922.html
(From OE-Core rev: ce8f7bb81598aecafe486a461c94254b97518832)
Signed-off-by: Randolph Sapp <rs@ti.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The intention of the mirror code was to allow faster cloning of glibc.
This wasn't quite working since the full clone was needed by glibc-tests
or cross-localedef-native. I noticed this when I updated a build and it
was trying to unpack glibc but fetch cross-localedef-native.
Add entries to cover these and restore the faster cloning for glibc
and realted recipes.
(From OE-Core rev: 7891b5fb96f757274bebbb77df634e7af9054298)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* e.g. for DISTROs which define extra user-defined fields with
PACKAGE_ADD_METADATA/PACKAGE_ADD_METADATA_RPM
as undefined fields in packagedata are fatal error for rpmbuild:
"error: line 9: Unknown tag: Author: Unspecified"
as shown in:
http://errors.yoctoproject.org/Errors/Details/751706/
with "Author" field added with:
PACKAGE_CLASSES = "package_rpm"
PACKAGE_ADD_AUTHOR_METADATA = "test-author"
PACKAGE_ADD_METADATA = "Author: ${PACKAGE_ADD_AUTHOR_METADATA}"
to fix rpm build you can use:
RPMBUILD_EXTRA_PARAMS = " --define '_Author Author'"
keep in mind that this doesn't cause this Author field to be
added in .rpm, it just avoids the BUILDSPEC failure.
and for ipk build:
OPKG_MAKE_INDEX_EXTRA_PARAMS = "-f"
alternatively you can avoid additional packagedata fields ending
in the package manager (if you use them only with buildhistory
or packagedata) with:
PACKAGE_ADD_METADATA_RPM = ""
PACKAGE_ADD_METADATA_IPK = ""
PACKAGE_ADD_METADATA_DEP = ""
(From OE-Core rev: 31030e7a19a27ad424b997fad36ee4f633ce0d63)
Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since f3eaeeb734
rpm does its own parsing of /etc/passwd and /etc/group instead of relying on getpwnam() and friends.
This has an unfortunate effect of leaking build host uid/gid values for users and groups
into the cpio header inside rpm file (where previously those were always zero).
Installation of rpm packages relies on rpm header to set files ownership, and that
is a different structure that is build from .spec information, so we can avoid host
contamination by setting the paths to something bogus.
(From OE-Core rev: ac8ea64bb39a5e56599e078c7e6cd056a2aa4144)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
