Since oe-core 66f8a7 merged in 2023[1], ld sets DT_RUNPATH instead of
DT_RPATH when -rpath is specified, which we don't check for.
Update the insane tests to look at both RPATH and RUNPATH.
[1] oe-core 66f8a745668a067d8d763fa2af3e65f26c9c1ebe
(From OE-Core rev: d6c5076d179a3d5ebb74b719ec4d523c197c1918)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reduces the number of files that need to be swept by not scanning
eg the library symlinks, and means we can remove the explicit islink()
checks in many of the tests.
(From OE-Core rev: aa9ec4b5c719bf610ad953095d1111e4c257747e)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If user namespaces are not available (typically because AppArmor is
blocking them), alert the user.
We consider network isolation sufficiently important that this is a fatal
error, and the user will need to configure AppArmor to allow bitbake to
create a user namespace.
[ YOCTO #15592 ]
(From OE-Core rev: b6af956fe6e876957a49d4abf425e8c789bf0459)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The TCLIBC value is already encoded into build paths through the triplet
so no need to encode it here where it can cause problems for allarch output
that span multiple libcs.
(From OE-Core rev: ea8c7a457a79589c35ca80b2f265799164855674)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The sstate functions currently pull in STAMP and SSTATE_PKG which
end up pulling in DEFAULTTUNE and other variables. The location on
disk encodes all the "architecture" information we need so clean up
the dependencies of these tasks and make them non-architecture specific.
(From OE-Core rev: 65df61ccff6781906449bfea386a8dd13112a51c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are several issues:
a) pointless empty directories were being created as a path wasn't
fixed in a previous commit
b) SSTATE_PKGARCH wasn't being captured into the task signature
since it is in the ignore list by default. We want to capture
the absolute value, not the dependencies
c) with those issues fixed, cross/native issues became apparent so
those need to be fixed too.
(From OE-Core rev: f68aed302a0e4b86fb8c16a6ef4e7295bed48b86)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was deprecated with the introduction of postfunc support for tasks
in general and only used by buildhistory. Now that usage has been removed,
drop the code from sstate.bbclass. Any other users should be able to use
postfuncs too.
(From OE-Core rev: 74e08170a5584d83f5f03cd8a71978b5e0895c1d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We planned to drop SSTATEPOSTINSTFUNC some time ago with the introduction of
postfuncs. Finally get around to doing that which should make the buildhistory
code a little more readable.
Unfortunately ordering the buildhistory function calls after the sstate ones is
difficult without coding that into the sstate class. This patch does that to
ensure everything functions as expected until we can find a better way. This is
still likely preferable than the generic sstate postfuncs support since the function
flow is much more readable.
(From OE-Core rev: c9e2a8fa2f0305ef1247ec405555612326f798f8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately some recent patches caused non-deterministic output.
One input hash lead to both good and bad output and whilst that patch
has been fixed, the problematic hash 'cross' linkage remains. Bump to
a new sstate and hash equivalence version to avoid this and work from
a clean slate.
(From OE-Core rev: 639e42b9c14dff516688964dba4ab25bba7b8a55)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some redundant return statements were left over from
insane: Drop oe.qa.add_message usage
(From OE-Core rev: 1e49635f802b04acad14115640ce9fcd63cc32a7)
Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want ERROR_QA to operate using the "contains" optimizations which means
accessing the variable only using the contains function.
To do this, remove usage of ALL_QA, open coding the few references to check
both WARN_QA and ERROR_QA.
Move the function table generation to a separate function where we can exclude
the ERROR_QA and WARN_QA variables since they are handled by the handle_error()
function calls.
Ensure all the chain of functions to the handle_error calls is correctly
recognised in the variable dependencies.
(From OE-Core rev: 384e9a6b2e7943b6a3ade1215ed79351c78a0b0d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now handle_error is used, we can further simplify the QA test execution
as we don't need seperate function lists for warnings and errors.
(From OE-Core rev: 6896c9fcfc57f007c0ce15f7804e79b6b88f5ded)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we improve the function dependency visibility in insane.bbclass, it
exposes some dependencies which were previously not seen causing variances
in the do_package_qa task checksums. Update vardepsexclude in a couple of
test cases to ensure the sstate hash selftests pass and the taskhashes
don't vary when we don't expect them too.
(From OE-Core rev: 9b6dae2771ed86bd2946548004f4da58e8c0b44c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop the oe.qa.add_message() usage in favour of oe.qa.handle_error() which has
code allowing it to be optimised with contains usage.
The patch also drops unused return values which we stopped using a while ago
and drops the now unneeded function parameters, generally leading to cleaner
code.
The code should be functionally equivalent.
(From OE-Core rev: 9b2eea9fd4eab4f5e12e955738db22091b91f698)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new contains code can't inspect variable references in handle_error()
calls. Expand what is effectively a hardcoded reference anyway so the code
can optimise it.
(From OE-Core rev: 51f767d92efb3daeb4aa3b91d72e6d2993cb0f46)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The npm recipetool handler redefines the license code the could be
unified. In order to do this refactoring, extract the bits we'll
need into separate functions.
guess_license() is renamed to find_licenses() and is split into
find_license_files() and match_licenses().
(From OE-Core rev: f1ec28feaea8ea6a2df894dd4ddba561c8a04ed2)
Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is currently both an incompatible-license and a
license-incompatible QA message. This is very confusing.
However, license-incompatible is only used to output a message when a
package is included in an image despite it having a license that is
normally incompatible (by using the INCOMPATIBLE_LICENSE_EXCEPTIONS
variable). To better match how it is used and to distinguish it from
incompatible-license, rename it to license-exception.
(From OE-Core rev: d309eed66f5a4a4bce082536e51207fe65725fab)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you are running your builds inside an environment where you don't
have access to the build tree (e.g. an autobuilder where you can only
download final artifacts such as images), then debugging build failures
can be difficult - you can't examine log files, the source tree or
output files. When enabled, by default this class will retain the work
directory for any recipe that has a task failure in the form of a
tarball, and can also be configured to save other directories on failure
or always.
It puts these tarballs in a configurable location (${TMPDIR}/retained by
default), where they can be picked up by a separate process and made
available as downloadable artifacts.
(From OE-Core rev: e2030c0d747eb990b9ad10098c6b74d6f8f4e74e)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Those were removed quite some time ago:
- perms: 5da7ad1a483d0840a9a2e3b95fa62a1901be73f2
- split-strip: bcc03ea19e103f6aa93bada2f49fcc5cc7bc0790
- (compile|install)-host-path: a67e9ebfd5b8002fd4a7d8d27ff0d997817f76e1
(From OE-Core rev: 068d3821430734132c3eb70fd95461e0917fd1e8)
Signed-off-by: Michal Sieron <michalwsieron@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable the new pep517-backend warning from setuptools3, initially as a
warning so as not to break builds straight away.
(From OE-Core rev: 27597d986ad7b3a6c2d36150a163951be7c640f1)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids searching through ${S} multiple times if unimplemented-ptest
QA check is disabled (the default case).
(From OE-Core rev: 8ee42430a91d13de2b7a53c2ae04aa54bd76fad0)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
S=WORKDIR is not supported anymore, so the check is now redundant.
This reverts commit 9a2d2f7c2b7236667a6d80355f73db4c27e6582e (in OE-Core).
(From OE-Core rev: 71c4bba0235b4cd45dc88844263e7b3f8ad9f079)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Most of these warnings have been around and enabled for a long time. In particular,
buildpaths has been like this for two years. I'm aware some layers still have not
been able to resolve all the warnings but I believe that regardless, it is still
time to raise the bar. If the warnings don't get fixed, it is probably a sign
that nobody cares about the recipe and it should be dropped.
For anyone coming here to find out what changed and how to disable it, if
you are going to remove from ERROR_QA and add back to WARN_QA (or just ignore
the warnings), please do it with a layer specific override rather than making
it global. We have fixed these issues in core and intend to keep them all fixed.
If you globally disable the errors, it just means we get patches which end up
regressing things.
You can do things like:
ERROR_QA:remove:layer-mylayername = "buildpaths"
not that I'd recommend it.
Also note that the next version of Yocto Project Compatible will only be
available to layers which are not disabling some set of these errors.
(From OE-Core rev: b79b191cc43a45dde2adb61ea349b426cb2461d1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
do_create_spdx is a outlier in that it doesn't need the RSS to be
extended just because it depends on do_populate_sysroot. In fact, it
only depends on do_populate_sysroot so it can see the actual recipes
sysroot, and attempting to extend the sysroot can cause problems for
some recipes (e.g. if a recipe does do_populate_sysroot[noexec] = "1")
As such, explicitly exclude do_create_spdx from extending the sysroot
just because it depends on do_populate_sysroot.
(From OE-Core rev: 8b506f327838b885bfeb2cf144f43c8be68b8591)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent pkgconfig change and subsqeuent fixes have left the hash equivalence
server in a corrupted state with hashes linking the changes beofore and after
the pkgconfig change, breaking reproducibile builds.
Bump the appropriate version numbers to allow us to move on and avoid
build failures and corrupt equivalence data now the underlying issue
was fixed.
(From OE-Core rev: 7b2cdd257132645534642a1461fe14b835eee6e8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
LLVM has changed the ELF header dump format [1], the code in oe_libinstall
relied upon the format and processed the SONAME inside square brackets
e.g.
0x000000000000000e (SONAME) Library soname: libreadline.so.8
with older readelf from ( llvm <19 or GNU binutils objdump ) we get
0x000000000000000e (SONAME) Library soname: [libreadline.so.8]
The check in oe_libinstall will now trip over ELF files read by llvm-readelf
from llvm19+
To make it portable which works across GNU binutils and LLVM tools
switch to using objdump -p to dump the ELF file and modify the regexp
accordingly, as an aside, the post processing expression is simplified
too
[1] https://github.com/llvm/llvm-project/pull/96562
(From OE-Core rev: 11ea8dc57f275057e19db564e6c55d2baea980b0)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This recipe needs to be renamed because the "-cross-" substring in the
name triggers the cross architecture detection in sstate, but this
recipe is not actually a cross recipe.
(From OE-Core rev: 812c114a8a872ad59b19c7ffb8c1f230fc64c823)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A few tests were still manually cleaning their build paths, change them
to use package_qa_clean_path().
(From OE-Core rev: f6550c3ee1bc076015d85db36b3d281e6a7ace9d)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
as this will clear WORKDIR and create race conditions
across various handling tasks
(From OE-Core rev: 1cf99ce3f79b2c96bdef5aa9b69c2b3ead7e46f1)
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
libtool auto detects the sysroot from gcc's parameters or configuration so we
don't need to pass in this configuration separately to libtool.
Whilst the option names do conflict with gcc/binutils, that is an issue for those
projects to resolve, not us. Upstream libtool did reject the patch. We can
drop this patch and simplify our code.
(From OE-Core rev: 7c8553f81bccc3e8c2bb1116ee1e89f5f8af4c9e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code that used these variable and the comment was introduced in
commit b44d32ef41 ("insane.bbclass: Portions of code were not running,
fix this and sync with OE.dev. Also add tests for bad sysroot rpaths in
binaries"). Later, in commit 17dae13fabe2 ("insane.bbclass: Fix ELF
bitsize comparison"), some of that code was removed again, but not the
variables and the comment.
(From OE-Core rev: 730d00b0d1d1d617b62900be12fa034bb41fc48b)
Signed-off-by: Emil Kronborg <emil.kronborg@protonmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit cd25e5544ca3 ("insane: use HOST_ variables, not TARGET_ to
determine the cross system") updated the variables themselves, but not
their names. To prevent confusion, match the Python variable name to the
BitBake variable name.
(From OE-Core rev: f5bebc96580ec74d10bc96b4265357ebc9bcd6ad)
Signed-off-by: Emil Kronborg <emil.kronborg@protonmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After commit cd25e5544ca3 ("insane: use HOST_ variables, not TARGET_ to
determine the cross system"), this check is no longer necessary. The
introduction of HOST_ variables ensures architecture compatibility is
correctly checked.
(From OE-Core rev: 6e1ddeb05dcd5ff77e0f5526a6e56a484daa4864)
Signed-off-by: Emil Kronborg <emil.kronborg@protonmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds API to reverse the removal of special characters from file names so
it can be correctly done in multiple places without open-coding it.
Replace the translation done in the package_rpm.bbclass with the new API
(From OE-Core rev: 4cb7e93c624987d146aaf626ce8e99568e938a70)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In sanity.bbclass the tar version is checked as tar needs to be recent enough
for reproducible builds. Tar could also be provided by other means then gnutar,
but we mean the version of gnutar in the check. Hence we also should ensure
that the installed tar is gnutar.
[YOCTO #14205]
(From OE-Core rev: bdef30bd887cd208d7822dd7853d33e24a6b7a4c)
Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildhistory is showing issues where plaindirs installed files (such as package
listings) are not reliably being handled with installs from sstate.
The reason is that plaindirs is being handled after SSTATEPOSTINSTFUNCS
instead of before it, meaning the files visible in a non-sstate accelerated
code run are different to show from an accelerated run.
This can be observed by the missing files lists for packages in buildhistory, both
in from scratch builds and in builds from sstate. In builds where sstate is installed
over an existing build directory, the files are present though, so there is a
determinism problem.
Fix this by moving the code into sstate_install, this is the only call
site for the funciton.
Since the move needs prepdir, move that as well as it's call site,
being careful to handle the two different definitions of SSTATE_INSTDIR. The
version originally in the function was obsolete and was causing the postinstfuncs
to run in an incorrect directory. The only user is buildhistory and it wasn't
sensitive to cwd however so this happened not to cause a problem. Fix the
code to use the correct location.
(From OE-Core rev: 62ee349cf18532dac8736488752c00e89de78fcd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The only user was siteconfig which has been removed. The API was horrible
and we don't want to encourage this kind of usage.
(From OE-Core rev: cfbfd0b2e89eb71783c55a1be4a7e63e6cd82c2f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now S and B can't be set to WORKDIR, add to PSEUDO_IGNORE_PATHS unconditionally
and simplify the code.
(From OE-Core rev: 26cd2d56261827ad8d07e2145e95f82422accac2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was never a good idea and would have mostly happened from S = WORKDIR
however explictly disallow it and error if anyone tries.
(From OE-Core rev: e3c2c1fac904bb518d85e10a2ac0177c81cbf7e8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Where a recipe uses WORKDIR as S, exit with a fatal error since the
code is no longer safe for this layout.
(From OE-Core rev: 32cba1cc916ad530c5e6630a927e74ca6f06289b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change do_unpack to unpack files to a subdirectory of WORKDIR instead of WORKDIR
itself. There are several good reasons for this but it is mainly about being able
to isolate the output of the unpack task and tell the files apart from other things
which are created in workdir (logs, sysroots, temp dirs and more).
This means that when the do_unpack task reruns, we can clean UNPACKDIR and know
we have a standard point to start builds from.
It also makes code in tools like devtool and recipetool easier.
To reduce the impact to users, if a subdirectory under UNPACKDIR matches
the first subdirectory under WORKDIR of S, that directory is moved into position
inside WORKDIR. This preserves the behaviour of S = "${WORKDIR}/git",
S = "${WORKDIR}/${BPN}" and other commonly used source directory setups.
The directory is moved since sadly many autotools based projects can't cope with
symlinks in their paths.
The patch also updates reproducible and SOURCE_DATE_EPOCH handling to
match the new potential source locations. We can get rid of the horrible
list of hardcoded directories in WORKDIR to ignore from that code.
(From OE-Core rev: b84eec5c4cbf4b39d6712800dd0d2fe5337721cb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If S doesn't exist, do_qa_patch would fail. Fix the code to not fail
in this situation.
(From OE-Core rev: 4041d91b63ff2315657499e22c74ec90adbf9e19)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having the unpack directory hardcoded to WORKDIR makes it really hard to
make any changes to the unpack process to try and allow for cleanup for example.
As a first step toward unraveling the intertwined location usages, add a
variable, UNPACKDIR which is where the fetcher is asked to unpack fetched
sources. It defaults to the existing value of WORKDIR at this point.
(From OE-Core rev: e022d62ba917790af2121da57646271ef17c03fa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using tinfoil to control the build, multiple commands (serially) could
trigger an error such as:
When reparsing ....bb:do_package, the basehash value changed from ... to .... The metadata is not deterministic and this needs to be fixed.
ERROR: The following commands may help:
ERROR: $ bitbake esw-conf -cdo_package -Snone
ERROR: Then:
ERROR: $ bitbake esw-conf -cdo_package -Sprintdiff
However following these commands it was not able to be reproduced. Forcing
bitbake to dump the signatures and then running bitbake-diffsigs showed
that the value of _SSTATE_EXCLUDEDEPS_SYSROOT was being set in one run, but
was blank is a different version.
Upon inspecting the code in sstate.bbclass, one usage (without the _) is
already excludes, the leading _ version is used as a cache, only if set but
is not actually required to be defined. So ignoring the value should work
properly.
(From OE-Core rev: 4ec704ed6a1cfaf0a6c20f2038e7192e361ef590)
Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "libdir" QA check tries to open every file it finds as an ELF. If
it finds a dangling symlink that looks like a library by the filename it
will try to open it and fail with FileNotFoundError error. As this
dangling symlink probably points to a real file, silently absorb the
error.
[ YOCTO #13949 ]
(From OE-Core rev: f044290f98ea66f2cecfbffd7d392dbc3d986da9)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In testing websocket hashequivalence, corrupted sstate was injected into the
autobuilder extensively. With the new release/LTS, being able to clearly
differentiate between old and new sstate is probably desireable anyway
so bump the appropriate versions.
(From OE-Core rev: 7f107c180f592be29f57f580c60a6adbbebd7714)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
meld fallback assumes that user has a gnome desktop
its better to fallback to good old diff tool which
will most distributions on build host will carry
(From OE-Core rev: d027236eac2a136b06903d4a47adbc50ccd6b7c6)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
