Rather than reporting each invalid PACKAGECONFIG with a separate error
message, report them all with one error message.
(From OE-Core rev: bf9366583f53fe2498d7aa9192ebfe6562887cf3)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes sure invalid PACKAGECONFIGs are reported also for recipes
that have no do_configure task, e.g., packagegroups.
(From OE-Core rev: d3325c384a7df54c564cae093659cf7b692629f2)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This prevented package_qa sstate from being reusable unless host uid/gid
values would match exactly (and they unfortunately do on the yocto autobuilder
worker machines which all share a 'pokybuild' user).
I noticed this when testing CDN sstate reuse, which otherwise works well.
(From OE-Core rev: 6ea8b4b10b0549c858427a8411bf2a4cd5c0eb7b)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The check to see if a provider of a given package is listed first
checks for an exact match of the provider name. However, if this match
existed, but didn't match in the task dependencies, it would not
continue to look for other providers of package. This would manifest if
one (non-virtual) recipe package RPROVIDES the name of a package
produced by another recipe.
Fix this, and also clean up the code to make it more readable by using a
function to check if a runtime dependency is in the task dependencies.
In addition, if no provider is found, list all the possible providers
instead of the last one that was looked at.
(From OE-Core rev: f13de6ab616eb1e38960a2296111febe2a9f4a28)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We'd like to start requiring some set of checks are enabled for Yocto Project
Compatible Status and to pass yocto-check-layer. Start by splitting ERROR_QA
into two sets, the ones we think can be required and the ones we know have
challenges to implement (e.g. with prebuilt binaries).
To change the required list, the YP TSC would need to approve.
(From OE-Core rev: ce2e42ace2d15fb6745437cf0a7f07d28398ca12)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, if you "bitbake XXX" and XXX depends on something else,
the do_package_qa teask for that something may not run. Users would
generally expect it to have though.
Add in the missing dependency to ensure that do_build does trigger
the right package_qa tasks.
(From OE-Core rev: e0beb64c6d3cf1d649f79a8704fb25cdf83b4a8b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a race condition when iterating directories which are being
altered whilst iterating, which is something that can and does happen
when do_package_qa runs at the same time as eg do_package_write_ipkg
(the opkg metadata is written inside the build tree). The race is that
naive code will list a directory contents and then stat() each name to
determine if its a directory or file. The classic failure that we see
is that CONTROL/ is found on a listdir but deleted by the time the stat
happens, so is incorrectly listed as a file (because it is not a
directory).
Since Python 3.5, os.walk() uses scandir() instead of listdir() which
mitigates this race by returning the file type alongside the name, so
a stat is no longer needed to identify the type.
However, cachedpath.walk() was copied from Python before this, so it
uses listdir() and has this race condition. Since I changed insane to
use cachedpath.walk()[1] I inadvertently reintroduced this race.
I believe there's actually no need to use cachedpath.walk() and a
logical fix is to simply use os.walk():
With os.walk() each directory is listed and categorised in a single
os.scandir() as the underlying syscall, getdents64, returns the type.
However, cachedpath.walk() uses os.listdir() which ignores the type
field returned and has to do a stat() on every file to determine the
type.
Thus, we should switch users of cachedpath.walk() to os.walk(): there's
no real gain in what is effectively just a prefetch for the stat cache,
but depending on what the calling code does may result in more stat()
calls than needed.
In the future we may want to redesign cachedpath to reimplement walk so
that it can also cache the DirEntry instances as returned by scandir()
as that will avoid needing to call stat() at all in many cases. However
I believe we should instead use a caching pathlib.Path instance instead.
[1] cad3c8 insane: use oe.cachedpath.CachedPath instead of os.path
(From OE-Core rev: 22e4486d65e4874bf48d89160d69118f318278e8)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The functions behave slightly differently to the functions they're
caching and the use in insane.bbclass isn't compatible. For now, to
avoid build failures, switch back to the stat calls. We may be able
to improve cachedpath or change the call sites.
(From OE-Core rev: fa771ae887ab5152f043748cf3419735831bcf7b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reorder and comment the architecture checks to make it clearer what they
are actually checking.
(From OE-Core rev: 78db9e79e1a307ffb8436e26656bfb98efb513bc)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The insane QAPATHTESTs make many os.stat() calls, the majority of which
are redundant with caching as the initial sweep does a stat() on every
entry to determine if it is a file or a directory, and from then on each
test that does further stat()s is redundant as the tree doesn't change.
Switch os.stat() and friends (os.path.isfile(), etc) to use a common
oe.cachedpath.CachedPath() instance that is shared between all of the
functions, meaning only one stat is done.
In my test case of ltp:do_package_qa, this reduces the time taken from
44s to 37s.
(From OE-Core rev: cad3c889439fd6a007debd6f2f6578f4a1e16c9c)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check whether the elf is not None first, before doing os.stat() calls
on disk. Also don't check anything that isn't a file, not just FIFOs.
(From OE-Core rev: 38454a2675f38c7db55efcb67bbb8b9fef7e0bf1)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't actively do more work:
- Exit early if there are no packages being generated
- Don't iterate repeatedly when removing CONTROL and DEBIAN
- Extend a list with another list instead of appending item by item
- Remove unused variables
(From OE-Core rev: 79ffb8896d570dd935d3aea9d28ee20b52e1674a)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the prepopulate function out of global scope, and access the
dictionary once instead of repeatedly.
This still results in each ELF being opened twice, but this avoids
opening all of the files at once and the ELFFile.open() call is fairly
fast.
(From OE-Core rev: cda3647b32703f43c4fe2af3bab977e5698633f6)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since oe-core 66f8a7 merged in 2023[1], ld sets DT_RUNPATH instead of
DT_RPATH when -rpath is specified, which we don't check for.
Update the insane tests to look at both RPATH and RUNPATH.
[1] oe-core 66f8a745668a067d8d763fa2af3e65f26c9c1ebe
(From OE-Core rev: d6c5076d179a3d5ebb74b719ec4d523c197c1918)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reduces the number of files that need to be swept by not scanning
eg the library symlinks, and means we can remove the explicit islink()
checks in many of the tests.
(From OE-Core rev: aa9ec4b5c719bf610ad953095d1111e4c257747e)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The TCLIBC value is already encoded into build paths through the triplet
so no need to encode it here where it can cause problems for allarch output
that span multiple libcs.
(From OE-Core rev: ea8c7a457a79589c35ca80b2f265799164855674)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some redundant return statements were left over from
insane: Drop oe.qa.add_message usage
(From OE-Core rev: 1e49635f802b04acad14115640ce9fcd63cc32a7)
Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want ERROR_QA to operate using the "contains" optimizations which means
accessing the variable only using the contains function.
To do this, remove usage of ALL_QA, open coding the few references to check
both WARN_QA and ERROR_QA.
Move the function table generation to a separate function where we can exclude
the ERROR_QA and WARN_QA variables since they are handled by the handle_error()
function calls.
Ensure all the chain of functions to the handle_error calls is correctly
recognised in the variable dependencies.
(From OE-Core rev: 384e9a6b2e7943b6a3ade1215ed79351c78a0b0d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now handle_error is used, we can further simplify the QA test execution
as we don't need seperate function lists for warnings and errors.
(From OE-Core rev: 6896c9fcfc57f007c0ce15f7804e79b6b88f5ded)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we improve the function dependency visibility in insane.bbclass, it
exposes some dependencies which were previously not seen causing variances
in the do_package_qa task checksums. Update vardepsexclude in a couple of
test cases to ensure the sstate hash selftests pass and the taskhashes
don't vary when we don't expect them too.
(From OE-Core rev: 9b6dae2771ed86bd2946548004f4da58e8c0b44c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop the oe.qa.add_message() usage in favour of oe.qa.handle_error() which has
code allowing it to be optimised with contains usage.
The patch also drops unused return values which we stopped using a while ago
and drops the now unneeded function parameters, generally leading to cleaner
code.
The code should be functionally equivalent.
(From OE-Core rev: 9b2eea9fd4eab4f5e12e955738db22091b91f698)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new contains code can't inspect variable references in handle_error()
calls. Expand what is effectively a hardcoded reference anyway so the code
can optimise it.
(From OE-Core rev: 51f767d92efb3daeb4aa3b91d72e6d2993cb0f46)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is currently both an incompatible-license and a
license-incompatible QA message. This is very confusing.
However, license-incompatible is only used to output a message when a
package is included in an image despite it having a license that is
normally incompatible (by using the INCOMPATIBLE_LICENSE_EXCEPTIONS
variable). To better match how it is used and to distinguish it from
incompatible-license, rename it to license-exception.
(From OE-Core rev: d309eed66f5a4a4bce082536e51207fe65725fab)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Those were removed quite some time ago:
- perms: 5da7ad1a483d0840a9a2e3b95fa62a1901be73f2
- split-strip: bcc03ea19e103f6aa93bada2f49fcc5cc7bc0790
- (compile|install)-host-path: a67e9ebfd5b8002fd4a7d8d27ff0d997817f76e1
(From OE-Core rev: 068d3821430734132c3eb70fd95461e0917fd1e8)
Signed-off-by: Michal Sieron <michalwsieron@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable the new pep517-backend warning from setuptools3, initially as a
warning so as not to break builds straight away.
(From OE-Core rev: 27597d986ad7b3a6c2d36150a163951be7c640f1)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids searching through ${S} multiple times if unimplemented-ptest
QA check is disabled (the default case).
(From OE-Core rev: 8ee42430a91d13de2b7a53c2ae04aa54bd76fad0)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
S=WORKDIR is not supported anymore, so the check is now redundant.
This reverts commit 9a2d2f7c2b7236667a6d80355f73db4c27e6582e (in OE-Core).
(From OE-Core rev: 71c4bba0235b4cd45dc88844263e7b3f8ad9f079)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Most of these warnings have been around and enabled for a long time. In particular,
buildpaths has been like this for two years. I'm aware some layers still have not
been able to resolve all the warnings but I believe that regardless, it is still
time to raise the bar. If the warnings don't get fixed, it is probably a sign
that nobody cares about the recipe and it should be dropped.
For anyone coming here to find out what changed and how to disable it, if
you are going to remove from ERROR_QA and add back to WARN_QA (or just ignore
the warnings), please do it with a layer specific override rather than making
it global. We have fixed these issues in core and intend to keep them all fixed.
If you globally disable the errors, it just means we get patches which end up
regressing things.
You can do things like:
ERROR_QA:remove:layer-mylayername = "buildpaths"
not that I'd recommend it.
Also note that the next version of Yocto Project Compatible will only be
available to layers which are not disabling some set of these errors.
(From OE-Core rev: b79b191cc43a45dde2adb61ea349b426cb2461d1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A few tests were still manually cleaning their build paths, change them
to use package_qa_clean_path().
(From OE-Core rev: f6550c3ee1bc076015d85db36b3d281e6a7ace9d)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
as this will clear WORKDIR and create race conditions
across various handling tasks
(From OE-Core rev: 1cf99ce3f79b2c96bdef5aa9b69c2b3ead7e46f1)
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
libtool auto detects the sysroot from gcc's parameters or configuration so we
don't need to pass in this configuration separately to libtool.
Whilst the option names do conflict with gcc/binutils, that is an issue for those
projects to resolve, not us. Upstream libtool did reject the patch. We can
drop this patch and simplify our code.
(From OE-Core rev: 7c8553f81bccc3e8c2bb1116ee1e89f5f8af4c9e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code that used these variable and the comment was introduced in
commit b44d32ef41 ("insane.bbclass: Portions of code were not running,
fix this and sync with OE.dev. Also add tests for bad sysroot rpaths in
binaries"). Later, in commit 17dae13fabe2 ("insane.bbclass: Fix ELF
bitsize comparison"), some of that code was removed again, but not the
variables and the comment.
(From OE-Core rev: 730d00b0d1d1d617b62900be12fa034bb41fc48b)
Signed-off-by: Emil Kronborg <emil.kronborg@protonmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit cd25e5544ca3 ("insane: use HOST_ variables, not TARGET_ to
determine the cross system") updated the variables themselves, but not
their names. To prevent confusion, match the Python variable name to the
BitBake variable name.
(From OE-Core rev: f5bebc96580ec74d10bc96b4265357ebc9bcd6ad)
Signed-off-by: Emil Kronborg <emil.kronborg@protonmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After commit cd25e5544ca3 ("insane: use HOST_ variables, not TARGET_ to
determine the cross system"), this check is no longer necessary. The
introduction of HOST_ variables ensures architecture compatibility is
correctly checked.
(From OE-Core rev: 6e1ddeb05dcd5ff77e0f5526a6e56a484daa4864)
Signed-off-by: Emil Kronborg <emil.kronborg@protonmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was never a good idea and would have mostly happened from S = WORKDIR
however explictly disallow it and error if anyone tries.
(From OE-Core rev: e3c2c1fac904bb518d85e10a2ac0177c81cbf7e8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Where a recipe uses WORKDIR as S, exit with a fatal error since the
code is no longer safe for this layout.
(From OE-Core rev: 32cba1cc916ad530c5e6630a927e74ca6f06289b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If S doesn't exist, do_qa_patch would fail. Fix the code to not fail
in this situation.
(From OE-Core rev: 4041d91b63ff2315657499e22c74ec90adbf9e19)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "libdir" QA check tries to open every file it finds as an ELF. If
it finds a dangling symlink that looks like a library by the filename it
will try to open it and fail with FileNotFoundError error. As this
dangling symlink probably points to a real file, silently absorb the
error.
[ YOCTO #13949 ]
(From OE-Core rev: f044290f98ea66f2cecfbffd7d392dbc3d986da9)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit f673d3d239799fb1ab50f4aa5d44187666aa0cd7 introduced a warning for
virtual/ being used in RPROVIDES and RDEPENDS. Make it possible to
disable the warning by removing "virtual-slash from WARN_QA.
(From OE-Core rev: 968ffdb9fee5017eecce36ce878ea604c869ce95)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should be pointing people at VIRTUAL-RUNTIME, not virtual so tweak
the warning. Try and make it clear the difference between the build
dependencies and the runtime ones.
(From OE-Core rev: 01d815aa2c0bea113fb79b51bf67c0ff90d57dd2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #14538]
Recipes shouldn't use "virtual/" in RPROVIDES and RDEPENDS. This was
addressed already in recipes in meta-oe and oe-core. Add a test for
this in insane.bbclass to ensure no regressions occur.
(From OE-Core rev: f673d3d239799fb1ab50f4aa5d44187666aa0cd7)
Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Uses the new foreach_runtime_provider_pkgdata() API to look up all
possible runtime providers of a given dependency when resolving
file-rdeps. This allows the check to correctly handle RPROVIDES for
non-virtual dependencies
(From OE-Core rev: 018fa1b7cb5e6a362ebb45b93e52b0909a782ac9)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When showing paths to the user we don't want to include the whole build
directory. Passing the package name to package_qa_clean_path strips
this completely.
(From OE-Core rev: 7f1a862d2a432f216e37bf63648bef787422a43d)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The SRC_URI tests are a better fit for the new do_recipe_qa task, move them
there.
(From OE-Core rev: 5afde8e24e74c7b73c1da312cca65b3277a6c355)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the distro feature usrmerge is set, all files from /bin are moved to
/usr/bin, i.e. /usr/bin/sh is the same as /bin/sh and should be allowed be
ignored, because it's always present.
(From OE-Core rev: 330dc61053afae8a1812bda6f9e01e2f09d1f08f)
Signed-off-by: Jörg Sommer <joerg.sommer@navimatix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
