When using tinfoil to control the build, multiple commands (serially) could
trigger an error such as:
When reparsing ....bb:do_package, the basehash value changed from ... to .... The metadata is not deterministic and this needs to be fixed.
ERROR: The following commands may help:
ERROR: $ bitbake esw-conf -cdo_package -Snone
ERROR: Then:
ERROR: $ bitbake esw-conf -cdo_package -Sprintdiff
However following these commands it was not able to be reproduced. Forcing
bitbake to dump the signatures and then running bitbake-diffsigs showed
that the value of _SSTATE_EXCLUDEDEPS_SYSROOT was being set in one run, but
was blank is a different version.
Upon inspecting the code in sstate.bbclass, one usage (without the _) is
already excludes, the leading _ version is used as a cache, only if set but
is not actually required to be defined. So ignoring the value should work
properly.
(From OE-Core rev: 4138f0a048149195d39347d97f8155199b6cf0cb)
Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4ec704ed6a1cfaf0a6c20f2038e7192e361ef590)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
In testing websocket hashequivalence, corrupted sstate was injected into the
autobuilder extensively. With the new release/LTS, being able to clearly
differentiate between old and new sstate is probably desireable anyway
so bump the appropriate versions.
(From OE-Core rev: 7f107c180f592be29f57f580c60a6adbbebd7714)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The purpose of the change is to never sign a package not created by
the build itself.
sstate_create_package is refactored into Python and re-designed
to handle signing inside the function. Thus, the signing should never apply
to existing sstate packages. The function is therefore renamed into
sstate_create_and_sign_package.
The creation of the archive remains in a separate shellscript function.
Co-authored-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
(From OE-Core rev: ba223f8fff19ea59440d56cf3fe46200f3f71e22)
Signed-off-by: Tobias Hagelborn <tobias.hagelborn@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The vardepsexclude was subtly wrong in that it referenced STATE_MANMACH
when the actual variable name is SSTATE_MANMACH.
(From OE-Core rev: ec5054396f7fafea2a071d2695ae111fc585d6e6)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We currently use mkdir -p to create missing parent directories within SSTATE_DIR.
Reading the man page for mkdir mentions that parent directories are created with
the current umask, *not* the mode passed upon the commandline.
We could fix this by setting and resetting the umask but since we already have
decent python code able to do this, move to using that injecting a python function
into the chain of functions already present.
This should help fix the occasional sstate directory creation with the wrong
permissions.
[YOCTO #14385]
(From OE-Core rev: ae642a4b038c6946e6c8aa9778bf09099d938a31)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With recent changes to runqueue in bitbake, this horrible hack is no longer needed
and we can drop it.
(From OE-Core rev: f7f50b6013f7b636fe46448b1cbe1473de891527)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commands like "bitbake XXX -S printdiff" search for sstate files but don't download
them. This means that local files aren't touched as the download code would do, meaning
the sstate cleanup scripts can delete them. This can then lead to obtuse build failures.
Have the search code touch local files in the same way as the main code paths would to
avoid these files disappearing.
Move the function to a common touch() function in lib/oe instead of duplicating code.
(From OE-Core rev: a27fc0bd5706ab5b9c68a0271fcf57377a678cdf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When originally implemented, overlapping symlinks in DEPLOY_DIR were common. That
is no longer the case and these overlapping links are causing bugs in other areas
(e.g. bug 14123).
Therefore start showing errors for overlapping symlinks in shared areas. Whilst here,
fix a broken file reference in the grep command to match current file layouts and
update the message shown to users to match current times. Most of the message content
is obsolete now due to other advances and changes in the way the staging code
now works.
(From OE-Core rev: 15f98f72e1730fed5164fcd539737a7a7b96a1a2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently DEPLOY_DIR/licenses is added to SSTATE_ALLOW_OVERLAP_FILES. This
leads to bugs since when one MACHINE_ARCH recipes is cleaned, it removes the
files for another which then results in later build failures as license files
disappear.
The solution is to include SSTAGE_PKGARCH in the path names to the license files.
That does mean a search has to be used to find the correct license files for a
given PN but that can be done via SSTATE_ARCHS.
The implication for other tools is the layout has changed so tools will need to
adapt to the new paths. The benefit is no more strange build failures such as from
patterns like:
MACHINE=qemux86-64 bitbake core-image-minimal
MACHINE=genericx86-64 bitbake core-image-minimal
MACHINE=qemux86-64 bitbake linux-yocto -c clean
MACHINE=genericx86-64 bitbake core-image-minimal -C rootfs
[YOCTO #14123]
For anyone finding this commit, I'd question how much people should be relying on
this code for tooling and suggest the SPDX manifests should be the preferred data
format going forward anyway.
(From OE-Core rev: 1a4ab9fc26659507e678e87312b514e8ea515673)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When building multilibs, we need to inject the multilib sstate pkgarch
into SSTATE_ARCHS so the list forms a complete search path. Add a tweak
to do this.
PACKAGE_ARCH defaults to TUNE_PKGARCH so this is equivalent and just
guards against recipes changing the value which may have other unwanted
side effects.
(From OE-Core rev: 37126ffc7ccbd3df57ebbd8e581d158f03bb3b4c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PACKAGE_ARCH gets refined by each target so this value isn't valid in all contexts.
Tweak to use underlying variables to build it so that it remains valid in wider
contexts and works with new usage in sdpx and license classes.
(From OE-Core rev: ca90379147cb75755d205670781d8922161e9317)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that SRCPV isn't needed we can simplify things in a few places...
(From OE-Core rev: 843f82a246a535c353e08072f252d1dc78217872)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A recent package.py change has highlighted some corruption issues with -dev
pkgconfig package dependencies. Bump the output versions to trigger a rebuild
and ensure everything is consistent.
Take the opportunity to also drop all HASHEQUIV_HASH_VERSION entries since the
main version is changing.
(From OE-Core rev: f45ddfbf007de858327eef0ffefd5840ef4c69b8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add in the new 'check' calls which mean that the code can exit from longer
running code loops if requested by the user. This covers sstate checks and
sstate manifest cleanup code which currently couldn't be interrupted by the
user from the UI.
Increase the minimum bitbake version which brings in this API.
(From OE-Core rev: cb6c50689175668b66ffbe09ff8f250ba9a9034b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For the case of a global shared state cache and a local sstate cache
with a mix of signed and un-signed entries, there is a case where
a .sig is missing locally, but may exist in the global sstate cache.
For this case, do not just fail sstate fetch, but rather backfill
the .sig file from the global sstate cache.
If this case is detected, re-run the fetch operation to get the
global .sig file (if it exists).
(From OE-Core rev: d5ea1a5396bf9fd4303cae46bc0e042be8de8d67)
Signed-off-by: Tobias Hagelborn <tobias.hagelborn@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Transition to ThreadPoolExecutor (eb6a6820928472ef194b963b606454e731f9486f)
broke the
| Checking sstate mirror object availability: ...
progress bar because the removed 'thread_worker' was still referenced
in an asynchronous function. As the result of the future is never
read, the resulting backtrace is silently discarded.
Replace the information given to 'ProcessProgress' by a counter.
(From OE-Core rev: eacd52216815e0c75e5f257b6db792afd8b1b6df)
Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
do_deploy_archives tasks don't need their dependencies so we can optimistion
this as we do for some other tasks.
(From OE-Core rev: 3dd9f6e398844380d3765c54d35afe0d2ccf82e7)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After a recent fix in bitbake, the checksums may be checked for local file
fetches at an earlier code point. The underlying recipe checksums won't
match sstate values so ensure these aren't set via SRC_URI flags when
reconfiguring the fetch for sstate.
(From OE-Core rev: 315cfebe77a46c868f307919d3540046a6cb8587)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move classes to classes-global or classes-recipe as appropriate to take
advantage of new bitbake functionality to check class scope/usage.
(From OE-Core rev: f5c128008365e141082c129417eb72d2751e8045)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
