The existence of missed files on the local sstate cache is the
condition to search for it on the sstate mirror.
The missed collection are populated when we search the files
on the local sstate cache.
- When we have sstate mirrors we don't need to iterate all files
again because we already know what's missing.
- When we found a file on the sstate mirror is because of they
are missing on the local sstate mirror.
So avoid checking if the hash exists on the missed collection.
- When we can't find the file on the sstate mirror we don't need
to add the hash again to the missed collection as it already exists.
(From OE-Core rev: 46e1f9a900e2be1abeda0dc540fc14040ecd0d7c)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We only have the progress bar when there are more than 100 objects.
So we can check for this and store the result to show the progress bar.
(From OE-Core rev: 18e7a77aa1626e11820804ad64b86a32f97da2ca)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We exclude allarch packagegroups from rebuilding when their dependencies change.
The reasoning is that we are just depending on a name so having these rebuild
lots is just pointless and inefficient. We also don't want them duplicated for
multiple machines for efficiency.
In general this works fine, as long as the package names don't change. That
is also rare but there is one corner case which does catch users out - debian
package renaming. When this does break, users question sstate and so on and
lose faith in the system even if this is a known choice we made.
This commit adds an error message if an allarch packagegroup depends on any
package which shows package renaming in action (through the PKG variable
being set).
If you run into this issue you either need to remove the dependency from the
packagegroup or mark the packagegroup as tune specific, i.e. set:
PACKAGE_ARCH = "${TUNE_PKGARCH}"
before the packagegroup inherit.
[YOCTO #7298]
(From OE-Core rev: 5bf3e447d2f5064495d83a8fad30229bcf1ecc9b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Optionally allow to set QB_KERNEL_ROOT to e.g. "/dev/vda ro" to mount the
rootfs reay-only in Qemu.
(From OE-Core rev: 448eb1fd21287ba16b17e9402ce040b86ae3638c)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
kernel.org now has a mirror of the downloads.yoctoproject.org sources
archive so include this in our mirrors list.
(From OE-Core rev: f602b6c2046bbc52a95dcc68a754f1cbb2db6761)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
meson now does not allow unknown options which means we have to be
mindful of adding -Dintrospection unconditionally and provide an option
to recipes to set GIR_MESON_OPTION = "" and when recipe does that then
do not add -D<Dintrospection> option to meson. This will help recipes to
use meson 60.x+ just by adding
GIR_MESON_OPTION = ""
(From OE-Core rev: c75a9439154834d45b2be1ac95682586b4409fb4)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Cc: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Meson 0.60 onwards no longer simply warns when an unrecognised option is
passed, it errors out instead. This means we can remove the logic in
insane.bbclass to check for the warnings.
(From OE-Core rev: 472acf29abd4cdc9d8ef10a1940a541275d508cf)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
uninative works via hashes and doesn't need the version in the tarball name but
it does make things easier to inspect in DL_DIR. There were reasons such as
ease of publication of the build tarballs but we can handle those differently
now and the signature issues from the early code aren't an issue now. From 3.4
onwards we can use a version'd name.
[YOCTO #12970]
(From OE-Core rev: dadba70d6a24d8ebb5576598efffa973151c7218)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Since commit ce0579dc in bitbake (fetch2: Revert the regexp removal
for the type field and instead anchor regexp) the type regexp has
been automatically anchored at the end.
* Since commit 044fb04d in bitbake (fetch2: Allow whitespace only mirror
entries) there is no need to separate the entries in PREMIRRORS with
"\n".
(From OE-Core rev: 1edc5e689d54b788c35ece14e0dbd76ec545f3ee)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Since commit ce0579dc in bitbake (fetch2: Revert the regexp removal
for the type field and instead anchor regexp) the type regexp has
been automatically anchored at the end.
* Since commit 044fb04d in bitbake (fetch2: Allow whitespace only mirror
entries) there is no need to separate the entries in MIRRORS with "\n".
(From OE-Core rev: ec806f1e3b08d73524515aa83c5ee8dea7a40215)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we allow the kernel package name to be varied, we should
allow those packages to provide virtua/kernel, which allows them
to be used as primary kernels.
This change drops the use of regex, since any regex would enforce
naming (and limit what can be a provider) and add complexity./
There are currently no recipes that I found, that inherit kernel
that are not kernel recipes (kernel-base, etc, provide other
facilities), so making this provides simpler shouldn't cause
problems.
Multiple kernel providers can be dealt with in the same way as
any multiple provider.
[YOCTO: #13172]
(From OE-Core rev: 365dc1471a1b67c45de58aadf29844a9ff83d30f)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
squash with provider
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
lnr is a script in oe-core that creates relative symlinks, with the same
behaviour as `ln --relative --symlink`. It was added back in 2014[1] as
not all of the supported host distributions at the time shipped
coreutils 8.16, the first release with --relative.
However the oldest coreutils release in the supported distributions is
now 8.22 in CentOS 7, so lnr can be deprecated and users switched to ln.
[1] 6ae3b85eaffd1b0b6914422e8de7c1230723157d
(From OE-Core rev: 1ca455a98de4c713f58df0a537d4c982d256cd68)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At the last nas outage, we realised that we don't have good mirrors of the
uninative tarball if our main system can't be accessed. kernel.org mirrors
some Yocto Project data so we've ensured uninative is there. Add the appropriate
mirror url to make use of that.
(From OE-Core rev: 1833cb0c5841afafb468b963b74b63366b09a134)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
SCons has a slightly unusual implementation of out-of-tree builds where
there is no standard way of doing it. However, if a recipe sets B to
the idiomatic ${WORKDIR}/build and passes the right arguments so that this
is used then scons will fail as it is trying to find the SConstruct file
in ${B} not ${S}.
Always pass --directory=${S} to scons so that the SConstruct file can be
found, and don't call --clean if out-of-tree builds are being used as
we would have just deleted the entire build tree already.
(From OE-Core rev: 88d77d82ee506576988936e06b8d624879a80f2e)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch is a fixup for 676757f "sstate: fix touching files inside pseudo"
running the 'id' command inside the sstate_unpack_package
function shows that this funcion run inside the pseudo:
uid=0(root) gid=0(root) groups=0(root)
The check for [ -w ${SSTATE_PKG} ] and [ -O ${SSTATE_PKG}.siginfo ]
will always return true and the touch can fail when the real user
don't have permission or in readonly filesystem.
As the documentation refers:
- the file test operator "-w" check if the file has write permission
(for the user running the test).
- the file test operator "-O" check if you are owner of file
We can avoid this test running the touch and mask any return errors
that we have.
(From OE-Core rev: 29fc85997ade490ae46ffca37ef8e1a56957c876)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the "Organization" field for SBOMs is hard-coded in
create-spdx. Create a new variable SPDX_ORG to make this field more
generic.
(From OE-Core rev: f239814f3f5d9bd54de54b0f2a5081067336e32b)
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The empty-dirs QA check verifies that all directories specified in
QA_EMPTY_DIRS are empty. It is possible to specify why a directory is
expected to be empty by defining QA_EMPTY_DIRS_RECOMMENDATION:<path>,
which will then be included in the error message if the directory is
not empty. If it is not specified for a directory, then "but it is
expected to be empty" will be used.
(From OE-Core rev: a454fda9c3d6b11cfdc54a49a7bb3f8a76a5ed58)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, when shortening sstate filenames, the reserved
characters for .siginfo were not considered, when siginfo=False,
resulting in differently shortend filenames for the sstate and siginfo
files. With this change, the filenames of the truncated sstate and
siginfo files have the same basename, just as is already the case for
untruncated filenames.
Making sure that the .siginfo files always have the filename of the
corresponding sstate file plus its .siginfo suffix, also when being
truncated, makes it easier to manage the sstate cache and an sstate
mirror outside of Bitbake/Yocto.
(From OE-Core rev: c2e0e43b7123cf5149833e0072c8edaea3629112)
Signed-off-by: Manuel Leonhardt <mleonhardt@arri.de>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
qemu-native 6.1.0 requires at least gcc v7.5:
ERROR: You need at least GCC v7.5 or Clang v6.0 (or XCode Clang v10.0)
(From OE-Core rev: 0137c3da41f8d9328888167fb497539436ead4e9)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Application can depend on several overlayfs mount points. Provide a
systemd unit application can depend on to make sure all overlays are
mounted before it is started to avoid any race conditions
(From OE-Core rev: b38e194db0c6825f28c56123cf88af94d3f52beb)
Signed-off-by: Bruno Knittel <Bruno.Knittel@bruker.com>
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should default to xz or zstd instead of gz, pick xz for now as it is more tested.
(From OE-Core rev: 658f274c161ec67b91f133ee9b5c3767c2925787)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is uncertainty about the default branch name in git going forward.
To try and cover the different possible outcomes, add branch names to all
git:// and gitsm:// SRC_URI entries.
This update was made with the script added to contrib in this patch which
aims to help others convert other layers.
(From OE-Core rev: b51c405faf6f8c0365f7533bfaf470d79152a463)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, there is no clean way to pass extra parameters to the go tool link,
which is passed by the go build ldflags flag, the append needs to happen inside
the quotes of the ldflags parameter
See [YOCTO #14554].
Add a variable to allow adding extra parameters to -ldflags in the GO_LDFLAGS
variable, one of the main usecases is setting the application version.
For example, adding to the recipe something like
GO_EXTRA_LDFLAGS="-X main.Version=v1.0.0"
or
GO_EXTRA_LDFLAGS="-X main.Version=${PV}"
(From OE-Core rev: eaa7a61dab9a1d7bb039f16abdd9aacb44faa595)
Signed-off-by: Ahmed Hossam <Ahmed.Hossam@opensynergy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
sstate.bbclass no longer removes empty directories to avoid a race (see
commit 4f94d929 "sstate/staging: Handle directory creation race issue").
Unfortunately Python apparently treats an empty egg-info directory as if
the version it previously contained still exists and fails if a newer
version is required, which Meson does. To avoid this, make sure there
are no empty egg-info directories from previous versions left behind.
(From OE-Core rev: 47d9d90b4ec7d04d6f3f1a9b97c0ab7f1264a88e)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The exceptions generated with repre are more detailed but escaped the
newlines making them unreadable. Fix this.
(From OE-Core rev: 26a7012e6e17e6b4b4478a25b1b2d5608fe77cfc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When rebuilds are triggered, autoconf-native can fail with:
| DEBUG: Executing shell function update_gnu_config
| install: cannot stat '[BUILDPATH]tmp/work/x86_64-linux/autoconf-native/2.71-r0/recipe-sysroot-native/usr/share/gnu-config/config.guess': No such file or directory
which is due to update_gnu_config running before extend_recipe_sysroot.
This only happens rarely since usually the prepare_recipe_sysroot
function would already have set things up and only in the invalidated
task hash cases does this rebuild in this way from configure only.
Fix the code to prepend this function instead of appending which
resolves the ordering issue.
(From OE-Core rev: b9535f513366536b13d0522058f517d2e04451b5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
do_populate_sdk was added to SSTATETASKS, but had no _setscene task
created to allow it to actually run from sstate. Add it so that SDKs can
be restored from sstate.
Note that like do_image_complete, do_populate_sdk is marked with
SSTATE_SKIP_CREATION by default so sstate is not used for them; adding
this task will allow it to work if the user overrides this default
though.
(From OE-Core rev: 1f204592903a2fd9375b0f3c9c52e7dde0467460)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
running the 'id' command inside the sstate_create_package
function shows that this funcion run inside the pseudo:
uid=0(root) gid=0(root) groups=0(root)
The check for touch files [ ! -w ${SSTATE_PKG} ]
will always return true and the touch can fail
when the real user don't have permission or
in readonly filesystem.
As the documentation refers, the file test operator "-w"
check if the file has write permission (for the user running the test).
We can avoid this test running the touch and mask any return errors
that we have.
(From OE-Core rev: f6e7445c94443544e92fda97a017ce93393c5f84)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the function multiubi_mkfs doesn't allow using anything else than
UBIFS over UBI.
Here, we propose to introduce two variables that allow to customize the ubinize
configuration and the image dependency, in order to be able to build UBI
images using the ubiblock abstraction layer and block filesystems.
For example, with this change it is possible to build a UBI volume using a
compressed squashfs, with the following configuration:
UBI_VOLTYPE ?= "static"
UBI_IMGTYPE ?= "squasfs-lz4"
(From OE-Core rev: 6f0c6a0cc9b4685716f0991ce350d046810eb8ec)
Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need extra python collections to count the found files
on the sstate cache and sstate mirrors.
The main found collections provides all the files that were found.
Then we only need to store the nunber of files that found on the
local sstate cache and the files that found on the sstate cache
mirror is derived from that.
(From OE-Core rev: 0fe71f31593e87b9188b388860d2ba94bf8266bc)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Even if initramfs_bundle_path was used, a wrong compression was reflected
in output its template file. Use linux.bin as universal kernel image.
The linux.bin file covers both cases because it's beying created from
vmlinux.
We know, that vmlinux is created inside compressed directory already,
so no external compression will be used.
(From OE-Core rev: 27fbbeaf972bb7e2535c1b23375cfa9d66b69db6)
Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Walter Schweizer <walter.schweizer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- use bash variable notation without {} where possible
- just to make sure it looks like bash variable not bitbake variable one
- fix indent style in "cat" commands
- replace "! -z" -> "-n"
- make debug info in ramdisk section creation more verbose
(From OE-Core rev: f44bb458884da64356ee188917094b5515d3b159)
Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If checkstatus returns an exception we should silently handle
FetchError, as this means the fetch failed for 'normal' reasons such as
file not found. However, other exceptions may be raised, and these
should be made visible.
(From OE-Core rev: 50d99faf88a1d82cbd939b9bd6e33ebed2b1ffd8)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the same WARN_WA and ERROR_QA variables as insane.bbclass to allow
individual recipes, the distro or other configuration to determine
whether the various detected license errors should be treated as a
warning (as now) or as an error.
oe.qa.handle_error isn't immediately fatal, so oe.qa.exit_if_errors must
be called at the end of do_populate_lic to fail the task.
(From OE-Core rev: bb164adca94b5a43751aabe6b6d702a3d60dfdc7)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extract package_qa_write_error, package_qa_handle_error and
package_qa_add_message functions from insane.bbclass to lib/oe/qa.py and
drop the package_qa_ prefixes.
Update various bbclasses to use the new functions. No import is required
since base.bbclass puts oe.qa in OE_IMPORTS.
Stop requiring callers to manually track whether a fatal error has been
encountered via a "sane" flag. Instead replace the QA_SANE variable with
QA_ERRORS_FOUND and call oe.qa.exit_if_errors or
oe.qa.exit_with_message_if_errors at the end of each task.
Inspired by discussion resulting from
https://lists.openembedded.org/g/openembedded-core/message/156793 and
https://lists.openembedded.org/g/openembedded-core/message/156900
(From OE-Core rev: f0ad152ef4cc15c042bc9eeefb6af096d054b220)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Add extra SRC variables to uboot-config.class for source cmd file
* Add DEPENDS on u-boot-mkimage-native if UBOOT_ENV_SUFFIX is scr
* Compile cmd -> scr in do_compile if UBOOT_ENV_SUFFIX is scr
(From OE-Core rev: 0ea02ca5c1fc4e15f640b1c26c0a5ce34fc08c05)
Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In 2017 we added changes to pass the BUILD_CFLAGS into the kernel
via BUILD_CC. This isn't really correct and the upstream kernel now has
places to pass build cflags, ldflags and more. Update our kernel
make flags to correctly use the kernel's variables. This addresses
concerns raised by kernel developers.
If this breaks some usecase please report it so we can work out how
to fix it properly.
(From OE-Core rev: 7fd06a57a1d91d8534721923f6e3951ec8220cec)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want things to be reproduicble and the variable doesn't really change
much any more. Drop the remaining uses and make those code paths always
active.
(From OE-Core rev: d15fb02c7ee7da50e322d74bc6a545234e20c7f3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change the "binary reproducibility" configuration within the kernel to
work off a separate variable, defaulting to reproducible builds.
This allows kernel developers wanting timestamps in their images to enable
it easily and clearly without changing the rest of the reproduciblity
code which they likely don't need to change anyway.
(From OE-Core rev: 0725ca18af7a2835aeb9616592a45ead2ee87987)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reproducibility is here to stay and needs to be part of our default workflow.
Move the remaining code to base.bbclass so it is always a first class citizen
and it is clear people need to be mindful of it.
(From OE-Core rev: abb0671d2cebfd7e8df94796404bbe9c7f961058)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
