* otherwise it ends '<unknown>' inside esdk, because of parsing order:
# $METADATA_REVISION [3 operations]
# set /OE/build/test-D/conf/local.conf:43
# "f2da54ef432eac89b0f18eaad68e602b6990b5de"
# immediate /OE/build/test-D/layers/poky/meta/classes/metadata_scm.bbclass:9
# "${@oe.buildcfg.detect_revision(d)}"
# set /OE/build/test-D/layers/poky/meta/classes/metadata_scm.bbclass:10
# [vardepvalue] "${METADATA_REVISION}"
# pre-expansion value:
# "<unknown>"
METADATA_REVISION="<unknown>"
* This causes base-files.do_install and following tasks to have different
signatures between esdk and the build directory where this esdk was created:
bitbake-diffsigs {test-D,poky/build-uninative-disabled}/tmp/stamps/qemux86_64-poky-linux/base-files/*do_install*sigdata*
NOTE: Starting bitbake server...
basehash changed from 5b6981cf58bfd57d416b0e31611b73a26baae635dd1ac31c08d46f95064c3ffc to dbdce042da4d7813d632b6d1cc87a16f728ad20e55fecbc392830e6acf72babd
Variable METADATA_REVISION value changed from '<unknown>' to 'f2da54ef432eac89b0f18eaad68e602b6990b5de'
and an warning from "python3 /OE/build/test-D/ext-sdk-prepare.py" when eSDK is being prepared for use:
WARNING: The base-files:do_install sig is computed to be 83b9c9a6ef1145baac5a1e0d08814b9156af239c58fc42df95c25a9cd8a7f201,
but the sig is locked to 3dc22233059075978e5503691e98e79e7cc60db94259dfcd886bca2291c0add7 in SIGGEN_LOCKEDSIGS_t-qemux86-64
[RP: Add commit about why we need the override for future reference]
(From OE-Core rev: 8f0e2e505d11697d0a1280b0a77a3fe8d72a4d49)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit 675ea7281c17f77bf5dea17cfd4d9da0928382a0)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The CVE product name for PyPI packages is (usually) the same as the PyPI
package name (and not our recipe name), so use that as the default.
(From OE-Core rev: 1c37b96cd4fdfad21bf24b8b883e371c3bca56de)
Signed-off-by: Alex Kiernan <alexk@zuma.ai>
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 61f6b0ad09bf87cdc2d3f08770b7c44cad1d0e58)
Signed-off-by: Sanjay Chitroda <schitrod@cisco.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Add SDK_ZIP_OPTIONS to remove symbolic link creation in zip archive or add options, e.g. for encryption of the zip archive.
(From OE-Core rev: 04b62f9459b401c276255f166d0738b6f902a576)
(From OE-Core rev: b9e0c3ced645cab74b2488a26b8f656a94b2a6f5)
Signed-off-by: Christoph Lauer <christoph.lauer@xtronic.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
NVD DB store version and update in the same value, separated by '_'.
The proposed patch check if the version from NVD DB contains a "_",
ie 9.2.0_p1 is convert to 9.2.0p1 before version comparison.
[YOCTO #14127]
Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
(From OE-Core rev: eb439b1283b60e6665694ff28c89fbd633eda6b0)
Signed-off-by: Geoffrey GIRY <geoffrey.giry@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 7d00f6ec578084a0a0e5caf36241d53036d996c4)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The master version of extended package data uses zstd for efficient
compression, but it relies on the zstd tool to be present on the host
system. Since dunfell supports older distros, we don't want to add this
tool as an additional requirement so switch to using gzip instead.
(From OE-Core rev: 1c7d555379c4b0962bccd018870989050d87675f)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The master version of the SPDX classes uses zstd for efficient
compression, but it relies on the zstd tool to be present on the host
system. Since dunfell supports older distros, we don't want to add this
tool as an additional requirement so switch to using gzip instead.
(From OE-Core rev: 55b08fd641767a3d2e080727828ca37d9e610109)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds extended package data which is encoded as JSON which allows it to
encode more structure than the "flat" package data files. The extended
data might be much larger than the standard package data, so it is not
read by default and instead requires
oe.packagedata.read_subpkgdata_extended() to be called
Currently, the file sizes and ELF debug sources are saved off into the
extended package data
(cherry picked from commit db9cf430e54ae68da80fbc3fba80ce88d8df164d)
(From OE-Core rev: 2318cb79ef1652fb19d696a8d43826e39c22834e)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Reviewed-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Backports the create-spdx classes from the latest versions on master.
This backport is a simple copy with no modifications, as its too
complex to cherry-pick all the corresponding changes. This will give an
appropriate base commit for subsequent changes and if necessary
additional backport cherry-picks from master in the future.
(From OE-Core rev: 48fbddf32ffa3ec44a788f42895c1730a84b5a91)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The script generated by the sdk_ext_postinst function was not quoting
the user existing PATH when updating it causing the export command to
fail.
Add necessary double quotes around $PATH.
(From OE-Core rev: b8298401c710146ae35f282048b66d5773a513ed)
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 00e96bf250eaaded839caf465dbc0af5b604aed7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The environment-setup script generated by the recipe was not quoting the
user existing PATH when updating it causing the export command to fail.
Add necessary double quotes around $PATH.
(From OE-Core rev: ad6aaa77f1d08b4ca66f46e318f1a09b645d9e23)
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 42177ff2d45ee70ad00917bb6fbabca49dae4f59)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous fix wasn't enough to address all the possible ways the
manifests might be ordered. Rework the previous fix so it is tied
to the multilib cross-canadian code which is causing the problem.
RECIPE_SYSROOT_MANIFEST_SUBDIR is not documented as I'd hope nobody
ever needs to use this outside the core multilib code.
(From OE-Core rev: 271ac31e15d00d636e888a1f3b7d4e22e0b8267a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit beab42e00713880cd95a04729c892f8662fbcbed)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"""
require conf/multilib.conf
MACHINE = "qemuarm64"
MULTILIBS = "multilib:lib32"
DEFAULTTUNE:virtclass-multilib-lib32 = "armv7athf-neon"
bitbake gcc-cross-canadian-arm
"""
and then inspecting the lib32* manifest files under recipe-sysroot-native shows
them referencing lib32-recipe-sysroot instead of recipe-sysroot as used by
gcc-cross-canadian recipes.
To fix this separate out the manifest by multilib. It is caching mechanism to
optimise disk usage so this doesn't break anything, just separated out some files.
(From OE-Core rev: eb9c6f7ac91431a6296789c23900f2a3016ef619)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 891d3faa3ed3d1cc231da58e5fa1325f05d5ade5)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
pkgconfig is being required to find dependencies for building kernel
native tools, move "inherit pkgconfig" to kernel.bbclass so BSP kernel
recipes can also benefit from it.
(From OE-Core rev: a5e62888768be641d5da00fd847e0cac96820c75)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 8a84bd98e3fbc16c782f83064801e469d086911e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The use of scripts/git-intercept was introduced in commit 3266c327df
(install/devshell: Introduce git intercept script due to fakeroot
issues) and later reverted in commit af27c81eaf (scripts: Make git
intercept global).
(From OE-Core rev: 51424b9955374196307aaf73cf4b6c184ce4fb6d)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit f6c260c8e2a33e282a35afc99de4ef8cc1791b08)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For the QA checks in `image.bbclass`, all exceptions
other than `oe.utils.ImageQAFailed` always print the
following generic message:
"Image QA function func_name failed"
This can be very misleading, as it may hide
python syntax errors and other kind of issues that are
hard to detect without more explicit error messages.
This change makes sure that the error message of all
exceptions are displayed.
Before this change:
"Image QA function func_name failed"
After this change:
"Image QA function func_name failed: f-string: empty expression not allowed (<string>, line 13)"
(From OE-Core rev: c4e3b86f2bcb2b445efc72bd8e06b1b89d88daa2)
Signed-off-by: Mauro Queiros <maurofrqueiros@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3d85b30d8704d38b86f5b006748cebc74bd2a4fa)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If on target locale generation is used, it fails at first boot showing
errors about a missing directory. Ensure the directory exists.
(From OE-Core rev: 63141bd8c19285bbef33f3d32a89ca127da0c658)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f2844c9f1bbb729562063d96a3d1cc9d44dafa0a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default return value from subprocess.check_output is an encoded byte.
The applied fix will decode the value to a string.
(From OE-Core rev: 622fea338a0c43f95185469b54cc72c545b73100)
Signed-off-by: Pawel Zalewski <pzalewski@thegoodpenguin.co.uk>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 046769fa952a511865c416b80d10af6287147fb7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We disable the useradd code for nativesdk targets since we don't support
postinstalls or multiple users in those cases. This means any usage
of chown/chgrp inside do_install tasks won't work and would have to be
conditional. Rather than require all recipes to do that, add intercepts
of the calls and map those to root/root user/groups. We can't just ignore
them as some calls are used to remove host contamination from the host
user ID so they need to be made, just as root.
(From OE-Core rev: a05c116c517e0700837c335d0691c3fcc4462dda)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e1f40670c438e33cae87678425de72ca03566888)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mitigate occurences where ':append' operator is used and leading
whitespace character is obviously missing, risking inadvertent
string concatenation.
(From OE-Core rev: fcd340ec53ff8352b8cae0eb351810072b025a08)
(From OE-Core rev: b21e5524e3326c77598d30f6fe9268a5fdac91dd)
Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a .gitmodules file exists but is empty then devtool finish fails.
Add an additional check for this.
[YOCTO #14999]
(From OE-Core rev: 077edd3992683985f0779afc73d4207a795ced39)
Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b4f0f7c4934bade9e4d4a1086f9d8b29d8e9ad45)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed an error when Bash's unbound variable protection is enabled (set -u) and variable "LD_LIBRARY_PATH" does not exist.
(From OE-Core rev: 2c7309eea616ec88338d508bc9f284e3bd44f5a1)
Signed-off-by: Jan Kircher <openembedded@hetsh.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 85685370b0ad93291cda59fb091a15eeecf5e0d5)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When building an image cve_check_write_rootfs_manifest() would sometimes fail
with a FileNotFoundError when writing the manifest.cve due to the parent
directory (DEPLOY_DIR_IMAGE) not (yet) existing.
The image task will provide the manifest in the deploy directory afterwards,
so other recipes depending on the manifest being in DEPLOY_DIR_IMAGE should
continue to function properly.
(From OE-Core rev: c25a5c5f6fede29893a2ac1502216e9deccfa6c3)
Signed-off-by: Jermain Horsman <jermain.horsman@nedap.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 00fb2aae22ce0d7ff5f3f8766fa770eeb4e73483)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The do_rm_work() task is using the first available 'rm' binary
available in PATH to remove files and folders.
However, depending on the PATH setup and RECIPE_SYSROOT_NATIVE
contents, the function can be using the 'rm' binary available
in RECIPE_SYSROOT_NATIVE, a folder that will get removed.
This causes a sporadic race-condition when trying to access the
'rm' binary of a folder already deleted.
Solve this by exclusively using the HOSTTOOLS 'rm' binary, as
this folder will not get removed.
(From OE-Core rev: 75b7e86c9d9931c9e4e114af026b51710f1920a2)
Signed-off-by: Luis Martins <luis.pinto.martins@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit edcd9ad333bc4e504594e8af83e8cb7007d2e35c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous code had 2 issues:
1. make hosttools/ccache always link to host's ccache (/usr/bin/ccache)
even we have one buildtools
2. make hosttools/gcc etc, link to host's gcc event we have one
buildtools when keyword ccache in buildtools's path, eg:
/mnt/ccache/bin/buildtools
This patch is for fix above issues.
(From OE-Core rev: f2f70bf8d93b33b65875828c0402a98e943f660a)
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1b7c81414cf252a7203d95703810a770184d7e4d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The QB_DEFAULT_KERNEL is set to pick bundled initramfs kernel image
if the Linux kernel image is generated with INITRAMFS_IMAGE_BUNDLE="1".
This makes runqemu to automatically pick bundled initramfs kernel image
instead of explicitly mentioning bundled initramfs kernel image in
runqemu.
[YOCTO #14748]
(From OE-Core rev: 0c63018ef3843cfefd2be31c0a6693181037410d)
Signed-off-by: Jagadeesh Krishnanjanappa <workjagadeesh@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 52371624313184e1a825519160c3833e282df8b9)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit 0533edac27 broke
bitbake parsing when bitbake is executed from directory with existing .gitmodules
and the recipe in externalsrc does not have .gitmodules
The check needs to search for .gitmodules in sources path, not cwd.
iParsing recipes...ERROR: ExpansionError during parsing <path to recipe>
...
bb.data_smart.ExpansionError: Failure expanding variable do_compile[file-checksums], expression was ${@srctree_hash_files(d)} which triggered exception CalledProcessError: Command '['git', 'config', '--file', '.gitmodules', '--get-regexp', 'path']' returned non-zero exit status 1.
(From OE-Core rev: c58d82b98348b167b60ec3c8b9651d73b1bdfbdc)
Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 66ff3d1f65cd2e7f5319e98fa41f47a59b714c72)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently TOOLCHAIN is strictly set to gcc in kernel-arch.bbclass.
And this prevents any TOOLCHAIN changes for any kernel recipe.
This change makes TOOLCHAIN configurable as usual.
(From OE-Core rev: 7d45d4696970c45534b905dc7a65418073108f97)
Signed-off-by: Alexey Smirnov <pyih.soft@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit be1634fc35dcc81f0301d942064a6eed584e0704)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the KERNEL_DEBUG_TIMESTAMPS is not working as expected
at rebuild. That is, even if we set it to "1", the kernel build time
is not changed. The problem could be reproduced by the following steps.
1. bitbake core-image-minimal; start image and check `uname -a` output.
2. set in local.conf: KERNEL_DEBUG_TIMESTAMPS = "1"
3. bitbake core-image-minimal; start image and check `uname -a` output.
It's expected that after enabling KERNEL_DEBUG_TIMESTAMPS, the kernel
build time will be set to current date. But it's not. This is because
the compile.h was not re-generated when do_compile task was re-executed.
In mkcompile_h, we have:
"""
# Only replace the real compile.h if the new one is different,
# in order to preserve the timestamp and avoid unnecessary
# recompilations.
# We don't consider the file changed if only the date/time changed,
# unless KBUILD_BUILD_TIMESTAMP was explicitly set (e.g. for
# reproducible builds with that value referring to a commit timestamp).
# A kernel config change will increase the generation number, thus
# causing compile.h to be updated (including date/time) due to the
# changed comment in the
# first line.
"""
It has made it very clear that it will not be re-generated unless
we have KBUILD_BUILD_TIMESTAMP set explicitly. So we set this variable
explicitly in do_compile to fix this issue.
(From OE-Core rev: e44f0cda8176186d42a752631810c1cb5f1971eb)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 1b68c2d2d385013a1c535ef81172494302a36d74)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In 526bdd88ccd758204452579333ba188e29270bde the imageType loop in
kernel_do_deploy was changed to use KERNEL_IMAGETYPE_FOR_MAKE rather
than KERNEL_IMAGETYPES. This broke the special handling for fitImage
immediately below because KERNEL_IMAGETYPE_FOR_MAKE never contains
fitImage.
It has always been my understanding that KERNEL_IMAGETYPE_FOR_MAKE
controlled what was passed to make, but KERNEL_IMAGETYPE controlled what
was installed/deployed. When the two are different then it's the
responsibility of whoever set KERNEL_IMAGETYPE_FOR_MAKE to ensure that
whatever comes out of the kernel build system has been transformed in to
the requested form by the time of installation. This is what happens for
kernel.bbclass's own support for vmlinux.gz.
I think this means that for KERNEL_IMAGETYPE vmlinux.gz, kernel.bbclass
is responsible for generating vmlinux.gz.initramfs[1] so that
kernel_do_deploy can deploy it. This means that the change in
526bdd88ccd758204452579333ba188e29270bde can be reverted, fixing
KERNEL_IMAGETYPE = "fitImage".
In addition, it ought to be possible for recipes and other classes that
use kernel.bbclass to hook into this mechanism by setting
KERNEL_IMAGETYPE_FOR_MAKE and performing their own transformations.
do_bundle_initramfs calls kernel_do_compile and we don't want it to
transform vmlinux to vmlinux.gz at that point, since it will fight
against the careful renaming and preserving that do_bundle_initramfs
does. Let's separate the transformation out of kernel_do_compile to a
new do_transform_kernel task that can be run at the right time. This
means that it's also logical to perform the equivalent translation for
the kernel with the initramfs in a separate
do_transform_bundled_initramfs task too.
This leaves two clear customisation points for recipes and other classes
to hook into the process and perform their transformations:
do_transform_kernel and do_transform_bundled_initramfs.
(I care about this because our recipes that use kernel.bbclass also set
KERNEL_IMAGETYPE_FOR_MAKE and transform vmlinux into a form suitable for
our bootloader after do_compile and do_bundle_initramfs into the format
matching KERNEL_IMAGETYPE. I'm unable to successfully bundle an
initramfs after 526bdd88ccd758204452579333ba188e29270bde, but I didn't
want to just revert that change to reintroduce the bug that it was
fixing.)
I can't say that I'm entirely happy with this change, but I'm unsure
what to do to improve it. I find the way that both the bare kernel and
the one with the initramfs both get deployed to be confusing, and a
waste of build time. I would like to not actually generate a publishable
kernel image at all during do_compile when an initramfs is in use, but I
suspect that this would affect valid use cases that I'm not aware of.
(From OE-Core rev: d9e9653616638f2b187d5e04540071ac34d99f56)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
[1] It could be argued that this should be vmlinux.initramfs.gz, but
that would require another special case in kernel_do_deploy and the
filename is only visible within this class and the recipes that use it
anyway.
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 10a4a132e87e835726bf5da81a60f6f509b90765)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, when shortening sstate filenames, the reserved
characters for .siginfo were not considered, when siginfo=False,
resulting in differently shortend filenames for the sstate and siginfo
files. With this change, the filenames of the truncated sstate and
siginfo files have the same basename, just as is already the case for
untruncated filenames.
Making sure that the .siginfo files always have the filename of the
corresponding sstate file plus its .siginfo suffix, also when being
truncated, makes it easier to manage the sstate cache and an sstate
mirror outside of Bitbake/Yocto.
(From OE-Core rev: 408bf1b4bb4f4ed126c17fb3676f9fa0513065ba)
Signed-off-by: Manuel Leonhardt <mleonhardt@arri.de>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c2e0e43b7123cf5149833e0072c8edaea3629112)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need the system tar to be GNU tar, as we reply on --xattrs. Some
distributions may be using libarchive's tar binary, which is definitely
not as featureful, so check for this and abort early with a clear
message instead of later with mysterious errors.
(From OE-Core rev: 8f852648fe730615c99bcdaace8a4748ef4e96a5)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 7dd2b1cd1bb10e67485dab8600c0787df6c2eee7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The trailing slash in ${B} caused -fdebug-prefix-map=${B}=... to not
match as intended, resulting in ${TMPDIR} ending up in files in
${PN}-dbg when externalsrc was in use, which in turn triggered buildpath
QA warnings.
(From OE-Core rev: fa89e048e223ae2e96d0f55979f93f29904b5229)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 9b5031ed5a0d102905fa75acc418246c23df6eef)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fix issue introduced in:
https://git.openembedded.org/openembedded-core/commit/?id=95fbac8dcad6c93f4c9737e9fe13e92ab6befa09
* it added check for s_dir + git-dir (typically '.git') isn't
the same as ${TOPDIR} + git-dir, but due to copy-paste issue
it was just comparing it with s_dir + git-dir again, resulting
in most external repos (where git-dir is '.git') to be processed
as regular directory (not taking advantage of git write-tree).
* normally this wouldn't be an issue, but for big repo with a lot of
files this added a lot of checksums in:
d.setVarFlag('do_compile', 'file-checksums', '${@srctree_hash_files(d)}')
and I mean *a lot, e.g. in chromium build it was 380227 paths
which still wouldn't that bad, but the checksum processing in
siggen.py isn't trivial and just looping through all these
checksums takes very long time (over 1000sec on fast NVME drive
with warm cache) and then
https://git.openembedded.org/bitbake/commit/?id=b4975d2ecf615ac4c240808fbc5a3f879a93846b
made the processing a bit more complicated and the loop in
get_taskhash() function took 6448sec and to make things worse
there was no output from bitbake during that time, so even with -DDD
it looks like this:
DEBUG: virtual/libgles2 resolved to: mesa (langdale/oe-core/meta/recipes-graphics/mesa/mesa_22.2.0.bb)
Bitbake still alive (no events for 600s). Active tasks:
Bitbake still alive (no events for 1200s). Active tasks:
Bitbake still alive (no events for 1800s). Active tasks:
Bitbake still alive (no events for 2400s). Active tasks:
Bitbake still alive (no events for 3000s). Active tasks:
Bitbake still alive (no events for 3600s). Active tasks:
Bitbake still alive (no events for 4200s). Active tasks:
Bitbake still alive (no events for 4800s). Active tasks:
Bitbake still alive (no events for 5400s). Active tasks:
Bitbake still alive (no events for 6000s). Active tasks:
DEBUG: Starting bitbake-worker
without -DDD it will get stuck for almost 2 hours in:
"Initialising tasks..."
before it finally writes sstate summary like:
"Sstate summary: Wanted 3102 Local 0 Mirrors 0 Missed 3102 Current 1483 (0% match, 32% complete)"
* fix the copy&paste typo to use git work-tree in most cases, but
be aware that this issue still exists for huge local source
trees not in git
[YOCTO #14942]
(From OE-Core rev: 1f0e4de8d92edd7438d462c779d917ac0ccd5499)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 9102e5a94b8146cb1da27afbe41d3db999a914ff)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a UBOOT_MKIMAGE_SIGN_ARGS variable to enable passing additional
options to uboot-mkimage when it is run the second time to perform
signing.
(From OE-Core rev: fd39b936b03a6416e7a66408c47578daa2a08cf1)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8fd7ee7414b45a1feeef7982af3583475902a677)
Signed-off-by: Massimiliano Minella <massimiliano.minella@se.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Git has removed support for "git submodule--helper list".
31955475d1
This change provides an alternate method for gathering the submodules
information.
Tested:
Build recipes with and without submodules
(From OE-Core rev: 6c50d83af0af677c2dff864ac40c580ae446372b)
Signed-off-by: Carson Labrado <clabrado@google.com>
Signed-off-by: John Edward Broadbent <jebr@google.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6d9364e5f3535954f65cbbc694ee7933ac1d664f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can have multiple processes reading the database at the same time, and
cursors only release their locks when they're garbage collected.
This might be the cause of random sqlite errors on the autobuilder, so
explicitly close the cursors when we're done with them.
(From OE-Core rev: 48742ddf4d0acd419c8ffb8f22124ed525efc2d9)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit 5d2e90e4a58217a943ec21140bc2ecdd4357a98a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The CVE check database needs to have a shared lock acquired on it before
it is accessed. This to prevent cve-update-db-native from deleting the
database file out from underneath it.
[YOCTO #14899]
(From OE-Core rev: 374dd13db2c4fa92793f12c93d68d09304f77c17)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 20a9911b73df62a0d0d1884e57085f13ac5016dd)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is to ensure host-user-contaminated.txt would be removed before
do_rootfs runs, since ${S} is in cleandirs of do_rootfs, otherwise, a
host-user-contaminated.txt file that generated from previous builds
could be used which is wrong.
(From OE-Core rev: 06cfa8be54c9aee23bd8570a370a974b463a0a1a)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 54a3fd63e684d070fad962be97e549f3af7ac111)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes the bin_package.bbclass work properly with the native class.
(From OE-Core rev: 0bf78a8e0e1cf7e74b55aca4db0e62dd9dfa55ce)
Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ad330b6d4b6e2ba051b5c6c437e07a183831f757)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Building external kernel modules like lttng-modules was showing build paths
inside the debug symbols for the modules and breaking build reproducibility.
Fix this by adding in the mapping needed to map the kernel build directory
to something more approriate on target.
(From OE-Core rev: c4d8834ed3d200f25f12fec8acfa2b954f3240e0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b56dc9009ba93174de6bf4c01e17808ef249dc5c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If enabled, the buildpaths test hangs in psplash as it tries to open
a fifo and read from it, hanging indefinitely.
Tweak the test to ignore fifo/socket/device files.
(From OE-Core rev: 0106c6a629d0a9f07d76ffaad2dc92e48021e1b0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2567edb7e0a8c5ca9a88d6940491bf33bfe0eff9)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moving the function will allow other classes to capture which CVEs have
been patched, in particular SBoM generation.
Also add a function to capture the CPE ID from the CVE Product and
Version
(From OE-Core rev: 75d34259a715120be1d023e4fd7b6b4b125f2443)
(From OE-Core rev: bba069463ca3813666d084643b0239b9af0199e1)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit fa6c07bc1a)
Signed-off-by: Akash Hadke <akash.hadke@kpit.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #14851]
Under some circumstances it is not desirable to create a combined locale
archive (/usr/lib/locale/locale-archive).
The new variable IMAGE_LOCALES_ARCHIVE defaults to '1', so the default
behaviour is not changed.
Modified to work with code before move to lib/oe/package_manager
(From OE-Core rev: af32908dfcebbc0f617ad828d895f504c37ee2d1)
Signed-off-by: Michael Thalmeier <michael.thalmeier@hale.at>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8d78b819c2ec33fce3a34254fa90864ee5fa7617)
Signed-off-by: Jate Sujjavanich <jatedev@gmail.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The cve-check class writes temporary files to preserve state across the
build, and cleans them up in a CookerExit handler.
However, in memory-resident builds the cooker won't exit in between
builds, so the state isn't cleared and the CVE report generation fails:
NOTE: Generating JSON CVE summary
ERROR: Error adding the same package twice
Easily solved by hooking to BuildCompleted, instead of CookerExit.
(From OE-Core rev: ee3270709158aff463fec6798f3b8968268b4d4b)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit fccdcfd301de281a427bfee48d8ff47fa07b7259)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current home path that is compared against is incorrect as it is missing the
package name, this patch adds it.
[YOCTO #14553]
(From OE-Core rev: b75caf4a985e3c20996531785125eaffdc832104)
Signed-off-by: Ahmed Hossam <Ahmed.Hossam@opensynergy.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit ae8f22d9e2694eea5ede3b31c6f3bca404ea4a5a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ignored CVEs aren't patched, but do not apply in our configuration
for some reason. Up till now they were only partially supported
and reported as "Patched".
This patch adds separate reporting of Ignored CVEs. The variable
CVE_CHECK_REPORT_PATCHED now manages reporting of both patched
and ignored CVEs.
(From OE-Core rev: 14b3c0ca46a0aa97565a24b7a5116306237d7cfe)
Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry-picked from c773102d4828fc4ddd1024f6115d577e23f1afe4)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using multiconfig with the same TMP folder we can have
races because the shared recipes like gcc-source run twice.
ARCHIVER_OUTDIR = ${ARCHIVER_TOPDIR}/${TARGET_SYS}/${PF}/
which includes TARGET_SYS and between the two different MACHINE values,
this changes from 'arm-poky-linux-gnueabi' to 'aarch64-poky-linux'.
This leads to the task running twice, once for each multiconfig.
To solve this we need to store the shared output in a common place
for all machines and in this way the stamps will be the same for each
machine so the gcc-source will on run once regardless of the machine used.
(From OE-Core rev: 135adeb82c9303c26193bb6f6bd3bc696793aa62)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5abe497aad39a6ce8d72556fcdda1938a0f8c1bc)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
