pkgconfig is being required to find dependencies for building kernel
native tools, move "inherit pkgconfig" to kernel.bbclass so BSP kernel
recipes can also benefit from it.
(From OE-Core rev: a5e62888768be641d5da00fd847e0cac96820c75)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 8a84bd98e3fbc16c782f83064801e469d086911e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The use of scripts/git-intercept was introduced in commit 3266c327df
(install/devshell: Introduce git intercept script due to fakeroot
issues) and later reverted in commit af27c81eaf (scripts: Make git
intercept global).
(From OE-Core rev: 51424b9955374196307aaf73cf4b6c184ce4fb6d)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit f6c260c8e2a33e282a35afc99de4ef8cc1791b08)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For the QA checks in `image.bbclass`, all exceptions
other than `oe.utils.ImageQAFailed` always print the
following generic message:
"Image QA function func_name failed"
This can be very misleading, as it may hide
python syntax errors and other kind of issues that are
hard to detect without more explicit error messages.
This change makes sure that the error message of all
exceptions are displayed.
Before this change:
"Image QA function func_name failed"
After this change:
"Image QA function func_name failed: f-string: empty expression not allowed (<string>, line 13)"
(From OE-Core rev: c4e3b86f2bcb2b445efc72bd8e06b1b89d88daa2)
Signed-off-by: Mauro Queiros <maurofrqueiros@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3d85b30d8704d38b86f5b006748cebc74bd2a4fa)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If on target locale generation is used, it fails at first boot showing
errors about a missing directory. Ensure the directory exists.
(From OE-Core rev: 63141bd8c19285bbef33f3d32a89ca127da0c658)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f2844c9f1bbb729562063d96a3d1cc9d44dafa0a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default return value from subprocess.check_output is an encoded byte.
The applied fix will decode the value to a string.
(From OE-Core rev: 622fea338a0c43f95185469b54cc72c545b73100)
Signed-off-by: Pawel Zalewski <pzalewski@thegoodpenguin.co.uk>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 046769fa952a511865c416b80d10af6287147fb7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We disable the useradd code for nativesdk targets since we don't support
postinstalls or multiple users in those cases. This means any usage
of chown/chgrp inside do_install tasks won't work and would have to be
conditional. Rather than require all recipes to do that, add intercepts
of the calls and map those to root/root user/groups. We can't just ignore
them as some calls are used to remove host contamination from the host
user ID so they need to be made, just as root.
(From OE-Core rev: a05c116c517e0700837c335d0691c3fcc4462dda)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e1f40670c438e33cae87678425de72ca03566888)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mitigate occurences where ':append' operator is used and leading
whitespace character is obviously missing, risking inadvertent
string concatenation.
(From OE-Core rev: fcd340ec53ff8352b8cae0eb351810072b025a08)
(From OE-Core rev: b21e5524e3326c77598d30f6fe9268a5fdac91dd)
Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a .gitmodules file exists but is empty then devtool finish fails.
Add an additional check for this.
[YOCTO #14999]
(From OE-Core rev: 077edd3992683985f0779afc73d4207a795ced39)
Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b4f0f7c4934bade9e4d4a1086f9d8b29d8e9ad45)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed an error when Bash's unbound variable protection is enabled (set -u) and variable "LD_LIBRARY_PATH" does not exist.
(From OE-Core rev: 2c7309eea616ec88338d508bc9f284e3bd44f5a1)
Signed-off-by: Jan Kircher <openembedded@hetsh.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 85685370b0ad93291cda59fb091a15eeecf5e0d5)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When building an image cve_check_write_rootfs_manifest() would sometimes fail
with a FileNotFoundError when writing the manifest.cve due to the parent
directory (DEPLOY_DIR_IMAGE) not (yet) existing.
The image task will provide the manifest in the deploy directory afterwards,
so other recipes depending on the manifest being in DEPLOY_DIR_IMAGE should
continue to function properly.
(From OE-Core rev: c25a5c5f6fede29893a2ac1502216e9deccfa6c3)
Signed-off-by: Jermain Horsman <jermain.horsman@nedap.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 00fb2aae22ce0d7ff5f3f8766fa770eeb4e73483)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The do_rm_work() task is using the first available 'rm' binary
available in PATH to remove files and folders.
However, depending on the PATH setup and RECIPE_SYSROOT_NATIVE
contents, the function can be using the 'rm' binary available
in RECIPE_SYSROOT_NATIVE, a folder that will get removed.
This causes a sporadic race-condition when trying to access the
'rm' binary of a folder already deleted.
Solve this by exclusively using the HOSTTOOLS 'rm' binary, as
this folder will not get removed.
(From OE-Core rev: 75b7e86c9d9931c9e4e114af026b51710f1920a2)
Signed-off-by: Luis Martins <luis.pinto.martins@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit edcd9ad333bc4e504594e8af83e8cb7007d2e35c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous code had 2 issues:
1. make hosttools/ccache always link to host's ccache (/usr/bin/ccache)
even we have one buildtools
2. make hosttools/gcc etc, link to host's gcc event we have one
buildtools when keyword ccache in buildtools's path, eg:
/mnt/ccache/bin/buildtools
This patch is for fix above issues.
(From OE-Core rev: f2f70bf8d93b33b65875828c0402a98e943f660a)
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1b7c81414cf252a7203d95703810a770184d7e4d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The QB_DEFAULT_KERNEL is set to pick bundled initramfs kernel image
if the Linux kernel image is generated with INITRAMFS_IMAGE_BUNDLE="1".
This makes runqemu to automatically pick bundled initramfs kernel image
instead of explicitly mentioning bundled initramfs kernel image in
runqemu.
[YOCTO #14748]
(From OE-Core rev: 0c63018ef3843cfefd2be31c0a6693181037410d)
Signed-off-by: Jagadeesh Krishnanjanappa <workjagadeesh@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 52371624313184e1a825519160c3833e282df8b9)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit 0533edac27 broke
bitbake parsing when bitbake is executed from directory with existing .gitmodules
and the recipe in externalsrc does not have .gitmodules
The check needs to search for .gitmodules in sources path, not cwd.
iParsing recipes...ERROR: ExpansionError during parsing <path to recipe>
...
bb.data_smart.ExpansionError: Failure expanding variable do_compile[file-checksums], expression was ${@srctree_hash_files(d)} which triggered exception CalledProcessError: Command '['git', 'config', '--file', '.gitmodules', '--get-regexp', 'path']' returned non-zero exit status 1.
(From OE-Core rev: c58d82b98348b167b60ec3c8b9651d73b1bdfbdc)
Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 66ff3d1f65cd2e7f5319e98fa41f47a59b714c72)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently TOOLCHAIN is strictly set to gcc in kernel-arch.bbclass.
And this prevents any TOOLCHAIN changes for any kernel recipe.
This change makes TOOLCHAIN configurable as usual.
(From OE-Core rev: 7d45d4696970c45534b905dc7a65418073108f97)
Signed-off-by: Alexey Smirnov <pyih.soft@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit be1634fc35dcc81f0301d942064a6eed584e0704)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the KERNEL_DEBUG_TIMESTAMPS is not working as expected
at rebuild. That is, even if we set it to "1", the kernel build time
is not changed. The problem could be reproduced by the following steps.
1. bitbake core-image-minimal; start image and check `uname -a` output.
2. set in local.conf: KERNEL_DEBUG_TIMESTAMPS = "1"
3. bitbake core-image-minimal; start image and check `uname -a` output.
It's expected that after enabling KERNEL_DEBUG_TIMESTAMPS, the kernel
build time will be set to current date. But it's not. This is because
the compile.h was not re-generated when do_compile task was re-executed.
In mkcompile_h, we have:
"""
# Only replace the real compile.h if the new one is different,
# in order to preserve the timestamp and avoid unnecessary
# recompilations.
# We don't consider the file changed if only the date/time changed,
# unless KBUILD_BUILD_TIMESTAMP was explicitly set (e.g. for
# reproducible builds with that value referring to a commit timestamp).
# A kernel config change will increase the generation number, thus
# causing compile.h to be updated (including date/time) due to the
# changed comment in the
# first line.
"""
It has made it very clear that it will not be re-generated unless
we have KBUILD_BUILD_TIMESTAMP set explicitly. So we set this variable
explicitly in do_compile to fix this issue.
(From OE-Core rev: e44f0cda8176186d42a752631810c1cb5f1971eb)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 1b68c2d2d385013a1c535ef81172494302a36d74)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In 526bdd88ccd758204452579333ba188e29270bde the imageType loop in
kernel_do_deploy was changed to use KERNEL_IMAGETYPE_FOR_MAKE rather
than KERNEL_IMAGETYPES. This broke the special handling for fitImage
immediately below because KERNEL_IMAGETYPE_FOR_MAKE never contains
fitImage.
It has always been my understanding that KERNEL_IMAGETYPE_FOR_MAKE
controlled what was passed to make, but KERNEL_IMAGETYPE controlled what
was installed/deployed. When the two are different then it's the
responsibility of whoever set KERNEL_IMAGETYPE_FOR_MAKE to ensure that
whatever comes out of the kernel build system has been transformed in to
the requested form by the time of installation. This is what happens for
kernel.bbclass's own support for vmlinux.gz.
I think this means that for KERNEL_IMAGETYPE vmlinux.gz, kernel.bbclass
is responsible for generating vmlinux.gz.initramfs[1] so that
kernel_do_deploy can deploy it. This means that the change in
526bdd88ccd758204452579333ba188e29270bde can be reverted, fixing
KERNEL_IMAGETYPE = "fitImage".
In addition, it ought to be possible for recipes and other classes that
use kernel.bbclass to hook into this mechanism by setting
KERNEL_IMAGETYPE_FOR_MAKE and performing their own transformations.
do_bundle_initramfs calls kernel_do_compile and we don't want it to
transform vmlinux to vmlinux.gz at that point, since it will fight
against the careful renaming and preserving that do_bundle_initramfs
does. Let's separate the transformation out of kernel_do_compile to a
new do_transform_kernel task that can be run at the right time. This
means that it's also logical to perform the equivalent translation for
the kernel with the initramfs in a separate
do_transform_bundled_initramfs task too.
This leaves two clear customisation points for recipes and other classes
to hook into the process and perform their transformations:
do_transform_kernel and do_transform_bundled_initramfs.
(I care about this because our recipes that use kernel.bbclass also set
KERNEL_IMAGETYPE_FOR_MAKE and transform vmlinux into a form suitable for
our bootloader after do_compile and do_bundle_initramfs into the format
matching KERNEL_IMAGETYPE. I'm unable to successfully bundle an
initramfs after 526bdd88ccd758204452579333ba188e29270bde, but I didn't
want to just revert that change to reintroduce the bug that it was
fixing.)
I can't say that I'm entirely happy with this change, but I'm unsure
what to do to improve it. I find the way that both the bare kernel and
the one with the initramfs both get deployed to be confusing, and a
waste of build time. I would like to not actually generate a publishable
kernel image at all during do_compile when an initramfs is in use, but I
suspect that this would affect valid use cases that I'm not aware of.
(From OE-Core rev: d9e9653616638f2b187d5e04540071ac34d99f56)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
[1] It could be argued that this should be vmlinux.initramfs.gz, but
that would require another special case in kernel_do_deploy and the
filename is only visible within this class and the recipes that use it
anyway.
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 10a4a132e87e835726bf5da81a60f6f509b90765)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, when shortening sstate filenames, the reserved
characters for .siginfo were not considered, when siginfo=False,
resulting in differently shortend filenames for the sstate and siginfo
files. With this change, the filenames of the truncated sstate and
siginfo files have the same basename, just as is already the case for
untruncated filenames.
Making sure that the .siginfo files always have the filename of the
corresponding sstate file plus its .siginfo suffix, also when being
truncated, makes it easier to manage the sstate cache and an sstate
mirror outside of Bitbake/Yocto.
(From OE-Core rev: 408bf1b4bb4f4ed126c17fb3676f9fa0513065ba)
Signed-off-by: Manuel Leonhardt <mleonhardt@arri.de>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c2e0e43b7123cf5149833e0072c8edaea3629112)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need the system tar to be GNU tar, as we reply on --xattrs. Some
distributions may be using libarchive's tar binary, which is definitely
not as featureful, so check for this and abort early with a clear
message instead of later with mysterious errors.
(From OE-Core rev: 8f852648fe730615c99bcdaace8a4748ef4e96a5)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 7dd2b1cd1bb10e67485dab8600c0787df6c2eee7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The trailing slash in ${B} caused -fdebug-prefix-map=${B}=... to not
match as intended, resulting in ${TMPDIR} ending up in files in
${PN}-dbg when externalsrc was in use, which in turn triggered buildpath
QA warnings.
(From OE-Core rev: fa89e048e223ae2e96d0f55979f93f29904b5229)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 9b5031ed5a0d102905fa75acc418246c23df6eef)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fix issue introduced in:
https://git.openembedded.org/openembedded-core/commit/?id=95fbac8dcad6c93f4c9737e9fe13e92ab6befa09
* it added check for s_dir + git-dir (typically '.git') isn't
the same as ${TOPDIR} + git-dir, but due to copy-paste issue
it was just comparing it with s_dir + git-dir again, resulting
in most external repos (where git-dir is '.git') to be processed
as regular directory (not taking advantage of git write-tree).
* normally this wouldn't be an issue, but for big repo with a lot of
files this added a lot of checksums in:
d.setVarFlag('do_compile', 'file-checksums', '${@srctree_hash_files(d)}')
and I mean *a lot, e.g. in chromium build it was 380227 paths
which still wouldn't that bad, but the checksum processing in
siggen.py isn't trivial and just looping through all these
checksums takes very long time (over 1000sec on fast NVME drive
with warm cache) and then
https://git.openembedded.org/bitbake/commit/?id=b4975d2ecf615ac4c240808fbc5a3f879a93846b
made the processing a bit more complicated and the loop in
get_taskhash() function took 6448sec and to make things worse
there was no output from bitbake during that time, so even with -DDD
it looks like this:
DEBUG: virtual/libgles2 resolved to: mesa (langdale/oe-core/meta/recipes-graphics/mesa/mesa_22.2.0.bb)
Bitbake still alive (no events for 600s). Active tasks:
Bitbake still alive (no events for 1200s). Active tasks:
Bitbake still alive (no events for 1800s). Active tasks:
Bitbake still alive (no events for 2400s). Active tasks:
Bitbake still alive (no events for 3000s). Active tasks:
Bitbake still alive (no events for 3600s). Active tasks:
Bitbake still alive (no events for 4200s). Active tasks:
Bitbake still alive (no events for 4800s). Active tasks:
Bitbake still alive (no events for 5400s). Active tasks:
Bitbake still alive (no events for 6000s). Active tasks:
DEBUG: Starting bitbake-worker
without -DDD it will get stuck for almost 2 hours in:
"Initialising tasks..."
before it finally writes sstate summary like:
"Sstate summary: Wanted 3102 Local 0 Mirrors 0 Missed 3102 Current 1483 (0% match, 32% complete)"
* fix the copy&paste typo to use git work-tree in most cases, but
be aware that this issue still exists for huge local source
trees not in git
[YOCTO #14942]
(From OE-Core rev: 1f0e4de8d92edd7438d462c779d917ac0ccd5499)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 9102e5a94b8146cb1da27afbe41d3db999a914ff)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a UBOOT_MKIMAGE_SIGN_ARGS variable to enable passing additional
options to uboot-mkimage when it is run the second time to perform
signing.
(From OE-Core rev: fd39b936b03a6416e7a66408c47578daa2a08cf1)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8fd7ee7414b45a1feeef7982af3583475902a677)
Signed-off-by: Massimiliano Minella <massimiliano.minella@se.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Git has removed support for "git submodule--helper list".
31955475d1
This change provides an alternate method for gathering the submodules
information.
Tested:
Build recipes with and without submodules
(From OE-Core rev: 6c50d83af0af677c2dff864ac40c580ae446372b)
Signed-off-by: Carson Labrado <clabrado@google.com>
Signed-off-by: John Edward Broadbent <jebr@google.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6d9364e5f3535954f65cbbc694ee7933ac1d664f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can have multiple processes reading the database at the same time, and
cursors only release their locks when they're garbage collected.
This might be the cause of random sqlite errors on the autobuilder, so
explicitly close the cursors when we're done with them.
(From OE-Core rev: 48742ddf4d0acd419c8ffb8f22124ed525efc2d9)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit 5d2e90e4a58217a943ec21140bc2ecdd4357a98a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The CVE check database needs to have a shared lock acquired on it before
it is accessed. This to prevent cve-update-db-native from deleting the
database file out from underneath it.
[YOCTO #14899]
(From OE-Core rev: 374dd13db2c4fa92793f12c93d68d09304f77c17)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 20a9911b73df62a0d0d1884e57085f13ac5016dd)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is to ensure host-user-contaminated.txt would be removed before
do_rootfs runs, since ${S} is in cleandirs of do_rootfs, otherwise, a
host-user-contaminated.txt file that generated from previous builds
could be used which is wrong.
(From OE-Core rev: 06cfa8be54c9aee23bd8570a370a974b463a0a1a)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 54a3fd63e684d070fad962be97e549f3af7ac111)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes the bin_package.bbclass work properly with the native class.
(From OE-Core rev: 0bf78a8e0e1cf7e74b55aca4db0e62dd9dfa55ce)
Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ad330b6d4b6e2ba051b5c6c437e07a183831f757)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Building external kernel modules like lttng-modules was showing build paths
inside the debug symbols for the modules and breaking build reproducibility.
Fix this by adding in the mapping needed to map the kernel build directory
to something more approriate on target.
(From OE-Core rev: c4d8834ed3d200f25f12fec8acfa2b954f3240e0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b56dc9009ba93174de6bf4c01e17808ef249dc5c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If enabled, the buildpaths test hangs in psplash as it tries to open
a fifo and read from it, hanging indefinitely.
Tweak the test to ignore fifo/socket/device files.
(From OE-Core rev: 0106c6a629d0a9f07d76ffaad2dc92e48021e1b0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2567edb7e0a8c5ca9a88d6940491bf33bfe0eff9)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moving the function will allow other classes to capture which CVEs have
been patched, in particular SBoM generation.
Also add a function to capture the CPE ID from the CVE Product and
Version
(From OE-Core rev: 75d34259a715120be1d023e4fd7b6b4b125f2443)
(From OE-Core rev: bba069463ca3813666d084643b0239b9af0199e1)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit fa6c07bc1a)
Signed-off-by: Akash Hadke <akash.hadke@kpit.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #14851]
Under some circumstances it is not desirable to create a combined locale
archive (/usr/lib/locale/locale-archive).
The new variable IMAGE_LOCALES_ARCHIVE defaults to '1', so the default
behaviour is not changed.
Modified to work with code before move to lib/oe/package_manager
(From OE-Core rev: af32908dfcebbc0f617ad828d895f504c37ee2d1)
Signed-off-by: Michael Thalmeier <michael.thalmeier@hale.at>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8d78b819c2ec33fce3a34254fa90864ee5fa7617)
Signed-off-by: Jate Sujjavanich <jatedev@gmail.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The cve-check class writes temporary files to preserve state across the
build, and cleans them up in a CookerExit handler.
However, in memory-resident builds the cooker won't exit in between
builds, so the state isn't cleared and the CVE report generation fails:
NOTE: Generating JSON CVE summary
ERROR: Error adding the same package twice
Easily solved by hooking to BuildCompleted, instead of CookerExit.
(From OE-Core rev: ee3270709158aff463fec6798f3b8968268b4d4b)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit fccdcfd301de281a427bfee48d8ff47fa07b7259)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current home path that is compared against is incorrect as it is missing the
package name, this patch adds it.
[YOCTO #14553]
(From OE-Core rev: b75caf4a985e3c20996531785125eaffdc832104)
Signed-off-by: Ahmed Hossam <Ahmed.Hossam@opensynergy.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit ae8f22d9e2694eea5ede3b31c6f3bca404ea4a5a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ignored CVEs aren't patched, but do not apply in our configuration
for some reason. Up till now they were only partially supported
and reported as "Patched".
This patch adds separate reporting of Ignored CVEs. The variable
CVE_CHECK_REPORT_PATCHED now manages reporting of both patched
and ignored CVEs.
(From OE-Core rev: 14b3c0ca46a0aa97565a24b7a5116306237d7cfe)
Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry-picked from c773102d4828fc4ddd1024f6115d577e23f1afe4)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using multiconfig with the same TMP folder we can have
races because the shared recipes like gcc-source run twice.
ARCHIVER_OUTDIR = ${ARCHIVER_TOPDIR}/${TARGET_SYS}/${PF}/
which includes TARGET_SYS and between the two different MACHINE values,
this changes from 'arm-poky-linux-gnueabi' to 'aarch64-poky-linux'.
This leads to the task running twice, once for each multiconfig.
To solve this we need to store the shared output in a common place
for all machines and in this way the stamps will be the same for each
machine so the gcc-source will on run once regardless of the machine used.
(From OE-Core rev: 135adeb82c9303c26193bb6f6bd3bc696793aa62)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5abe497aad39a6ce8d72556fcdda1938a0f8c1bc)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that begin_idx (i.e., beginline - 1) and end_idx (i.e.,
endline) are positive numbers in copy_license_files(). This makes sure
the same lines are copied as populate_lic_qa_checksum() uses when it
calculates the checksum. Before, beginline=0 would typically lead to
that no lines were copied at all.
(From OE-Core rev: 27cd074d747c5ef4b475c8a62a8ede2bbe58f996)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ab3cc3651d08d226675c461da760cda0bb6c0ce0)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Several tasks deactivate exiting on non-zero return codes via set +e because
they run subcommands that have legitimate non-zero return codes. However when
appending to those tasks, this behavior is not expected and can lead to builds
silently proceeding in case of an error. Therefore reset the default behavior
at the end of the respective tasks via set -e.
(From OE-Core rev: 7777cc853db7200b819095be6f6a3561738ac94d)
Signed-off-by: Stefan Wiehler <stefan.wiehler@nokia.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 83a6f28d2e464f00202090e998a63045adba9e4e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Until now the CVE checker was giving information about CVEs found for
a product (or more products) contained in a recipe. However, there was
no easy way to find out which products or recipes have no CVEs. Having
no reported CVEs might mean there are simply none, but can also mean
a product name (CPE) mismatch.
This patch adds CVE_CHECK_COVERAGE option enabling a new type of
statistics. Then we use the new JSON format to report the information.
The legacy text mode report does not contain it.
This option is expected to help with an identification of recipes with
mismatched CPEs, issues in the database and more.
This work is based on [1], but adding the JSON format makes it easier
to implement, without additional result files.
[1] https://lists.openembedded.org/g/openembedded-core/message/159873
(From OE-Core rev: c63d06becc340270573bdef2630749db1f5230d4)
Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit d1849a1facd64fa0bcf8336a0ed5fbf71b2e3cb5)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the cve-check text mode output, we didn't write fragment
files if there are no CVEs (if CVE_CHECK_REPORT_PATCHED is 1),
or no unpached CVEs otherwise.
However, in a system after multiple builds,
cve_check_write_rootfs_manifest might find older files and use
them as current, what leads to incorrect reporting.
Fix it by always writing a fragment file, even if empty.
(From OE-Core rev: 4c10ee956f21ea2f805403704ac3c54b7f1be78c)
Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f1b7877acd0f6e3626faa57d9f89809cfcdfd0f1)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running CVE checks in CI we're usually not interested in warnings on the
console for any CVEs present. Add a configuration option CVE_CHECK_SHOW_WARNINGS
to allow this to be disabled (it is left enabled by default).
(From OE-Core rev: d009233f36fb866f6bdaa12fb6deedf5e253e9c9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1054d3366ba528f2ad52585cf951e508958c5c68)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
(cherry picked from commit 8fd6a9f521ea6b1e10c80fe33968943db30991ba)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before this the rootfs manifest and the summary were identical.
We should separate the summary and rootfs manifest more clearly,
now the summary is for all CVEs and the rootfs manifest is only for
things in that image. This is even more useful if you build multiple
images.
(From OE-Core rev: 2bacd7cc67b2f624885ce9c9c9e48950b359387d)
Signed-off-by: Ernst Sjöstrand <ernstp@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3b8cc6fc45f0ea5677729ee2b1819bdc7a441ab1)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
(cherry picked from commit 65498411d73e8008d5550c2d0a1148f990717587)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As product, sdk should do cve check as well as rootfs.
(From OE-Core rev: df09cd71b4cd3f830fced9ce91aa202c1609bfc5)
Signed-off-by: Lei Maohui <leimaohui@fujitsu.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit cc17753935c5f9e08aaa6c5886f059303147c07b)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The addition of summary output caused two issues: error when building
an image and the fact that JSON output was generated even when
CVE_CHECK_FORMAT_JSON.
When generating an image it caused an error like:
ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_func_python() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:cve_check_write_rootfs_manifest(d)
0003:
File: '/home/alexk/poky/meta/classes/cve-check.bbclass', lineno: 213, function: cve_check_write_rootfs_manifest
0209:
0210: link_path = os.path.join(deploy_dir, "%s.json" % link_name)
0211: manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
0212: bb.note("Generating JSON CVE manifest")
*** 0213: generate_json_report(json_summary_name, json_summary_link_name)
0214: bb.plain("Image CVE JSON report stored in: %s" % link_path)
0215:}
0216:
0217:ROOTFS_POSTPROCESS_COMMAND:prepend = "${@'cve_check_write_rootfs_manifest; ' if d.getVar('CVE_CHECK_CREATE_MANIFEST') == '1' else ''}"
Exception: NameError: name 'json_summary_name' is not defined
The fix is to pass the d variable to the pure python function generate_json_report
to get correct values of variables and add conditions for the JSON
output where needed.
In addition clarify the message presenting the summary JSON file,
which isn't related to an image.
Uses partial fixes from Alex Kiernan, Ernst Sjöstrand (ernstp),
and Davide Gardenal.
Fixes: f2987891d315 ("cve-check: add JSON format to summary output")
(From OE-Core rev: 665f981fccbb09d51349c4bd4cfe4ca91001e3bd)
Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9015dec93233c7d45fd0c9885ff5d4ec23ad377d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We'd like to intercept git calls but we don't want circular references
and HOSTTOOLS currently sets them up. Tweak to avoid them.
(From OE-Core rev: 1567b7cec5ccbe198bfd0cca9ee8a2b1cf6dbf42)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 52c37e133fa55846aca2248ffcf3a10648dbb8d7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
