Useful to work around shebang relocation issues, where
shebangs are too long or have arguments in them, thus preventing them
from using the /usr/bin/env shebang.
(From OE-Core rev: 6edc1fffcbe1405d8c309a75643d7d6cd9a92848)
Signed-off-by: Paulo Neves <ptsneves@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As reported in the bug report [1], there was no check for shebang
sizes on native scripts and now this is fixed.
The path scope of the qa_staging was increased from just checking
libdir to all the relevant SYSROOT_DIRS.
It is possible to skip this check through INSANE_SKIP.
[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053
(From OE-Core rev: 377fe11bc0d6939ab1aaebab1bf4e55adca1ab15)
Signed-off-by: Paulo Neves <ptsneves@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The cve-check class writes temporary files to preserve state across the
build, and cleans them up in a CookerExit handler.
However, in memory-resident builds the cooker won't exit in between
builds, so the state isn't cleared and the CVE report generation fails:
NOTE: Generating JSON CVE summary
ERROR: Error adding the same package twice
Easily solved by hooking to BuildCompleted, instead of CookerExit.
(From OE-Core rev: fccdcfd301de281a427bfee48d8ff47fa07b7259)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The last time of sampling would be updated within the SystemStats class
but not re-recorded into the datastore, leading to multiple samples being
collected in the same second in the sample function of buildstats.py.
Fix this to collect and store only one sample per second within a
certain tolerance to deal with variation in the arrival time.
This fix elimates the spikiness of sampled data, in cases where the difference
between the current and the last sample is taken. Previously, since many
samples per second were recorded, certain types of data would result in a
very small elapsed time and hence a small numerical difference. For example,
the CPU usage from /proc/stat is a running total of usage and taking the
difference between data collected 0.1 seconds apart would result in usage
appearing lower than it actually was.
(From OE-Core rev: 0e2df45ab066bb4ad2c4f8622ee9c1a8ecdea9cb)
Signed-off-by: Aryaman Gupta <aryaman.gupta@windriver.com>
Signed-off-by: Randy MacLeod <randy.macleod@windriver.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With nodejs 16, the simple 'npm cache add' approach does not work
anymore because its fetcher implementation downloads also meta
information from the registry.
We have to generate these information and add them to the cache.
There is no direct support in 'npm' for task so we have to implement
it manually.
This implementation consists of a openembedded python module (in
oe-core) and a nodejs version specific helper (in oe-meta).
(From OE-Core rev: 019b9c341d539939098962c228c1fd5c99331312)
Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We know the content of 'package.json' from earlier patches; there is
no need to parse the tarball name to extract the version.
(From OE-Core rev: f553e528e76f7e3925ed1c0950d96e73aec37da9)
Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have to read 'package.json' to calculate the name of the tarball.
This content is interesting for later patches.
(From OE-Core rev: d67367e389c492ae90f9021066d6a4d5ebcf68e5)
Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'npm pack' is a maintainer tool which tries to execute 'prepare'
and similar scripts. This fails usually in OE because it requires
completely installed 'node_modules'.
Earlier nodejs versions supported an undocumented 'ignore-scripts'
option. This has been removed in nodejs 16.
We could patch 'package.json' and remove the unwanted scripts. But
this might complicate local workflows (applying patches) and installed
packages will contain the modified 'package.json'.
Instead of, package it manually by 'tar czf'. As a sideeffect,
'do_configure' is running much faster now.
(From OE-Core rev: 68b480d64ffb6750699cc8fa00d2ac0bc6a2e58a)
Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test for obsolete licenses used in INCOMPATIBLE_LICENSE_EXCEPTIONS
tried to match the "<package>:<license>" tuples with the obsolete
licenses and thus never matched anything.
(From OE-Core rev: 3ad994d95815eefed2a72b675c7a323b3ed38191)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since oe_runmake wrapper doesn't by default involve PARALLEL_MAKE
outside do_compile() nor PARALLEL_MAKEINST outside do_install(),
enable parallellized make by default when oe_runmake is invoked from
do_compile_ptest() or do_install_ptest() by declaring wrapper task
specific EXTRA_OEMAKE overrides in fashion similar to do_compile and
do_install overrides in meta/conf/bitbake.conf.
Parallel make can still be disabled by resetting bbclass specific
PTEST_PARALLEL_MAKE and PTEST_PARALLEL_MAKEINST variables in recipe
e.g. if a race issue needs to be avoided without modifying source code.
Tested by issuing following command sequence on a 32-core build host:
$ bitbake -c clean util-linux && bitbake --skip-setscene -c compile util-linux && time bitbake --skip-setscene -c compile_ptest_base util-linux
and found that before this change the result was
real 0m34.684s
user 0m0.753s
sys 0m0.131s
and after this change
real 0m9.868s
user 0m0.749s
sys 0m0.150s
(From OE-Core rev: 5c09d2a017add8bd34142c6c1543e2d0f9ead286)
Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current home path that is compared against is incorrect as it is missing the
package name, this patch adds it.
[YOCTO #14553]
(From OE-Core rev: ae8f22d9e2694eea5ede3b31c6f3bca404ea4a5a)
Signed-off-by: Ahmed Hossam <Ahmed.Hossam@opensynergy.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Recipes in core are now needing to use the Hatch build system, so
move the hatchling recipe and class from meta-oe to oe-core.
(From OE-Core rev: 846e806181f1349be29cbce78c5041735dfd7e6f)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ignored CVEs aren't patched, but do not apply in our configuration
for some reason. Up till now they were only partially supported
and reported as "Patched".
This patch adds separate reporting of Ignored CVEs. The variable
CVE_CHECK_REPORT_PATCHED now manages reporting of both patched
and ignored CVEs.
(From OE-Core rev: c773102d4828fc4ddd1024f6115d577e23f1afe4)
Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using multiconfig with the same TMP folder we can have
races because the shared recipes like gcc-source run twice.
ARCHIVER_OUTDIR = ${ARCHIVER_TOPDIR}/${TARGET_SYS}/${PF}/
which includes TARGET_SYS and between the two different MACHINE values,
this changes from 'arm-poky-linux-gnueabi' to 'aarch64-poky-linux'.
This leads to the task running twice, once for each multiconfig.
To solve this we need to store the shared output in a common place
for all machines and in this way the stamps will be the same for each
machine so the gcc-source will on run once regardless of the machine used.
(From OE-Core rev: 5abe497aad39a6ce8d72556fcdda1938a0f8c1bc)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rename the default file from "build" to "buildinfo" since this is more
obvious to anyone looking in an image.
Add SDK_BUILDINFO_FILE and allow the same information to be written into
SDK images. This will be useful for buildtools-tarball and uninative-tarball.
(From OE-Core rev: 38b9525f1d5377288922e7cbac3f30afe6046242)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reduce code duplication to a common function in oe.buildcfg.
(From OE-Core rev: 0b191a1df4c3722defb09dde0c16b1d9e7fe7ef6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The comments even say this was copy/paste code. Move to a
shared library function.
(From OE-Core rev: ac3de2f850a418673b87e1c454970cb099e191b0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a load of duplicated git/layer/revision code which makes
most sesne as a python library, not bbclass code. Start to refactor as such.
(From OE-Core rev: 439cdf8a1e52fd2c4dc81dc40ce7e6af282ce7ac)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Someone reported that if some other shell function has left i or j set,
the concat_dtb_helper function could fail. Add a small tweak to avoid this.
[YOCTO #14815]
(From OE-Core rev: d219c97bdf5d30be89795fbf9b66ddc367bef384)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If IMAGE_LINK_NAME and IMAGE_MANIFEST are equal
don't create a link otherwise it will create it
to itself.
(From OE-Core rev: 8fd1a37c38dbfc688d840d12c61d76f151f9f605)
Signed-off-by: Davide Gardenal <davide.gardenal@huawei.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can't build glibc without make 4.0 and we don't work on older distros with
older versions of make without buildtools tarball so increase the minimum
version to 4.0.
(From OE-Core rev: e9a449aeec72a698206323c3471c9268f949b1ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that begin_idx (i.e., beginline - 1) and end_idx (i.e.,
endline) are positive numbers in copy_license_files(). This makes sure
the same lines are copied as populate_lic_qa_checksum() uses when it
calculates the checksum. Before, beginline=0 would typically lead to
that no lines were copied at all.
(From OE-Core rev: ab3cc3651d08d226675c461da760cda0bb6c0ce0)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make empty return types in check_cvs the same for all code paths.
(From OE-Core rev: f86393c93dec47b24e837d0c4c5761a716ecdbb6)
Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the cve-check text mode output, we didn't write fragment
files if there are no CVEs (if CVE_CHECK_REPORT_PATCHED is 1),
or no unpached CVEs otherwise.
However, in a system after multiple builds,
cve_check_write_rootfs_manifest might find older files and use
them as current, what leads to incorrect reporting.
Fix it by always writing a fragment file, even if empty.
(From OE-Core rev: f1b7877acd0f6e3626faa57d9f89809cfcdfd0f1)
Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the function to a library, it could be useful in other places.
(From OE-Core rev: debd37abcdde8788761ebdb4a05bc61f7394cbb8)
Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A previous fix for zero length bb_unihashes.dat files wasn't complete
as there is a second copy of the unihashes file made. Change this second
call site to match the first to fully fix the zero length file issue.
(From OE-Core rev: 962bd78af1acc86b3d900dce9081ed310616e0fa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Several tasks deactivate exiting on non-zero return codes via set +e because
they run subcommands that have legitimate non-zero return codes. However when
appending to those tasks, this behavior is not expected and can lead to builds
silently proceeding in case of an error. Therefore reset the default behavior
at the end of the respective tasks via set -e.
(From OE-Core rev: 83a6f28d2e464f00202090e998a63045adba9e4e)
Signed-off-by: Stefan Wiehler <stefan.wiehler@nokia.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OverlayFS systemd helper unit might require more pre-processing
commands. It gets more complicated to embed them in a unit file, because
systemd shell subset is limited and might require additional escaping.
Move the command to a separate script, thus simplifying systemd unit.
(From OE-Core rev: 86a457016e7f3fc7acacf86cd87f5d8d882132dd)
Signed-off-by: Vyacheslav Yurkov <v.yurkov@precitec.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a race were the bb_unihashes.dat file may end up zero sized due to
concurrent builds. Use recently added API within bitbake to copy the file
safely. Also use the opportunity to remove hardcoded filepaths internal to
bitbake from OE-Core.
Bump the minimum bitbake version to match the API being used.
(From OE-Core rev: 106a7e07057e4681e6fb06dfe9e28d2e183d667d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running CVE checks in CI we're usually not interested in warnings on the
console for any CVEs present. Add a configuration option CVE_CHECK_SHOW_WARNINGS
to allow this to be disabled (it is left enabled by default).
(From OE-Core rev: 1054d3366ba528f2ad52585cf951e508958c5c68)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Building native Go tools results in the tool pointing to the wrong
location of dynamic linker (see below). The linker is looked up in the
temporary dir, which can be removed if rm_work is inherited. This
results in being unable to execute the program with the 'No such file or
directory' error. Override linker specificiation for native recipes (and
let Go build environment to pick up a correct one on it's own).
The error is observed in case the distro doesn't use uninative.bbclass.
If uninative.bbclass is used, the binary will be patched automatically
to use the uninative loader instead of the system one.
Without this patch:
$ ldd tmp-rpb-glibc/sysroots-components/x86_64/go-md2man-native/usr/bin/go-md2man
linux-vdso.so.1 (0x00007ffe945ec000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3a7490e000)
/home/lumag/Projects/RPB/build-rpb/tmp-rpb-glibc/work/x86_64-linux/go-md2man-native/1.0.10+gitAUTOINC+f79a8a8ca6-r0/recipe-sysroot-native/usr/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007f3a74d13000)
$ tmp-rpb-glibc/sysroots-components/x86_64/go-md2man-native/usr/bin/go-md2man --help
-bash: tmp-rpb-glibc/sysroots-components/x86_64/go-md2man-native/usr/bin/go-md2man: No such file or directory
With the patch
$ ldd tmp-rpb-glibc/sysroots-components/x86_64/go-md2man-native/usr/bin/go-md2man
linux-vdso.so.1 (0x00007ffd19dbf000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f2d44181000)
/lib64/ld-linux-x86-64.so.2 (0x00007f2d44586000)
$ tmp-rpb-glibc/sysroots-components/x86_64/go-md2man-native/usr/bin/go-md2man --help
Usage of tmp-rpb-glibc/sysroots-components/x86_64/go-md2man-native/usr/bin/go-md2man:
-in string
Path to file to be processed (default: stdin)
-out string
Path to output processed file (default: stdout)
(From OE-Core rev: 44b397daa68b4d0a461225fe9ff7db8b5fcfdb7b)
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* this doesn't fix any issue (at least AFAIK),
just keeps the log files more deterministic to avoid unnecessary churn like in:
--- /OE/build/oe-core/tmp-glibc/work/qemux86_64-oe-linux/keymaps/1.0-r31/temp/log.do_patch.2213051 2022-05-26 11:35:44.110063749 +0200
+++ /OE/build/oe-core/tmp-glibc/work/qemux86_64-oe-linux/keymaps/1.0-r31/temp/log.do_patch.2213372 2022-05-26 11:35:54.553062630 +0200
@@ -1,7 +1,7 @@
DEBUG: Executing python function extend_recipe_sysroot
-NOTE: Direct dependencies are ['virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-devtools/patch/patch_2.7.6.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_populate_sysroot']
+NOTE: Direct dependencies are ['/OE/build/oe-core/openembedded-core/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_populate_sysroot', 'virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-devtools/patch/patch_2.7.6.bb:do_populate_sysroot']
NOTE: Installed into sysroot: []
-NOTE: Skipping as already exists in sysroot: ['patch-native', 'quilt-native', 'attr-native', 'libtool-native', 'gettext-minimal-native', 'texinfo-dummy-native']
+NOTE: Skipping as already exists in sysroot: ['quilt-native', 'patch-native', 'attr-native', 'libtool-native', 'gettext-minimal-native', 'texinfo-dummy-native']
DEBUG: Python function extend_recipe_sysroot finished
DEBUG: Executing python function do_patch
DEBUG: Executing python function patch_do_patch
(From OE-Core rev: d3ebb37b97da15166d452bf51f5f7e0c312ae42e)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 47d9d90b4ec7d04d6f3f1a9b97c0ab7f1264a88e.
This is no longer needed with a patch to fix importlib in python.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before this the rootfs manifest and the summary were identical.
We should separate the summary and rootfs manifest more clearly,
now the summary is for all CVEs and the rootfs manifest is only for
things in that image. This is even more useful if you build multiple
images.
(From OE-Core rev: 3b8cc6fc45f0ea5677729ee2b1819bdc7a441ab1)
Signed-off-by: Ernst Sjöstrand <ernstp@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow recipe to set a custom postinst for kernel image.
Use prependVar in order to allow custom postinst to be
ran after default postinst, potentially modifying the
installed file/symlink.
(From OE-Core rev: 579906b9f4ea32c8d4e0e11a87ee6c9e27cdf0dc)
Signed-off-by: Raphael Teller <raphael.teller@se.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Building with MACHINE=qemuarm and MACHINE=qemux86-64 causes rust-native
to rebuild. The reason is all the ingredients into the RUST_*_SYS
variables are being tracked but that doesn't really make sense in this
case:
$ bitbake-diffsigs tmp/stamps/x86_64-linux/rust-native/1.60.0-r0.do_rust_gen_targets.sigdata.63* tmp/stamps/x86_64-linux/rust-native/1.60.0-r0.do_rust_gen_targets.sigdata.34*
NOTE: Starting bitbake server...
basehash changed from 860b8f11b10182dc5b2737f62cdb697477f714adb63eeb4d4b932d67cac8eec2 to 9379e8b9df9696e8056fec7d1534661f34dda073f6d816e241b09a2dff76ae2d
Variable rust_base_triple value changed:
@@ -36,4 +36,4 @@
# In some cases uname and the toolchain differ on their idea of the arch name
-TUNE_FEATURES{callconvention-hard} = Set
+TUNE_FEATURES{callconvention-hard} = Unset
Switch to using the final variable value for sstate signature purposes
which avoids this issue.
(From OE-Core rev: 38b61aeaf877f6a34c3c0060c175119975ebd782)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Isolate the writing of the UBI configuration in a separate function.
This allows to use a custom configuration easily by overriding the
function.
Replace the echo functions by a cat to have a more readable configuration.
(From OE-Core rev: 601294b35c83faa134e26844c65db59b42d5052f)
Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To support crate:// fetcher on externalsrc, we need to remove "-z
${EXTERNALSRC} check of bitbake vendoring. It is possible to disable
vendoring by CARGO_DISABLE_BITBAKE_VENDORING = "1" if externalsrc-ed
project does not want to enablt it.
(From OE-Core rev: 3d8a388ba540598fec83d505d62ac36642bb90c7)
Signed-off-by: Chanho Park <chanho61.park@samsung.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
