We've some long standing bugs where the RDEPENDS from -dev packages causes
problems, e.g. dropbear and openssh components on an image working fine together
but then the SDK failing to build as the main openssh and dropbear packages
conflict with each other (pulled in by openssh-dev and dropbear-dev).
We propose changing the behavour of complementary package installation to
ignore RRECOMMENDS. If we then change the ${PN}-dev dependency on ${PN}
to a RRECOMMENDS, we can avoid many of the issues people run into yet still
have the desired behaviour of ${PN}-dev pulling in ${PN}.
This therefore changes the package manager code so that it doesn't follow
RRECOMMENDS for completementary package globs.
[RP: Added deb support]
(From OE-Core rev: b44b0b9294675f89aa51ff84f532664f4c479677)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Helper module to:
- generate meta information from package.json content. This data has
a format as provided by https://registry.npmjs.org
- put this meta information and the corresponding tarball in the
nodejs cache. This uses an external, nodejs version specific helper
script (oe-npm-cache) shipped in oe-meta
To avoid further nodejs version dependencies, future versions of this
module might omit the caching completely and serve meta information
and tarball by an http server.
(From OE-Core rev: 6cd5886ad05fee704e8a5892bd370c360c8c3b54)
Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reduce code duplication to a common function in oe.buildcfg.
(From OE-Core rev: 0b191a1df4c3722defb09dde0c16b1d9e7fe7ef6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This isn't used anywhere and everyone is using git now, the functions
are now hardcoded as such too. Drop this function.
(From OE-Core rev: 09ba96a2d7fa02c7a82758bbf4109b04ffca2c55)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The comments even say this was copy/paste code. Move to a
shared library function.
(From OE-Core rev: ac3de2f850a418673b87e1c454970cb099e191b0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a load of duplicated git/layer/revision code which makes
most sesne as a python library, not bbclass code. Start to refactor as such.
(From OE-Core rev: 439cdf8a1e52fd2c4dc81dc40ce7e6af282ce7ac)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With CONFIG_MODULE_COMPRESS_ZSTD enabled, kernel modules will have a
.ko.zst extension. This fixes depmod not being run.
Fixes: 1b696a45ddb ("rootfs.py: Add check for kernel modules before running depmod")
(From OE-Core rev: 425efac7110f0f42d70643e0a448e834d0f01a7a)
Signed-off-by: Sean Anderson <sean.anderson@seco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the function to a library, it could be useful in other places.
(From OE-Core rev: debd37abcdde8788761ebdb4a05bc61f7394cbb8)
Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
sstate filename generation was changed a while ago and taskname has to be
passed into core functions for the correct filename to be generated. Update
find_siginfo to match those changes and pass in taskname via SSTATE_CURRTASK.
Thanks to Gregory Lumen <gregorylumen@microsoft.com> for spotting.
[YOCTO #14774]
(From OE-Core rev: 51d7310b6cf8ef9033d461868c07f686656070ba)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a remote patch is compressed we need to have run the unpack task for
the file to exist locally. Currently cve_check only depends on fetch so
instead of erroring out, emit a warning that this file won't be scanned
for CVE references.
Typically, remote compressed patches won't contain our custom tags, so
this is unlikely to be an issue.
(From OE-Core rev: cefc8741438c91f74264da6b59dece2e31f9e5a5)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In bitbake commit 1ecc1d94 (process: Do not mix stderr with stdout),
bb.process.Popen() was changed to no longer combine stdout and stderr by
default. However, the Terminal class was not updated to reflect this and
subsequently only output stdout in case of failures.
(From OE-Core rev: 116d0bb07ba044cf8847bf3d5c3996ad7e58b7ae)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implement debian package repository signature.
For each Release file created in repository subdirectory, a signature
Release.gpg is created.
Signature is performed using gpg backend when the following variables
are set in local.conf:
PACKAGE_CLASSES += "sign_package_feed"
PACKAGE_FEED_GPG_NAME = "<Id of GPG key>"
PACKAGE_FEED_GPG_PASSPHRASE_FILE="<path to password file>"
(From OE-Core rev: fcc3cee276999efe6402959eb295e7a0e1e96f96)
Signed-off-by: Xavier Berger <xavier.berger@bio-logic.net>
Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
output_suffix: If defined, add output_suffix as file name extension.
use_sha256: If True, use sha256 for gpg as digest algorithm
(From OE-Core rev: cfcaa54dc73925df448099fb60f75b18350b2a3b)
Signed-off-by: Xavier Berger <xavier.berger@bio-logic.net>
Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add an option to output the CVE check in a JSON-based format.
This format is easier to parse in software than the original
text-based one and allows post-processing by other tools.
Output formats are now handed by CVE_CHECK_FORMAT_TEXT and
CVE_CHECK_FORMAT_JSON. Both of them are enabled by default.
The JSON output format gets generated in a similar way to the
text format with the exception of the manifest: appending to
JSON arrays requires parsing the file. Because of that we
first write JSON fragments and then assemble them in one pass
at the end.
(From OE-Core rev: df567de36ae5964bee433ebb97e8bf702034994a)
Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add ${IMAGE_ROOTFS} and ${IMGDEPLOYDIR} to do_rootfs[dirs] and
do_rootfs[cleandirs], this ensures do_rootfs run from a clean
workspace, with this change, we can now remove two bb.utils.mkdirhier
lines from meta/lib/oe/rootfs.py.
(From OE-Core rev: cb8b6f7eee4e059bb311330c57068e11bc477366)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Layers could be located anywhere. The eSDK should work with them even if
they are not located in TOPDIR or in the same parent directory as
COREBASE.
For layers located in the same parent directory as COREBASE this preserves
the intent from the previous
copy_buildsystem: include layer tree during build structure creation
commit.
Related OE-Core rev: 5a59a6997f41e606d088e3e86812de56f72f543b
(From OE-Core rev: 16d330d42e03085769eddb1b60ba1df7228baf36)
Signed-off-by: Daniel Wagenknecht <dwagenknecht@emlix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On modern Power systems `uname -m` yields 'ppc64le' while the toolchain
knows the architecture as 'powerpc64le'. Provide a mapping from one to
the other to integrate with the existing architecture configuration
flags.
arch_to_rust_arch() only exists to map the OE *_ARCH variables before
any further processing, unlike arch_to_rust_target_arch() which is
specific to the internal triple handling of rust.
On Linux ppc64le systems the changes give the following config:
```
$ cat ./tmp/work/ppc64le-linux/rust-native/1.58.0-r0/targets/ppc64le-linux.json
{
"llvm-target": "powerpc64le-unknown-linux-gnu",
"data-layout": "e-m:e-i64:64-n32:64-v256:256:256-v512:512:512",
"max-atomic-width": 64,
"target-pointer-width": "64",
"target-c-int-width": "64",
"target-endian": "little",
"arch": "powerpc64",
"os": "linux",
"env": "gnu",
"vendor": "unknown",
"target-family": "unix",
"linker": "gcc",
"cpu": "generic",
"dynamic-linking": true,
"executables": true,
"linker-is-gnu": true,
"linker-flavor": "gcc",
"has-rpath": true,
"has-elf-tls": true,
"position-independent-executables": true,
"panic-strategy": "unwind"
}
```
Change-Id: Ief0c01189185d7d4da31d307270bec4e1de674ca
(From OE-Core rev: 9ab61e3cfef0157393cb870d606c2f362e190889)
Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want to use SPDX identifiers in LICENSE variables. There is now a
conversion script to make most of the translations. Add a list of
strings which have been replaced so we can show warnings to users
if they're still used anywhere.
Add checks to the package as insane check. This is currently a warning
by default but can be turned off or made an error as per the other standard
checks.
(From OE-Core rev: 9379f80f484f94686a4d494e9e237fadfb72a938)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This re-writes the INCOMPATIBLE_LICENSE checking code to replace
the WHITELIST_<lic> with
INCOMPATIBLE_LICENSE_EXCEPTIONS = '<pkg>:<lic> <pkg>:<lic> ...'
This initial change leaves most of the code structure in place,
but the code in base.bbclass needs to be re-written to make
the check more consistent around packages (PKGS) and not recipe
names (PN). This also is taking into account the changes for SPDX
licenses.
The aim is to provide a mode consistent variable where the variable
name is known and can easily be queried.
(From OE-Core rev: 0d19c45ba6cf43518f380ca5afe9753a2eda0691)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is redundant (target arch is already in PN), and breaks
compiling a cross-canadian toolchain, as that needs populating the
sysroot with two different native-hosted toolchains built from
cross recipes. Inserting TARGET_ARCH allows only one or the other.
(From OE-Core rev: 33fc1792cd782feb8dbb4285e3006bb588f7978f)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous commit introduced a small logic error. Fix the renaming issue.
(From OE-Core rev: b01e10b27d23ea1b4ac58376c2423505a70832d6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There were few bugs in the _isInitialized() function which might trigger
git repo to be reinitialized and patches failing to apply.
(From OE-Core rev: 80500ecda4c1bc8812e6e078b6b0db5ec46624de)
Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, SPDX SBOMs are only created for images. Add support for
SDKs.
(From OE-Core rev: c3acbb936a339636153903daf127eec9f36de79b)
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updates within bitbake now require this call to be triggered somewhere with
floating git source revs. Add the missing call. Issue can be reproduced with:
devtool check-upgrade-status dbus-wait
(From OE-Core rev: d7088b2b5012f9e386914a6837685f5869379cc5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was prompted by ltp's unreasonably long package_qa times; it has
a massive amount of executables and insane runs objdump for all of
them, serially.
This reduces the time from 4 minutes to 1m20s on my machine.
(From OE-Core rev: fac984b99fdb46949879516cb87153860f402c75)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds the set helper for list properties to allow list members to be
replaced with a new list. This is necessary as it is (currently) the
only way to replace the default value for a list member.
(From OE-Core rev: 8280b857c510646cd10167dd2ea9a0c2d5f1e800)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prelink is being dropped by glibc in 2.36. It already causes issues with
binary corruption, has a number of open bugs and is of questionable benefit
without disabling load address randomization and PIE executables.
We disabled it by default a while back but left people able to use it.
We would be unable to maintain it alone without glibc support so remove
the remaining pieces.
(From OE-Core rev: 23c0be78106f1d1e2bb9c724174a1bb8c56c2469)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Native task outputs are directly run on the build system during the build
after being built. Even if the output of a native recipe doesn't change, a
change in one of its dependencies may cause a change in the output it
generates (e.g. rpm output depends on the output of its dependent zstd
library).
This can cause poor interactions with hash equivalence, since this
recipe's output-changing dependency is "hidden" and downstream tasks only
see that this recipe has the same outhash and therefore is equivalent.
This can result in different output in different cases and issues with
reproducible builds in parcular (e.g. rpm compression changes for the same
content).
To resolve this, unhide the output-changing dependency by adding it's
unihash to this tasks outhash calculation. Unfortunately, we don't know
specifically know which dependencies are output-changing, so we have to
add all of them.
[YOCTO #14685]
(From OE-Core rev: d6c7b9f4f0e61fa6546d3644e27abe3e96f597e2)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extend runstrip() to accept additional argument to enable
sharing it with the kernel do_strip() so that
KERNEL_IMAGE_STRIP_EXTRA_SECTIONS can be passed.
Since is_elf() understands kernel modules there is no need to keep a
seperate list for kernmodules or hardcode the values to runstrip.
(From OE-Core rev: e09a8fa931fe617afc05bd5e00dca5dd3fe386e8)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since honister host manifest stopped to generate, i.e.
manifest file is empty but all ipks/files into sdk is
ok.
(From OE-Core rev: 79a2392f5d2a4cb6509a83afb40bca01bac59914)
Signed-off-by: Oleksiy Obitotskyy <oobitots@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If PATCHTOOL="git" has been specified but workdir is not git repo
bitbake fails to apply the patches with error message:
Command Error: 'git rev-parse --show-toplevel' exited with 0 Output:
fatal: not a git repository (or any of the parent directories): .git
Fix this by initializing the repo before patching.
This allows binary git patches to be applied.
(From OE-Core rev: 6184b56a7a0fc6f5d19fdfb81e7453667f7da940)
Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
distutils is going away and we have functionality in bitbake which can
handle these comparisions so switch to the bb.utils function.
(From OE-Core rev: fe624b520e6c75e16a8f394785ab0216341402f9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Larger systems may have large numbers of cores, but beyond a certain
point they can't all be used for compiling: whilst purely
compute-intensive jobs can be parallelised to hundreds of cores,
operations such as compressing (needs lots of RAM) or compiling (lots of
I/O) don't scale linearly.
For example, the Marvel ThunderX2 has 32 cores, each capable of
executing four threads, and can be configured with two sockets, making
256 CPUs according to Linux. Zstd using 256 threads has been seen to
fail to allocate memory during even small recipes such as iso-codes.
Add a default cap of 64 CPUs to the cpu_count() method so that extreme
parallisation is limited. 64 is high enough that meaningful gains
beyond it are unlikely, but high enough that most systems won't be
effected.
(From OE-Core rev: 765d0f25ce48636b1838a5968e2dc15de2127428)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When looking at logs involving thread pools it is useful if the threads
can be named.
(From OE-Core rev: 18342945b021608794d83ecf567afd43f4379b24)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Application can depend on several overlayfs mount points. Provide a
systemd unit application can depend on to make sure all overlays are
mounted before it is started to avoid any race conditions
(From OE-Core rev: b38e194db0c6825f28c56123cf88af94d3f52beb)
Signed-off-by: Bruno Knittel <Bruno.Knittel@bruker.com>
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having annotations on relationship can provide additional information
about the relationship such as how it was derived.
(From OE-Core rev: d98585aa89e1d3819f8139a07fb7376ef89b37f8)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the function a ThreadedWorker is executing raises an exception, don't
use print() as that mostly disappears. Instead, output it to the logger.
This is done using bb.mainlogger.debug directly instead of bb.debug() as
this allows us to pass the exception instance directly, which is then
incorporated into the log stream.
(From OE-Core rev: 2f1ea25c222b344dd8b784b2bc73a6540ab30274)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extract package_qa_write_error, package_qa_handle_error and
package_qa_add_message functions from insane.bbclass to lib/oe/qa.py and
drop the package_qa_ prefixes.
Update various bbclasses to use the new functions. No import is required
since base.bbclass puts oe.qa in OE_IMPORTS.
Stop requiring callers to manually track whether a fatal error has been
encountered via a "sane" flag. Instead replace the QA_SANE variable with
QA_ERRORS_FOUND and call oe.qa.exit_if_errors or
oe.qa.exit_with_message_if_errors at the end of each task.
Inspired by discussion resulting from
https://lists.openembedded.org/g/openembedded-core/message/156793 and
https://lists.openembedded.org/g/openembedded-core/message/156900
(From OE-Core rev: f0ad152ef4cc15c042bc9eeefb6af096d054b220)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want things to be reproduicble and the variable doesn't really change
much any more. Drop the remaining uses and make those code paths always
active.
(From OE-Core rev: d15fb02c7ee7da50e322d74bc6a545234e20c7f3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reproducibility is here to stay and needs to be part of our default workflow.
Move the remaining code to base.bbclass so it is always a first class citizen
and it is clear people need to be mindful of it.
(From OE-Core rev: abb0671d2cebfd7e8df94796404bbe9c7f961058)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
