This function does something similar to the bitbake fetcher file search
paths and is an old remnant of how this could have once been handled. There
are no users in OE-Core and we'd always prefer the fetcher to handle this now
as it handles signatures correctly. The direct OVERRIDES reference would
likely break any user of the function. Remove it to be safe and ensure it
isn't being used anywhere.
(From OE-Core rev: d49ba0243e3d28672d16cd02753eb7e85d91bbab)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of putting the full license text for non-SPDX licenses into the
recipe and every package, use links to the recipe from the packages if
possible.
(From OE-Core rev: 9220d35dc9071ebbe991117af8261ad99f321bb3)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For licenses which are not known to SPDX, find and embed the actual
license text in an ExtractedLicesingInfo block.
(From OE-Core rev: c638f4ae579520cab8cf30e1b4306bd0e2d2a788)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
add_extracted_document() is only called if the license isn't known to
SPDX, so there's no need to check again.
(From OE-Core rev: 03f084251510c810c8434d9ddd459187827c1ca8)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
spdx-licenses.json contains an array of licenses objects. As we'll be
searching it often, convert that to a dictionary when we parse it.
(From OE-Core rev: 3405d9114ca9fe4ba820e0025c91670d1a5150b1)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the previously added general git repo fallback rule the server
specific fallback mirrors for git.savannah.gnu.org and
git.yoctoproject.org are redundant. Remove them.
(From OE-Core rev: da864dbed0c29b3e3f0d1676ec1bb974f332ff3f)
Signed-off-by: Daniel Wagenknecht <dwagenknecht@emlix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Git hosting servers that require a slightly different URL when accessing
repositories via https protocol instead of the git native protocol
are not uncommon (servers using cgit as web UI). Provide a general rule
to try HOST/git/PATH via https as git repo fallback.
(From OE-Core rev: 0682697987873ab395360035b70630acfe66cd3a)
Signed-off-by: Daniel Wagenknecht <dwagenknecht@emlix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't want this warning causing problems on the AB, so leave it
comment out for now
(From OE-Core rev: d41018311abf91899645b8f3160db540abdb5398)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Native recipes do not produce packages and should not process them,
otherwise it can trigger an error in read_subpackage_metadata
(From OE-Core rev: ea077b1ff6a4ccd5241a085bf9a3f67011e0f7bf)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Stop filtering the runtime dependencies based on do_create_sdpx (makes
it only pick up things in DEPENDS) and instead include all task
dependencies that are not the current PN. This allows other dependency
methods to be picked up correctly, for example the dependency on the
kernel used by kernel modules.
(From OE-Core rev: c9659562cf88a820c668ff1eb20814dcb4829392)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Public Domain license (PD) needs a special exception in the license
processing since there is no common license text to be extracted for
these licenses.
(From OE-Core rev: fe5b757712aa99ff1ff10d2304ac320100635200)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
file:// URIs should not be included as the downloadLocation. Instead,
loop until a non-file:// URI is found, or set the location to
NOASSERTION if none is found
(From OE-Core rev: 19c61ec3986213b336d965d90b1329085e7d6179)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the bb.utils.sha* utilities to hash files since they are much faster
than the loops we were rolling ourselves
(From OE-Core rev: a6d9de5350937c7e25899491db59f473345f0b69)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Licenses reported in the SPDX documents should be either:
A) A valid SPDX identifier cross referenced from the SPDX license
database
B) A "LicenseRef" to a license described in the SPDX document
The licensing code will now add a placeholder extracted license with
corresponding "LicenseRef" for any licenses that are not matched to the
SPDX database
Parenthesis in the license expression are now handled correctly
(From OE-Core rev: 28d9d035c0ff8fcaf28bc96a976a43a602a47e94)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will create a more uniq DocumentRef, which will allow
the individual spdx files to be merged into a single SBOM
file reflecting the image. Do the same with the runtime dependencies
also
(From OE-Core rev: df7c88a48621d32c02f328eedc314f10d475b758)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes another creator that was missed earlier
(From OE-Core rev: 046c05fec9fc5162d7a14971ed1402d86605d229)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a debug source cannot be found, mark it as NOASSERTION so that other
tools at least know we were unable to locate it.
(From OE-Core rev: 0e6bdd3f208c50153087c2baca67e9fd64a458d0)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the index to DEPLOYDIR in addition to adding it to the SPDX archive
(From OE-Core rev: 374dc08c0f22e98a267676f71308592d17f77d64)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the creator name since this no longer lives in meta-doubleopen
(From OE-Core rev: 6fdea64ef53eb4de3d0e58e70ae6b391fdff6cd0)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds extended package data which is encoded as JSON which allows it to
encode more structure than the "flat" package data files. The extended
data might be much larger than the standard package data, so it is not
read by default and instead requires
oe.packagedata.read_subpkgdata_extended() to be called
Currently, the file sizes and ELF debug sources are saved off into the
extended package data
(From OE-Core rev: db9cf430e54ae68da80fbc3fba80ce88d8df164d)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Reviewed-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When building cargo-native on centos7 with buildtools tarball installed,
we see failures:
/bin/sh: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by /home/pokybuild/yocto-worker/reproducible-centos/build/build-st/tmp/work/x86_64-linux/cargo-native/1.54.0-r0/recipe-sysroot-native/usr/lib/libtinfo.so.5)
We also see this for libstd-rs once cargo-native is fixed.
The reason for this is that the wrapper script
cargo-native/1.54.0-r0/wrapper/target-rust-ccld has /bin/sh as it's
interpreter and cargo calls this with LD_LIBRARY_PATH set to include the
recipe-sysroot-native. The host /bin/sh links to libtinfo from the host
but it finds the version in the sysroot which needs a newer libc. This
results in the above error since the loader is an older libc and the two
are incompatible.
Our ccld wrapper calls gcc/ld which don't need the LD_LIBRARY_PATH
variable set. We can't patch this out the source since we're using
a prebuilt binary to generate a new cargo binary so this is impossible
to bootstrap.
Instead, put a binary wrapper into place which removes LD_LIBRARY_PATH
from the environment before calling the original wrapper (left in shell
as it is simpler to maintain).
(From OE-Core rev: 8feeeb7f76c6725e5226458c8f22999b67c52694)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
ptest.bbclass has a sanity check that all recipes in oe-core which
inherit ptest are also listed in the ptest-packagelists.inc file, and
the build fails if this is not the case.
Whilst this is a laudable goal, it is over-zealous as if the recipe has
a bbappend in another layer which inherits ptest, the build will fail.
By changing the combination of anonymous Python and bb.error() to a
recipe-scope QA test, this can be handled with the other sanity checks
and bbappends can skip the test if desired.
(From OE-Core rev: abe45c8c0a6da56a278796654d0520250dfd2a97)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make the u-boot binary name configurable. Use the existing variable
UBOOT_BINARYNAME which is evaluated from the UBOOT_BINARY.
(From OE-Core rev: e588cde6aed1e699a055e3481df0f3dc719c9774)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make the SPL suffix configurable via SPL_SUFFIX variable to support SPL
binaries with suffix. The suffix is optional and empty per default. The
delimiter in front of the suffix is added automatically if the suffix
is not empty. A new variable SPL_BINARYFILE contains the binary file
name inclusive optional delimiter and suffix and the old variable
SPL_BINARYNAME contains only the name of the binary without directory,
delimiter and specified suffix. This behavior is backward compatible
with empty SPL_SUFFIX variable.
(From OE-Core rev: e4cfec633c20d6406523da905530e887b853f7ed)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In addition to checking DISTRO_, MACHINE_, and COMBINED_ FEATURES for required
or conflicting features, extend this functionality in order to check
IMAGE_FEATURES in the same way.
(From OE-Core rev: 96a9702a24a73c152c8e7398c3774e63c53a92a9)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, depends.dot includes per-file dependencies but not the packages
providing those files. This makes it hard to obtain all package
dependencies by just looking at depends.dot.
Parse the RPROVIDES and FILERPROVIDES fields from pkgdata to map each of
their values to the package providing the component. Include runtime
packages as dependencies in depends.dot, together with the component
provided by the package as a label.
(From OE-Core rev: 2ba33093017574bbe29eeba699eb90628614d03a)
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Running 'rustc --print <foo>' in -c devshell environment
the target specifications are not known, unless the
RUST_TARGET_PATH (absolute path to rustlib/) is defined.
(From OE-Core rev: 6d0603ffe18c1a5999db854abb668b05aff47f67)
Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable zstd PACKAGECONFIG knob for squashfs-tools and add support for zstd
compressed initramfs and squashfs.
(From OE-Core rev: fa146049eec5f9730781830270cc9dc55a2146f7)
Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
EXTENDPKGV can contain AUTOINC so use an expanded form in the variables allowing
for later expansion during packaging to avoid race issues over the variable and
the build failures that can result.
(From OE-Core rev: 1f93d2426d22c74893e8daec9bb939133c5737c2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, buildhistory does not produce a single file combining relevant
information of installed packages. Produce an output file
"installed-package-info.txt" listing a package's runtime name, buildtime name,
its recipe, version, and size to avoid having to look up each package externally.
Leave the existing package list files as-is for backwards compatibility.
In order to support this efficiently, extend oe-pkgdata-util to accept multiple keys
for its read-value argument.
(From OE-Core rev: 1e18b514bf1f960d324a21db608c8e8e5af007ef)
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Setting the Meson buildtype to "debug" will by default only enable the
-g option to the compiler, which should not matter for OE-Core as it is
already enabled by setting DEBUG_BUILD to 1 in the first place. However,
if the package uses get_option('debug') in its meson.build files to
enable package specific debug code, this will now trigger as intended.
(From OE-Core rev: 7c7c353858a020153be5dba78480b91a28c7a05b)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the kernel configuration enables module signing but no key
is provided, then the kernel generates one during the kernel build.
The current runtime-dependency references (with only package names
without full versions) allow mixed package installations from different
rebuilds of the same kernel version.
This creates an issue because then the modules either don't work
or taint the kernel.
Tighten RDEPENDS with the full package version, i.e. use (= ${EXTENDPKGV})
markers for inter-package dependencies.
The kernel will pull in the kernel-modules subpackage of the same
exact version automatically if KERNEL_SPLIT_MODULES="0" is set.
Otherwise the situation is the same as with the old default with
one subpackage per kernel module where they have to be upgraded
manually.
(From OE-Core rev: 56428d15803733e14d6f1faf5f77eb933ef966f3)
Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For some use cases, a monolithic kernel-modules package containing
all modules built from the kernel sources may be preferred.
For one, download time is shorter and installation time is faster.
Set KERNEL_SPLIT_MODULES="0" for this in. The default is one subpackage
per module.
Also, adapt kernel.bbclass to KERNEL_SPLIT_MODULES != "1" case
Extra RDEPENDS and other inter-package references are needed in
this case.
(From OE-Core rev: 4b13409d82e9d576e32b978f7a42a143127ab894)
Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The -cc wrapper can be used for linking and can fail if key flags are missing:
| error: linking with `[path]/tmp/work/x86_64-linux/rust-native/1.54.0-r0/wrapper/target-rust-cc`
Add the flags to fix builds even if that is counter-intuitive (cc would
normally be used for compiling and ccld for linking).
(From OE-Core rev: 62242e83c49b81a9ea65c9a1f5957a7c309d910a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The oeqa selftest test:
sstatetests.SStateTests.test_sstate_sametune_samesigs
which checks if the sstate checksums of two identical machines
(using the same tune) are the same, apart from changes within
the machine specific stamps directory, fails on the assertion:
self.assertCountEqual(files1, files2)
due to the signature of various 32 bit package builds such as:
x86_64-linux/lib32-rust-cross-i686
x86_64-linux/rust-cross-i686
x86-pokymllib32-linux/lib32-libstd-rs
x86-pokymllib32-linux/lib32-rust
differing. Jumping down the rabbit hole past all the bitbake-diffsig
outputs that differ due to dependent hashes, you come to a diff of:
-Variable MACHINEOVERRIDES value is ${@bb.utils.contains('TUNE_FEATURES', 'mx32', 'x86-x32:', '', d)}${@bb.utils.contains('TUNE_FEATURES', 'm32', 'x86:', '', d)}qemuall:${MACHINE}
+Variable MACHINEOVERRIDES value is ${@bb.utils.contains('TUNE_FEATURES', 'mx32', 'x86-x32:', '', d)}${@bb.utils.contains('TUNE_FEATURES', 'm32', 'x86:', '', d)}qemuall:${MACHINE}:qemux86
in
stamps/x86_64-linux/rust-cross-i686/1.54.0-r0.do_rust_gen_target.<sig>
This is because there are two rust functions referencing OVERRIDES
related variables (target_is_armv7 and llvm_features_from_tune). These
indirectly influnce the build and should be excluded from the signatures
directly as is done in other toolchain recipes, e.g.:
39bfa0dd32 recipes/*-cross recipes: ignore TARGET_ARCH sstate hash
(From OE-Core rev: 72d67410e92207a98a801ddf0cb9f1297a752975)
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the test_sstate_noop_samesigs oe-selftest that
produces an error like:
core2-64-poky-linux/libstd-rs/1.54.0-r0.do_configure.sigdata differs:
basehash changed from <hash-a> to <hash-b>
Variable http_proxy value changed from '' to 'http://example.com/'
by simply removing the proxy config option. This may be added back
after merge to oe-core if needed and if the diffsigs check passes.
(From OE-Core rev: 81ad8aceaacda726361e953778294de20eeac24e)
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch is a follow on of the the PR server rework in bitbake to add
read-only support. The shift to using the bb.asyncrpc code in the PR
server and client brings issues with respect to reuse of the same
asyncio loop in different processes. This patch removes the PR service
connection caching to avoid one source of this problem. It is believed
that in practice this should have little impact on overall performance.
(From OE-Core rev: 0fc3055027e2a76ac863f1c0e0d52e95748066aa)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
utils.bbclass contains create_cmdline_wrapper() function that
creates wrapper script with additional arguments for any passed
"$cmd" command, and uses several calls to "dirname".
Because "dirname" is an external command, in cases of lots of
calls to wrapped "$cmd", each call of "dirname" will incur
significant overhead.
There are three same calls to "dirname": one for saving it`s
output to "realdir" variable, and other two in "exec" command.
So last two "dirname" calls can be replaced with cached value
from "realdir" variable.
(From OE-Core rev: 4b9cf2c80fd14386e0b88a2e6c40a9fa3f1ae0f7)
Signed-off-by: Oleksandr Popovych <opopovyc@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we have:
$ rpm -qp --provides tmp/deploy/rpm/core2_64/bash-5.1.8-r0.core2_64.rpm
bash = 5.1.8-r0
This is incorrect as bash provides /bin/bash and /bin/sh. This is caused by
incomplete conversion of new override syntax, which breaks the per-file
parsing of package_rpm.
With this patch, we have:
$ rpm -qp --provides tmp/deploy/rpm/core2_64/bash-5.1.8-r0.core2_64.rpm
/bin/bash
/bin/sh
bash = 5.1.8-r0
(From OE-Core rev: f186d32ff8bf176ad0bd8d49e47cdb017c701a46)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Small golang applications create massive memory overhead if go-runtime
is deployed. So it is helpfull to be able to disable the GO_DYNLINK
option on a per distro/local.conf basis by making it a ?= instead of =.
(From OE-Core rev: 2a7f5843c4ad2a3bd44bf9c262aacab2931b5677)
Signed-off-by: Oliver Kranz <o.kranz.88@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- bitbake BB_NUMBER_THREADS uses cpu_count from oe utils that uses
the python os.sched_getaffinity and it is more acurrate.
grep -nH ^BB_NUMBER_THREADS meta/conf/bitbake.conf
meta/conf/bitbake.conf:806:BB_NUMBER_THREADS ?= "${@oe.utils.cpu_count()}"
- multiprocessing.cpu_count() returns the number of CPUs on the host,
not the number of usable CPUs on the host. If the user is using
scheduler affinity then the number of usable CPUs may be less,
so when determining how many cores we can use check the affinity instead.
(From OE-Core rev: 45c52f08a289c6eb2329de50634a0406204d1d8e)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With glibc 2.34, the libraries were renamed. Tweak the glob to support both
as this is needed for newer uninative versions.
[RP: tweak commit message]
(From OE-Core rev: 98248306e4b5f023e96375293b60524574ebb686)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes pkgdata PKGSIZE info after the overrides change.
(From OE-Core rev: 6964f06e48c7002c9ad788aa04bd8873fb3ee024)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
