There have been errors seen when assembling root file system SPDX
documents where they will references files that don't exist in the
package SPDX.
The speculation is that this is caused by os.walk() ignoring errors when
walking, causing files to be omitted. Improve the code by adding an
error handler to os.walk() to report errors when they occur.
In addition, sort the files and directories while walking to ensure
consistent ordering of the file SPDX IDs.
(From OE-Core rev: 86b581e80637cd8136ce7a7e95db94d9553d2f60)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a "contains" relationship that relates the root file system package
to the files contained in it. If a package provides a file with a
matching hash and path, it will be linked, otherwise a new File element
will be created
(From OE-Core rev: e6fe754aef93e834e5226c8b13fdf75e03080ba2)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes a bug in the code that created the hash dictionary that was
accidentally excluding items with a SHA256 hash instead of including
them
(From OE-Core rev: 90765467e42241d8b572d035389d7062a0316a9f)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
create_image_spdx() implementation assumes that image is indeed a file.
If image recipe deploys a directory (for example, which contains an
hierarchy of flash artifacts, that is used by SoC vendor-specific
flashing tool) which follows ${IMAGE_NAME}.${IMAGE_TYPE} naming scheme,
create_image_spdx() function will fail after trying to hash a directory:
*** 0002:do_create_image_spdx(d)
0003:
File: '.../meta/classes-recipe/create-spdx-image-3.0.bbclass', lineno: 48, function: do_create_image_spdx
0044:addtask do_create_rootfs_spdx_setscene
0045:
0046:python do_create_image_spdx() {
0047: import oe.spdx30_tasks
*** 0048: oe.spdx30_tasks.create_image_spdx(d)
0049:}
0050:addtask do_create_image_spdx after do_image_complete do_create_rootfs_spdx before do_build
0051:SSTATETASKS += "do_create_image_spdx"
...
File: '.../bitbake/lib/bb/utils.py', lineno: 536, function: _hasher
0532:
0533:def _hasher(method, filename):
0534: import mmap
0535:
*** 0536: with open(filename, "rb") as f:
0537: try:
0538: with mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as mm:
0539: for chunk in iter(lambda: mm.read(8192), b''):
0540: method.update(chunk)
Exception: IsADirectoryError: [Errno 21] Is a directory: '...'
(From OE-Core rev: a0d63082a4db375a55586c7864e280cd8f45ff7b)
Signed-off-by: Igor Opaniuk <igor.opaniuk@foundries.io>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The idea of the base class dependency is to say "yes, I need a C cross compiler"
and this was never meant to be gcc specific. Looking at the codebase, whilst we
code triplets into this, it does overcomplicate things as there are only ever
limited, "target", "sdk" and the class extended versions like mutlilib.
After much thought, we can simplify this to virtual/cross-cc and virtual/nativesdk-cross-cc.
This lets us remove the "gcc" specific element as well as removing the over
complicated triplet usage.
At the same time, change the much less widely used "g++" variant to "c++" for
similar reasons and remove the triplet from virtual/XXX-binutils too.
Backwards compatibility mappings could be left but are just going to confuse
things in future so we'll just require users to update.
This simplification, whilst disruptive for any toolchain focused layers, will
make improved toolchain selection in the future much easier.
Since we no longer have overlapping variables, some code for that can just
be removed. The class extension code does need to start remapping some variables
but not the crosssdk target recipe names.
This patch is in two pieces, this one handles the renaming with the functional
changes separate in a second for easier review even if this breaks bisection.
(From OE-Core rev: 4ccc3bc8266c327bcc18c9a3faf7536210dfb9f0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 3.0.1 release of SPDX has been officially released with a few minor
modifications. Regenerate the bindings to use this version.
(From OE-Core rev: 54233a7d6fe414d22449fb02fac26b66a820b17a)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to commit [cve-check: Rework patch parsing] applied, it missed to
update patched_cves dictionary if cve_id not in patched_cves
(From OE-Core rev: 08796a8153666d93bb622c6a7497a85cef4def42)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The cve_check functionality to parse CVE IDs from the patch filename and
patch contents have been reworked to improve parsing and also utilize
tests. This ensures that the parsing works as intended.
Additionally, the new patched_cves dict has a few issues I tried to fix
as well. If multiple patch files exist for a single CVE ID, only the
last one will show up with the "resource" key. The value for the
"resource" key has been updated to hold a list and return all patch
files associated with a given CVE ID. Also, at the end of
get_patch_cves, CVE_STATUS can overwrite an existing entry in the dict.
This could cause an issue, for example, if a CVE has been addressed via
a patch, but a CVE_STATUS line also exists that ignores the given CVE
ID. A warning has been added if this ever happens.
(From OE-Core rev: 87c6da681609b4f8e048eca2a27ae8e068c724e1)
Signed-off-by: Colin McAllister <colinmca242@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Two warning lines in cve_check.py reference a variable that doesn't
exist. These would cause a runtime error if the conditions they are
hidden in were to be entered. The log lines have been updated to no
longer reference an undefined variable.
(From OE-Core rev: c9d059e8a362b3c9d604f7ebe8fd1dd994f0af6b)
Signed-off-by: Colin McAllister <colinmca242@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
rpm2cpio has been deprecated upstream, so this prepares for its
eventual removal.
rpm2archive produces a tar archive which can be uncompressed
with tar executable from the host.
(From OE-Core rev: ed824d3fb23f0c89d8dfdacb2c4ef0b7c21a5144)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The SPDX code needs to be able to look up an Element by its SPDX ID,
locating the file that (should) contain the SPDX ID and opening it for
parsing. Previously, the code would do this be hashing each Element
SPDX ID and Alias, and the creating a symbolic link to the file that
contains the element with a name of the hash.
This worked well as it was possible to look up any arbitrary SPDX ID or
alias by simply hashing it and following the symbolic link to get the
file. However, the down side of this approach is that it creates a lot
of symbolic links, since it will make one or two per Element in the
document. This can be a problem when using SPDX_INCLUDE_SOURCES, for
example.
This change reworks this strategy so that the only Element that gets a
symbolic link based on the hash is the singular SpdxDocument that is
create for each file. All other Elements are assigned an alias with a
special prefix that encodes the hash of SpdxDocument alias. Thus, when
attempting to look up an arbitrary alias, the code sees the special
prefix, extract the hash, opens the file based on the symlink with that
hash name, then finds the matching Element in the file. This drastically
reduces the number of symbolic links by making only one per file.
This also means that the custom link extension can be removed since it
is now superfluous.
(From OE-Core rev: 838d64c09657ac53175737fc4e7fd6f01f3dcf47)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a test for several of the extra options provided by the SPDX
classes. In particular, these are the options that can produce
non-reproducible results, so are not enabled by default in OE core. This
test takes care to configure the build so that the tests do run in a
reproducible manner so that pre-built test objects can be pulled from
sstate
(From OE-Core rev: 14f534f15f7fe6362723d7f064d39783c5bd758f)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The call to make a new import for the SPDX_BUILD_HOST was incorrect,
since it was asking for the key named "SPDX_BUILD_HOST", instead of the
key name contained in that variable
(From OE-Core rev: 250c2bcc557040b4cd67cecf25f8351638cff45d)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a agent reference was being used, the code was not using the
correct base variable to look up the agent
(From OE-Core rev: f3f13f48e214b25cf302b8ce397b630f5aa283fa)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These two files are the only ones that are left in meta/lib.
They logically belong to meta/lib/oe, so move them there.
(From OE-Core rev: c65dd0e3e463d6072b9364ac74e1fef0d998068f)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code in this file uses oe.qa, but it was not imported resulting an
an exception when a license error was detected
(From OE-Core rev: e44cb7c5b7281d614ed51fdec06dad0a7211528a)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
While the SPDX ID aliases are primarily intended for internal use by the
SPDX code, they are used in places where a valid SPDX ID is expected. In
order to make sure that they still conform to this, prefix them with
"http://spdx.org/spdxdocs/openembedded-alias"
(From OE-Core rev: 5e0ff36e025f5e842fa90b8219b53257d65ea66a)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes the build parameters code, since the build_parameter property was
changed to be singular in SPDX 3.0.1
(From OE-Core rev: e6e1e4585198b819319bab61ac2676e82f177020)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previous commit added logic to move certain tasks to the end, but these
had no effect, because the result of the make_last() function was not
used to update the post_process_cmds variable.
Also, once this is fixed, it becomes evident that the commands need to
be joined using whitespace, otherwise they all run together, and cannot
be executed as individual commands anymore.
Fixes: 0ffff2c1f8 ("rootfs-postcommands: Try and improve ordering constraints")
(From OE-Core rev: 2a4e8f06bac1bff0a167f775f7babab94b32732a)
Signed-off-by: Ralph Siemsen <ralph.siemsen@linaro.org>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When CPE is not provided and character ":" is in cve status description,
current code takes only last part of split function.
This works only if there is no ":" in description, otherwise it drops
the other split parts.
Do a new split of the original string to take the whole description unchanged.
This fixes following entries from world build of poky+meta-oe+meta-python:
tiff-4.6.0-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2015-7313
CVE_STATUS: fixed-version: Tested with check from https://security-tracker.debian.org/tracker/CVE-2015-7313 and already 4.3.0 doesn't have the issue
description: //security-tracker.debian.org/tracker/CVE-2015-7313 and already 4.3.0 doesn't have the issue
corrected: Tested with check from https://security-tracker.debian.org/tracker/CVE-2015-7313 and already 4.3.0 doesn't have the issue
gnupg-2.5.0-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2022-3219
CVE_STATUS: upstream-wontfix: Upstream doesn't seem to be keen on merging the proposed commit - https://dev.gnupg.org/T5993
description: //dev.gnupg.org/T5993
corrected: Upstream doesn't seem to be keen on merging the proposed commit - https://dev.gnupg.org/T5993
libyaml-0.2.5-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2024-35325
CVE_STATUS: upstream-wontfix: Upstream thinks this is a misuse (or wrong use) of the libyaml API - https://github.com/yaml/libyaml/issues/303
description: //github.com/yaml/libyaml/issues/303
corrected: Upstream thinks this is a misuse (or wrong use) of the libyaml API - https://github.com/yaml/libyaml/issues/303
libyaml-0.2.5-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2024-35326
CVE_STATUS: upstream-wontfix: Upstream thinks there is no working code that is exploitable - https://github.com/yaml/libyaml/issues/302
description: //github.com/yaml/libyaml/issues/302
corrected: Upstream thinks there is no working code that is exploitable - https://github.com/yaml/libyaml/issues/302
libyaml-0.2.5-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2024-35328
CVE_STATUS: upstream-wontfix: Upstream thinks there is no working code that is exploitable - https://github.com/yaml/libyaml/issues/302
description: //github.com/yaml/libyaml/issues/302
corrected: Upstream thinks there is no working code that is exploitable - https://github.com/yaml/libyaml/issues/302
cpio-2.15-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2023-7216
CVE_STATUS: disputed: intended behaviour, see https://lists.gnu.org/archive/html/bug-cpio/2024-03/msg00000.html
description: //lists.gnu.org/archive/html/bug-cpio/2024-03/msg00000.html
corrected: intended behaviour, see https://lists.gnu.org/archive/html/bug-cpio/2024-03/msg00000.html
openssh-9.9p1-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2023-51767
CVE_STATUS: upstream-wontfix: It was demonstrated on modified sshd and does not exist in upstream openssh https://bugzilla.mindrot.org/show_bug.cgi?id=3656#c1.
description: //bugzilla.mindrot.org/show_bug.cgi?id=3656#c1.
corrected: It was demonstrated on modified sshd and does not exist in upstream openssh https://bugzilla.mindrot.org/show_bug.cgi?id=3656#c1.
cups-2.4.10-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2021-25317
CVE_STATUS: not-applicable-config: This concerns /var/log/cups having lp ownership, our /var/log/cups is root:root, so this doesn't apply.
description: root, so this doesn't apply.
corrected: This concerns /var/log/cups having lp ownership, our /var/log/cups is root:root, so this doesn't apply.
unzip-1_6.0-r0 do_cve_check: CVE_STATUS with 3 parts for CVE-2008-0888
CVE_STATUS: fixed-version: Patch from https://bugzilla.redhat.com/attachment.cgi?id=293893&action=diff applied to 6.0 source
description: //bugzilla.redhat.com/attachment.cgi?id=293893&action=diff applied to 6.0 source
corrected: Patch from https://bugzilla.redhat.com/attachment.cgi?id=293893&action=diff applied to 6.0 source
syslog-ng-4.7.0-r0 do_cve_check: CVE_STATUS with 6 parts for CVE-2022-38725
CVE_STATUS: cpe-incorrect: cve-check wrongly matches cpe:2.3🅰️oneidentity:syslog-ng:*:*:*:*:premium:*:*:* < 7.0.32
description: syslog-ng:*:*:*:*:premium:*:*:* < 7.0.32
corrected: cve-check wrongly matches cpe:2.3🅰️oneidentity:syslog-ng:*:*:*:*:premium:*:*:* < 7.0.32
(From OE-Core rev: cc33dd9176726cb4b2d2f142ed1bc655da8e0a9f)
Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Correct maxsplit parameter from 5 to 4 to not drop text if
description contains ":".
Example:
>>> "detail: cpe:vendor:product:description:cont".split(':', 5)
['detail', ' cpe', 'vendor', 'product', 'description', 'xxx']
>>> "detail: cpe:vendor:product:description:cont".split(':', 4)
['detail', ' cpe', 'vendor', 'product', 'description:xxx']
(From OE-Core rev: 3c4d8ca41ac0b429af92bf0ea84f1dfd0cda9e1f)
Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Originally, while SPDX_INCLUDE_SOURCES = "1" [1], there is bug in scan
for gcc, libgcc in which the sources locates in work-share directory.
Copy source from ${WORKDIR} to ${SPDXWORK} did not satisfy the situation
while ${S} was not included in ${WORKDIR}
This commit aim to support SPDX include source for work-share directory
1. If is_work_shared_spdx, Copy source from ${S} to ${SPDXWORK},
normally the dest dir in ${SPDXWORK} has the same basename dir of ${S};
but for kernel source, rename basename dir 'kernel-source' to ${BP} (${BPN}-${PV})
2. For SPDX source copy, do hard link copy to save copy time
3. Move do_patch to no work shared situation along with do_unpack
4. Tweak task do_create_spdx dependencies to assure the patched source
in work share is ready for SPDX source copy
5. Remove bb.data.inherits_class('kernel', d) from is_work_shared_spdx,
the kernel source locates in 'work-shared', test kernel.bbclass is not
necessary
[1] https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-SPDX_INCLUDE_SOURCES
(From OE-Core rev: 64454b1956a9b50d6c89a3f3d7c594c1272cb289)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Reviewed-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Call function ${@create_spdx_source_deps(d)} or ${create_spdx_source_deps(d)}
along with addtask not working, use task do_create_spdx flag 'depends'
to instead
Move function create_spdx_source_deps to spdx-common.bbclass for both of
create-spdx-2.2.bbclass and create-spdx-3.0.bbclass
(From OE-Core rev: dda4b7ccd416ef370634babd150e944f2aa9116e)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Reviewed-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code was changing the timestamps of the files in the do_package output,
particularly the files added for debug sources. This was to do two things:
a) make do_package sstate more reproducible
b) ensure better hash equivalence matching
Unfortuately the debug source files are hardlinks into the source tree for
efficiency so touching these, touches a lot of files in ${B} and ${S}. This
causes unpredictable effects if compile is run again for example, or could
cause compiling in the install task.
The hash equivalence matching is of key importance but we can mimic that
using clamping of the file timestamps in the depsig output used to generate
the hashes.
This patch drops the global timestamp clamping, instead allowing the files
to retain their creation timestamps into sstate. This makes do_package sstate
slightly less reproducibile. We could clamp the sstate timestamps but that
would lead to two different sets of timestamps depending on whether the
data came from sstate or not. I'd prefer to have consistent code behaviour,
rather than differing behavhour depending on whether data came from sstate
or not.
If we wanted to have reproducibiliy and fix the "corruption" of S/B and have
consistent codepaths, the only other option would be two copies of the
sources, which could end up huge and seems the least desireable option.
This patch therefore drops the timestamp clamping in the sstate files
and tweaks the depsig data generation to clamp the timestamps for do_package
instead since this seems the best compromise.
I validated that rpm/deb/ipk files still generate correctly as before.
(From OE-Core rev: 475759fdab7200488b2a568b2ba1aa31a456d113)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When creating split locales, copy the license from LICENSE:${PN}-locale
if set, otherwise leave it unspecified (which will result in falling
back to LICENSE)
(From OE-Core rev: 34b844d3a920574d81affbdd2096ffd7dea14067)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of checking for incompatible licenses in the anonymous python
and setting '_exclude_incompatible-', (re)check all the packages in
populate_packages(). This ensures that all packages are processed, even
dynamically generated ones.
The use of the '_exclude-incompatible-' variable set in base.bbclass has
been the mechanism used for per-packages licenses since it was added as
a feature (although with different names for the variable throughout
history). However, since this misses dynamic packages, calling
oe.license.skip_incompatible_package_licenses() a second time on the
actual final package set is a better solution.
(From OE-Core rev: 1816c5a5e724a7f907c9afe4a47b6699ea081ace)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moves the code that skips packages with incompatible licenses to the
library code so that it can be called in other locations
(From OE-Core rev: 86eb409e3c1b30110869ec5a0027ae2d48bbfe7f)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moves several of the functions in license.bbclass to be library code
New function dependencies were manually verified using bitbake-dumpsigs
to ensure that bitbake identified the same dependencies even though they
are now in library code (although the new function names mean that the
task hashes still change)
(From OE-Core rev: 0333e04e353991260c5f67a72f80f3ab9dcf526a)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The license information and Build created by do_create_spdx are changed
to be referenced by their link alias instead of the actual SPDX ID. This
fixes a case where do_create_package_spdx would pull these from
mismatching sstate, and then the SPDX IDs would be unresolved when
assembling the final document
(From OE-Core rev: c0fcdc72a7c8fca86a874d1b04298fe9e500c796)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The functions behave slightly differently to the functions they're
caching and the use in insane.bbclass isn't compatible. For now, to
avoid build failures, switch back to the stat calls. We may be able
to improve cachedpath or change the call sites.
(From OE-Core rev: fa771ae887ab5152f043748cf3419735831bcf7b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can skip processing the installed-pkgs file if globs is empty.
This is the case if self.d.getVar for IMAGE_INSTALL_COMPLEMENTARY
returns an empty string. If globs is an empty string the result from
processing with empty glob in oe-pkgdata-util will always be 0 packages
to install.
Instead of return early on this we just skip and still generate the
locale archive if needed.
(From OE-Core rev: 160c45c83d5addf01e4834cf896af871bd6fca7f)
Signed-off-by: Claus Stovgaard <claus.stovgaard@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Every time oe.elf.machine_dict() is called a large dictionary is created
and returned. However, the "arch" package_qa test will call this method
for every binary in a package, which results in a lot of dictionary
creation.
Concrete exmaple: in running ltp:do_package_qa, the arch test takes 25%
of the runtime, and opitimising the machine_dict() call to cache the
generated dictionary reduces the runtime from 57s to 44s.
(From OE-Core rev: dca4455aa4f870bad900a0214c4e95e09a055295)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upgrades the SPDX 3.0 implementation from 3.0.0 -> 3.0.1. This version
introduced some breaking changes. Effectively, 3.0.0 was a pre-release
version that we do not need to support any longer.
(From OE-Core rev: 7e6509a19f6146f0dad0be7037ff4b3268d0128c)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"gitsm" is not a recognized URI protocol (outside of bitbake), so map it
to "git" when writing. This should be OK since we report all of the
submodule source code (if enabled), and it's still possible for 3rd
party analyzers to determine that submodules are in use by looking at
.gitmodules.
The code to do the mapping is moved to a common location so it covers
SPDX 2.2 also
[YOCTO #15582]
(From OE-Core rev: 6ecf89c75b1a74515266085acc5d3621a0fb2fa1)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Include the local files when searching for matching debug sources
(From OE-Core rev: 3ff2cc453d1ec3e12876e64f7dc84d79d25a7ea9)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After commit 905e224849fbbed1719e0add231b00e2d570b3b4 (image_qa: fix
error handling), any unexpected exceptions in do_image_qa() would result
in a variable being set, but never used, effectively hiding the error.
Since image_qa now calls oe.qa.exit_if_errors(), remove the support for
oe.utils.ImageQAFailed and instead rely on the called functions to call
oe.qa.handle_error() themselves. This matches what do_package_qa() does.
Also update the description of do_image_qa() to explain that the called
functions are expected to call oe.qa.handle_error() themselves.
[ YOCTO #15601 ]
(From OE-Core rev: 0c3e111c965af2bc56533633c376b70b7fa5e1de)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make ImageQAFailed inherit BBHandledException so exceptions raised in tests are
catched when the actual test function is executed by bb.utils.better_exec.
Change the do_image_qa tasks so errors are handled with oe.qa.handle_error. Add
some comment to explain this requires to list the test in ERROR_QA or WARN_QA.
[YOCTO #14807]
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14807
(From OE-Core rev: 905e224849fbbed1719e0add231b00e2d570b3b4)
Signed-off-by: Louis Rannou <louis.rannou@non.se.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Method is_file() was wrongly called as isfile()
(From OE-Core rev: 356c52a45db139bf1fdfcf5b6e0903ece7d1dd46)
Signed-off-by: Daniil Batalov <dbatalov@deltard.ru>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
