Commit Graph

7734 Commits

Author SHA1 Message Date
Joshua Watt
dfb279f49e classes: create-spdx: Fix variable dependencies
A few variable dependencies were incorrect from the SPDX tasks, which
prevented rebuilds from occurring, or caused them to occur when they
should not.

(From OE-Core rev: 38a5dd136b3a3713e63d23598011efe1c4b0f459)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-12-05 17:07:10 +00:00
Denys Dmytriyenko
2d10a8f522 yocto-check-layer: expand to cover all required QA checks
insane.bbclass now defines CHECKLAYER_REQUIRED_TESTS list with required
QA checks that are becoming mandatory for Yocto Project Compatible layers.

Update yocto-check-layer.bbclass in order to catch when packages from such
layers try to skip any of the required QA checks.

(From OE-Core rev: 9c3ba88628853b20fb4c98c99cf3fe8349024016)

Signed-off-by: Denys Dmytriyenko <denys@konsulko.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-12-03 11:42:35 +00:00
Richard Purdie
e11cf39bf9 buildhistory: Drop git-gc call as obsolete
buildhistory used to result in many individual objects in the repo which
could affect performance. This gc call was added over a decade ago to avoid
those performance issues. Modern git calls git-gc when needed and this kind
of workaround should no longer be needed.

There is a small chance this is contributing to problems with the autobuilder
buildhistory repositories, so simplifying the code may help that too.

(From OE-Core rev: 0a88f9109f7670a6b188463a19392d445af6ff40)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-12-03 11:21:34 +00:00
Peter Marko
cc8ba2f80b cve-check: fix cvesInRecord
Currently flag cvesInRecord is set to false if all CVEs are ignored or
patched. This is inconsistent as it shows false if a CVE was fixed via
patch and true if this CVE was fixed by upgrade. In both cases the CVE
is valid and was fixed.

As I understand this flag, it should say if any CVE exists for
particular component's product (regardless of how this CVE is handled)
and can be used to validate if a product is correctly set.

Note that skipping ignored CVEs may make sense in some cases, as ignored
may mean that NVD DB is wrong, but in many cases it is ignored for other
reasons. Further patch can be done to evaluate ignore subtype but that
would be against my understanding of this flag as described above.

(From OE-Core rev: c5d499693672ec9619392011b765941cf94aa319)

Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-28 12:27:12 +00:00
Peter Marko
275aec49b6 cve-check: restore CVE_CHECK_SHOW_WARNINGS functionality
Commit 05ef4f2a7b225c8d230eaca8d333ffb921729d79 removed this
functionality by accident. It was implemented in text exporter, while it
should have been a global feature independent on exporter type to avoid
such accidental deletion.

(From OE-Core rev: 2996b11596afca288a6b7f409a5287063d331f3b)

Signed-off-by: Peter Marko <peter.marko@siemens.com>
Cc: Marta Rybczynska <marta.rybczynska@ygreky.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-28 12:27:12 +00:00
Joshua Watt
09eb747db8 classes/go-vendor: Use UNPACKDIR instead of WORKDIR
modules.txt is provided using file:// in SRC_URI, therefore it will be
found in UNPACKDIR instead of WORKDIR

(From OE-Core rev: 114fe16f96280f40a9cea20b02b2743be42645dd)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-28 00:07:30 +00:00
Joshua Watt
9ca3716437 spdx: Fix SPDX tasks not running when code changes
The SPDX code makes heavy use of python classes. While this works very
well, the bitbake dependency parser is unable to understand how to deal
with them, and thus changes to the class code do not cause rebuilds to
occur. To correct this, add the library files that include SPDX code as
file checksums for the SPDX tasks. If this method works well for SPDX,
we will look at implementing something similar in the bitbake dependency
parser that should allow correct checksums without having to explicitly
add them to each class.

(From OE-Core rev: 6ac3033b77a0d1f7ab15801c5c65931adede3923)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-23 14:44:54 +00:00
Hongxu Jia
486ed07bd9 meta/classes/spdx-common.bbclass: fix sstate cache racing issue on kernel source while SPDX_INCLUDE_SOURCES = "1"
While two projects share one sstate cache, the 1st project builds kernel-devsrc without
setting SPDX_INCLUDE_SOURCES, and 2nd project build kernel-devsrc with setting
SPDX_INCLUDE_SOURCES = "1". Then the 2nd build failed with kernel-source not found

1. In 1st build
$ cd path-to-first-build
$ bitbake kernel-devsrc

2. In 2nd build, share sstate cache of 1st build and SPDX_INCLUDE_SOURCES = "1"
$ cd path-to-second-build
$ echo 'SSTATE_DIR = "path-to-first-build/sstate-cache"' >> conf/local.conf
$ echo 'SPDX_INCLUDE_SOURCES = "1"'  >> conf/local.conf
$ bitbake kernel-devsrc
...
NOTE: copyhardlinktree path-to-second-build/tmp-glibc/work-shared/axxiaarm64/kernel-source to path-to-second-build/tmp-glibc/work/axxiaarm64-wrs-linux/kernel-devsrc/1.0/spdx/3.0.1/work/kernel-source
ERROR: Error executing a python function in exec_func_python() autogenerated:
...
     0208:            bb.note(f"copyhardlinktree {share_src} to {src_dir}")
 *** 0209:            oe.path.copyhardlinktree(share_src, src_dir)
...
tar: path-to-second-build/tmp-glibc/work-shared/axxiaarm64/kernel-source: Cannot open: No such file or directory
...

The recipe kernel-devsrc or the recipe to inherit kernelsrc.bbclass, they do not
have task do_shared_workdir but depends on virtual/kernel:do_shared_workdir.

In this situation(the ${S} == ${STAGING_KERNEL_DIR}), explicitly make do_create_spdx
depends on virtual/kernel:do_shared_workdir could fix the racing issue

(From OE-Core rev: b05ff49de1e58f5696e8ee28fceaf9319be70e1f)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-22 16:53:37 +00:00
Robert Yang
242e47dd09 multilib_global.bbclass: Avoid expanding native recipes in NON_MULTILIB_RECIPES
Fixed:
require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE:virtclass-multilib-lib32 = "x86"
MACHINE = "qemux86-64"

$ bitbake lib32-grub-native
The build would go on before the patch which was incorrect. Now:

ERROR: Nothing PROVIDES 'lib32-grub-native'. Close matches:
[snip]

The nativesdk doesn't have such an issue, so only skip native.

(From OE-Core rev: 73bea15bb5d0f70b587d2ad1007cc0282652eed7)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-12 11:19:44 +00:00
Hongxu Jia
ce192383d5 create-spdx-{2.2,3.0}: support SPDX include source for work-share directory
Originally, while SPDX_INCLUDE_SOURCES = "1" [1], there is bug in scan
for gcc, libgcc in which the sources locates in work-share directory.
Copy source from ${WORKDIR} to ${SPDXWORK} did not satisfy the situation
while ${S} was not included in ${WORKDIR}

This commit aim to support SPDX include source for work-share directory

1. If is_work_shared_spdx, Copy source from ${S} to ${SPDXWORK},
normally the dest dir in ${SPDXWORK} has the same basename dir of ${S};
but for kernel source, rename basename dir 'kernel-source' to ${BP} (${BPN}-${PV})

2. For SPDX source copy, do hard link copy to save copy time

3. Move do_patch to no work shared situation along with do_unpack

4. Tweak task do_create_spdx dependencies to assure the patched source
in work share is ready for SPDX source copy

5. Remove bb.data.inherits_class('kernel', d) from is_work_shared_spdx,
the kernel source locates in 'work-shared', test kernel.bbclass is not
necessary

[1] https://docs.yoctoproject.org/dev/ref-manual/variables.html#term-SPDX_INCLUDE_SOURCES

(From OE-Core rev: 64454b1956a9b50d6c89a3f3d7c594c1272cb289)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Reviewed-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-01 12:07:12 +00:00
Hongxu Jia
8f4759806e create-spdx-{2.2,3.0}: fix do_create_spdx dependency while spdx include sources
Call function ${@create_spdx_source_deps(d)} or ${create_spdx_source_deps(d)}
along with addtask not working, use task do_create_spdx flag 'depends'
to instead

Move function create_spdx_source_deps to spdx-common.bbclass for both of
create-spdx-2.2.bbclass and create-spdx-3.0.bbclass

(From OE-Core rev: dda4b7ccd416ef370634babd150e944f2aa9116e)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Reviewed-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-01 12:07:12 +00:00
Peter Marko
3b551fc466 cve-check: add support for cvss v4.0
https://nvd.nist.gov/general/news/cvss-v4-0-official-support

CVSS v4.0 was released in November 2023
NVD announced support for it in June 2024

Current stats are:
* cvss v4 provided, but also v3, so cve-check showed a value
sqlite> select count(*) from nvd where scorev4 != 0.0 and scorev3 != 0.0;
2069
* only cvss v4 provided, so cve-check did not show any
sqlite> select count(*) from nvd where scorev4 != 0.0 and scorev3 = 0.0;
260

(From OE-Core rev: 358dbfcd80ae1fa414d294c865dd293670c287f0)

Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-25 15:25:33 +01:00
Katawann
d32bab0137 cve-check: add field "modified" to JSON report
Added the "modified" field to the JSON export in the
cve-check.class. This field captures the last modification date of each
CVE, providing more detailed information on changes and updates within
the exported data.

(From OE-Core rev: 740b8a0b23c4021d07c3714420e3ea8b46e61454)

Signed-off-by: Katawann <quent_55@hotmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-18 15:01:04 +01:00
Richard Purdie
35c3b9132d migrate_localcount: Drop long obsolete code
If BB_URI_LOCALCOUNT isn't set, the code does nothing. That code was removed in 2012:

https://git.yoctoproject.org/poky/commit/?id=d0f35207f9e19b440393a79ebf621649c495738d

Therefore drop the rest of it!

(From OE-Core rev: fca25fc4d7721f85f64c942307ebe7ba9f2fad3e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-08 13:26:39 +01:00
Joshua Watt
2e59418325 create-spdx-3.0: Upgrade to SPDX 3.0.1
Upgrades the SPDX 3.0 implementation from 3.0.0 -> 3.0.1. This version
introduced some breaking changes. Effectively, 3.0.0 was a pre-release
version that we do not need to support any longer.

(From OE-Core rev: 7e6509a19f6146f0dad0be7037ff4b3268d0128c)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-30 17:07:18 +01:00
Joshua Watt
07836a9684 spdx 3.0: Map gitsm URI to git
"gitsm" is not a recognized URI protocol (outside of bitbake), so map it
to "git" when writing. This should be OK since we report all of the
submodule source code (if enabled), and it's still possible for 3rd
party analyzers to determine that submodules are in use by looking at
.gitmodules.

The code to do the mapping is moved to a common location so it covers
SPDX 2.2 also

[YOCTO #15582]

(From OE-Core rev: 6ecf89c75b1a74515266085acc5d3621a0fb2fa1)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-30 17:07:18 +01:00
Richard Purdie
ead03cae66 buildhistory: Simplify intercept call sites and drop SSTATEPOSTINSTFUNC usage
We planned to drop SSTATEPOSTINSTFUNC some time ago with the introduction of
postfuncs. Finally get around to doing that which should make the buildhistory
code a little more readable.

Unfortunately ordering the buildhistory function calls after the sstate ones is
difficult without coding that into the sstate class. This patch does that to
ensure everything functions as expected until we can find a better way. This is
still likely preferable than the generic sstate postfuncs support since the function
flow is much more readable.

(From OE-Core rev: c9e2a8fa2f0305ef1247ec405555612326f798f8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-04 12:38:44 +01:00
Marta Rybczynska
f06fbe9bd8 cve-check: remove the TEXT format support
Remove the TEXT format support, as the JSON format offers more functions.
Users who do automation should have migrated already.

Support of both formats makes the code more complex than necessary.

Users can convert JSON files to TEXT files with cve-json-to-text.py
in scripts/

(From OE-Core rev: 05ef4f2a7b225c8d230eaca8d333ffb921729d79)

Signed-off-by: Marta Rybczynska <marta.rybczynska@ygreky.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-25 15:36:30 +01:00
Marta Rybczynska
3859ff5915 vex.bbclass: add a new class
The "vex" class generates the minimum information that is necessary
for VEX generation by an external CVE checking tool. It is a drop-in
replacement of "cve-check". It uses the same variables from recipes
to make the migration and backporting easier.

The goal of this class is to allow generation of the CVE list of
an image or distribution on-demand, including the latest information
from vulnerability databases. Vulnerability data changes every day,
so a status generated at build becomes out-of-date very soon.

Research done for this work shows that the current VEX formats (CSAF
and OpenVEX) do not provide enough information to generate such
rolling information. Instead, we extract the needed data from recipe
annotations (package names, CPEs, versions, CVE patches applied...)
and store for later use in the format that is an extension of the
CVE-check JSON output format.

This output can be then used (separately or with SPDX of the same
build) by an external tool to generate the vulnerability annotation
and VEX statements in standard formats.

(From OE-Core rev: 6352ad93a72e67d6dfa82e870222518a97c426fa)

Signed-off-by: Marta Rybczynska <marta.rybczynska@syslinbit.com>
Signed-off-by: Samantha Jalabert <samantha.jalabert@syslinbit.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-20 14:12:40 +01:00
Marta Rybczynska
fb3f440b7d cve-check: annotate CVEs during analysis
Add status information for each CVE under analysis.

Previously the information passed between different function of the
cve-check class included only tables of patched, unpatched, ignored
vulnerabilities and the general status of the recipe.

The VEX work requires more information, and we need to pass them
between different functions, so that it can be enriched as the
analysis progresses. Instead of multiple tables, use a single one
with annotations for each CVE encountered. For example, a patched
CVE will have:

{"abbrev-status": "Patched", "status": "version-not-in-range"}

abbrev-status contains the general status (Patched, Unpatched,
Ignored and Unknown that will be added in the VEX code)
status contains more detailed information that can come from
CVE_STATUS and the analysis.

Additional fields of the annotation include for example the name
of the patch file fixing a given CVE.

We also use the annotation in CVE_STATUS to filter out entries
that do not apply to the given recipe

(From OE-Core rev: 452e605b55ad61c08f4af7089a5a9c576ca28f7d)

Signed-off-by: Marta Rybczynska <marta.rybczynska@syslinbit.com>
Signed-off-by: Samantha Jalabert <samantha.jalabert@syslinbit.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-20 14:12:40 +01:00
Marta Rybczynska
bf34db1439 cve-check: encode affected product/vendor in CVE_STATUS
CVE_STATUS contains assesment of a given CVE, but until now it didn't have
include the affected vendor/product. In the case of a global system include,
that CVE_STATUS was visible in all recipes.

This patch allows encoding of affected product/vendor to each CVE_STATUS
assessment, also for groups. We can then filter them later and use only
CVEs that correspond to the recipe.

This is going to be used in meta/conf/distro/include/cve-extra-exclusions.inc
and similar places.

(From OE-Core rev: abca80a716e92fc18d3085aba1a15f4bac72379c)

Signed-off-by: Marta Rybczynska <marta.rybczynska@syslinbit.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-20 14:12:40 +01:00
Enrico Jörns
9b2c1ef319 archiver.bbclass: fix BB_GENERATE_MIRROR_TARBALLS checking
The variable 'have_mirror_tarballs' is used as a boolean while it is
actually the result of 'd.getVar('BB_GENERATE_MIRROR_TARBALLS')' and
thus a string.

Fix this by converting it into a boolean before using i
t.

(From OE-Core rev: f6185d51b84d8d2eb578dbd322c1c61537efabf6)

Signed-off-by: Enrico Jörns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-09 22:33:38 +01:00
Richard Purdie
a211f058cc sdpx: Avoid loading of SPDX_LICENSE_DATA into global config
Loading a load of json files into a memory structure and stashing in a bitbake
variable is relatively anti-social making bitbake -e output hard to read for
example as well as other potential performance issues.

Defer loading of that data until it is actually needed/used in a funciton
where it is now passed as a parameter.

(From OE-Core rev: 6f21cc9598178288784ff451ab3c40b174c0ef3e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-07 15:47:15 +01:00
Pedro Ferreira
7355465f9e buildhistory: Restoring files from preserve list
This fix will ensure that, when we activate feature
`BUILDHISTORY_RESET`, files marked to keep on feature
`BUILDHISTORY_PRESERVE` will indeed exist is buildhistory
final path since they are moved to buildhistory/old but
not restored at any point.

(From OE-Core rev: 9f68a45aa238ae5fcdfaca71ba0e7015e9cb720e)

Signed-off-by: Pedro Ferreira <Pedro.Silva.Ferreira@criticaltechworks.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-07 15:47:15 +01:00
Pedro Ferreira
738b3b1e65 buildhistory: Fix intermittent package file list creation
The directory that buildhistory_list_pkg_files writes to during do_package
is created by do_packagedata so a clean buildhistory doesn't have
files-in-package written during the first build since packagedata happens
after do_package.

Ensure the output package folder is created to avoid missing
files-in-package.txt files.

Also it ensures that in case of `find` fails we leave with
a hard error instead of hiding the error on the for loop.

(From OE-Core rev: 8de9b8c1e199896b9a7bc5ed64967c6bfbf84bea)

Signed-off-by: Pedro Silva Ferreira <Pedro.Silva.Ferreira@criticaltechworks.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-07 15:47:15 +01:00
Richard Purdie
0731d49014 cve_check: Use a local copy of the database during builds
Rtaher than trying to use a sqlite database over NFS from DL_DIR, work from
a local copy in STAGING DIR after fetching.

(From OE-Core rev: 03596904392d257572a905a182b92c780d636744)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-05 14:31:57 +01:00
Richard Purdie
7c1de3118f create-spdx-3.0/populate_sdk_base: Add SDK_CLASSES inherit mechanism to fix tarball SPDX manifests
Currently, "tarball" sdk based recipes don't generate SPDX manifests as they
don't include the rootfs generation classes. Split the SPDX 3.0 image class into
two so the SDK components can be included where needed.

To do this, introduce an SDK_CLASSES variable similar to IMAGE_CLASSES which
the SDK code can use.

Migrate testsdk usage to this.

Also move the image/sdk spdx classes to classes-recipe rather than the general classes
directory since they'd never be included on a global level.

For buildtools-tarball, it has its own testsdk functions so disable the class there as
a deferred inherit would overwrite it.

(From OE-Core rev: 662396533177b72cc1d83e95841b27f7e42dcb20)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-26 12:28:42 +01:00
Mark Hatle
edc44fcf13 create-spdx-*: Support multilibs via SPDX_MULTILIB_SSTATE_ARCHS
When a create-spdx-* classes is processing documents, it needs to
find the document in a path that is related to the SSTATE_ARCH
when a packge is generated.  The SSTATE_ARCH can be affected by
multilib configurations, resulting is something like armv8a-mlib.

When the image (or SDK) is being generated and the components are
collected, the system has no knowledge of the multilib arch and
will fail to find it, such as:

  ERROR: meta-toolchain-1.0-r0 do_populate_sdk: No SPDX file found
   for package libilp32-libgcc-dbg,
   False sstate:libilp32-libgcc:armv8a-ilp32-mllibilp32-elf:14.1.0:r0:armv8a-ilp32:12:
   sstate:libilp32-libgcc::14.1.0:r0::12:

Adding in the new SPDX_MULTILIB_SSTATE_ARCHS will provide a full
set of SSTATE_ARCHS including ones that contain the multilib
extension which will allow create-spdx-* to correctly find the
document it is looking for.  This would also be valuable to any
other function doing a similar search through SSTATE_ARCH that may
have been extended with multilib configurations.

(From OE-Core rev: f1499c36c1054fc90f7b7268cc95285f2eca72f7)

Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-26 12:28:42 +01:00
Mark Hatle
a798d00d54 create-sdpx-2.2.bbclass: Switch from exists to isfile checking debugsrc
While debugsrc is almost always a file (or link), there are apparently
cases where a directory could be returned from the dwarfsrcfiles
processing.  When this happens, the hashing fails and an error results
when building the SPDX documents.

(From OE-Core rev: 02e262c291c0b2066132b4cb2ca5fda8145284a9)

Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-18 17:21:39 +01:00
Joshua Watt
3f76c3d2c2 Switch default spdx version to 3.0
Changes the default SPDX version to 3.0

(From OE-Core rev: beef4c1a3e9fd8fa9cdbbb61e1bc3931012735b3)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-16 15:00:59 +01:00
Joshua Watt
87c60b9a5a classes/create-spdx-3.0: Move tasks to library
Move the bulk of the python code in the SPDX 3.0 classes into a library
file

(From OE-Core rev: aed6f8c1c2e291bde4d7172742790fa535b2fc7d)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-16 14:55:53 +01:00
Joshua Watt
9850df1b60 classes/spdx-common: Move to library
Moves the bulk of the code in the spdx-common bbclass into library code

(From OE-Core rev: 3f9b7c7f6b15493b6890031190ca8d1a10f2f384)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-16 14:55:53 +01:00
Joshua Watt
8426e027e8 classes/create-spdx-3.0: Add classes
Adds a class to generate SPDX 3.0 output and an image class that is used
when generating images

(From OE-Core rev: b63f6f50458fc6898e4deda5d6739e7bf3639c15)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-16 14:55:53 +01:00
Joshua Watt
2c2e1becd6 classes/create-spdx-2.2: Handle empty packages
When combining an SPDX document, the package list might be empty (e.g.
a baremetal image). Handle this case instead of erroring out

(From OE-Core rev: baf4e360f6e65a5e9aff2def69d2a720f38f92b2)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-13 23:28:31 +01:00
Ulrich Ölmann
b2bc061f7c buildhistory: fix typos
Fix some simple typos found while looking through the code.

(From OE-Core rev: c23cb2d89fcd014d1f08944468c7e2d8a7f3e8b0)

Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-04 22:56:15 +01:00
Joshua Watt
226ae9288e classes/create-spdx-2.2: Handle SemVer License List Versions
SPDX transitioned the license list to use SemVer visioning, (e.g.
"MAJOR.MINOR.MICRO"), but SPDX 2 only allows "MAJOR.MINOR". For maximum
compatibility, only keep the first two version numbers and discard the
rest which allows it to work with either scheme

(From OE-Core rev: 8757a5eaeaf2b9d7345212d003e5622289b123e6)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-02 22:29:13 +01:00
Joshua Watt
10917abebf classes/spdx-common: Move SPDX_SUPPLIER
Move the SPDX_SUPPLIER variable to create-spdx-2.2 since it's format
only has meaning in SPDX 2.2 (SPDX 3 uses SPDX_PACKAGE_SUPPLIER with a
different format)

(From OE-Core rev: 628c1e04072178d2c8095e53d5f6600c45a2679f)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-27 13:03:35 +01:00
Aleksandar Nikolic
edba6889b4 cve-check: Introduce CVE_CHECK_MANIFEST_JSON_SUFFIX
The variable contains the suffix of the CVE JSON manifest file.
By default, this variable is set to 'json', so the current behavior
is not changed, but enables developers to use some other suffix,
e.g., cve.json (similar to spdx.json).

(From OE-Core rev: d99eee76923659c0b95bf9ef415ae5d44f736d01)

Signed-off-by: Aleksandar Nikolic <an010@live.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-24 12:43:40 +01:00
Benjamin Szőke
dbe070b602 archiver.bbclass: Fix work-shared checking for kernel recipes
Source dir can be a symbolic link in some BSP's linux kernel recipe
which points to work-shared path (like linux-fslc in meta-freescale).
Change to use os.path.realpath() in order to get real path of
source dir.

(From OE-Core rev: 9191aa685418af32f003e067ef7c5737a271e3a5)

Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-18 08:44:41 +01:00
Joshua Watt
717a427807 classes/spdx-common: Return empty list from extract_licenses
This is nicer as the normal return type is a list, so the calling code
doesn't have to deal with a None sometimes and a list others.

(From OE-Core rev: e200aa9cc6ceb8ca58ef239a1a5565287b38ce55)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-12 16:16:03 +01:00
Joshua Watt
37c9adaf80 classes/spdx-common: Add SPDX version to path
Since multiple versions of SPDX are available, use the version in the
path

(From OE-Core rev: 35c061a4514905b3ebbb7f0633a584927519445c)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-12 16:16:03 +01:00
Joshua Watt
92b202c246 classes/spdx-common: Move common SPDX to new class
Moves SPDX code that can be shared between different SPDX versions into
a common class

(From OE-Core rev: 769a390adc9fc0b52978abe0f19f885967af0117)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-12 16:16:03 +01:00
Kai Kang
b46075a150 multilib.bbclass: replace deprecated e.data with d
Replace deprecated e.data with d in multilib.bbclass and
multilib_global.bbclass.

Remove event check in function multilib_virtclass_handler_vendor in
multilib_global.bbclass. The function flag 'eventmask' has been set
with 'bb.event.ConfigParsed', so no need to check the event any more.

(From OE-Core rev: 8d87662d676a2b9ef921cb49ea6edc07b37410e2)

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-06 11:56:04 +01:00
Richard Purdie
6b9eb4e9f0 siteconfig: Drop siteconfig class/code/support
The siteconfig code was only used for 5 cache values. The complexity added to sstate
to support this code was considerable and the runtime much more significant than
any benefit the cache files would have added. Drop the support for this which
was only used minimally for ncurses and zlib.

(From OE-Core rev: f3766dc038f7ba9780ddaf5eb8d27385ea31d7d0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-31 16:58:36 +01:00
Richard Purdie
6d77296d83 devtool: Drop oe-local-files and simplify
The only real reason for oe-local-files was to support S = WORKDIR. With changes to
drop support for that, it makes sense to simplify devtool and to try and make both
the code and the processes/workflows simpler.

This patch drops support for S = WORKDIR, removes oe-local-files and then updates
the test cases to match this new situation.

At the code level, we assume we can always now track code changes using git and
that things committed into git are handled as patches (as before) but delta against
HEAD is saved as specific file level changes to the recipe.

One test is disabled as it is no longer approproate. It is being keped until we can
make WORKDIR != UNPACKDIR at which point it should be revisited.

(From OE-Core rev: ce8190c519052fed10b5233697b69a75868db45a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-21 12:08:04 +01:00
joshua Watt
3bf65ca15f classes/create-spdx-2.2: Fix SPDX dependencies for ABI Safe recipes
SPDX Documents can currently get into a state where they reference SPDX
IDs that do not exist (locally). The reason for this is that some
tasks/recipes are marked as ABI safe, and thus are excluded from
taskhash. This means that when SPDX creates a document, it will refer to
the SPDX ID in the dependency at the time when the document is created,
but if the dependency changes and gets a new SPDX ID, the document will
not rebuild to reference the new SPDX ID, causing it to be dangling.

Fix this by using the new field in BB_TASKDEPDATA to skip dependencies
that are not part of the taskhash calculation.

(From OE-Core rev: 8393557b5229c7a0cea18a586848384bff1d7424)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-09 09:55:27 +01:00
joshua Watt
8d79ca1478 classes/create-spdx-2.2: Fix SPDX Namespace Prefix
According to the SPDX documentation, it should be "spdxdocs" not
"spdxdoc"

[YOCTO #15398]

(From OE-Core rev: d1b25413ced62dc2927dae57b8d16e67d15dc220)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-02 15:14:07 +01:00
Simone Weiß
296fdb6643 cve-check: Log if CVE_STATUS set but not reported for component
Log if the CVE_STATUS is set for a CVE, but the cve is not reported for a
component. This should hopefully help to clean up not needed CVE_STATUS
settings.

(From OE-Core rev: 013d531a84fa08b6ae8a47bdf3ba1fa8f18ba270)

Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-24 16:10:23 +00:00
Piotr Łobacz
f6daeba2e9 useradd.bbclass: Fix order of postinst-useradd-*
postinst-useradd-* haven't been running in order of dependency.

This patch is reworked from Piotr Łobacz's patch and fixes:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15084
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13904

basepasswd_sysroot_postinst in base-passwd can install postinst-useradd-*
scripts with any order. Sometimes this means, for example a useradd postinst
will attempt to run without the corresponding group postinst causing errors.
This patch ensures that we first run groupadd, then useradd and then
group membership.

[RP: Tweaked to avoid removing previous fixes and for whitespace/style issues
     Also ensure the scripts are changed to execute with -e to highlight errors]

(From OE-Core rev: 322ef726132a47d977d2c6ee41de5358f1e85994)

Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
Signed-off-by: Jan Górski <j.gorski@welotec.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-24 16:10:22 +00:00
Siong W.LIM
9813515ff2 useradd.bbclass: Fix missing space when appending vardeps.
(From OE-Core rev: c4658401f1331026dc47a859dd665aab74918bba)

Signed-off-by: Siong <wslim87@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-19 11:34:33 +00:00