Commit Graph

29 Commits

Author SHA1 Message Date
Andres Beltran
7869000827 create-spdx: Fix key errors in do_create_runtime_spdx
Currently, the do_create_runtime_spdx task fails with a Key Error if a
dependency is not contained in the package providers dictionary. Add a
check before using "dep" as a key in "providers".

(From OE-Core rev: 140ce5ef5e8f10251091660e3ef76f315f409076)

Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-21 11:05:02 +00:00
Saul Wold
3b836f5817 create-spdx: Protect against None from LICENSE_PATH
If LICENSE_PATH is not set, then the split() will fail on a NoneType.

(From OE-Core rev: d6260decae6d2654f6e058f12ca02d582a8ef5a4)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-21 11:05:02 +00:00
Saul Wold
e1e8531c64 create_spdx: ensure is_work_shared() is unique
There is a function with the same name is_work_shared() in the archiver class
this causes a conflict when both classes are included. Use work-shared as the
check in WORKDIR to allow for other packages beyond the kernel and gcc that
use a common shared-work source directory.

(From OE-Core rev: 18eab77ee65c73b17225e69c7ba446ab1c69fa92)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-21 11:05:02 +00:00
Andres Beltran
d313f53126 create-spdx: Set the Organization field via a variable
Currently, the "Organization" field for SBOMs is hard-coded in
create-spdx. Create a new variable SPDX_ORG to make this field more
generic.

(From OE-Core rev: f239814f3f5d9bd54de54b0f2a5081067336e32b)

Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-05 11:41:00 +00:00
Saul Wold
540182dd5b create-spdx: cross recipes are native also
Recipes that inherit cross should also be categorized as isNative

(From OE-Core rev: ee113e3894deb1cfb18622085a3fe0600e1ef01d)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-28 14:16:31 +01:00
Saul Wold
36c184344e create-spdx: add create_annotation function
This allows code reuse and future usage with relationship annotations

(From OE-Core rev: 1f8fdb7dc9d02d0ee3c42674ca16e03f0ec18cba)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-28 14:16:31 +01:00
Saul Wold
6d98af48f4 create-spdx: Use SPDXAnnotation to track native recipes
Create a small function that checks for 'isNative' as part of an Annotation

When the collect_dep_sources() runs, it collects sources from both native
and non-native recipes. Later when the GENERATED_FROM matching occurs it
may find the file (via checksum) from the native recipe since it's the
same checksum as the target file. The that are generated DocumentRefs
point to the native recipe rather than the target recipe DocumentRef.

(From OE-Core rev: ba5ec03986a0bce8dccbc381fd34e44b8a135e66)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-28 17:55:04 +01:00
Peter Kjellerstedt
ba7f322a3e create-spdx.bbclass: Search all license directories for licenses
Before, even if the code was seemingly written to search through all
licenses in ${COMMON_LICENSE_DIR} and ${LICENSE_PATH}, it would
actually bail out after only searching ${COMMON_LICENSE_DIR} due to
the exception handling.

Also refrain from using f-strings.

(From OE-Core rev: f58d54b31a1ddb4e60eb07365bfb7dfe78ed56af)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-16 09:50:34 +01:00
Ross Burton
d42e12a0bd create-spdx: handle CLOSED license
The special CLOSED license means that this is closed source code with
no other licensing assertions.  It's not a generic license in oe-core,
and not a SPDX license, so transform it to NONE explicitly.

(From OE-Core rev: b71bfba1b5076ad1b9a10c08ef0e3393ccbed92a)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-10 09:25:38 +01:00
Saul Wold
745bc89574 create-spdx: remove trailing comma
The trailing comma turns ref_id into an object instead of a string
when added to the relationship table.

(From OE-Core rev: ec20e1a0e0ba61a7615b361bfc9d9dc58a91f9b3)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-08 15:32:55 +01:00
Ross Burton
4d3692b5a7 create-spdx: don't duplicate license texts in each package
Instead of putting the full license text for non-SPDX licenses into the
recipe and every package, use links to the recipe from the packages if
possible.

(From OE-Core rev: 9220d35dc9071ebbe991117af8261ad99f321bb3)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-04 08:44:11 +01:00
Ross Burton
41a29f0b96 create-spdx: embed unknown license texts
For licenses which are not known to SPDX, find and embed the actual
license text in an ExtractedLicesingInfo block.

(From OE-Core rev: c638f4ae579520cab8cf30e1b4306bd0e2d2a788)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-04 08:44:11 +01:00
Ross Burton
734f5badef create-spdx: remove redundant test
add_extracted_document() is only called if the license isn't known to
SPDX, so there's no need to check again.

(From OE-Core rev: 03f084251510c810c8434d9ddd459187827c1ca8)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-04 08:44:11 +01:00
Ross Burton
3c9102f3b6 create-spdx: transform license list into a dict for faster lookups
spdx-licenses.json contains an array of licenses objects. As we'll be
searching it often, convert that to a dictionary when we parse it.

(From OE-Core rev: 3405d9114ca9fe4ba820e0025c91670d1a5150b1)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-04 08:44:11 +01:00
Joshua Watt
acec2d26d6 classes/create-spdx: Comment out placeholder license warning
We don't want this warning causing problems on the AB, so leave it
comment out for now

(From OE-Core rev: d41018311abf91899645b8f3160db540abdb5398)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03 09:53:29 +01:00
Joshua Watt
cf707e8323 classes/create-spdx: Skip package processing for native recipes
Native recipes do not produce packages and should not process them,
otherwise it can trigger an error in read_subpackage_metadata

(From OE-Core rev: ea077b1ff6a4ccd5241a085bf9a3f67011e0f7bf)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03 09:53:29 +01:00
Joshua Watt
b8b45c35fa classes/create-spdx: Collect all task dependencies
Stop filtering the runtime dependencies based on do_create_sdpx (makes
it only pick up things in DEPENDS) and instead include all task
dependencies that are not the current PN. This allows other dependency
methods to be picked up correctly, for example the dependency on the
kernel used by kernel modules.

(From OE-Core rev: c9659562cf88a820c668ff1eb20814dcb4829392)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03 09:53:29 +01:00
Joshua Watt
45ea7b7970 classes/create-spdx: Add special exception for Public Domain license
The Public Domain license (PD) needs a special exception in the license
processing since there is no common license text to be extracted for
these licenses.

(From OE-Core rev: fe5b757712aa99ff1ff10d2304ac320100635200)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03 09:53:29 +01:00
Joshua Watt
de3b871786 classes/create-spdx: Fix file:// in downloadLocation
file:// URIs should not be included as the downloadLocation. Instead,
loop until a non-file:// URI is found, or set the location to
NOASSERTION if none is found

(From OE-Core rev: 19c61ec3986213b336d965d90b1329085e7d6179)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03 09:53:29 +01:00
Joshua Watt
beeb0f9e53 classes/create-spdx: Speed up hash calculations
Use the bb.utils.sha* utilities to hash files since they are much faster
than the loops we were rolling ourselves

(From OE-Core rev: a6d9de5350937c7e25899491db59f473345f0b69)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03 09:53:28 +01:00
Joshua Watt
902480107d classes/create-spdx: Fix up license reporting
Licenses reported in the SPDX documents should be either:
 A) A valid SPDX identifier cross referenced from the SPDX license
    database
 B) A "LicenseRef" to a license described in the SPDX document

The licensing code will now add a placeholder extracted license with
corresponding "LicenseRef" for any licenses that are not matched to the
SPDX database

Parenthesis in the license expression are now handled correctly

(From OE-Core rev: 28d9d035c0ff8fcaf28bc96a976a43a602a47e94)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03 09:53:28 +01:00
Saul Wold
0d5c2d9f35 classes/create-spdx: extend DocumentRef to include name
This will create a more uniq DocumentRef, which will allow
the individual spdx files to be merged into a single SBOM
file reflecting the image. Do the same with the runtime dependencies
also

(From OE-Core rev: df7c88a48621d32c02f328eedc314f10d475b758)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03 09:53:28 +01:00
Joshua Watt
25e41200c7 classes/create-spdx: Fix another creator
Fixes another creator that was missed earlier

(From OE-Core rev: 046c05fec9fc5162d7a14971ed1402d86605d229)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03 09:53:28 +01:00
Joshua Watt
bdda626481 classes/create-spdx: Add NOASSERTION for unknown debug sources
If a debug source cannot be found, mark it as NOASSERTION so that other
tools at least know we were unable to locate it.

(From OE-Core rev: 0e6bdd3f208c50153087c2baca67e9fd64a458d0)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03 09:53:28 +01:00
Joshua Watt
0b871c57b8 classes/create-spdx: Add runtime dependency mapping
(From OE-Core rev: 82f3229bce41dc101c79865033432161dac269d8)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03 09:53:28 +01:00
Joshua Watt
69faca3df6 classes/create-spdx: Add index to DEPLOYDIR
Add the index to DEPLOYDIR in addition to adding it to the SPDX archive

(From OE-Core rev: 374dc08c0f22e98a267676f71308592d17f77d64)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03 09:53:28 +01:00
Joshua Watt
f1cd4f264d classes/create-spdx: Add SHA1 to index file
(From OE-Core rev: ebfe78ad26b643ce0fb22ba5b3ede43da4a78987)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03 09:53:28 +01:00
Joshua Watt
f3796b4524 classes/create-spdx: Change creator
Update the creator name since this no longer lives in meta-doubleopen

(From OE-Core rev: 6fdea64ef53eb4de3d0e58e70ae6b391fdff6cd0)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03 09:53:28 +01:00
Joshua Watt
f1a34a63e4 classes/create-spdx: Add class
Adds a class as a first attempt to create SPDX SBoM documents during the
build. This initial work was influenced by [meta-doubleopen][1],
although almost completely rewritten.

[1]: https://github.com/doubleopen-project/meta-doubleopen

(From OE-Core rev: 78c79821ae7e2f060b9a74ea9aefce98102bb00e)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03 09:53:28 +01:00