In expectation of SPDX 3.0 support, move the create-spdx.bbclass ->
create-spdx-2.2.bbclass. The create-spdx.bbclass class still exists and
can be used if a user doesn't care about which specific version of SPDX
they get.
(From OE-Core rev: 9ec01fe3e59be66331c14ab5391ecb0b6f140c22)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a link is pointing to location inaccessible to build user (e.g. "/root/something"),
filepath.is_file() throws "PermissionError: [Errno 13] Permission denied".
Fix this by first checking if it is a link.
(From OE-Core rev: e105befbe4ee0d85e94c2048a744f0373e2dbcdf)
Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Where there isn't a copyright statement, add one to make it explicit.
Also drop editor config lines where they were present.
(From OE-Core rev: 880c1ea3edc8edef974e65b2d424fc36809ea034)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise spdx can have references for data that is not packed
in the package delivered because this contol data is temporarly
and only exist while the package is been write.
During do_package_write_ipk task in do_package_ipk the control
files is cleaned up at the end. This can create a race condiction
when the do_create_spdx task runs the add_package_files function
and these files is been deleted at same time in the
task do_package_write_ipk.
ERROR: alsa-topology-conf-1.2.5.1-r0 do_create_spdx: Error executing a python function in exec_func_python() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:do_create_spdx(d)
0003:
File: '/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass', lineno: 567, function: do_create_spdx
0563: package_doc.add_relationship(package_doc, "DESCRIBES", spdx_package)
0564:
0565: package_archive = deploy_dir_spdx / "packages" / (package_doc.name + ".tar.zst")
0566: with optional_tarfile(package_archive, archive_packaged) as archive:
*** 0567: package_files = add_package_files(
0568: d,
0569: package_doc,
0570: spdx_package,
0571: pkgdest / package,
File: '/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass', lineno: 234, function: add_package_files
0230: info.mtime = source_date_epoch
0231:
0232: archive.addfile(info, f)
0233:
*** 0234: sha1 = bb.utils.sha1_file(filepath)
0235: sha1s.append(sha1)
0236: spdx_file.checksums.append(oe.spdx.SPDXChecksum(
0237: algorithm="SHA1",
0238: checksumValue=sha1,
File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 559, function: sha1_file
0555: """
0556: Return the hex string representation of the SHA1 checksum of the filename
0557: """
0558: import hashlib
*** 0559: return _hasher(hashlib.sha1(), filename)
0560:
0561:def sha384_file(filename):
0562: """
0563: Return the hex string representation of the SHA384 checksum of the filename
File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 528, function: _hasher
0524:
0525:def _hasher(method, filename):
0526: import mmap
0527:
*** 0528: with open(filename, "rb") as f:
0529: try:
0530: with mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as mm:
0531: for chunk in iter(lambda: mm.read(8192), b''):
0532: method.update(chunk)
Exception: FileNotFoundError: [Errno 2] No such file or directory: '/srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa-topology-conf/1.2.5.1-r0/packages-split/alsa-topology-conf/CONTROL/control'
ERROR: Logfile of failure stored in: /srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa-topology-conf/1.2.5.1-r0/temp/log.do_create_spdx.998864
INFO: recipe alsa-topology-conf-1.2.5.1-r0: task do_create_spdx: Failed
(From OE-Core rev: 0bfe403f46fd568910a727982c3ff528e3d5c8bc)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The correct field name is "supplier" according to SPDX schema.
The "supplier" field translates to "PackageSupplier", but that's for
tag-value format.
(From OE-Core rev: ca8db0e0a2860ac1e3f537471fa71b43c3be0a58)
Signed-off-by: Mihai Lindner <mihai.lindner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When building FreeRTOS this dependency generates an error because
bitbake cannot find any provider for "virtual/kernel".
>From a dependency analysis the task is executed independently from
this so it can be safely removed.
This patch has been discussed in this ML thread:
https://lists.openembedded.org/g/openembedded-core/topic/90602531
(From OE-Core rev: 1c02b768a71ec88bfe1cc0c4443683de8b66056e)
Signed-off-by: Davide Gardenal <davide.gardenal@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If IMAGE_NAME and IMAGE_LINK_NAME are the same an
exception is raised in image_combine_spdx because
it cannot create a symlink with the same exact
path of the original file. In that case there is
no point in creating a link in the first place.
Refactor image_combine_spdx to avoid code duplication
(From OE-Core rev: 88f76bba705007a7fcda506c79f743060d8f2e52)
Signed-off-by: Davide Gardenal <davide.gardenal@huawei.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no reason to first search for all available licenses using
avail_licenses() and then search through the same paths looking for
the actual license texts. Just look for the license texts directly
instead.
(From OE-Core rev: 59eb405a80f0a0acf9b754b2b78399bacb0094ae)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can directly call the function rather than using the variable indirection.
As this is the last user of the variable, it then allows removal of it in a
followup patch.
(From OE-Core rev: 3ed84651b2f4eff9409bfecba2a080e244124880)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no reason to include a path in foo[dirs] if it is also in
foo[cleandirs] (except if it is the last path in foo[dirs]).
(From OE-Core rev: cb30a6cfa58072bbf5f5def36103fbcac920e168)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, SPDX SBOMs are only created for images. Add support for
SDKs.
(From OE-Core rev: c3acbb936a339636153903daf127eec9f36de79b)
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch will read the begining of source files and try to find
the SPDX-License-Identifier to populate the licenseInfoInFiles
field for each source file. This does not populate licenseConcluded
at this time, nor rolls it up to package level.
We read as binary file since some source code seem to have some
binary characters, the license is then converted to ascii strings.
(From OE-Core rev: 51e5f328635eb022143178c3169bae719509697a)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A superfluous package was being created, but it was never used.
(From OE-Core rev: ddfa61f691fc51c1968e8f1ff43f69dccd1632ee)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Per NIST SBoM recommendations, include the Package Supplier field for
all SPDX packages that are created. This field should generally be set
to the person or organization that is performing the build, since they
would be considered the "supplier" of the SPDX packages.
(From OE-Core rev: ca48349501e0ec93dc2448d064e1567fca390bf5)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the kernel source is stored in work-shared, we need to add it
to the search path so the kernel and kernel-modules source code can
be found correctly.
(From OE-Core rev: 93a94890b9f2679945e96973937b209aa12302b7)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If IMAGE_LINK_NAME is set to "" (supported in other classes) then skip
creating symlinks for the spdx manifest files.
(From OE-Core rev: c2d5e2dbbba78d8d7036f6418360b2e84bcdc169)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the do_create_runtime_spdx task fails with a Key Error if a
dependency is not contained in the package providers dictionary. Add a
check before using "dep" as a key in "providers".
(From OE-Core rev: 140ce5ef5e8f10251091660e3ef76f315f409076)
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If LICENSE_PATH is not set, then the split() will fail on a NoneType.
(From OE-Core rev: d6260decae6d2654f6e058f12ca02d582a8ef5a4)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a function with the same name is_work_shared() in the archiver class
this causes a conflict when both classes are included. Use work-shared as the
check in WORKDIR to allow for other packages beyond the kernel and gcc that
use a common shared-work source directory.
(From OE-Core rev: 18eab77ee65c73b17225e69c7ba446ab1c69fa92)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the "Organization" field for SBOMs is hard-coded in
create-spdx. Create a new variable SPDX_ORG to make this field more
generic.
(From OE-Core rev: f239814f3f5d9bd54de54b0f2a5081067336e32b)
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a small function that checks for 'isNative' as part of an Annotation
When the collect_dep_sources() runs, it collects sources from both native
and non-native recipes. Later when the GENERATED_FROM matching occurs it
may find the file (via checksum) from the native recipe since it's the
same checksum as the target file. The that are generated DocumentRefs
point to the native recipe rather than the target recipe DocumentRef.
(From OE-Core rev: ba5ec03986a0bce8dccbc381fd34e44b8a135e66)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before, even if the code was seemingly written to search through all
licenses in ${COMMON_LICENSE_DIR} and ${LICENSE_PATH}, it would
actually bail out after only searching ${COMMON_LICENSE_DIR} due to
the exception handling.
Also refrain from using f-strings.
(From OE-Core rev: f58d54b31a1ddb4e60eb07365bfb7dfe78ed56af)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The special CLOSED license means that this is closed source code with
no other licensing assertions. It's not a generic license in oe-core,
and not a SPDX license, so transform it to NONE explicitly.
(From OE-Core rev: b71bfba1b5076ad1b9a10c08ef0e3393ccbed92a)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The trailing comma turns ref_id into an object instead of a string
when added to the relationship table.
(From OE-Core rev: ec20e1a0e0ba61a7615b361bfc9d9dc58a91f9b3)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of putting the full license text for non-SPDX licenses into the
recipe and every package, use links to the recipe from the packages if
possible.
(From OE-Core rev: 9220d35dc9071ebbe991117af8261ad99f321bb3)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For licenses which are not known to SPDX, find and embed the actual
license text in an ExtractedLicesingInfo block.
(From OE-Core rev: c638f4ae579520cab8cf30e1b4306bd0e2d2a788)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
add_extracted_document() is only called if the license isn't known to
SPDX, so there's no need to check again.
(From OE-Core rev: 03f084251510c810c8434d9ddd459187827c1ca8)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
spdx-licenses.json contains an array of licenses objects. As we'll be
searching it often, convert that to a dictionary when we parse it.
(From OE-Core rev: 3405d9114ca9fe4ba820e0025c91670d1a5150b1)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't want this warning causing problems on the AB, so leave it
comment out for now
(From OE-Core rev: d41018311abf91899645b8f3160db540abdb5398)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Native recipes do not produce packages and should not process them,
otherwise it can trigger an error in read_subpackage_metadata
(From OE-Core rev: ea077b1ff6a4ccd5241a085bf9a3f67011e0f7bf)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Stop filtering the runtime dependencies based on do_create_sdpx (makes
it only pick up things in DEPENDS) and instead include all task
dependencies that are not the current PN. This allows other dependency
methods to be picked up correctly, for example the dependency on the
kernel used by kernel modules.
(From OE-Core rev: c9659562cf88a820c668ff1eb20814dcb4829392)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Public Domain license (PD) needs a special exception in the license
processing since there is no common license text to be extracted for
these licenses.
(From OE-Core rev: fe5b757712aa99ff1ff10d2304ac320100635200)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
file:// URIs should not be included as the downloadLocation. Instead,
loop until a non-file:// URI is found, or set the location to
NOASSERTION if none is found
(From OE-Core rev: 19c61ec3986213b336d965d90b1329085e7d6179)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the bb.utils.sha* utilities to hash files since they are much faster
than the loops we were rolling ourselves
(From OE-Core rev: a6d9de5350937c7e25899491db59f473345f0b69)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Licenses reported in the SPDX documents should be either:
A) A valid SPDX identifier cross referenced from the SPDX license
database
B) A "LicenseRef" to a license described in the SPDX document
The licensing code will now add a placeholder extracted license with
corresponding "LicenseRef" for any licenses that are not matched to the
SPDX database
Parenthesis in the license expression are now handled correctly
(From OE-Core rev: 28d9d035c0ff8fcaf28bc96a976a43a602a47e94)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will create a more uniq DocumentRef, which will allow
the individual spdx files to be merged into a single SBOM
file reflecting the image. Do the same with the runtime dependencies
also
(From OE-Core rev: df7c88a48621d32c02f328eedc314f10d475b758)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes another creator that was missed earlier
(From OE-Core rev: 046c05fec9fc5162d7a14971ed1402d86605d229)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a debug source cannot be found, mark it as NOASSERTION so that other
tools at least know we were unable to locate it.
(From OE-Core rev: 0e6bdd3f208c50153087c2baca67e9fd64a458d0)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the index to DEPLOYDIR in addition to adding it to the SPDX archive
(From OE-Core rev: 374dc08c0f22e98a267676f71308592d17f77d64)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the creator name since this no longer lives in meta-doubleopen
(From OE-Core rev: 6fdea64ef53eb4de3d0e58e70ae6b391fdff6cd0)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
