Commit Graph

7708 Commits

Author SHA1 Message Date
Richard Purdie
7c1de3118f create-spdx-3.0/populate_sdk_base: Add SDK_CLASSES inherit mechanism to fix tarball SPDX manifests
Currently, "tarball" sdk based recipes don't generate SPDX manifests as they
don't include the rootfs generation classes. Split the SPDX 3.0 image class into
two so the SDK components can be included where needed.

To do this, introduce an SDK_CLASSES variable similar to IMAGE_CLASSES which
the SDK code can use.

Migrate testsdk usage to this.

Also move the image/sdk spdx classes to classes-recipe rather than the general classes
directory since they'd never be included on a global level.

For buildtools-tarball, it has its own testsdk functions so disable the class there as
a deferred inherit would overwrite it.

(From OE-Core rev: 662396533177b72cc1d83e95841b27f7e42dcb20)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-26 12:28:42 +01:00
Mark Hatle
edc44fcf13 create-spdx-*: Support multilibs via SPDX_MULTILIB_SSTATE_ARCHS
When a create-spdx-* classes is processing documents, it needs to
find the document in a path that is related to the SSTATE_ARCH
when a packge is generated.  The SSTATE_ARCH can be affected by
multilib configurations, resulting is something like armv8a-mlib.

When the image (or SDK) is being generated and the components are
collected, the system has no knowledge of the multilib arch and
will fail to find it, such as:

  ERROR: meta-toolchain-1.0-r0 do_populate_sdk: No SPDX file found
   for package libilp32-libgcc-dbg,
   False sstate:libilp32-libgcc:armv8a-ilp32-mllibilp32-elf:14.1.0:r0:armv8a-ilp32:12:
   sstate:libilp32-libgcc::14.1.0:r0::12:

Adding in the new SPDX_MULTILIB_SSTATE_ARCHS will provide a full
set of SSTATE_ARCHS including ones that contain the multilib
extension which will allow create-spdx-* to correctly find the
document it is looking for.  This would also be valuable to any
other function doing a similar search through SSTATE_ARCH that may
have been extended with multilib configurations.

(From OE-Core rev: f1499c36c1054fc90f7b7268cc95285f2eca72f7)

Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-26 12:28:42 +01:00
Mark Hatle
a798d00d54 create-sdpx-2.2.bbclass: Switch from exists to isfile checking debugsrc
While debugsrc is almost always a file (or link), there are apparently
cases where a directory could be returned from the dwarfsrcfiles
processing.  When this happens, the hashing fails and an error results
when building the SPDX documents.

(From OE-Core rev: 02e262c291c0b2066132b4cb2ca5fda8145284a9)

Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-18 17:21:39 +01:00
Joshua Watt
3f76c3d2c2 Switch default spdx version to 3.0
Changes the default SPDX version to 3.0

(From OE-Core rev: beef4c1a3e9fd8fa9cdbbb61e1bc3931012735b3)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-16 15:00:59 +01:00
Joshua Watt
87c60b9a5a classes/create-spdx-3.0: Move tasks to library
Move the bulk of the python code in the SPDX 3.0 classes into a library
file

(From OE-Core rev: aed6f8c1c2e291bde4d7172742790fa535b2fc7d)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-16 14:55:53 +01:00
Joshua Watt
9850df1b60 classes/spdx-common: Move to library
Moves the bulk of the code in the spdx-common bbclass into library code

(From OE-Core rev: 3f9b7c7f6b15493b6890031190ca8d1a10f2f384)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-16 14:55:53 +01:00
Joshua Watt
8426e027e8 classes/create-spdx-3.0: Add classes
Adds a class to generate SPDX 3.0 output and an image class that is used
when generating images

(From OE-Core rev: b63f6f50458fc6898e4deda5d6739e7bf3639c15)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-16 14:55:53 +01:00
Joshua Watt
2c2e1becd6 classes/create-spdx-2.2: Handle empty packages
When combining an SPDX document, the package list might be empty (e.g.
a baremetal image). Handle this case instead of erroring out

(From OE-Core rev: baf4e360f6e65a5e9aff2def69d2a720f38f92b2)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-13 23:28:31 +01:00
Ulrich Ölmann
b2bc061f7c buildhistory: fix typos
Fix some simple typos found while looking through the code.

(From OE-Core rev: c23cb2d89fcd014d1f08944468c7e2d8a7f3e8b0)

Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-04 22:56:15 +01:00
Joshua Watt
226ae9288e classes/create-spdx-2.2: Handle SemVer License List Versions
SPDX transitioned the license list to use SemVer visioning, (e.g.
"MAJOR.MINOR.MICRO"), but SPDX 2 only allows "MAJOR.MINOR". For maximum
compatibility, only keep the first two version numbers and discard the
rest which allows it to work with either scheme

(From OE-Core rev: 8757a5eaeaf2b9d7345212d003e5622289b123e6)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-02 22:29:13 +01:00
Joshua Watt
10917abebf classes/spdx-common: Move SPDX_SUPPLIER
Move the SPDX_SUPPLIER variable to create-spdx-2.2 since it's format
only has meaning in SPDX 2.2 (SPDX 3 uses SPDX_PACKAGE_SUPPLIER with a
different format)

(From OE-Core rev: 628c1e04072178d2c8095e53d5f6600c45a2679f)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-27 13:03:35 +01:00
Aleksandar Nikolic
edba6889b4 cve-check: Introduce CVE_CHECK_MANIFEST_JSON_SUFFIX
The variable contains the suffix of the CVE JSON manifest file.
By default, this variable is set to 'json', so the current behavior
is not changed, but enables developers to use some other suffix,
e.g., cve.json (similar to spdx.json).

(From OE-Core rev: d99eee76923659c0b95bf9ef415ae5d44f736d01)

Signed-off-by: Aleksandar Nikolic <an010@live.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-24 12:43:40 +01:00
Benjamin Szőke
dbe070b602 archiver.bbclass: Fix work-shared checking for kernel recipes
Source dir can be a symbolic link in some BSP's linux kernel recipe
which points to work-shared path (like linux-fslc in meta-freescale).
Change to use os.path.realpath() in order to get real path of
source dir.

(From OE-Core rev: 9191aa685418af32f003e067ef7c5737a271e3a5)

Signed-off-by: Benjamin Szőke <egyszeregy@freemail.hu>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-18 08:44:41 +01:00
Joshua Watt
717a427807 classes/spdx-common: Return empty list from extract_licenses
This is nicer as the normal return type is a list, so the calling code
doesn't have to deal with a None sometimes and a list others.

(From OE-Core rev: e200aa9cc6ceb8ca58ef239a1a5565287b38ce55)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-12 16:16:03 +01:00
Joshua Watt
37c9adaf80 classes/spdx-common: Add SPDX version to path
Since multiple versions of SPDX are available, use the version in the
path

(From OE-Core rev: 35c061a4514905b3ebbb7f0633a584927519445c)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-12 16:16:03 +01:00
Joshua Watt
92b202c246 classes/spdx-common: Move common SPDX to new class
Moves SPDX code that can be shared between different SPDX versions into
a common class

(From OE-Core rev: 769a390adc9fc0b52978abe0f19f885967af0117)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-12 16:16:03 +01:00
Kai Kang
b46075a150 multilib.bbclass: replace deprecated e.data with d
Replace deprecated e.data with d in multilib.bbclass and
multilib_global.bbclass.

Remove event check in function multilib_virtclass_handler_vendor in
multilib_global.bbclass. The function flag 'eventmask' has been set
with 'bb.event.ConfigParsed', so no need to check the event any more.

(From OE-Core rev: 8d87662d676a2b9ef921cb49ea6edc07b37410e2)

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-06 11:56:04 +01:00
Richard Purdie
6b9eb4e9f0 siteconfig: Drop siteconfig class/code/support
The siteconfig code was only used for 5 cache values. The complexity added to sstate
to support this code was considerable and the runtime much more significant than
any benefit the cache files would have added. Drop the support for this which
was only used minimally for ncurses and zlib.

(From OE-Core rev: f3766dc038f7ba9780ddaf5eb8d27385ea31d7d0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-31 16:58:36 +01:00
Richard Purdie
6d77296d83 devtool: Drop oe-local-files and simplify
The only real reason for oe-local-files was to support S = WORKDIR. With changes to
drop support for that, it makes sense to simplify devtool and to try and make both
the code and the processes/workflows simpler.

This patch drops support for S = WORKDIR, removes oe-local-files and then updates
the test cases to match this new situation.

At the code level, we assume we can always now track code changes using git and
that things committed into git are handled as patches (as before) but delta against
HEAD is saved as specific file level changes to the recipe.

One test is disabled as it is no longer approproate. It is being keped until we can
make WORKDIR != UNPACKDIR at which point it should be revisited.

(From OE-Core rev: ce8190c519052fed10b5233697b69a75868db45a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-21 12:08:04 +01:00
joshua Watt
3bf65ca15f classes/create-spdx-2.2: Fix SPDX dependencies for ABI Safe recipes
SPDX Documents can currently get into a state where they reference SPDX
IDs that do not exist (locally). The reason for this is that some
tasks/recipes are marked as ABI safe, and thus are excluded from
taskhash. This means that when SPDX creates a document, it will refer to
the SPDX ID in the dependency at the time when the document is created,
but if the dependency changes and gets a new SPDX ID, the document will
not rebuild to reference the new SPDX ID, causing it to be dangling.

Fix this by using the new field in BB_TASKDEPDATA to skip dependencies
that are not part of the taskhash calculation.

(From OE-Core rev: 8393557b5229c7a0cea18a586848384bff1d7424)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-09 09:55:27 +01:00
joshua Watt
8d79ca1478 classes/create-spdx-2.2: Fix SPDX Namespace Prefix
According to the SPDX documentation, it should be "spdxdocs" not
"spdxdoc"

[YOCTO #15398]

(From OE-Core rev: d1b25413ced62dc2927dae57b8d16e67d15dc220)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-02 15:14:07 +01:00
Simone Weiß
296fdb6643 cve-check: Log if CVE_STATUS set but not reported for component
Log if the CVE_STATUS is set for a CVE, but the cve is not reported for a
component. This should hopefully help to clean up not needed CVE_STATUS
settings.

(From OE-Core rev: 013d531a84fa08b6ae8a47bdf3ba1fa8f18ba270)

Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-24 16:10:23 +00:00
Piotr Łobacz
f6daeba2e9 useradd.bbclass: Fix order of postinst-useradd-*
postinst-useradd-* haven't been running in order of dependency.

This patch is reworked from Piotr Łobacz's patch and fixes:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15084
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13904

basepasswd_sysroot_postinst in base-passwd can install postinst-useradd-*
scripts with any order. Sometimes this means, for example a useradd postinst
will attempt to run without the corresponding group postinst causing errors.
This patch ensures that we first run groupadd, then useradd and then
group membership.

[RP: Tweaked to avoid removing previous fixes and for whitespace/style issues
     Also ensure the scripts are changed to execute with -e to highlight errors]

(From OE-Core rev: 322ef726132a47d977d2c6ee41de5358f1e85994)

Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
Signed-off-by: Jan Górski <j.gorski@welotec.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-24 16:10:22 +00:00
Siong W.LIM
9813515ff2 useradd.bbclass: Fix missing space when appending vardeps.
(From OE-Core rev: c4658401f1331026dc47a859dd665aab74918bba)

Signed-off-by: Siong <wslim87@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-19 11:34:33 +00:00
Chen Qi
6cc6246fd0 multilib_global.bbclass: fix parsing error with no kernel module split
The problem could be reproduced with the following settings:
MACHINE = "qemux86-64"
KERNEL_SPLIT_MODULES = "0"
require conf/multilib.conf
MULTILIBS ?= "multilib:lib32"
DEFAULTTUNE:virtclass-multilib-lib32 ?= "core2-32"

The error message is as below:
  bb.data_smart.ExpansionError: Failure expanding variable KERNEL_VERSION_PKG_NAME, expression was ${@legitimize_package_name(d.getVar('KERNEL_VERSION'))} which triggered exception TypeError: expected string or bytes-like object
  The variable dependency chain for the failure is: KERNEL_VERSION_PKG_NAME -> RPROVIDES:kernel-modules

This is because multilib_virtclass_handler_global function in
multilib_global.bbclass deletes KERNEL_VERSION. So we need to handle
such situation. We'll also need to delete KERNEL_VERSION_PKG_NAME
to avoid this parsing error.

(From OE-Core rev: 43dd497bc161ac44faecfdff052db03679dbb4f8)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-06 10:32:19 +00:00
Alexander Kanavin
55bfa99451 classes/multilib: expand PACKAGE_WRITE_DEPS in addition to DEPENDS
Otherwise, PACKAGE_WRITE_DEPS would contain non-multilib variants
of dependencies even when building multilib items, resulting in
sysroots being populated with entirely wrong versions of them.

This hasn't been noticed until now through sheer (bad) luck, I think,
except in the cpio recipe, but the previous commit shows that the issues
did occur, quietly. Every other recipe in oe-core and meta-oe does not
prepend the multilib prefix.

(From OE-Core rev: 234965cb88ccfa9c3a357928f7155b119044e8fc)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-27 08:20:02 +00:00
Alexander Kanavin
ed08180608 sysroot user management postinsts: run with /bin/sh -e to report errors when they happen
This exposes the following failure in a multilib setup, when everything
up to do_package_write_rpm is in sstate, but do_package_write_rpm is not
(there's a similar fail for lib64-man-db, and the failures themselves will be fixed separately
in the next commit):

Exception: subprocess.CalledProcessError: Command '/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/recipe-sysroot/usr/bin/postinst-base-passwd' returned non-zero exit status 1.

Subprocess output:
install: cannot stat '/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/lib64-recipe-sysroot/usr/share/base-passwd/passwd.master': No such file or directory

(From OE-Core rev: 5ffa333db28bc5d8e440c983fdf95589d332461d)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-27 08:20:02 +00:00
Julien Stephan
8934057ce8 externalsrc: fix task dependency for do_populate_lic
do_populate_lic dependencies are defined inside license.bbclass such as:

  addtask populate_lic after do_patch before do_build

but externalsrc deletes the do_patch task, so the only dependency left for
do_populate_lic is "before do_build"

On a devtool context, when doing devtool modify, sources are extracted inside
build/workspace/sources/${BPN}/ and local files inside
build/workspace/sources/${BPN}/oe-local-files

When building the recipe after a devtool modify, do_unpack is called again to
unpack (possibly modified) local files from
build/workspace/sources/${BPN}/oe-local-files into ${WORKDIR}.

Since the only left dependency for do_populate_lic is do_build, the
do_populate_lic can be called BEFORE do_unpack. Most of the time this is not a
problem, because license files are generally located inside ${S}, which
corresponds to build/workspace/sources/${BPN} (and is already unpacked),
but this can lead to an issue if recipe sets LIC_FILES_CHKSUM to look for
files in ${WORKDIR} (example from init-ifupdown_1.0.bb):

  LIC_FILES_CHKSUM = "file://${WORKDIR}/copyright;md5=3dd6192d306f582dee7687da3d8748ab"

So devtool modify init-ifupdown && bitbake init-ifupdown gives the following
error:

  WARNING: init-ifupdown-1.0-r0 do_populate_lic: Could not copy license file <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/copyright to <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/license-destdir/qemux86_64/init-ifupdown/copyright: [Errno 2] No such file or directory: '<...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/copyright'
  ERROR: init-ifupdown-1.0-r0 do_populate_lic: QA Issue: init-ifupdown: LIC_FILES_CHKSUM points to an invalid file: <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/copyright [license-checksum]
  ERROR: init-ifupdown-1.0-r0 do_populate_lic: Fatal QA errors were found, failing task.
  ERROR: Logfile of failure stored in: <...>/build/tmp/work/qemux86_64-poky-linux/init-ifupdown/1.0/temp/log.do_populate_lic.838584
  ERROR: Task (<...>/poky/meta/recipes-core/init-ifupdown/init-ifupdown_1.0.bb:do_populate_lic) failed with exit code '1'

Fix this by forcing the do_populate_lic task to run after do_unpack

(From OE-Core rev: ea6a0cccdd274534809df62a0a196bf83489a1e5)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-24 15:46:19 +00:00
Vyacheslav Yurkov
4eca05e15b classes: go-vendor: Unlink vendor dir later
Vendor directory might still be required during install stage, so defer
the removal until later stage.

(From OE-Core rev: f4538e3884a0cfa07e16e6a2c986271cc7322dd8)

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-19 12:21:22 +00:00
Vyacheslav Yurkov
3e579c303e classes: go-vendor: Handle modules from the same repo
Take into account module version when populating vendor directory,
because a module with the same URL but with a different version tag
could be used as an indirect dependency.

(From OE-Core rev: 8f6320c0858941b2441e290ef3586b48c2700cd1)

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-19 12:21:22 +00:00
Vyacheslav Yurkov
d5e2279d21 classes: go-vendor: Reference local modules
Create symlinks for local modules, which are usually not referenced in
the SRC_URI, but still expected to be found in the vendor directory
during the build.

(From OE-Core rev: 16da5d9ad448aafd8b5fd63480727bd1b09ec9f1)

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-19 12:21:22 +00:00
Richard Purdie
c917323a39 classes/recipes: Switch to use inherit_defer
Now that bitbake supports the use of inherit_defer, switch all conditional
(variable based) inherits to use this instead. This leads to more a more
deterministic user experience since there is no longer an immediate expansion
and later changes to the variables in question (e.g. a bbappend) are
accounted for.

This patch tries to ensure the behaviour before/after remains as unchanged
as it reasonably can, e.g. by always inherting populate_sdk_base. native
and nativesdk continue to need to be inherited last, hence being used
with inherit_defer in a handful of very specific cases.

(From OE-Core rev: 451363438d38bd4552d5bcec4a92332f5819a5d4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-18 10:15:58 +00:00
Etienne Cordonnier
3db106c41b chrpath.bbclass: fix Darwin support
Ported from the meta-darwin layer:
The call to out.split("\n") expects a string, thus the parameter text=True is
needed (otherwise Popen returns a bytes object).

Note that "text" is just a more readable alias for universal_newlines.

(From OE-Core rev: 0abaa7bf7f7d9a5ac96e6fdbe99334cb2fb0e4db)

Signed-off-by: Dominik Schnitzer <dominik@snap.com>
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-15 21:42:15 +00:00
Robert Yang
fdef56a596 archiver.bbclass: Drop tarfile module to improve performance
* The tarfile module doesn't support xz options or environment varible
  XZ_DEFAULTS, this makes do_ar_patched incrediblely slow when the file is
  large, for example, chromium-x11 is about 3GB:
  - "bitbake chromium-x11 -car_patched" hasn't been done after 3 hours on my
     host, I checked the partial tar.xz file is only 1.5GB, so maybe more than 6
     hours is required to complete the task.

  - Now only less than 4 minutes is needed on the same host.

* Need add xz to HOSTTOOLS when archiver.bbclass is enabled and compression is xz.

(From OE-Core rev: 6548354f049b173e8d443bc547d35c9d9fc05259)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-23 08:46:01 +00:00
Jeremy A. Puhlman
a49de3ccc1 create-spdx-2.2: combine spdx can try to write before dir creation
On occasion a file is attmpeded to be opened prior to the
creation of the spdx_workdir. Create the directory before
the open, just in case.

File: '/build/layers/poky/meta/classes/create-spdx-2.2.bbclass', lineno: 1081, function: combine_spdx
     1077:        )
     1078:
     1079:    image_spdx_path = spdx_workdir / (rootfs_name + ".spdx.json")
     1080:
 *** 1081:    with image_spdx_path.open("wb") as f:
     1082:        doc.to_json(f, sort_keys=True, indent=get_json_indent(d))

(From OE-Core rev: bb9f2a9c0ff5dcdeaf1a0beb6a614d0d022a2481)

Signed-off-by: Jeremy A. Puhlman <jpuhlman@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-17 19:07:22 +00:00
Robert Yang
bb0566eafd archiver.bbclass: Improve work-shared checking
There are other recipes except the listed ones which use work-shared, improve
the checking to make other recipes such as llvm-project-source work with
do_ar_patched.

(From OE-Core rev: 5fbb4ca8da4f4f1ea426275c45634802dcb5a575)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-13 11:34:27 +00:00
Richard Purdie
4bb222e0d7 useradd: Fix useradd do_populate_sysroot dependency bug
If a task is adde which has a dependency on the do_populate_sysroot task of
the recipe, it will cause it to be installed into the sysroot (similar to
do_addto_recipe_sysroot). This fails since the postinst script is an overlapping
file:

Exception: FileExistsError: [Errno 17] File exists:
'tmp/sysroots-components/all/useraddbadtask/usr/bin/postinst-useradd-useraddbadtask'
  ->
'tmp/work/all-poky-linux/useraddbadtask/1.0/recipe-sysroot/usr/bin/postinst-useradd-useraddbadtask'

The copy written out at do_prepare_recipe_sysroot time is just for debug so
rename it, meaning there are no longer overlapping files and the installation
can be successful, removing the error.

[YCOTO #14961]

With the bug fixed, enable the test.

(From OE-Core rev: 564339afb73fc52a66c1a08437587cad1c4d46e7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-10 09:24:01 +00:00
Eilís 'pidge' Ní Fhlannagáin
ecef665062 useradd: Fix issues with useradd dependencies
If recipe A requires the useradd actions of recipe B we need to
ensure that recipe B is part of the recipe A dependancy chain. In
order to do that, we introduce USERADD_DEPENDS. This makes sure
that the do_populate_sysroot_setscene of recipe B exists for
recipe A in case of a missing TMPDIR. This requires changes made in
runqueue.py by RP.

This commit along with the runqueue fixes effects:
Bug 13419 - recipes that add users to groups cannot rely on other recipes creating those groups (when population from sstate happens)
Bug 13904 - do_prepare_recipe_sysroot: postinst-useradd-* does not run in order of dependency and sometimes fails
Bug 13279 - Make sure users/groups exist for package_write_* tasks
Bug 15084 - For some reason using of same user in two recipes does not work properly

I've included the start of self-testing for useradd by adding tests for
13419 (which ends up testing 13904, 13279, 15084 by virtue of them all
      having the same root cause)

(From OE-Core rev: b47f2352376bd16b7e7087b4dab143403e67e094)

Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-08 17:51:10 +00:00
Ross Burton
028b6f6226 Revert "cve-check: Modify judgment processing using "=" in version comparison"
This change introduced a warning if version comparisons failed, but
this is far too common an issue in data that we don't control, so this
shouldn't cause a warning:

WARNING: automake-native-1.16.5-r0 do_cve_check: automake: Failed to compare 1.16.5 = branch_1-9 for CVE-2009-4029
WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m1 for CVE-2010-4539
WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m2 for CVE-2010-4539
WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m3 for CVE-2010-4539
WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m4\/m5 for CVE-2010-4539
WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m1 for CVE-2010-4644
WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m2 for CVE-2010-4644
WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m3 for CVE-2010-4644
WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m4\/m5 for CVE-2010-4644
WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m1 for CVE-2011-0715
WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m2 for CVE-2011-0715
WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m3 for CVE-2011-0715
WARNING: subversion-1.14.2-r0 do_cve_check: subversion: Failed to compare 1.14.2 = m4\/m5 for CVE-2011-0715
WARNING: automake-1.16.5-r0 do_cve_check: automake: Failed to compare 1.16.5 = branch_1-9 for CVE-2009-4029
WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s for CVE-2003-0577
WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s for CVE-2004-0982
WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s for CVE-2004-1284
WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s_r11 for CVE-2006-3355
WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s for CVE-2007-0578
WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s_r11 for CVE-2007-0578
WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s for CVE-2009-1301
WARNING: mpg123-1.32.3-r0 do_cve_check: mpg123: Failed to compare 1.32.3 = pre0.59s_r11 for CVE-2009-1301

This reverts commit a1989e4197178c2431ceca499e0b4876b233b131.

(From OE-Core rev: c7c7dbdd5474002cfd9ec24864e77a0df2b790ea)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-05 21:16:43 +00:00
Matsunaga-Shinji
291bc9e96a cve-check: Modify judgment processing using "=" in version comparison
Judgment processing of vulnerable using "=" compares characters as strings rather than numbers,
and misjudges "cases that do not match in strings but do match in numbers" as "Patched".
(e.g. PV = "1.2.0" and Vulnerabilities Affected Versions (registered with NVD) = "1.2")

Therefore, if the comparison operator used in the judgment processing of vulnerable is "=",
add numeric comparison processing.

(From OE-Core rev: a1989e4197178c2431ceca499e0b4876b233b131)

Signed-off-by: Shinji Matsunaga <shin.matsunaga@fujitsu.com>
Signed-off-by: Shunsuke Tokumoto <s-tokumoto@fujitsu.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-02 17:18:57 +00:00
Julien Stephan
17427db136 devtool: tag all submodules
In the case of a repository with submodules, we need to add the
"devtool-base" and "devtool-patched" tag on all submodules in order to
properly detect the added/removed/modified patches

(From OE-Core rev: 241da68805d177d4ec4b302c8a997645cc645286)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-01 11:48:25 +00:00
Adam Johnston
cdd525b150 useradd_base: Fix sed command line for passwd-expire
A previous commit tried to add the --follow-symlinks option to
the perform_passwd_expire function in useradd_base.bbclass, however it used
a single -.

This is interpreted as --file=ollow-symlinks which results in...

sed: couldn't open file ollow-symlinks: No such file or directory

and...

ERROR: <image name>: passwd --expire operation did not succeed.

Fix by adding the missing -

(From OE-Core rev: 3c0deafcfcea3f610c7dd9a2d2884a16fbfe0497)

Signed-off-by:  Adam Johnston <adam.johnston@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-21 21:25:41 +00:00
Joakim Tjernlund
a2ea2cb8d9 sed -i destroys symlinks
If /etc/passwd is a symlink, sed -i on same file will replace the
symlink with a new file. Prevent that by adding --follow-symlinks
option to sed

(From OE-Core rev: 6ec004b2e7b4342465af8e5e6cc66041834821a0)

Signed-off-by: Joakim Tjernlund <joakim.tjernlund@infinera.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-20 15:30:52 +00:00
Richard Purdie
7e0e0ef000 go-vendor: Minor style tweaks
Drop a len() usage that isn't needed, drop a comment that isn't needed
and use bb.fatal() to end the task with an error.

(From OE-Core rev: 1b9cf9cc2a52902e1181d2dac2ba9f2c3172835a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-08 10:56:35 +00:00
Lukas Funke
22f8fdbe7f classes: go-vendor: Add go-vendor class
(From OE-Core rev: d61bdf392e10140671ca56f2a2b0dc824be8ab80)

Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-06 16:41:11 +00:00
Ross Burton
a9986a2184 cve-check: sort the package list in the JSON report
The JSON report generated by the cve-check class is basically a huge
list of packages.  This list of packages is, however, unsorted.

To make things easier for people comparing the JSON, or more
specifically for git when archiving the JSON over time in a git
repository, we can sort the list by package name.

(From OE-Core rev: e9861be0e5020830c2ecc24fd091f4f5b05da036)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-26 15:29:34 +01:00
Peter Kjellerstedt
d066ec92dc externalsrc.bbclass: Support specifying patterns in CONFIGURE_FILES
This allows, e.g., *.cmake to be added to CONFIGURE_FILES to make the
do_configure task depend on changes to any cmake file.

(From OE-Core rev: 09873b3fb24a00cfbd73282d29e4c5821774f579)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06 11:51:11 +01:00
Joshua Watt
7b7a439953 classes/create-spdx-2.2: Show error if document is not found
As in other places, print a more helpful error if a SPDX document is not
found when assembling documents for the final SPDX archive.

(From OE-Core rev: 0a3f7e0b41552e113347d6f5edefe84fd4e47fdf)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-29 23:11:59 +01:00
Antoine Lubineau
8e680771d2 cve-check: add CVSS vector string to CVE database and reports
This allows building detailed vulnerability analysis tools without
relying on external resources.

(From OE-Core rev: 048ff0ad927f4d37cc5547ebeba9e0c221687ea6)

Signed-off-by: Antoine Lubineau <antoine.lubineau@easymile.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-28 12:37:46 +01:00
Richard Purdie
f69017a713 create-spdx: Ensure it is clear where the message comes from
This message can appear in do_rootfs and the users tend to think this is
some kind of packaging issue. Add SPDX to the message so users like me
think in the right direction.

(From OE-Core rev: ddcf15134b5ec4bf4c728156461c2e45194d71ca)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-26 10:25:42 +01:00