Commit Graph

1208 Commits

Author SHA1 Message Date
Richard Purdie
bc883e4991 rootfs-postcommands: Try and improve ordering constraints
The current code is in race to see who can set things last. This isn't scalable or
sustainable and problemtic in the face of inherit ordering changes.

Move the ordering issue into the actual code execution, which isn't ideal but
the best of several bad options and at least lets us drop the anonymous python.

(From OE-Core rev: 0ffff2c1f80a9b79b133d787764bab164d9abd70)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-18 10:15:58 +00:00
Richard Purdie
12619deabd rootfs: Fix MULTILIB_RE_ALLOW to be inherit order independent
This variable is only used by the ipk backend with multilibs.

In order to make it work correctly regardless of inherit order, change
the string to be space delimeted, set using += and add in the regex '|'
sperator at the end of processing.

(From OE-Core rev: 72befdb12568fbc642022ef0a23b269c5b37a638)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-18 10:15:58 +00:00
Etienne Cordonnier
5031cf42ff package.py: fix Darwin support
- 'subprocess.Popen([d.expand("${HOST_PREFIX}otool)' requires text-mode (a more
  readable alias for the universal_newlines parameter), since otool produces
  text and the code 'out.split("\n")' expects a string, not a bytes object.
  otool is used on MacOS only, so this error isn't triggered on Linux.

- use 'startswith("darwin")' in order to support all darwin versions and not
  just specific versions (meta-darwin supports darwin21 at the moment).

(From OE-Core rev: 248ca79a6400e063c4965f9542c614bf837ff758)

Signed-off-by: Dominik Schnitzer <dominik@snap.com>
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-15 21:42:15 +00:00
Yang Xu
b11e3aa995 rootfs.py: check depmodwrapper execution result
The execution result of depmodwrapper is not checked which makes depmod
generation failed sliently and hard to detect.

So check exection result and stop building if depmodwrapper failed.

(From OE-Core rev: 2f88e7d331390c6aaecc4522253e24791aec299e)

Signed-off-by: Yang Xu <yang.xu@mediatek.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 17:01:28 +00:00
Alexander Kanavin
4675bbb757 lib/sstatesig/find_siginfo: raise an error instead of returning None when obtaining mtime
Suppressing the error and returning None can result in a delayed failure:
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/6254/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/6262/steps/14/logs/stdio

It is not clear why the os.stat() error occurs to begin with (it shouldn't), so rather than
adding further workarounds, let's get diagnostics at the source first, so we understand
what is going on.

(From OE-Core rev: 35483d4756ab53805507f72a9a0edb3f83759694)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-05 12:04:52 +00:00
Richard Purdie
a7aa37f2fd sstatesig: Add version information for find_sigingfo
Since we're changing the return values of the function, add a version so
bitbake can ensure it is using a compatible function.

(From OE-Core rev: 738bf3717002d124958a21a6925b0eca14fef131)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-05 11:59:08 +00:00
Alexander Kanavin
c45ffa9e94 sstatesig/find_siginfo: unify a disjointed API
find_siginfo() returns two different data structures depending
on whether its third argument (list of hashes to find) is empty or
not:
- a dict of timestamps keyed by path
- a dict of paths keyed by hash

This is not a good API design; it's much better to return
a dict of dicts that include both timestamp and path, keyed by
hash. Then the API consumer can decide how they want to use these
fields, particularly for additional diagnostics or informational
output.

I also took the opportunity to add a binary field that
tells if the match came from sstate or local stamps dir, which
will help prioritize local stamps when looking up most
recent task signatures.

(From OE-Core rev: 8721c52041e910bd4d8a9235b52f274f4f02c8a3)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-05 11:59:08 +00:00
Richard Purdie
52f41ff243 lib/prservice: Improve lock handling robustness
It is possible we could crash holding a lock whilst parsing in this code.
Switch to use utils.fileslocked() in the with expression to avoid this.

This may be causing some of our strange intermittent failures in PRServ
tests.

(From OE-Core rev: 4e59db15e5df2cc3d0ae042454812a2d54cef77b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-24 10:39:07 +00:00
Joshua Watt
dd5475d8e3 lib/packagedata.py: Add API to iterate over rprovides
Adds an API that makes it easier to iterate over the package data for a
all providers of a runtime dependency.

(From OE-Core rev: 68bdc219a4a819e83217f5b54c463624af8d3b9e)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-24 10:39:07 +00:00
Joshua Watt
72b6728f82 lib/packagedata.py: Fix broken symlinks for providers with a '/'
If a package had a provider with a '/' in it (e.g. "/bin/sh",
"/bin/bash", etc.), the generated symlinks were broken due to being at a
hard coded depth. Use oe.path.relsymlink() instead to make a correct
relative symbolic link

(From OE-Core rev: 8b1482a4e2adb7cf358d638265cf116b34078b84)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-23 08:46:00 +00:00
Joshua Watt
a22f7d7b53 lib/oe/path.py: Add relsymlink()
Adds API to make a relative symbolic link between two directories. The
arguments are the same as oe.path.symlink()

(From OE-Core rev: 3eeec7f3412e881e51763ef947c82772d3858f09)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-23 08:46:00 +00:00
Joshua Watt
df548e981d ipk: Switch to using zstd compression
Converts IPK package generation to use zstd instead of xz. zstd has a
much larger compression/speed tradeoff range allowing users to choose
what suits them best, and fast decompression speeds. It also continues
to support parallel compression as xz did.

A new variable called ZSTD_DEFAULTS is provided to set the defaults for
places that want to use zstd for compression; the zst image conversion
command is also modified to use this.

Finally, in order for this to function properly, opkg must include zstd
support, so it is enabled all the time with no PACKAGECONFIG to turn it
off.

(From OE-Core rev: 1bc3e9bbaa670b6128c74c76b4b5264e60ce3463)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-23 08:46:00 +00:00
Alexander Kanavin
95c187e54e sstatesig/find_siginfo: special-case gcc-source when looking in sstate caches
This is already done for local stamps just above, and will allow enabling
the full selftest that compares gcc-source signatures via printdiff
(that is, both local stamp and sstate variants).

(From OE-Core rev: 29775b5ecfc8d811293962f050fcfc3b3ad7efde)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-09 19:17:11 +00:00
Peter Kjellerstedt
3ef9ea88f1 lib/oe/recipeutils: Avoid wrapping any SRC_URI[sha*sum] variables
Before, a variable such as SRC_URI[sha512sum] would end up as:

SRC_URI[sha512sum] = "45ff3abce4dab24a8090409e6d7bb26afa7fa7812a51e067 \
28c2aa47d5b4de610d97ba4609cf13d9173087bd909fdf377235eee988a6fdcf52abb7 \
0341c40b5b"

when updated by patch_recipe_lines().

(From OE-Core rev: a67e2feed1420739504d2a59d018dff7e6e17e04)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-08 16:58:34 +00:00
Etienne Cordonnier
959b1f7de4 gdb/systemd: enable minidebuginfo support conditionally
Enabling minidebuginfo is not useful if gdb and systemd-coredump
are unable to parse it.

In order to parse it, gdb needs xz support. Systemd needs coredump enabled, as
well as elfutil enabled as well (systemd-coredump loads libdw which is part of elfutils using dlopen).

(From OE-Core rev: 0d2df803bebfd7e832ab7da54c4dacaaeeb424a9)

Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:50 +00:00
Julien Stephan
0ae9cf2373 recipeutils: bbappend_recipe: allow to patch the recipe itself
Add a new parameter update_original_recipe to allow to patch a recipe
instead of creating/updating a bbappend

(From OE-Core rev: 2f68ab2464bfad1b377df44a7b51203df59d66ce)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
165626f7b9 recipeutils: bbappend_recipe: remove old srcuri entry if parameters are different
Currently we do not add a new src_ury entry if the entry already exists
AND the parameters are the same.

I believe that when an entry already exist with different parameters,
we should remove it and add the new entry otherwise we end up with two
entries with different parameters

(From OE-Core rev: a4628fffcfecb5cd95dc2558dfd39ebd71121eab)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
11d4d437d5 recipeutils: bbappend_recipe: add a way to specify the name of the file to add
bbappend_recipe can take a dict of source files to add to SRC_URI where
the key is the full path to the file to be added and the value is a dict

Add a new optionnal entry "newname" to specify the name of the newly added file

(From OE-Core rev: e7bc09e5c9d7a0f4f8f4eba40730b68857b00677)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
b45cab4e1c recipeutils: bbappend_recipe: fix docstring
Add missing patchdir key in docstring description for srcfiles
parameters.

Also fix typo: value --> key in srcfile docstring description

(From OE-Core rev: 302dccaef46418d0e0731870a882b54a525739ed)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
eb0b664c8c recipeutils: bbappend_recipe: fix undefined variable
In the case get_bbappend_path returns None (could not find the layer
containing the recipe) the error message tries to print the recipefile,
but it is not defined. Fix it.

(From OE-Core rev: 234111fb67ffbcc5492cb0cd96db25ed8f5acea0)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Yoann Congal
2c59f5ad01 lib/oe/patch: handle creating patches for CRLF sources
Using devtool to patch CRLF based sources creates patch files which have
mixed end of lines : LF for headers and CRLF for source context and
modified lines.

Python open(..., newline=None) (default for newline arg)does detect
end-of-line in this mixed file but only outputs LF EOL data. This
result in patch files that does not apply on the original sources.

Switching to open(..., newline='') allows to detect end-of-line but keep
the original end-of-line intact. This generate correct patches for CRLF
based sources.

Fixes [YOCTO #15285]

(From OE-Core rev: 58f845499c0277a2b8069eefa235430b5f5f7661)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
89f1662484 devtool: add support for git submodules
Adding the support of submodules required a lot of changes on the
internal data structures:
* initial_rev/startcommit used as a starting point for looking at new
  / updated commits was replaced by a dictionary where the keys are the
  submodule name ("." for main repo) and the values are the
  initial_rev/startcommit

* the extractPatches function now extracts patch for the main repo and
  for all submodules and stores them in a hierarchical way describing the
    submodule path

* store initial_rev/commit also for all submodules inside the recipe
  bbappend file

* _export_patches now returns dictionaries that contains the 'patchdir'
  parameter (if any). This parameter is used to add the correct
  'patchdir=' parameter on the recipe

Also, recipe can extract a secondary git tree inside the workdir.

By default, at the end of the do_patch function, there is a hook in
devtool that commits everything that was modified to have a clean
repository. It uses the command: "git add .; git commit ..."

The issue here is that, it adds the secondary git tree as a submodule
but in a wrong way. Doing "git add <git dir>" declares a submodule but do
not adds a url associated to it, and all following "git submodule foreach"
commands will fail.

So detect that a git tree was extracted inside S and correctly add it
using "git submodule add <url> <path>", so that it will be considered as a
regular git submodule

(From OE-Core rev: 900129cbdf25297a42ab5dbd02d1adbea405c935)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-01 11:48:25 +00:00
Julien Stephan
c7ed37b5ff lib/oe/recipeutils.py: remove trailing white-spaces
Remove useless trailing white-spaces

(From OE-Core rev: 4aa28d5ea74016087979afd1faea478fda253cc3)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-30 08:43:04 +00:00
Ross Burton
f841d7918d lib/oe/patch: ensure os.chdir restoring always happens
If we chdir(), do the chdir back to the original directory in a finally
block so they always run.

(From OE-Core rev: cdc40292818683b6df1c814498c7589450a163fa)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-22 14:08:48 +00:00
Richard Purdie
0769e8ef6b bitbake.conf: Drop oldincludedir
Autoconf defines this as:

"The directory for installing C header files for non-GCC compilers."

Whilst this is something autoconf does allow changing, I find it hard
to believe it has much use in the wild now and that headers don't get
split like this in reality, it would probably only be useful on really
old unixes.. The values are the same in our configuration anyway.

Drop the value and just use includedir everywhere.

(From OE-Core rev: 506c91cbc6a604a84e37e53ccff430436369802e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-20 15:30:52 +00:00
Richard Purdie
3d35d09ff2 sstate: Ensure sstate searches update file mtime
Commands like "bitbake XXX -S printdiff" search for sstate files but don't download
them. This means that local files aren't touched as the download code would do, meaning
the sstate cleanup scripts can delete them. This can then lead to obtuse build failures.

Have the search code touch local files in the same way as the main code paths would to
avoid these files disappearing.

Move the function to a common touch() function in lib/oe instead of duplicating code.

(From OE-Core rev: a27fc0bd5706ab5b9c68a0271fcf57377a678cdf)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-08 10:56:35 +00:00
Vyacheslav Yurkov
914a921c33 lib/oe/path: Deploy files can start only with a dot
There might be only hidden files deployed. In that case we don't need a
generic wildcard present in copy command, otherwise it fails with:

Exception: subprocess.CalledProcessError: Command 'cp -afl --preserve=xattr ./.??* ./* <BUILDDIR>/tmp/deploy/images/qemux86-64' returned non-zero exit status 1.

Subprocess output:
cp: cannot stat './*': No such file or directory

(From OE-Core rev: f92c751281609ea6bd6b838307de4bc70bf26ab9)

Signed-off-by: Vyacheslav Yurkov <Vyacheslav.Yurkov@wika.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-05 11:28:35 +00:00
Jermain Horsman
a9063f845b lib/oe/buildcfg.py: Remove unused parameter
Several functions included the 'd' parameter but never used it,
additionally the value passed is always None.

(From OE-Core rev: 9e03ce0426576ebef3739dc1dfec4f7cd73ae094)

Signed-off-by: Jermain Horsman <jermain.horsman@nedap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-05 11:28:33 +00:00
Jermain Horsman
d012de568f lib/oe/buildcfg.py: Include missing import
get_layer_revisions() uses the os.path module but it is not included
in the imports.

(From OE-Core rev: 4339ef5b46c39cce402d1cc3526b39702837b839)

Signed-off-by: Jermain Horsman <jermain.horsman@nedap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-05 11:28:32 +00:00
Alexander Kanavin
a92d76e4da meta/lib/oe/copy_buildsystem.py: do not derefence symlinks
This was added (I think) for the purpose of supporting layers that refer
to items outside of the layer via relative symlinks:
https://git.yoctoproject.org/poky-contrib/commit/?id=d31d1ad4e566e42d0bbcf1f41ac25e33181fb517

I do not think copying the link target into the layer that references it is the
correct solution: rather the original target should be included
into the SDK with the same relative path.

This change is done for the sake of preserving symlinks
that are referencing things inside the layer as they are;
particularly the content of scripts/esdk-tools/.

(From OE-Core rev: 52a7bbd5c4875c5f61ea65dda38e495a2925a20d)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-05 08:44:06 +00:00
Ross Burton
7ffa4d4044 cve-check: don't warn if a patch is remote
We don't make do_cve_check depend on do_unpack because that would be a
waste of time 99% of the time.  The compromise here is that we can't
scan remote patches for issues, but this isn't a problem so downgrade
the warning to a note.

Also move the check for CVEs in the filename before the local file check
so that even with remote patches, we still check for CVE references in
the name.

(From OE-Core rev: 0251cad677579f5b4dcc25fa2f8552c6040ac2cf)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-03 16:58:40 +00:00
Alexander Kanavin
2cf13f2513 lib/oe/sstatesig.py: dump locked.sigs.inc only when explicitly asked via -S lockedsigs
This was writing out locked-sigs.inc into cwd with every
'bitbake -S' invocation. When the intent is only to to get task
stamps (-S none), or print the difference between them (-S printdiff),
the file is unnecessary clutter.

A couple of selftests/scripts were however relying on this, so they're
adjusted to explicitly request the file.

eSDK code calls dump_lockedsigs() separately via
oe.copy_buildsystem.generate_locked_sigs() and so isn't affected.

(From OE-Core rev: ad57c3cac2a8d3e60222e3cca0685f582dcea135)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 10:53:43 +01:00
Ross Burton
c1e7eed4f7 cve-check: slightly more verbose warning when adding the same package twice
Occasionally the cve-check tool will warn that it is adding the same
package twice.  Knowing what this package is might be the first step
towards understanding where this message comes from.

(From OE-Core rev: c1179faec8583a8b7df192cf1cbf221f0e3001fc)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-26 15:29:34 +01:00
Jose Quaresma
f784681413 sstatesig: be more precise and show the full path in exceptions
Also remove the warning than doesn't make sense as the code will generate
an exception and bitbake will abort.

Before:
| WARNING: core-image-minimal-initramfs-1.0-r0 do_image_complete: KeyError in .
| Exception: Exception: KeyError: 'getpwuid(): uid not found: xxxxx'
| Path . is owned by uid xxxxx, gid yyy, which doesn't match any user/group on target. This may be due to host contamination.

After:
| Exception: Exception: KeyError: 'getpwuid(): uid not found: xxxxx'
| Path /build/tmp/work/intel_corei7_64-lmp-linux/core-image-minimal-initramfs/1.0-r0/sstate-build-image_complete is owned by uid xxxxx, gid yyy, which doesn't match any user/group on target. This may be due to host contamination.

(From OE-Core rev: 90f84e345950a02bf91f823cc9c6d893e7cd1100)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-19 13:38:56 +01:00
Joshua Watt
70eddc70f8 goarch: Move Go architecture mapping to a library
Other spaces uses the Go architecture definitions as their own (for
example, container arches are defined to be Go arches). To make it
easier for other places to use this mapping, move the code that does the
translation of OpenEmbedded arches to Go arches to a library.

(From OE-Core rev: 3e86f72fc2e1cc2e5ea4b4499722d736941167ce)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-19 13:26:05 +01:00
Julien Stephan
043ca5f64d sstatesig: add a new info level for SIGGEN_LOCKEDSIGS_TASKSIG_CHECK
as of now, SIGGEN_LOCKEDSIGS_TASKSIG_CHECK can take 2 values: "warn" and
"error", displaying respectively a warning or a fatal error message
only when a task is locked and the task signature is different from
the locked one.

The "info" level is introduced to add a "note" message to remind the
user that a recipe is locked even if the signature is equivalent to the
locked one.

The "warn" and "error" level display the warn/error message for each
task having a mismatch of the signature. Doing this with the "info"
level would result in very verbose output if there are several tasks
locked, so the info level will only print once the list of recipes that
have locked signature.

(From OE-Core rev: 840402181d36ca3f60119984478979afb5bb3bbf)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-09 15:58:47 +01:00
Richard Purdie
374fd90865 lib: Import packagedata oe module by default
Variable dependencies for functions in the oe.packagedata module were
missing as it was not present in BBIMPORTS. Add it as the fact it was
missing is likely just historical oversight from base.bbclass history
and the dependencies are useful.

Add an exclusion to bitbake.conf to ensure BB_NUMBER_THREADS doesn't
change task checksums.

(From OE-Core rev: 7df9178766ee7939d139648e04f5747b0ffe1e13)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-26 10:25:42 +01:00
Michael Opdenacker
00ae1a5ab9 lib/oe/qa: update guidelines link for Upstream-Status
(From OE-Core rev: a8781a8f02c924d8a19e727e5d1bc6b89bf976da)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-22 07:45:17 +01:00
Michael Opdenacker
79cd9301bd lib/oe/qa: remove obsolete "Accepted" string for Upstream-Status
Replaced by "Backport" since release 3.2
https://docs.yoctoproject.org/migration-guides/migration-3.2.html#miscellaneous-changes

(From OE-Core rev: 4053fa9184fd016cfa59a67ff7857e17d88bf25e)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-22 07:45:17 +01:00
Richard Purdie
b0b0b89ad7 create-spdx/sbom: Ensure files don't overlap between machines
Currently the by-id and by-namespace SPDX files are created without reference
to PACKAGE_ARCH. This means that for two machines using a common package architecture
(e.g. genericx86-64 and qqemux86-64), there would be overlapping files. This means
that the build of one can remove files from the other leading to build failures. An
example would be:

MACHINE=qemux86-64 bitbake core-image-minimal
MACHINE=genericx86-64 bitbake core-image-minimal
MACHINE=qemux86-64 bitbake linux-yocto -c clean
MACHINE=genericx86-64 bitbake core-image-minimal -C rootfs

To fix this, add PACKAGE_ARCH to the path used for the files and use a search
path based upon PACKAGE_ARCHS to access them.

(From OE-Core rev: b2db10e966438071d00d2057b84d5f347613d841)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-22 07:45:17 +01:00
Eilís 'pidge' Ní Fhlannagáin
9ccde59c53 lib/oe/package_managegment: Add nativesdk-intercept PATH
[YOCTO #15023]

This patch adds (and removes after function execution) the
nativesdk-intercept/chown|chgrp PATH before target_pm.run_intercepts
calls during populate_sdk builds.

This has been tested with cleanall builds and testsdk and fails on deb
due to an issue where $D${localstatedir}/cache/man/ does not exist for
some reason. I've a work around for that in the next patch in this
series.

(From OE-Core rev: e7afdfe9da150209ab2676d09eae040de2155c6d)

Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-17 19:37:19 +01:00
Adrian Freihofer
b31f99a6cb lib/oe/package: Refactor to make strip_execs callable without d
This allows to call strip_execs function from devtool without going
via tinfoil and a bitbake server process.

(From OE-Core rev: 3bde26d64a0c8c3ef8ffbcb398f2a268759321af)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-12 12:44:47 +01:00
Adrian Freihofer
40fdebda20 lib/oe/utils: Refactor to make multiprocess_launch callable without d
This is a preparation for making the strip_execs function callable from
devtool without going via tinfoil and a bitbake server process.

(From OE-Core rev: af8ee73cdef90b83556a7ac5e139a08108706486)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-12 12:44:47 +01:00
Richard Purdie
6fd8af0d30 classes: Drop ';' delimiter from ROOTFS/IMAGE*COMMAND variables
Originally these were shell functions but they have long since been processed by
bb.build.exec_func(). Since we no longer need shell syntax, we can drop the ';'
delimiters and just use a space separated string.

This cleans up the variable and quietly removes any stray ';' that do happen to
still make it in.

(From OE-Core rev: c3365dfd9ddd7fbe70b62e0f11166e57a8ca6f73)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-09 22:14:41 +01:00
Alexander Kanavin
d3e4f25993 lib/oe/recipeutils.py: accommodate SRCPV being optional and deprecated in version check regex
(From OE-Core rev: 84794b59940cb08ec44e5ee7718a464d24136812)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-30 09:56:12 +01:00
Julien Stephan
060f285e58 patch.py: use --absolute-git-dir instead of --show-toplevel to retrieve gitdir
[YOCTO #14141]

Currently the gitdir is manually constructed using `git
rev-parse --show-toplevel` and appending `.git`. This is most of the time
correct but not always: `.git` can be a file with the following content:

gitdir: <some_folder>

This is the case for submodules, so when using devtool modify on a recipe
using submodules *and* patching files inside one of the submodules, do_patch
fails with the following error:

ERROR: Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:patch_do_patch(d)
     0003:
File: '<..>/poky/meta/classes-global/patch.bbclass', lineno: 157, function: patch_do_patch
     0153:        except Exception as exc:
     0154:            bb.utils.remove(process_tmpdir, True)
     0155:            bb.fatal("Importing patch '%s' with striplevel '%s'\n%s" % (parm['patchname'], parm['striplevel'], repr(exc).replace("\\n", "\n")))
     0156:        try:
 *** 0157:            resolver.Resolve()
     0158:        except bb.BBHandledException as e:
     0159:            bb.utils.remove(process_tmpdir, True)
     0160:            bb.fatal("Applying patch '%s' on target directory '%s'\n%s" % (parm['patchname'], patchdir, repr(e).replace("\\n", "\n")))
     0161:
File: '<..>/poky/meta/lib/oe/patch.py', lineno: 769, function: Resolve
     0765:    def Resolve(self):
     0766:        olddir = os.path.abspath(os.curdir)
     0767:        os.chdir(self.patchset.dir)
     0768:        try:
 *** 0769:            self.patchset.Push()
     0770:        except Exception:
     0771:            import sys
     0772:            os.chdir(olddir)
     0773:            raise
File: '<..>/poky/meta/lib/oe/patch.py', lineno: 274, function: Push
     0270:            else:
     0271:                next = 0
     0272:
     0273:            bb.note("applying patch %s" % self.patches[next])
 *** 0274:            ret = self._applypatch(self.patches[next], force)
     0275:
     0276:            self._current = next
     0277:            return ret
     0278:
File: '<..>/poky/meta/lib/oe/patch.py', lineno: 556, function: _applypatch
     0552:        if os.path.lexists(hooks_dir_backup):
     0553:            raise Exception("Git hooks backup directory already exists: %s" % hooks_dir_backup)
     0554:        if os.path.lexists(hooks_dir):
     0555:            shutil.move(hooks_dir, hooks_dir_backup)
 *** 0556:        os.mkdir(hooks_dir)
     0557:        commithook = os.path.join(hooks_dir, 'commit-msg')
     0558:        applyhook = os.path.join(hooks_dir, 'applypatch-msg')
     0559:        with open(commithook, 'w') as f:
     0560:            # NOTE: the formatting here is significant; if you change it you'll also need to
Exception: NotADirectoryError: [Errno 20] Not a directory: '<..>/build/tmp/work/core2-64-poky-linux/vulkan-samples/git/devtooltmp-n87_zx1i/workdir/git/third_party/spdlog/.git/hooks'

Using `git rev-parse --absolute-git-dir` instead of `git rev-parse
--show-toplevel` ensure we get the correct gitdir

(From OE-Core rev: f74879dd95b19504ce8a8554636d2310d0336806)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-29 09:41:07 +01:00
Richard Purdie
65318019cd recipes/classes/scripts: Drop SRCPV usage in OE-Core
Now that SRCPV isn't needed we can simplify things in a few places...

(From OE-Core rev: 843f82a246a535c353e08072f252d1dc78217872)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-24 16:50:24 +01:00
Jasper Orschulko
92983dba65 cve_check: Fix cpe_id generation
Use "*" (wildcard) instead of "a" (application)in cpe_id generation,
as the product is not necessarily of type application, e.g.
linux_kernel, which is of type "o" (operating system).

(From OE-Core rev: cae9528b002c06143bf048b991b9d7e93968cb6b)

(From OE-Core rev: e7c1def3c3c3a72249802ef6fb64292277a7a53e)

Signed-off-by: Jasper Orschulko <jasper@fancydomain.eu>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-22 15:13:54 +01:00
Richard Purdie
d724ec6f04 lib/package_manager: Improve repo artefact filtering
If you run an arm build followed by an x86 one and then ask for a
full repo to be created, it will include all of the arm and x86 packages.
testexport will then find the arm socat package rather than the x86 one
and try and run arm binaries within an x86 qemu image with no success.

The reproducer for this was:

oe-selftest -r fitimage.FitImageTests.test_initramfs_bundle runtime_test.TestImage.test_testimage_install

This patch only symlinks in the compatible package archictures rather
than all of them which fixes the failure and the resulting autobuilder
intermittent failure too.

[YOCTO #15190]

(From OE-Core rev: 30b45bcf49bf8207fd96bb45a55d7708661f3359)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-16 07:54:38 +01:00
Michael Opdenacker
a26970fa3a recipes: remove unused AUTHOR variable
No longer used in generating packages
Also creates a possible confusion with the recipe maintainer
name.

(From OE-Core rev: 9d5edd124b7dddb995ceddd79f8a7fc8cf44badf)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-10 09:18:53 +01:00