Commit Graph

1236 Commits

Author SHA1 Message Date
Changqing Li
dcf9e34e51 patch.py: set commituser and commitemail for addNote
When PATCHTOOL is set to 'git', and user don't setup
user.name and user.email for git, do_patch fail with
the following error, fix by passing -c options.
CmdError("git notes --ref refs/notes/devtool append -m 'original patch: 0001-PATCH-increase-to-cpp17-version.patch' HEAD", 0, 'stdout:
stderr: Author identity unknown
*** Please tell me who you are.
Run
  git config --global user.email "you@example.com"
  git config --global user.name "Your Name"

(From OE-Core rev: 9de38ac99c2b19f549c00ea5277faf621c6f4e65)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-04-16 06:41:24 -07:00
Richard Purdie
de0a865f2d do_package/sstate/sstatesig: Change timestamp clamping to hash output only
The code was changing the timestamps of the files in the do_package output,
particularly the files added for debug sources. This was to do two things:

a) make do_package sstate more reproducible
b) ensure better hash equivalence matching

Unfortuately the debug source files are hardlinks into the source tree for
efficiency so touching these, touches a lot of files in ${B} and ${S}. This
causes unpredictable effects if compile is run again for example, or could
cause compiling in the install task.

The hash equivalence matching is of key importance but we can mimic that
using clamping of the file timestamps in the depsig output used to generate
the hashes.

This patch drops the global timestamp clamping, instead allowing the files
to retain their creation timestamps into sstate. This makes do_package sstate
slightly less reproducibile. We could clamp the sstate timestamps but that
would lead to two different sets of timestamps depending on whether the
data came from sstate or not. I'd prefer to have consistent code behaviour,
rather than differing behavhour depending on whether data came from sstate
or not.

If we wanted to have reproducibiliy and fix the "corruption" of S/B and have
consistent codepaths, the only other option would be two copies of the
sources, which could end up huge and seems the least desireable option.

This patch therefore drops the timestamp clamping in the sstate files
and tweaks the depsig data generation to clamp the timestamps for do_package
instead since this seems the best compromise.

I validated that rpm/deb/ipk files still generate correctly as before.

(From OE-Core rev: 9df0bf5775ec96773e77c489ba6a443037d65ab6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 475759fdab7200488b2a568b2ba1aa31a456d113)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-12-09 06:54:14 -08:00
Gassner, Tobias.ext
aad028a455 rootfs: Ensure run-postinsts is not uninstalled for read-only-rootfs-delayed-postinsts
This patch ensures that pkg_postinst_ontarget task is executed for read only rootfs when
read-only-rootfs-delayed-postinsts is set as IMAGE_FEATURES. The issue was that run-postinsts
could be uninstalled at the end of rootfs construction and that shouldn't happen for
the delayed usecase.

In addition to the fix, a test in meta/lib/oeqa/selftest/cases/overlayfs.py testing
the fix has been implemented.

(From OE-Core rev: e0c4f2f69723f64a29ba35c4de5fb1a7c79ef718)

Signed-off-by: Gassner, Tobias.ext <tobias.gassner.ext@karlstorz.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 60f587475dda99eaa07848880058b69286b8900e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-12-06 05:50:25 -08:00
Peter Kjellerstedt
200d12b6a5 image.bbclass: Drop support for ImageQAFailed exceptions in image_qa
After commit 905e224849fbbed1719e0add231b00e2d570b3b4 (image_qa: fix
error handling), any unexpected exceptions in do_image_qa() would result
in a variable being set, but never used, effectively hiding the error.

Since image_qa now calls oe.qa.exit_if_errors(), remove the support for
oe.utils.ImageQAFailed and instead rely on the called functions to call
oe.qa.handle_error() themselves. This matches what do_package_qa() does.

Also update the description of do_image_qa() to explain that the called
functions are expected to call oe.qa.handle_error() themselves.

[ YOCTO #15601 ]

(From OE-Core rev: a1b28a88bc7697371ab166b18587b615d6d39c8e)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0c3e111c965af2bc56533633c376b70b7fa5e1de)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-10-18 06:04:41 -07:00
Louis Rannou
2c09c72cf1 image_qa: fix error handling
Make ImageQAFailed inherit BBHandledException so exceptions raised in tests are
catched when the actual test function is executed by bb.utils.better_exec.

Change the do_image_qa tasks so errors are handled with oe.qa.handle_error. Add
some comment to explain this requires to list the test in ERROR_QA or WARN_QA.

[YOCTO #14807]
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14807

(From OE-Core rev: 8fe7aef17eefa70e3f7c07077b8c695e5c00ed5e)

Signed-off-by: Louis Rannou <louis.rannou@non.se.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 905e224849fbbed1719e0add231b00e2d570b3b4)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-10-18 06:04:41 -07:00
Claus Stovgaard
9d9325d07b lib/oe/package-manager: skip processing installed-pkgs with empty globs
We can skip processing the installed-pkgs file if globs is empty.
This is the case if self.d.getVar for IMAGE_INSTALL_COMPLEMENTARY
returns an empty string. If globs is an empty string the result from
processing with empty glob in oe-pkgdata-util will always be 0 packages
to install.

Instead of return early on this we just skip and still generate the
locale archive if needed.

(From OE-Core rev: f944a1be484378c733d7fd835dc17e210f1c2705)

Signed-off-by: Claus Stovgaard <claus.stovgaard@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 160c45c83d5addf01e4834cf896af871bd6fca7f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-10-18 06:04:40 -07:00
Mark Hatle
0d017528d4 package.py: Fix static library processing
When PACKAGE_STRIP_STATIC is enabled the system did not pay attention to
hardlinks.  This could trigger a race condition during stripping of static
libraries where multiple strips (through hardlinks) could run at the same
time triggering a truncated or modified file error.

The hardlink breaking code is based on the existing code for elf files, but
due to the nature of the symlinks needed to be done in a separate block of
code.

Add support for static-library debugfs hardlinking through the existing
inode processing code.

Print a note to the logs if the link target can't be found.  This isn't
strictly an error, but may be useful for debugging an issue where a file
isn't present.

(From OE-Core rev: c2809691992dab48a360c9516d205ec031378cda)

Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ff371d69f60a1529ed456acb7d8e9305242e74bd)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-08-06 19:11:18 -07:00
Mark Hatle
648d2617ba package.py: Fix static debuginfo split
Fix:
  NameError: name 'shutil' is not defined

(From OE-Core rev: a9d8e3f12032c92fa8dd7f2ad40f618da5ee281a)

Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 13bdd750ae54d57a5f459e4b7d8636c864978241)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-08-06 19:11:18 -07:00
Philip Lorenz
d37ef8925a ipk: Fix clean up of extracted IPK payload
It turns out that the IPK payload tarball was actually cleaned up in the
concrete package manager implementation (most likely because at some
point Debian and IPK packages used different compression algorithms).

Globbing removes this ambiguity so move the removal of the payload into
the common extract method.

(From OE-Core rev: ec1c6bc79f5ca219e77a379bbabc2dad808d2020)

Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1e2b02a54f482159e21902eeb997b21e00e9588e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-06-05 05:57:12 -07:00
Philip Lorenz
fb4eb8a09d lib/package_manager/ipk: Do not hardcode payload compression algorithm
The chosen payload compression algorithm can be changed by overriding
`OPKGBUILDCMD`. Ensure that package extraction deals with this by
globbing for "data.tar.*" to select the actual payload tarball.

(From OE-Core rev: c2b00cd4148d4b08ba5673488a652f8daf3ac085)

Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2ad05635a6da403b4fadcc126fe7734067c12c73)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-06-05 05:57:12 -07:00
Ross Burton
0fe72b1a67 lib/oe/package-manager: allow including self in create_packages_dir
This function is typically used to construct a limited feed for image
creation, but there are other cases when you might want a limited feed
and include the current recipe's packages in it.

To ensure that existing behaviour is preserved, add a boolean to control
this behaviour and default it to False.

(From OE-Core rev: 20a6f55328733ad6f0c05b1353e8d525019aeea7)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit aada7fda2b118152d82b1ab295d92b8251afe4ac)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-05-23 08:55:01 -07:00
Jose Quaresma
110ee701b3 sstatesig: fix netrc.NetrcParseError exception
Looks like sometimes the e.filename and the e.lineno is not properly set by the netrc
and this can cause TypeError.

| File "/poky/meta/lib/oe/sstatesig.py", line 342, in init_rundepcheck
|     bb.warn("Error parsing %s:%d: %s" % (e.filename, e.lineno, e.msg))
|             ~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| TypeError: %d format: a real number is required, not NoneType

(From OE-Core rev: 486ed69939f1b7572f121960c2dde246032686e7)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-04 21:50:24 +01:00
lixiaoyong
66f2f710e6 oe/package: enhance objdump command call with llvm
Replace `${HOST_PREFIX}objdump` with `${OBJDUMP}`.

When utilizing llvm for compiling packages, the invocation of GNU objdump
will consistently occur during package precess. This behavior is unfriendly
to llvm. So prefer `${OBJDUMP}` over `${HOST_PREFIX}objdump`.

(From OE-Core rev: fac4d33041d8feb041e617b9b16689c4d3a522cb)

Signed-off-by: lixiaoyong <lixiaoyong19@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-30 22:25:43 +00:00
Jermain Horsman
9b839a835c bblayers/makesetup.py: Move git utility functions to oe.buildcfg module
This allows other classes to make use of these as well.

Includes a git describe and git toplevel function and functions
to get info for git remotes.

(From OE-Core rev: a04a084b6e513d15cb57ee103c6d6215ce1c75b9)

Signed-off-by: Jermain Horsman <jermain.horsman@nedap.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-30 22:25:43 +00:00
Joshua Watt
df60c6d3ee sstatesig: Warn on bad .netrc
If there is an error parsing .netrc, warn the user

(From OE-Core rev: 47ea9418dfa31a87bc0014c482be5bfdc9df98e1)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-26 17:10:51 +00:00
Joshua Watt
84cb5d2568 sstatesig: Set hash server credentials from bitbake variables
Allows the hash server credentials to be specified in bitbake variables.
If omitted, the users .netrc will be checked

(From OE-Core rev: ba391d39f2b888706e53028e9df3a37c5baedfc1)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-23 10:18:20 +00:00
Jonathan GUILLOT
ca25926d07 lib/oe/package: fix LOCALE_PATHS scan to create locale packages
split_locales() must only check subdirectories in paths added to
LOCALE_PATHS to avoid creating weird packages based on filenames also
present in paths.

Without such a filter, cups recipe adding ${datadir}/cups/templates to
LOCALE_PATHS creates the following incorrect packages:
- cups-locale-add-class.tmpl
- cups-locale-add-printer.tmpl
- cups-locale-admin.tmpl

(From OE-Core rev: ba3aee0d516bd066829d6edaa8d7bacdd75dd6ef)

Signed-off-by: Jonathan GUILLOT <jonathan@joggee.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-24 16:10:22 +00:00
Peter Kjellerstedt
4cfd0f7e4e lib/oe/patch: Use git notes to store the filenames for the patches
The old way of keeping track of the filenames for the patches that
correspond to the commits was to add a special comment line to the end
of the commit message, e.g., "%% original patch: <filename>", using a
temporary git hook. This method had some drawbacks, e.g.:

* It caused problems if one wanted to push the commits upstream as the
  comment line had to be manually removed.
* The comment line would end up in patches if someone used git
  format-path rather than devtool finish to generate the patches.
* The comment line could interfere with global Git hooks used to
  validate the format of the Git commit message.
* When regenerating patches with `devtool finish --force-patch-refresh`,
  the process typically resulted in adding empty lines to the end of the
  commit messages in the updated patches.

A better way of keeping track of the patch filenames is to use Git
notes. This way the commit messages remain unaffected, but the
information is still shown when, e.g., doing `git log`. A special Git
notes space, refs/notes/devtool, is used to not intefere with the
default Git notes. It is configured to be shown in, e.g., `git log` and
to survive rewrites (i.e., `git commit --amend` and `git rebase`).

Since there is no longer any need for a temporary Git hook, the code
that manipulated the .git/hooks directory has also been removed. To
avoid potential problems due to global Git hooks, --no-verify was added
to the `git commit` command.

To not cause troubles for those who have done `devtool modify` for a
recipe with the old solution and then do `devtool finish` with the new
solution, the code will fall back to look for the old strings in the
commit message if no Git note can be found.

While not technically motivated like above, the way to keep track of
ignored commits is also changed to use Git notes to avoid having
different methods to store similar information.

(From OE-Core rev: f5e6183b9557477bef74024a587de0bfcc2b7c0d)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-19 16:03:22 +00:00
Peter Kjellerstedt
ff63bc403d lib/oe/patch: Add GitApplyTree.commitIgnored()
This function can be used to create a commit that devtool will ignore
when creating/updating the patches.

(From OE-Core rev: 94f0838b9223b7ece7affaa707e54a5d784da25e)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-19 16:03:22 +00:00
Peter Kjellerstedt
24433ce8f9 lib/oe/patch: Make extractPatches() not extract ignored commits
If a commit is marked with "%% ignore" it means it is used by devtool to
keep track of changes to the source code that are not the result of
running do_patch(). These changes need to actually be ignored when
extracting the patches as they typically make no sense as actual patches
in a recipe.

This also adds a new test for oe-selftest that verifies that there are
no patches generated from ignored commits.

(From OE-Core rev: c3d43de7e54189bf09fbe8e87ddb976e42ebf531)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-19 16:03:22 +00:00
Joshua Watt
ba68f3132d sstatesig: Implement new siggen API
Implements the new API required for querying unihashes in parallel

(From OE-Core rev: 0c66d07238d20dc1b34355e4c472aa7457d2e5fe)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-19 11:58:12 +00:00
Jonathan GUILLOT
c2a7c008e1 lib/oe/package: add LOCALE_PATHS to add define all locations for locales
Some packages may contain localized files not located in default path
${datadir}/locale. Add the new variable LOCALE_PATHS to allow a recipe
to define extra paths or even fully override the scanned directories.
LOCALE_PATHS is set at ${datadir}/locale by default to keep the exact
same behavior for the recipes which did not need modification.

(From OE-Core rev: 0ffc7cf01225743789ac30dd325fca05b9203be1)

Signed-off-by: Jonathan GUILLOT <jonathan@joggee.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-14 13:53:36 +00:00
Jonathan GUILLOT
c381ca7243 lib/oe/package: replace in place PN-locale-* packages in PACKAGES
split_locales() removes PN-locale from PACKAGES and adds PN-locale-* to the end.
As the PN-locale package typically appears before PN base package, it may result
in paths not installed in PN-locale-* packages if already catched by PN. Now
insert PN-locale-* exactly where PN-locale was existing in list to avoid such
an issue.

(From OE-Core rev: 108bc167ed0d43505af3140947a0ab79c89f0a7b)

Signed-off-by: Jonathan GUILLOT <jonathan@joggee.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-14 13:53:36 +00:00
Philip Lorenz
87798f82db ipk: Remove temporary package lists during SDK creation
The temporary package lists used during SDK creation should not be
shipped as part of the SDK (in particular because due to the opkg local
file download optimization they are actually symlinks into the build
directory). Remove them by calling the respective helper method during
the SDK build.

(From OE-Core rev: c18ba66da3c77f247170efd2cb350686010bef57)

Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-13 13:51:41 +00:00
Martin Jansa
77f5f99fcf package_manager: ipk: add OPKG_MAKE_INDEX_EXTRA_PARAMS variable
* can be used to pass e.g. -f param to preserve user-defined fields
  in the index as added in:
  https://git.yoctoproject.org/opkg-utils/commit/opkg-make-index?id=13f6281d24e17199e0fef6c2984419372ea0f86f

* otherwise it will show a lot of messages like:
  "Lost field Author <value>"
  for every package in the feed

(From OE-Core rev: 6dc772fd9d5c2d90a6e32cfa4bc46ca0221f1b7f)

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-08 10:53:13 +00:00
Ross Burton
bfaea5f7ec cve_check: cleanup logging
Primarily list the number of patches found, useful when debugging.

Also clean up some bad escaping that caused warnings and use
re.IGNORECASE instead of manually doing case-insenstive rang matches.

(From OE-Core rev: 10acc75b7f3387b968bacd51aade6a8dc11a463f)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-23 11:53:41 +00:00
Ross Burton
ab375ea3fe cve_check: handle CVE_STATUS being set to the empty string
Handle CVE_STATUS[...] being set to an empty string just as if it was
not set at all.

This is needed for evaluated CVE_STATUS values to work, i.e. when
setting not-applicable-config if a PACKAGECONFIG is disabled.

(From OE-Core rev: 2c9f20f746251505d9d09262600199ffa87731a2)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-23 11:53:41 +00:00
Richard Purdie
2ebe7f60fd reproducible: Fix race with externalsrc/devtool over lockfile
We occasionally see races over the lockfile used by externalsrc/devtool
when walking files for the source_date_epock calculation. Skip this file
if present to avoid the issues and fix a real issue where SDE could be
contaminated too.

[YOCTO #14921]

(From OE-Core rev: 4bc0eb4bd90e6e6e46581a8ed367212bdd910a26)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-19 12:21:22 +00:00
Richard Purdie
bc883e4991 rootfs-postcommands: Try and improve ordering constraints
The current code is in race to see who can set things last. This isn't scalable or
sustainable and problemtic in the face of inherit ordering changes.

Move the ordering issue into the actual code execution, which isn't ideal but
the best of several bad options and at least lets us drop the anonymous python.

(From OE-Core rev: 0ffff2c1f80a9b79b133d787764bab164d9abd70)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-18 10:15:58 +00:00
Richard Purdie
12619deabd rootfs: Fix MULTILIB_RE_ALLOW to be inherit order independent
This variable is only used by the ipk backend with multilibs.

In order to make it work correctly regardless of inherit order, change
the string to be space delimeted, set using += and add in the regex '|'
sperator at the end of processing.

(From OE-Core rev: 72befdb12568fbc642022ef0a23b269c5b37a638)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-18 10:15:58 +00:00
Etienne Cordonnier
5031cf42ff package.py: fix Darwin support
- 'subprocess.Popen([d.expand("${HOST_PREFIX}otool)' requires text-mode (a more
  readable alias for the universal_newlines parameter), since otool produces
  text and the code 'out.split("\n")' expects a string, not a bytes object.
  otool is used on MacOS only, so this error isn't triggered on Linux.

- use 'startswith("darwin")' in order to support all darwin versions and not
  just specific versions (meta-darwin supports darwin21 at the moment).

(From OE-Core rev: 248ca79a6400e063c4965f9542c614bf837ff758)

Signed-off-by: Dominik Schnitzer <dominik@snap.com>
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-15 21:42:15 +00:00
Yang Xu
b11e3aa995 rootfs.py: check depmodwrapper execution result
The execution result of depmodwrapper is not checked which makes depmod
generation failed sliently and hard to detect.

So check exection result and stop building if depmodwrapper failed.

(From OE-Core rev: 2f88e7d331390c6aaecc4522253e24791aec299e)

Signed-off-by: Yang Xu <yang.xu@mediatek.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 17:01:28 +00:00
Alexander Kanavin
4675bbb757 lib/sstatesig/find_siginfo: raise an error instead of returning None when obtaining mtime
Suppressing the error and returning None can result in a delayed failure:
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/6254/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/6262/steps/14/logs/stdio

It is not clear why the os.stat() error occurs to begin with (it shouldn't), so rather than
adding further workarounds, let's get diagnostics at the source first, so we understand
what is going on.

(From OE-Core rev: 35483d4756ab53805507f72a9a0edb3f83759694)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-05 12:04:52 +00:00
Richard Purdie
a7aa37f2fd sstatesig: Add version information for find_sigingfo
Since we're changing the return values of the function, add a version so
bitbake can ensure it is using a compatible function.

(From OE-Core rev: 738bf3717002d124958a21a6925b0eca14fef131)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-05 11:59:08 +00:00
Alexander Kanavin
c45ffa9e94 sstatesig/find_siginfo: unify a disjointed API
find_siginfo() returns two different data structures depending
on whether its third argument (list of hashes to find) is empty or
not:
- a dict of timestamps keyed by path
- a dict of paths keyed by hash

This is not a good API design; it's much better to return
a dict of dicts that include both timestamp and path, keyed by
hash. Then the API consumer can decide how they want to use these
fields, particularly for additional diagnostics or informational
output.

I also took the opportunity to add a binary field that
tells if the match came from sstate or local stamps dir, which
will help prioritize local stamps when looking up most
recent task signatures.

(From OE-Core rev: 8721c52041e910bd4d8a9235b52f274f4f02c8a3)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-05 11:59:08 +00:00
Richard Purdie
52f41ff243 lib/prservice: Improve lock handling robustness
It is possible we could crash holding a lock whilst parsing in this code.
Switch to use utils.fileslocked() in the with expression to avoid this.

This may be causing some of our strange intermittent failures in PRServ
tests.

(From OE-Core rev: 4e59db15e5df2cc3d0ae042454812a2d54cef77b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-24 10:39:07 +00:00
Joshua Watt
dd5475d8e3 lib/packagedata.py: Add API to iterate over rprovides
Adds an API that makes it easier to iterate over the package data for a
all providers of a runtime dependency.

(From OE-Core rev: 68bdc219a4a819e83217f5b54c463624af8d3b9e)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-24 10:39:07 +00:00
Joshua Watt
72b6728f82 lib/packagedata.py: Fix broken symlinks for providers with a '/'
If a package had a provider with a '/' in it (e.g. "/bin/sh",
"/bin/bash", etc.), the generated symlinks were broken due to being at a
hard coded depth. Use oe.path.relsymlink() instead to make a correct
relative symbolic link

(From OE-Core rev: 8b1482a4e2adb7cf358d638265cf116b34078b84)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-23 08:46:00 +00:00
Joshua Watt
a22f7d7b53 lib/oe/path.py: Add relsymlink()
Adds API to make a relative symbolic link between two directories. The
arguments are the same as oe.path.symlink()

(From OE-Core rev: 3eeec7f3412e881e51763ef947c82772d3858f09)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-23 08:46:00 +00:00
Joshua Watt
df548e981d ipk: Switch to using zstd compression
Converts IPK package generation to use zstd instead of xz. zstd has a
much larger compression/speed tradeoff range allowing users to choose
what suits them best, and fast decompression speeds. It also continues
to support parallel compression as xz did.

A new variable called ZSTD_DEFAULTS is provided to set the defaults for
places that want to use zstd for compression; the zst image conversion
command is also modified to use this.

Finally, in order for this to function properly, opkg must include zstd
support, so it is enabled all the time with no PACKAGECONFIG to turn it
off.

(From OE-Core rev: 1bc3e9bbaa670b6128c74c76b4b5264e60ce3463)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-23 08:46:00 +00:00
Alexander Kanavin
95c187e54e sstatesig/find_siginfo: special-case gcc-source when looking in sstate caches
This is already done for local stamps just above, and will allow enabling
the full selftest that compares gcc-source signatures via printdiff
(that is, both local stamp and sstate variants).

(From OE-Core rev: 29775b5ecfc8d811293962f050fcfc3b3ad7efde)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-09 19:17:11 +00:00
Peter Kjellerstedt
3ef9ea88f1 lib/oe/recipeutils: Avoid wrapping any SRC_URI[sha*sum] variables
Before, a variable such as SRC_URI[sha512sum] would end up as:

SRC_URI[sha512sum] = "45ff3abce4dab24a8090409e6d7bb26afa7fa7812a51e067 \
28c2aa47d5b4de610d97ba4609cf13d9173087bd909fdf377235eee988a6fdcf52abb7 \
0341c40b5b"

when updated by patch_recipe_lines().

(From OE-Core rev: a67e2feed1420739504d2a59d018dff7e6e17e04)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-08 16:58:34 +00:00
Etienne Cordonnier
959b1f7de4 gdb/systemd: enable minidebuginfo support conditionally
Enabling minidebuginfo is not useful if gdb and systemd-coredump
are unable to parse it.

In order to parse it, gdb needs xz support. Systemd needs coredump enabled, as
well as elfutil enabled as well (systemd-coredump loads libdw which is part of elfutils using dlopen).

(From OE-Core rev: 0d2df803bebfd7e832ab7da54c4dacaaeeb424a9)

Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:50 +00:00
Julien Stephan
0ae9cf2373 recipeutils: bbappend_recipe: allow to patch the recipe itself
Add a new parameter update_original_recipe to allow to patch a recipe
instead of creating/updating a bbappend

(From OE-Core rev: 2f68ab2464bfad1b377df44a7b51203df59d66ce)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
165626f7b9 recipeutils: bbappend_recipe: remove old srcuri entry if parameters are different
Currently we do not add a new src_ury entry if the entry already exists
AND the parameters are the same.

I believe that when an entry already exist with different parameters,
we should remove it and add the new entry otherwise we end up with two
entries with different parameters

(From OE-Core rev: a4628fffcfecb5cd95dc2558dfd39ebd71121eab)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
11d4d437d5 recipeutils: bbappend_recipe: add a way to specify the name of the file to add
bbappend_recipe can take a dict of source files to add to SRC_URI where
the key is the full path to the file to be added and the value is a dict

Add a new optionnal entry "newname" to specify the name of the newly added file

(From OE-Core rev: e7bc09e5c9d7a0f4f8f4eba40730b68857b00677)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
b45cab4e1c recipeutils: bbappend_recipe: fix docstring
Add missing patchdir key in docstring description for srcfiles
parameters.

Also fix typo: value --> key in srcfile docstring description

(From OE-Core rev: 302dccaef46418d0e0731870a882b54a525739ed)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
eb0b664c8c recipeutils: bbappend_recipe: fix undefined variable
In the case get_bbappend_path returns None (could not find the layer
containing the recipe) the error message tries to print the recipefile,
but it is not defined. Fix it.

(From OE-Core rev: 234111fb67ffbcc5492cb0cd96db25ed8f5acea0)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Yoann Congal
2c59f5ad01 lib/oe/patch: handle creating patches for CRLF sources
Using devtool to patch CRLF based sources creates patch files which have
mixed end of lines : LF for headers and CRLF for source context and
modified lines.

Python open(..., newline=None) (default for newline arg)does detect
end-of-line in this mixed file but only outputs LF EOL data. This
result in patch files that does not apply on the original sources.

Switching to open(..., newline='') allows to detect end-of-line but keep
the original end-of-line intact. This generate correct patches for CRLF
based sources.

Fixes [YOCTO #15285]

(From OE-Core rev: 58f845499c0277a2b8069eefa235430b5f5f7661)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
89f1662484 devtool: add support for git submodules
Adding the support of submodules required a lot of changes on the
internal data structures:
* initial_rev/startcommit used as a starting point for looking at new
  / updated commits was replaced by a dictionary where the keys are the
  submodule name ("." for main repo) and the values are the
  initial_rev/startcommit

* the extractPatches function now extracts patch for the main repo and
  for all submodules and stores them in a hierarchical way describing the
    submodule path

* store initial_rev/commit also for all submodules inside the recipe
  bbappend file

* _export_patches now returns dictionaries that contains the 'patchdir'
  parameter (if any). This parameter is used to add the correct
  'patchdir=' parameter on the recipe

Also, recipe can extract a secondary git tree inside the workdir.

By default, at the end of the do_patch function, there is a hook in
devtool that commits everything that was modified to have a clean
repository. It uses the command: "git add .; git commit ..."

The issue here is that, it adds the secondary git tree as a submodule
but in a wrong way. Doing "git add <git dir>" declares a submodule but do
not adds a url associated to it, and all following "git submodule foreach"
commands will fail.

So detect that a git tree was extracted inside S and correctly add it
using "git submodule add <url> <path>", so that it will be considered as a
regular git submodule

(From OE-Core rev: 900129cbdf25297a42ab5dbd02d1adbea405c935)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-01 11:48:25 +00:00