The pipes library is deprecated in Python 3.11 and will be removed in
Python 3.13. pipes.quote is just an import of shlex.quote anyway.
(From OE-Core rev: d167661bceebebafb04dca3bf6a888003f46e6c9)
Signed-off-by: Ernst Persson <ernst.persson@non.se.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Reproducible builds are no longer a configuration option but are required.
We also rely on the prefix mapping capability of the compilers now.
As such, rewrite the source locating code to use the prefix maps instead
of taking a guess about WORKDIR which isn't correct for kernels, gcc,
externalsrc and probably more.
Instead, iterate the maps to locate any matching source code, keeping
in mind that multiple maps may map to one target location.
(From OE-Core rev: 80289f49d0c5ca98da1d1558728b8a468aab4326)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit cbd6144a9769d21371ae0fe04db2adc05f6eed02)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Currently, the latest version abseil-cpp contains a new library named "absl_log_internal_format", it's
basic package config(.pc file) as below:
prefix=/usr
exec_prefix=${prefix}
......
Requires: absl_config = 20230125, absl_core_headers = 20230125, absl_log_internal_append_truncated = 20230125,
absl_log_internal_config = 20230125, absl_log_internal_globals = 20230125, absl_log_severity = 20230125,
absl_strings = 20230125, absl_str_format = 20230125, absl_time = 20230125, absl_span = 20230125
......
Normally, the process_pkgconfig() would process variable data before field data in a .pc file, but in the
absl_log_internal_format, the field data in "Requires" section contains "xxxx = xxxx" format, the
process_pkgconfig() treats them as normal variable and using the setVar() in bitbake's data_smart.py
try to process. The absl_log_internal_format field data contains "_append_", this hit the setVar() checking
and finally bitbake stop building and reporting an error as below:
"Variable xxx contains an operation using the old override syntax. Please convert this layer/metadata before attempting to use with a newer bitbake."
This patch move the field data process before variable process to avoid the process_pkgconfig() treat the field
data as variable.
(From OE-Core rev: e7d3e02a624f7ce23d012bb11ad1df2049066b37)
Signed-off-by: Xiangyu Chen <xiangyu.chen@windriver.com>
(cherry picked from commit a73e269d3e591a10bb397b94b82e3fb960112d33)
Signed-off-by: Clément Péron <peron.clem@gmail.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Newly introduced kirkstone-only commit
https://git.openembedded.org/openembedded-core/commit/?h=kirkstone&id=80839835ec9fcb63069289225a3c1af257ffdef7
broke builds with externalsrc in Gitlab-CI.
This is yocto-4.0.9 regression.
It checks if directory starts with "build" instead of
if checking if it equals to "build".
Gitlab-CI uses directory "/builds" which matches the check
but directory /build does not exist, only /builds.
After successful check it tries to move this non-existent
directory which does not exists and thus do_package fails.
(From OE-Core rev: b67e714b367a08fdeeeff68c2d9495ec9bc07304)
Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
This is needed when the SDK or eSDK is installed in a /build top level
directory as it conflicts with the build directory within the existing
/usr/src/debug/build (which is really a link). Rename it and then do the
copy, this is not an issue with master currently due to some other
changes that occurred in master.
Fixes: [YOCTO #15026]
(From OE-Core rev: 80839835ec9fcb63069289225a3c1af257ffdef7)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Newer versions of binutils (2.38+) have changed how the
"--only-keep-debug" of objcopy behaves when stripping non-debug sections
from an ELF.
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=68f543154e92ab0f5d6c569e0fa143f5e8bd2d80
This change causes associated sections to be correctly marked as NOBITS
with the section contents removed from the output. The side effect is
that this causes issues with objcopy's ability to perform symbol and
relocation stripping (-S/--strip-all) on the debug split ELF, such that
with some object files (e.g. kernel modules) objcopy fails to strip
symbols/relocations with an error like the following:
.../.debug/nls_cp950.ko[.rodata]: file truncated
Because of this it is now problematic to generate minidebuginfo for
these types of ELF objects. However it is not typically useful to inject
minidebuginfo into these types of ELFs, and other distributions (e.g.
Fedora, referring to find-debuginfo.sh of debugedit) only insert
minidebuginfo into executables and shared libraries.
This change causes the minidebuginfo injection to only apply to EXEC/DYN
type ELFs, which limits the injection to executables and shared
libraires.
Additionally this change fixes the parsing of the sections from the
"readelf -W -S" output which was not accounting for the section index
column having leading spaces for single digit index values e.g. "[ 1]".
(From OE-Core rev: 9485559d269ed11bfcc90399c9282549ced35ce0)
Signed-off-by: Nathan Rossi <nathan.rossi@digi.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 2084cfcb3d15db3e02637f1cd63ab9c997f38a65)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previous commit c725bdb29b broke kernel source handling, this was due
to the code expecting the sources to be in a different directory, this did
not happen when using externalsrc since sources were found in the expected
directories.
Pass work-shared to the check to allow sources to be found in the proper
directory, allowing these to be packaged in the next step.
To test this we grabbed a commit where we knew the buildpaths
QA test should flag a file inside the kernel sources, with the previous
commit the QA warning wasnt flagged since no sources where there, with
this fix the buildpaths QA warning gets flagged properly.
(From OE-Core rev: 8efd5e31670235f7c59af2a5ee14646f029f4d18)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2714a8ef8c7b3c66d50f27f4f52fe2fe4db39b00)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since commit d756b346f2 kernel modules are stripped by the functions 'runstrip' and 'splitdebuginfo'. Signed modules must not be stripped. Function 'runstrip' avoids this by running is_kernel_module_signed. Apply the same check to splitdebuginfo.
(From OE-Core rev: 6859226652339b19cbc7bdfec074fe2016cdee60)
(From OE-Core rev: cefc6ec245ac1ca7a1a47d55aaec793e730a24a5)
Signed-off-by: Christoph Lauer <christoph.lauer@xtronic.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit dc0f0413eabfd50f78d887f73f808d40a314fbd8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
While executing do_package, bitbake checks for a list of
debug source files and uses a pattern to match the ones
to be included in copydebugsources.
Previously when externalsrc was in use either directly or by
using devtool, the source location changed and this pattern
no longer matched, hence debug source files failed to be
included in the corresponding package.
Check when the source directory isnt the default (based on
WORKDIR), and change the pattern used to match debug source
files if that is the case, allowing us to perform do_package
properly.
Workaround debugsource.list containing paths from the host by
moving debug source files away from the host directory
structure to avoid host contamination (this seems to happen
when packages use $TMPDIR/work-shared and externalsrc is
in use).
Test matrix included using:
- devtool to use externalsrc automatically
- externalsrc with a non-devtool based source directory
- No externalsrc at all
Tested the following packages to be working:
- glibc ($TMPDIR/work-shared based)
- libxcrypt ($TMPDIR/work based)
[YOCTO 8015]
(From OE-Core rev: 2772b77c0e3d0f6226a2d7c40a59900a6d7c942f)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a887bd96fd0a15398e8077ea79df5070971866e4)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the signatures for do_packagedata don't reflect whether PRServ
was active or not. This means that if you have mxiing of PRServ usage and
non PRServ usage against the same sstate cache it can rarely become
corrupted with one referencing the other.
This likely doesn't happen in general but does on the autobuilder as PRServ
is tested. Add in some variables to ensure the binary state of PRServ being
enabled or disabled is tracked (but not the server value). We continue to
assume one PRServ is used per sstate cache.
(From OE-Core rev: 4c2f429d6876c29b17931daa039c4899aacd7234)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit dd660e5c3fb74f7c4b7b8e863f7143066ae22813)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The files in /sysroot-only are intended to make it into the
recipes sysroot output, but not into the package. However, if
do_package is run before do_populate_sysroot, the files are
removed.
Use a smaller hammer to avoid copying the files into the package so
they are still around when do_populate_sysroot runs.
(From OE-Core rev: acc0a31305cb1f799e8a6b3ea2d2c5ac5a9f67ee)
Signed-off-by: Russ Dill <russ.dill@nikolamotor.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit 493145c6f1bc92ab2b7a23e181641b09df87c9ff)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no reason to include a path in foo[dirs] if it is also in
foo[cleandirs] (except if it is the last path in foo[dirs]).
(From OE-Core rev: 9d2dac56deda5afa4d77c5ddd7e1e8003f6d4725)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By renaming LICENSE_EXCLUSION to _exclude_incompatible, it makes it
clear that this is an internal variable.
(From OE-Core rev: 20a4cc2c2dcf345ef898abfe7735b7bc75ac0059)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This simplifies the APIs for splitdebuginfo(), splitstaticdebuginfo()
and inject_minidebuginfo()
(From OE-Core rev: 9b00cf049032a9ed7ba3c63d3c37c7414f94edbf)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It simplifies the API to return one dict instead of seven strings.
(From OE-Core rev: 610c790a94591da5c3a6b1db938cd85e2cbdbe5a)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The split_and_strip_files() function is big and hard to follow. This
takes a small step to improve that by splitting out the part that sets
up the variables used to implement the various debug splitting methods
specified by PACKAGE_DEBUG_SPLIT_STYLE.
(From OE-Core rev: e414c6ec8563f9c448f1421f33870a908a0353b4)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extend runstrip() to accept additional argument to enable
sharing it with the kernel do_strip() so that
KERNEL_IMAGE_STRIP_EXTRA_SECTIONS can be passed.
Since is_elf() understands kernel modules there is no need to keep a
seperate list for kernmodules or hardcode the values to runstrip.
(From OE-Core rev: e09a8fa931fe617afc05bd5e00dca5dd3fe386e8)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PACKAGE_SNAP_LIB_SYMLINKS renames libraries based on their SONAME so
that they can be found directly rather than going via symlinks that
would be created by ldconfig. For example, without
PACKAGE_SNAP_LIB_SYMLINKS in ${libdir} we have:
libharfbuzz.so.0 -> libharfbuzz.so.0.20600.4
libharfbuzz.so.0.20600.4
but with PACKAGE_SNAP_LIB_SYMLINKS="1" we have just:
libharfbuzz.so.0
Unfortunately, this renaming is done based on the SONAME which breaks
packages like mesa which install a single library with multiple hard
links:
-rwxr-xr-x root/root 13593488 2021-12-07 12:26 ./usr/lib/dri/i915_dri.so
-rwxr-xr-x root/root 13137328 2021-12-07 12:26 ./usr/lib/dri/i965_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/iris_dri.so link to ./usr/lib/dri/i915_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/kms_swrast_dri.so link to ./usr/lib/dri/i915_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/nouveau_vieux_dri.so link to ./usr/lib/dri/i965_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/r200_dri.so link to ./usr/lib/dri/i965_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/radeon_dri.so link to ./usr/lib/dri/i965_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/swrast_dri.so link to ./usr/lib/dri/i915_dri.so
The SONAME for i915_dri.so (and therefore all the other names that link
to the same file) is libgallium_dri.so. This means that
PACKAGE_SNAP_LIB_SYMLINKS causes do_package to successfully rename the
first name found to libgallium_dri.so. A similar thing happens to
i965_dri.so with its SONAME of libmesa_dri_drivers.so. The order is not
deterministic, so this means that although every build will be missing
one name, it's not always the same one.
-rwxr-xr-x root/root 13593488 2021-11-30 15:17 ./usr/lib/dri/i915_dri.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/kms_swrast_dri.so link to ./usr/lib/dri/i915_dri.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/libgallium_dri.so link to ./usr/lib/dri/i915_dri.so
-rwxr-xr-x root/root 13137328 2021-11-30 15:17 ./usr/lib/dri/libmesa_dri_drivers.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/nouveau_vieux_dri.so link to ./usr/lib/dri/libmesa_dri_drivers.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/r200_dri.so link to ./usr/lib/dri/libmesa_dri_drivers.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/radeon_dri.so link to ./usr/lib/dri/libmesa_dri_drivers.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/swrast_dri.so link to ./usr/lib/dri/i915_dri.so
This renaming means that the library cannot be found at runtime.
The simplest way to avoid this renaming is to only snap libraries that
would be processed by ldconfig.
(From OE-Core rev: 8571182ece1602ce8e030c98aef17cdc5718a037)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Phil Blundell <pb@pbcl.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We exclude allarch packagegroups from rebuilding when their dependencies change.
The reasoning is that we are just depending on a name so having these rebuild
lots is just pointless and inefficient. We also don't want them duplicated for
multiple machines for efficiency.
In general this works fine, as long as the package names don't change. That
is also rare but there is one corner case which does catch users out - debian
package renaming. When this does break, users question sstate and so on and
lose faith in the system even if this is a known choice we made.
This commit adds an error message if an allarch packagegroup depends on any
package which shows package renaming in action (through the PKG variable
being set).
If you run into this issue you either need to remove the dependency from the
packagegroup or mark the packagegroup as tune specific, i.e. set:
PACKAGE_ARCH = "${TUNE_PKGARCH}"
before the packagegroup inherit.
[YOCTO #7298]
(From OE-Core rev: 5bf3e447d2f5064495d83a8fad30229bcf1ecc9b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extract package_qa_write_error, package_qa_handle_error and
package_qa_add_message functions from insane.bbclass to lib/oe/qa.py and
drop the package_qa_ prefixes.
Update various bbclasses to use the new functions. No import is required
since base.bbclass puts oe.qa in OE_IMPORTS.
Stop requiring callers to manually track whether a fatal error has been
encountered via a "sane" flag. Instead replace the QA_SANE variable with
QA_ERRORS_FOUND and call oe.qa.exit_if_errors or
oe.qa.exit_with_message_if_errors at the end of each task.
Inspired by discussion resulting from
https://lists.openembedded.org/g/openembedded-core/message/156793 and
https://lists.openembedded.org/g/openembedded-core/message/156900
(From OE-Core rev: f0ad152ef4cc15c042bc9eeefb6af096d054b220)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the pkgconfig pclist files contain full paths which are build
host specific and the order of entries is not deterministic.
Fix both these issues so the files are deterministic.
(From OE-Core rev: e422e29bca4af3ab4073e04490f38b05cd7c38c0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
pkgdata output isn't entirely deterministic since the sorting of some of the
variables can change. This likely doesn't have any real world effect but fix
this with some additional sorting as it allows better hash equivalency of
task output.
(From OE-Core rev: a30c4de8a0b309c398c42fe710c45a4df7fcc64d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds extended package data which is encoded as JSON which allows it to
encode more structure than the "flat" package data files. The extended
data might be much larger than the standard package data, so it is not
read by default and instead requires
oe.packagedata.read_subpkgdata_extended() to be called
Currently, the file sizes and ELF debug sources are saved off into the
extended package data
(From OE-Core rev: db9cf430e54ae68da80fbc3fba80ce88d8df164d)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Reviewed-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch is a follow on of the the PR server rework in bitbake to add
read-only support. The shift to using the bb.asyncrpc code in the PR
server and client brings issues with respect to reuse of the same
asyncio loop in different processes. This patch removes the PR service
connection caching to avoid one source of this problem. It is believed
that in practice this should have little impact on overall performance.
(From OE-Core rev: 0fc3055027e2a76ac863f1c0e0d52e95748066aa)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes pkgdata PKGSIZE info after the overrides change.
(From OE-Core rev: 6964f06e48c7002c9ad788aa04bd8873fb3ee024)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a long standing bug where FILES_INFO isn't written into pkgdata
with a package suffix. This means if the files are read into the datastore
as intended, the last one "wins".
Fix this to work as intended. Most of the call sites using the data need
to be updated to handle this and the overrides change correctly.
Also fix some other problematic references noticed along the way.
(From OE-Core rev: a1190903e0a61a12c9854c96af918ae8d12c6327)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
pkgdata no longer includes FILERPROVIDES* nor
FILERDEPENDS*. Fix with new override syntax.
Also apply to FILERPROVIDESLIST and FILERDEPENDSLIST.
(From OE-Core rev: 8f96a2d13bbae8fb70ed7feafdcff26544e3710d)
Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The automated conversion of OE-Core to use the new override sytax isn't
perfect. This patches some mis-converted lines and some lines which were missed
by the automation.
(From OE-Core rev: 4e9a06b64b43131b731fb59a0305f78a98e27fbd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the result of automated script conversion:
scripts/contrib/convert-overrides.py <oe-core directory>
converting the metadata to use ":" as the override character instead of "_".
(From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Incremental build in Docker fails with:
OSError: [Errno 18] Invalid cross-device link
when source and destination are on different overlay filesystems.
Rather than adding fallback code to every call site, use a new wrapper
in bitbake which detects this case and falls back to shutil.move
which is slower but will handtle the overlay docker filesystems correctly.
[YOCTO #14301]
(From OE-Core rev: 656a65b2b84e7d529b89cf5de7eb838f902d84a2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The host may or may not have fc-cache which is used for find provides
information by rpmdeps. This lead to non-deterministic build output.
Disable the font provides code so we have deterministic builds,
we have nothing using/relying on it at this point.
Need to disable this in both the rpmdeps code and in package_rpm
itself although the latter shouldn't be being used.
(From OE-Core rev: 1c0cd8012a96fd4d9caf33c6de5fd39cea6db55d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In an earlier commit, libprocps was split into a separate package leaving
no shlibs in the main package. A bug was seen where igt-gpu-tools wouldn't
build correctly in some cases as it thought the librbary was still in the
main package, throwing qa errors as a result.
The issue was due to an extra file being left in the sstate output of
the do_packagedata task in the shlibs2/ folder which contained the bad
shlibs information.
The reason for this was that the temporary directory used in this
task wasn't being cleaned so files which were deleted were not handled
correctly. Add a missing cleandirs entry to fix this.
(From OE-Core rev: 50f17d0a655a3a2556f9fcad67259101c2814a36)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The '/sysroot-only' in SYSROOT_DIRS is to be used by recipes which
generate artifacts that are not included in the target filesystem.
Also, remove the ${D}/sysroot-only dir before copying D do PKGD to
generate the packages since it is not supposed to be included in
any package.
This will allow recipes to share non-target filesystem artifacts
without needing to use the DEPLOY_DIR and keep it tidy.
(From OE-Core rev: ed1c156cf46c2cdd8038d6bcf7ed58ebe275e3a1)
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If files are copied into /usr/src/debug directly from WORKDIR (e.g. makedevs)
we'd get the permissions from the checkout which would depend on the host umask.
Avoid this and be deterministic by setting the file modes consistently. Core
code copies the files in so we're responsible for the permissions.
Unfortunately to force this change to apply we need to invalidate both
the package tasks and the hash equivalance mappings since file mode
'corruption' already made it into the output hashes (both input options
were mapped to the output hashes).
(From OE-Core rev: 1f958bcd6c9cd12ec76d80586cba15f4d6ed17a7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the PR service is enabled a number of small changes may happen
to variables. In the do_package step a call to package_get_auto_pr
will end up setting PRAUTO and modifying PKGV (if AUTOINC is there).
PRAUTO is then used by EXTENDPRAUTO, which is then used to generate
PKGR.
Since this behavior typically happens BEFORE the BB_UNIHASH is
calculated for do_package, we need a way to defer the expansion
until after we have the unihash value.
Writing out the pkgdata files w/o AUTOPR and PKGV (AUTOINC) expanded
to placeholder values is the easiest way to deal with this. All other
variables are expanded as expected.
In the next task, typically do_packagedata, we will then use the
UNIHASH from the do_package to get the PR (AUTOPR) as well as
generate the AUTOINC replacement value (now PRSERV_PV_AUTOINC).
The do_packagedata then translates the placeholders to the final values
when copying the data from pkgdata to pkgdata-pdata-input.
Also update the prservice test case. With unihash, just changing the
do_package (via a _append) will not change the PR. So write the date
to a specific file that is incorporated into the unihash to ensure it
is always different for the test. Various assert messages were also
updated to make it easier to figure out where/why a problem occured.
(From OE-Core rev: 2e32f37b0e4abc438c8f60e673cd18a5cc110768)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
emit_pkgdata() creates symlinks for each of the RPROVIDES in
pkgdata/MACHINE/runtime-rprovides. However this string can contain
versions which results in directories called (=2.32), so pass the
RPROVIDES string through bb.utils.explode_deps() to strip the versions
out.
Helps mitigate - but not solve - #13999.
(From OE-Core rev: 6a5395dec34192db233bfb2a060e5ccc99708f03)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The output was unsorted, so different versions of python, different input
ordering could have have changed the files, and thus changed the hashes
making the system think the output was different, even when unmodified.
(From OE-Core rev: 23cc846c92219c5cbe3fc6a0024579195d3cbd78)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code was using TARGET_OS to try and detect what type of OS the
packages would be running on, but this is incorrect. TARGET_OS is the OS
for which the package will generate output (e.g. GCC output target or
GDB debugging target). The OS where the package will run is the HOST_OS.
Note that HOST_OS only differs from TARGET_OS when doing a
canadian cross compile, and even then in our case only when doing so for
a non-Linux host (e.g. MinGW).
Fix the code to use HOST_OS instead.
(From OE-Core rev: f284c5b2df220b520b025a59874e04ef4becd829)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The PACKAGE_ADD_METADATA can be used to define additional metadata for
packages. Changes to this variable should affect the packaging sstate hash
so packages are re-generated when this variable changes.
This variable is added to both PKGDATA_VARS and PACKAGEVARS. It is needed
in PACKAGEVARS to ensure changes invoke do_package to re-run. It is needed
in PKGDATA_VARS to ensure changes are written into the pkgdata so that the
hash equiv reports a change and does not skip over the package_write tasks.
(From OE-Core rev: de871a51cc450e3f90c923c5d80155453e5c241d)
Signed-off-by: Michael Ho <Michael.Ho@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Packages with a runtime dependency on a target package whose name is
changed by the PKG_* mechanism must rebuild when that mapping changes,
but we have no way of tracking this today, so
eg. packagegroup-machine-base ends up with a relationship on a
versioned kernel-image, and does not get rebuilt when that version
changes, leading to unsatisfiable dependency and reproducibility
issue.
OTOH there is no reason for the dependency to get rewritten if the
renamed package already has a RPROVIDES on the non-rewritten package
name, and if the dependency relationship is an unversionned one. This
is what this patch prevents.
Note that this may not cover all cases of rewritten package names.
Notably I had to let the rewrite be done in the case of versionned
dependencies, as package managers usually can follow "Provides" in
such case; this includes many dependencies against shared-lib packages
renamed to their soname, and those at least are OK, since the
dependent recipe should explicitly depend on the target recipe.
(From OE-Core rev: 920beaaeef62b558e046f32c8ef0332250969ef1)
Signed-off-by: Yann Dirson <yann@blade-group.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"Mini debuginfo" is a special section in ELF executables containing
minimal compressed debuginfo for non-exported symbols:
https://sourceware.org/gdb/onlinedocs/gdb/MiniDebugInfo.html
It lets debugging tools produce better stack traces, including local
function names, without incurring the space overhead of full debuginfo.
The feature was originally developed for Fedora:
https://fedoraproject.org/wiki/Features/MiniDebugInfo
but nowadays it is widely supported in the ecosystem, including in gdb
and elfutils (and therefore also in tools which use elfutils, such as
systemd-coredump).
This patch adds an optional extra step in package.bbclass to inject
minidebuginfo while stripping and splitting out debuginfo. It can be
enabled by setting PACKAGE_MINIDEBUGINFO=1. In my testing, this
increases the size of resulting binaries by roughly 5%.
The code for producing and re-injecting the minidebuginfo is my own
Python implementation but corresponds directly to the shell
implementation that RPM uses for doing the same:
https://github.com/rpm-software-management/rpm/blob/rpm-4.15.1-release/scripts/find-debuginfo.sh#L261
(From OE-Core rev: 4df992ce50c2d12e356b6d9fe7b23a6320c8b4df)
Signed-off-by: Dan Callaghan <dan.callaghan@opengear.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that the plain variable X is also added to the dependencies
as well as X_<pkg>.
Allow the funciton to be called with a different variable list too.
(From OE-Core rev: c392401e10ff43d10b7e57c9d552522a02c91878)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default, we won't copy and strip static libraries. However, this
functionality can be useful in some cases where people are doing
development on the target, and don't generally want the larger debug
capable static libraries. To enable the new functionality set:
PACKAGE_DEBUG_STATIC_SPLIT = '1'
Add a new function splitstaticdebuginfo. Thus function will copy the
unmodified static library into the specific debug directory location.
By keeping an unmodified version, it is possible for a user trying
to debug something to use -L /usr/lib/.debug-static and their existing
build commands to switch from stripped to full debug versions.
The PACKAGE_DEBUG_SPLIT_STYLE will select between two different
approaches, /usr/lib/debug-static or <path>/.debug-static.
Additionally you can now choose to strip static libraries to conserve
space. If either 'PACKAGE_DEBUG_STATIC_SPLIT' or 'PACKAGE_STRIP_STATIC'
is set to 1, the static library will be stripped. (This is not on by
default, as it could make diagnosing static library usage difficult in
some cases.)
Add to insane.bbclass a skip to the staticdev warning for the specific
-dbg package versions.
(From OE-Core rev: 17fa66c8199d73f0b59b2b3e609075933bf1e74b)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change the order of the skip processing to happen before any .ko and static
library processing. This will allow these types of files to be individually
skipped if necessary.
(From OE-Core rev: c50fcd54ffe60b63d042d05e6cf538a593dc410f)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Format of the sources list is the [ (file, [source, ...]), ... ] before
this change, the static libraries were processed but the items were
included incorrectly causing no sources for static libraries to be
included.
(From OE-Core rev: fa356b23c2f4599681693bba50d36659b07a8125)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When excluding a package from being packaged due to incompatible
licenses, report the license(s) that are actually incompatible.
(From OE-Core rev: 7d5c07e4a32a0968942ae538023c2891b59d8ab5)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that ldconfig may get installed from a feed, use it when it's
available on the target.
(From OE-Core rev: 0f09f19dc48edf2ac50b554c18c217b7f97d4ae3)
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the case where multiple packages provide the same file, we show an error.
Otherwise, python will generate a different build depending on which provider
appears first in the dictionary. On my system this order changes every time
I run bitbake causing intermittent build differences.
Add a sorted() to fix the determinism issue too.
(From OE-Core rev: 61c41369003444bfbf1c45e7cfd2752a4b7bc22f)
Signed-off-by: Jonathan Marler <johnnymarler@hp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
