The files in /sysroot-only are intended to make it into the
recipes sysroot output, but not into the package. However, if
do_package is run before do_populate_sysroot, the files are
removed.
Use a smaller hammer to avoid copying the files into the package so
they are still around when do_populate_sysroot runs.
(From OE-Core rev: acc0a31305cb1f799e8a6b3ea2d2c5ac5a9f67ee)
Signed-off-by: Russ Dill <russ.dill@nikolamotor.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit 493145c6f1bc92ab2b7a23e181641b09df87c9ff)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no reason to include a path in foo[dirs] if it is also in
foo[cleandirs] (except if it is the last path in foo[dirs]).
(From OE-Core rev: 9d2dac56deda5afa4d77c5ddd7e1e8003f6d4725)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By renaming LICENSE_EXCLUSION to _exclude_incompatible, it makes it
clear that this is an internal variable.
(From OE-Core rev: 20a4cc2c2dcf345ef898abfe7735b7bc75ac0059)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This simplifies the APIs for splitdebuginfo(), splitstaticdebuginfo()
and inject_minidebuginfo()
(From OE-Core rev: 9b00cf049032a9ed7ba3c63d3c37c7414f94edbf)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It simplifies the API to return one dict instead of seven strings.
(From OE-Core rev: 610c790a94591da5c3a6b1db938cd85e2cbdbe5a)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The split_and_strip_files() function is big and hard to follow. This
takes a small step to improve that by splitting out the part that sets
up the variables used to implement the various debug splitting methods
specified by PACKAGE_DEBUG_SPLIT_STYLE.
(From OE-Core rev: e414c6ec8563f9c448f1421f33870a908a0353b4)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extend runstrip() to accept additional argument to enable
sharing it with the kernel do_strip() so that
KERNEL_IMAGE_STRIP_EXTRA_SECTIONS can be passed.
Since is_elf() understands kernel modules there is no need to keep a
seperate list for kernmodules or hardcode the values to runstrip.
(From OE-Core rev: e09a8fa931fe617afc05bd5e00dca5dd3fe386e8)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PACKAGE_SNAP_LIB_SYMLINKS renames libraries based on their SONAME so
that they can be found directly rather than going via symlinks that
would be created by ldconfig. For example, without
PACKAGE_SNAP_LIB_SYMLINKS in ${libdir} we have:
libharfbuzz.so.0 -> libharfbuzz.so.0.20600.4
libharfbuzz.so.0.20600.4
but with PACKAGE_SNAP_LIB_SYMLINKS="1" we have just:
libharfbuzz.so.0
Unfortunately, this renaming is done based on the SONAME which breaks
packages like mesa which install a single library with multiple hard
links:
-rwxr-xr-x root/root 13593488 2021-12-07 12:26 ./usr/lib/dri/i915_dri.so
-rwxr-xr-x root/root 13137328 2021-12-07 12:26 ./usr/lib/dri/i965_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/iris_dri.so link to ./usr/lib/dri/i915_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/kms_swrast_dri.so link to ./usr/lib/dri/i915_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/nouveau_vieux_dri.so link to ./usr/lib/dri/i965_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/r200_dri.so link to ./usr/lib/dri/i965_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/radeon_dri.so link to ./usr/lib/dri/i965_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/swrast_dri.so link to ./usr/lib/dri/i915_dri.so
The SONAME for i915_dri.so (and therefore all the other names that link
to the same file) is libgallium_dri.so. This means that
PACKAGE_SNAP_LIB_SYMLINKS causes do_package to successfully rename the
first name found to libgallium_dri.so. A similar thing happens to
i965_dri.so with its SONAME of libmesa_dri_drivers.so. The order is not
deterministic, so this means that although every build will be missing
one name, it's not always the same one.
-rwxr-xr-x root/root 13593488 2021-11-30 15:17 ./usr/lib/dri/i915_dri.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/kms_swrast_dri.so link to ./usr/lib/dri/i915_dri.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/libgallium_dri.so link to ./usr/lib/dri/i915_dri.so
-rwxr-xr-x root/root 13137328 2021-11-30 15:17 ./usr/lib/dri/libmesa_dri_drivers.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/nouveau_vieux_dri.so link to ./usr/lib/dri/libmesa_dri_drivers.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/r200_dri.so link to ./usr/lib/dri/libmesa_dri_drivers.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/radeon_dri.so link to ./usr/lib/dri/libmesa_dri_drivers.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/swrast_dri.so link to ./usr/lib/dri/i915_dri.so
This renaming means that the library cannot be found at runtime.
The simplest way to avoid this renaming is to only snap libraries that
would be processed by ldconfig.
(From OE-Core rev: 8571182ece1602ce8e030c98aef17cdc5718a037)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Phil Blundell <pb@pbcl.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We exclude allarch packagegroups from rebuilding when their dependencies change.
The reasoning is that we are just depending on a name so having these rebuild
lots is just pointless and inefficient. We also don't want them duplicated for
multiple machines for efficiency.
In general this works fine, as long as the package names don't change. That
is also rare but there is one corner case which does catch users out - debian
package renaming. When this does break, users question sstate and so on and
lose faith in the system even if this is a known choice we made.
This commit adds an error message if an allarch packagegroup depends on any
package which shows package renaming in action (through the PKG variable
being set).
If you run into this issue you either need to remove the dependency from the
packagegroup or mark the packagegroup as tune specific, i.e. set:
PACKAGE_ARCH = "${TUNE_PKGARCH}"
before the packagegroup inherit.
[YOCTO #7298]
(From OE-Core rev: 5bf3e447d2f5064495d83a8fad30229bcf1ecc9b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extract package_qa_write_error, package_qa_handle_error and
package_qa_add_message functions from insane.bbclass to lib/oe/qa.py and
drop the package_qa_ prefixes.
Update various bbclasses to use the new functions. No import is required
since base.bbclass puts oe.qa in OE_IMPORTS.
Stop requiring callers to manually track whether a fatal error has been
encountered via a "sane" flag. Instead replace the QA_SANE variable with
QA_ERRORS_FOUND and call oe.qa.exit_if_errors or
oe.qa.exit_with_message_if_errors at the end of each task.
Inspired by discussion resulting from
https://lists.openembedded.org/g/openembedded-core/message/156793 and
https://lists.openembedded.org/g/openembedded-core/message/156900
(From OE-Core rev: f0ad152ef4cc15c042bc9eeefb6af096d054b220)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the pkgconfig pclist files contain full paths which are build
host specific and the order of entries is not deterministic.
Fix both these issues so the files are deterministic.
(From OE-Core rev: e422e29bca4af3ab4073e04490f38b05cd7c38c0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
pkgdata output isn't entirely deterministic since the sorting of some of the
variables can change. This likely doesn't have any real world effect but fix
this with some additional sorting as it allows better hash equivalency of
task output.
(From OE-Core rev: a30c4de8a0b309c398c42fe710c45a4df7fcc64d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds extended package data which is encoded as JSON which allows it to
encode more structure than the "flat" package data files. The extended
data might be much larger than the standard package data, so it is not
read by default and instead requires
oe.packagedata.read_subpkgdata_extended() to be called
Currently, the file sizes and ELF debug sources are saved off into the
extended package data
(From OE-Core rev: db9cf430e54ae68da80fbc3fba80ce88d8df164d)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Reviewed-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch is a follow on of the the PR server rework in bitbake to add
read-only support. The shift to using the bb.asyncrpc code in the PR
server and client brings issues with respect to reuse of the same
asyncio loop in different processes. This patch removes the PR service
connection caching to avoid one source of this problem. It is believed
that in practice this should have little impact on overall performance.
(From OE-Core rev: 0fc3055027e2a76ac863f1c0e0d52e95748066aa)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes pkgdata PKGSIZE info after the overrides change.
(From OE-Core rev: 6964f06e48c7002c9ad788aa04bd8873fb3ee024)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a long standing bug where FILES_INFO isn't written into pkgdata
with a package suffix. This means if the files are read into the datastore
as intended, the last one "wins".
Fix this to work as intended. Most of the call sites using the data need
to be updated to handle this and the overrides change correctly.
Also fix some other problematic references noticed along the way.
(From OE-Core rev: a1190903e0a61a12c9854c96af918ae8d12c6327)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
pkgdata no longer includes FILERPROVIDES* nor
FILERDEPENDS*. Fix with new override syntax.
Also apply to FILERPROVIDESLIST and FILERDEPENDSLIST.
(From OE-Core rev: 8f96a2d13bbae8fb70ed7feafdcff26544e3710d)
Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The automated conversion of OE-Core to use the new override sytax isn't
perfect. This patches some mis-converted lines and some lines which were missed
by the automation.
(From OE-Core rev: 4e9a06b64b43131b731fb59a0305f78a98e27fbd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the result of automated script conversion:
scripts/contrib/convert-overrides.py <oe-core directory>
converting the metadata to use ":" as the override character instead of "_".
(From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Incremental build in Docker fails with:
OSError: [Errno 18] Invalid cross-device link
when source and destination are on different overlay filesystems.
Rather than adding fallback code to every call site, use a new wrapper
in bitbake which detects this case and falls back to shutil.move
which is slower but will handtle the overlay docker filesystems correctly.
[YOCTO #14301]
(From OE-Core rev: 656a65b2b84e7d529b89cf5de7eb838f902d84a2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The host may or may not have fc-cache which is used for find provides
information by rpmdeps. This lead to non-deterministic build output.
Disable the font provides code so we have deterministic builds,
we have nothing using/relying on it at this point.
Need to disable this in both the rpmdeps code and in package_rpm
itself although the latter shouldn't be being used.
(From OE-Core rev: 1c0cd8012a96fd4d9caf33c6de5fd39cea6db55d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In an earlier commit, libprocps was split into a separate package leaving
no shlibs in the main package. A bug was seen where igt-gpu-tools wouldn't
build correctly in some cases as it thought the librbary was still in the
main package, throwing qa errors as a result.
The issue was due to an extra file being left in the sstate output of
the do_packagedata task in the shlibs2/ folder which contained the bad
shlibs information.
The reason for this was that the temporary directory used in this
task wasn't being cleaned so files which were deleted were not handled
correctly. Add a missing cleandirs entry to fix this.
(From OE-Core rev: 50f17d0a655a3a2556f9fcad67259101c2814a36)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The '/sysroot-only' in SYSROOT_DIRS is to be used by recipes which
generate artifacts that are not included in the target filesystem.
Also, remove the ${D}/sysroot-only dir before copying D do PKGD to
generate the packages since it is not supposed to be included in
any package.
This will allow recipes to share non-target filesystem artifacts
without needing to use the DEPLOY_DIR and keep it tidy.
(From OE-Core rev: ed1c156cf46c2cdd8038d6bcf7ed58ebe275e3a1)
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If files are copied into /usr/src/debug directly from WORKDIR (e.g. makedevs)
we'd get the permissions from the checkout which would depend on the host umask.
Avoid this and be deterministic by setting the file modes consistently. Core
code copies the files in so we're responsible for the permissions.
Unfortunately to force this change to apply we need to invalidate both
the package tasks and the hash equivalance mappings since file mode
'corruption' already made it into the output hashes (both input options
were mapped to the output hashes).
(From OE-Core rev: 1f958bcd6c9cd12ec76d80586cba15f4d6ed17a7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the PR service is enabled a number of small changes may happen
to variables. In the do_package step a call to package_get_auto_pr
will end up setting PRAUTO and modifying PKGV (if AUTOINC is there).
PRAUTO is then used by EXTENDPRAUTO, which is then used to generate
PKGR.
Since this behavior typically happens BEFORE the BB_UNIHASH is
calculated for do_package, we need a way to defer the expansion
until after we have the unihash value.
Writing out the pkgdata files w/o AUTOPR and PKGV (AUTOINC) expanded
to placeholder values is the easiest way to deal with this. All other
variables are expanded as expected.
In the next task, typically do_packagedata, we will then use the
UNIHASH from the do_package to get the PR (AUTOPR) as well as
generate the AUTOINC replacement value (now PRSERV_PV_AUTOINC).
The do_packagedata then translates the placeholders to the final values
when copying the data from pkgdata to pkgdata-pdata-input.
Also update the prservice test case. With unihash, just changing the
do_package (via a _append) will not change the PR. So write the date
to a specific file that is incorporated into the unihash to ensure it
is always different for the test. Various assert messages were also
updated to make it easier to figure out where/why a problem occured.
(From OE-Core rev: 2e32f37b0e4abc438c8f60e673cd18a5cc110768)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
emit_pkgdata() creates symlinks for each of the RPROVIDES in
pkgdata/MACHINE/runtime-rprovides. However this string can contain
versions which results in directories called (=2.32), so pass the
RPROVIDES string through bb.utils.explode_deps() to strip the versions
out.
Helps mitigate - but not solve - #13999.
(From OE-Core rev: 6a5395dec34192db233bfb2a060e5ccc99708f03)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The output was unsorted, so different versions of python, different input
ordering could have have changed the files, and thus changed the hashes
making the system think the output was different, even when unmodified.
(From OE-Core rev: 23cc846c92219c5cbe3fc6a0024579195d3cbd78)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code was using TARGET_OS to try and detect what type of OS the
packages would be running on, but this is incorrect. TARGET_OS is the OS
for which the package will generate output (e.g. GCC output target or
GDB debugging target). The OS where the package will run is the HOST_OS.
Note that HOST_OS only differs from TARGET_OS when doing a
canadian cross compile, and even then in our case only when doing so for
a non-Linux host (e.g. MinGW).
Fix the code to use HOST_OS instead.
(From OE-Core rev: f284c5b2df220b520b025a59874e04ef4becd829)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The PACKAGE_ADD_METADATA can be used to define additional metadata for
packages. Changes to this variable should affect the packaging sstate hash
so packages are re-generated when this variable changes.
This variable is added to both PKGDATA_VARS and PACKAGEVARS. It is needed
in PACKAGEVARS to ensure changes invoke do_package to re-run. It is needed
in PKGDATA_VARS to ensure changes are written into the pkgdata so that the
hash equiv reports a change and does not skip over the package_write tasks.
(From OE-Core rev: de871a51cc450e3f90c923c5d80155453e5c241d)
Signed-off-by: Michael Ho <Michael.Ho@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Packages with a runtime dependency on a target package whose name is
changed by the PKG_* mechanism must rebuild when that mapping changes,
but we have no way of tracking this today, so
eg. packagegroup-machine-base ends up with a relationship on a
versioned kernel-image, and does not get rebuilt when that version
changes, leading to unsatisfiable dependency and reproducibility
issue.
OTOH there is no reason for the dependency to get rewritten if the
renamed package already has a RPROVIDES on the non-rewritten package
name, and if the dependency relationship is an unversionned one. This
is what this patch prevents.
Note that this may not cover all cases of rewritten package names.
Notably I had to let the rewrite be done in the case of versionned
dependencies, as package managers usually can follow "Provides" in
such case; this includes many dependencies against shared-lib packages
renamed to their soname, and those at least are OK, since the
dependent recipe should explicitly depend on the target recipe.
(From OE-Core rev: 920beaaeef62b558e046f32c8ef0332250969ef1)
Signed-off-by: Yann Dirson <yann@blade-group.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"Mini debuginfo" is a special section in ELF executables containing
minimal compressed debuginfo for non-exported symbols:
https://sourceware.org/gdb/onlinedocs/gdb/MiniDebugInfo.html
It lets debugging tools produce better stack traces, including local
function names, without incurring the space overhead of full debuginfo.
The feature was originally developed for Fedora:
https://fedoraproject.org/wiki/Features/MiniDebugInfo
but nowadays it is widely supported in the ecosystem, including in gdb
and elfutils (and therefore also in tools which use elfutils, such as
systemd-coredump).
This patch adds an optional extra step in package.bbclass to inject
minidebuginfo while stripping and splitting out debuginfo. It can be
enabled by setting PACKAGE_MINIDEBUGINFO=1. In my testing, this
increases the size of resulting binaries by roughly 5%.
The code for producing and re-injecting the minidebuginfo is my own
Python implementation but corresponds directly to the shell
implementation that RPM uses for doing the same:
https://github.com/rpm-software-management/rpm/blob/rpm-4.15.1-release/scripts/find-debuginfo.sh#L261
(From OE-Core rev: 4df992ce50c2d12e356b6d9fe7b23a6320c8b4df)
Signed-off-by: Dan Callaghan <dan.callaghan@opengear.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that the plain variable X is also added to the dependencies
as well as X_<pkg>.
Allow the funciton to be called with a different variable list too.
(From OE-Core rev: c392401e10ff43d10b7e57c9d552522a02c91878)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default, we won't copy and strip static libraries. However, this
functionality can be useful in some cases where people are doing
development on the target, and don't generally want the larger debug
capable static libraries. To enable the new functionality set:
PACKAGE_DEBUG_STATIC_SPLIT = '1'
Add a new function splitstaticdebuginfo. Thus function will copy the
unmodified static library into the specific debug directory location.
By keeping an unmodified version, it is possible for a user trying
to debug something to use -L /usr/lib/.debug-static and their existing
build commands to switch from stripped to full debug versions.
The PACKAGE_DEBUG_SPLIT_STYLE will select between two different
approaches, /usr/lib/debug-static or <path>/.debug-static.
Additionally you can now choose to strip static libraries to conserve
space. If either 'PACKAGE_DEBUG_STATIC_SPLIT' or 'PACKAGE_STRIP_STATIC'
is set to 1, the static library will be stripped. (This is not on by
default, as it could make diagnosing static library usage difficult in
some cases.)
Add to insane.bbclass a skip to the staticdev warning for the specific
-dbg package versions.
(From OE-Core rev: 17fa66c8199d73f0b59b2b3e609075933bf1e74b)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change the order of the skip processing to happen before any .ko and static
library processing. This will allow these types of files to be individually
skipped if necessary.
(From OE-Core rev: c50fcd54ffe60b63d042d05e6cf538a593dc410f)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Format of the sources list is the [ (file, [source, ...]), ... ] before
this change, the static libraries were processed but the items were
included incorrectly causing no sources for static libraries to be
included.
(From OE-Core rev: fa356b23c2f4599681693bba50d36659b07a8125)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When excluding a package from being packaged due to incompatible
licenses, report the license(s) that are actually incompatible.
(From OE-Core rev: 7d5c07e4a32a0968942ae538023c2891b59d8ab5)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that ldconfig may get installed from a feed, use it when it's
available on the target.
(From OE-Core rev: 0f09f19dc48edf2ac50b554c18c217b7f97d4ae3)
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the case where multiple packages provide the same file, we show an error.
Otherwise, python will generate a different build depending on which provider
appears first in the dictionary. On my system this order changes every time
I run bitbake causing intermittent build differences.
Add a sorted() to fix the determinism issue too.
(From OE-Core rev: 61c41369003444bfbf1c45e7cfd2752a4b7bc22f)
Signed-off-by: Jonathan Marler <johnnymarler@hp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PRIVATE_LIBS is used to exclude 'private' libraries from getting added to
automatic runtime dependency resolution. This variable currently has to list
all libraries by name, which becomes a maintenance issue if the list
of such libraries frequently changes, or is very large.
This change allows using shell-style wildcards in the variable, similar
to how FILES lists what gets packaged.
(From OE-Core rev: 732db32714c208d8eeeb90308926dc886ef7b791)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
do_package has PKGDESTWORK as a cleandir and do_packagedata has it as an sstate-input
dir. This means do_package wipes out the directory at both do_package and do_package_setscene.
do_package_setscene and do_packagedata_setscene can run in parallel when installing from
sstate which means they can wipe out parts of each other leading to interesting
build failures.
We therefore have to add in a hardlink copy so that the directories can work independently
of each other.
(From OE-Core rev: 692b2046bb60806ee6bbde6ba15bd1637b00efb1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its possible in cases with multiple shlib providers we were not being
deterministic. Add in a couple of sorted() calls to fix the shlibs and
pkgconfig cases with this potential issue.
(From OE-Core rev: 5b9a4214ee17e1a39dd5a1742f2ac5ed25a11310)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a package uses PKG_ variables to map package names to version specific
variants, on target postinstall functionality will be broken.
i.e. something like the following casuses rootfs assembly errors:
d.setVar('pkg_postinst_ontarget_linux-source', 'cd /usr/src/; ln -sf %s linux-source' % source_pkg)
This breakage is due to the fact that the original package name (as specified by
the PACKAGES variable) is logged by the intercept scripts, but the mapped /
specific version is actually installed to the rootfs (and hence logged by the
package manager).
When the runtime listing of on-target scripts is performed, we get a package
manager error due to a missing package, since it checks the generic version
logged by the intercept scripts.
We can fix this by ensuring that the PKG_ variable mapped package name
is logged by the intercept phase, and hence the package manager can locate
and execute the on target postinst script.
This variable check is consistent with other places in the code, and has
no impact if PKG_ variables are not used.
(From OE-Core rev: a6af0886d1be584974086c0ddb4a5bc566eb7984)
Signed-off-by: Bruce Ashfield <bruce.ashfield@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This switches the code to build pkgdata specific to the current recipe
which means that its filtered to the recipes dependencies and can perform
better as we can drop the lockfile.
It uses a similar method to the staging code to do this, using BB_TASKDEPDATA
to construct a list of packagedata task output which this recipe should "see".
The original pkgdata store is left unaltered so existing code works.
The lock file was there to prevent files disappearing as they were read or as
directories were listed. Since we have a copy of the data and only access output
from completed tasks (as per their manifests), we can remove the lock.
The lock was causing starvation issues on systems with parallelism.
There was also a potential determinism problem as the current code could "see"
data from recipes which it doesn't depend upon.
[YOCTO #13412]
(From OE-Core rev: 1951132576bfb95675b4879287f8b3b7c47524fa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
populate_packages relies on ``mkdir`` to both create a directory and set
its permissions. However, ``mkdir`` honors the ``umask`` value.
Therefore, some bits may be lost in the operation. In our case, the
setgid bit on the directories were lost.
This commit fixes this by having a distinct call to create the directory
and to set the permissions.
(From OE-Core rev: 0f82b53a650e76e0129fae6ce7581a41d042315b)
Signed-off-by: Jean-Tiare Le Bigot <jean-tiare.le-bigot@easymile.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This introduces a variable, PKGDATA_VARS, that contains the names of
the variables that are to be output in the runtime pkgdata files.
(From OE-Core rev: 43e55bfa040425cf93d94ac626a31f6fd00a7a74)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sorts the list of detected ELF files by path before processing. This
ensures that when multiple files are hardlinked together the first one
found is always the same. This is required to have reproducible builds.
(From OE-Core rev: de86bfeda6e3845336a0b56c883b49219967128f)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Permanently adds the -src source package to ${PACKAGES} instead of
relying on creating it dynamically at packaging time. The source package
is now defined in bitbake.conf, just like -dev and -dbg packages.
For compatibility, the -src package is still added dynamically if it was
missing, since some recipes have a tendency to completely override
PACKAGES and do so without manually adding back the -src package.
This allows RDEPENDS on the -src packages, which wasn't previously
possible.
[YOCTO #13203]
(From OE-Core rev: b25e1edf0204fc2f64aa8d66e09b8e2d67b90e17)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
