as this will clear WORKDIR and create race conditions
across various handling tasks
(From OE-Core rev: 1cf99ce3f79b2c96bdef5aa9b69c2b3ead7e46f1)
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
libtool auto detects the sysroot from gcc's parameters or configuration so we
don't need to pass in this configuration separately to libtool.
Whilst the option names do conflict with gcc/binutils, that is an issue for those
projects to resolve, not us. Upstream libtool did reject the patch. We can
drop this patch and simplify our code.
(From OE-Core rev: 7c8553f81bccc3e8c2bb1116ee1e89f5f8af4c9e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code that used these variable and the comment was introduced in
commit b44d32ef41 ("insane.bbclass: Portions of code were not running,
fix this and sync with OE.dev. Also add tests for bad sysroot rpaths in
binaries"). Later, in commit 17dae13fabe2 ("insane.bbclass: Fix ELF
bitsize comparison"), some of that code was removed again, but not the
variables and the comment.
(From OE-Core rev: 730d00b0d1d1d617b62900be12fa034bb41fc48b)
Signed-off-by: Emil Kronborg <emil.kronborg@protonmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit cd25e5544ca3 ("insane: use HOST_ variables, not TARGET_ to
determine the cross system") updated the variables themselves, but not
their names. To prevent confusion, match the Python variable name to the
BitBake variable name.
(From OE-Core rev: f5bebc96580ec74d10bc96b4265357ebc9bcd6ad)
Signed-off-by: Emil Kronborg <emil.kronborg@protonmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After commit cd25e5544ca3 ("insane: use HOST_ variables, not TARGET_ to
determine the cross system"), this check is no longer necessary. The
introduction of HOST_ variables ensures architecture compatibility is
correctly checked.
(From OE-Core rev: 6e1ddeb05dcd5ff77e0f5526a6e56a484daa4864)
Signed-off-by: Emil Kronborg <emil.kronborg@protonmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds API to reverse the removal of special characters from file names so
it can be correctly done in multiple places without open-coding it.
Replace the translation done in the package_rpm.bbclass with the new API
(From OE-Core rev: 4cb7e93c624987d146aaf626ce8e99568e938a70)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In sanity.bbclass the tar version is checked as tar needs to be recent enough
for reproducible builds. Tar could also be provided by other means then gnutar,
but we mean the version of gnutar in the check. Hence we also should ensure
that the installed tar is gnutar.
[YOCTO #14205]
(From OE-Core rev: bdef30bd887cd208d7822dd7853d33e24a6b7a4c)
Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildhistory is showing issues where plaindirs installed files (such as package
listings) are not reliably being handled with installs from sstate.
The reason is that plaindirs is being handled after SSTATEPOSTINSTFUNCS
instead of before it, meaning the files visible in a non-sstate accelerated
code run are different to show from an accelerated run.
This can be observed by the missing files lists for packages in buildhistory, both
in from scratch builds and in builds from sstate. In builds where sstate is installed
over an existing build directory, the files are present though, so there is a
determinism problem.
Fix this by moving the code into sstate_install, this is the only call
site for the funciton.
Since the move needs prepdir, move that as well as it's call site,
being careful to handle the two different definitions of SSTATE_INSTDIR. The
version originally in the function was obsolete and was causing the postinstfuncs
to run in an incorrect directory. The only user is buildhistory and it wasn't
sensitive to cwd however so this happened not to cause a problem. Fix the
code to use the correct location.
(From OE-Core rev: 62ee349cf18532dac8736488752c00e89de78fcd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The only user was siteconfig which has been removed. The API was horrible
and we don't want to encourage this kind of usage.
(From OE-Core rev: cfbfd0b2e89eb71783c55a1be4a7e63e6cd82c2f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now S and B can't be set to WORKDIR, add to PSEUDO_IGNORE_PATHS unconditionally
and simplify the code.
(From OE-Core rev: 26cd2d56261827ad8d07e2145e95f82422accac2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was never a good idea and would have mostly happened from S = WORKDIR
however explictly disallow it and error if anyone tries.
(From OE-Core rev: e3c2c1fac904bb518d85e10a2ac0177c81cbf7e8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Where a recipe uses WORKDIR as S, exit with a fatal error since the
code is no longer safe for this layout.
(From OE-Core rev: 32cba1cc916ad530c5e6630a927e74ca6f06289b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change do_unpack to unpack files to a subdirectory of WORKDIR instead of WORKDIR
itself. There are several good reasons for this but it is mainly about being able
to isolate the output of the unpack task and tell the files apart from other things
which are created in workdir (logs, sysroots, temp dirs and more).
This means that when the do_unpack task reruns, we can clean UNPACKDIR and know
we have a standard point to start builds from.
It also makes code in tools like devtool and recipetool easier.
To reduce the impact to users, if a subdirectory under UNPACKDIR matches
the first subdirectory under WORKDIR of S, that directory is moved into position
inside WORKDIR. This preserves the behaviour of S = "${WORKDIR}/git",
S = "${WORKDIR}/${BPN}" and other commonly used source directory setups.
The directory is moved since sadly many autotools based projects can't cope with
symlinks in their paths.
The patch also updates reproducible and SOURCE_DATE_EPOCH handling to
match the new potential source locations. We can get rid of the horrible
list of hardcoded directories in WORKDIR to ignore from that code.
(From OE-Core rev: b84eec5c4cbf4b39d6712800dd0d2fe5337721cb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If S doesn't exist, do_qa_patch would fail. Fix the code to not fail
in this situation.
(From OE-Core rev: 4041d91b63ff2315657499e22c74ec90adbf9e19)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having the unpack directory hardcoded to WORKDIR makes it really hard to
make any changes to the unpack process to try and allow for cleanup for example.
As a first step toward unraveling the intertwined location usages, add a
variable, UNPACKDIR which is where the fetcher is asked to unpack fetched
sources. It defaults to the existing value of WORKDIR at this point.
(From OE-Core rev: e022d62ba917790af2121da57646271ef17c03fa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using tinfoil to control the build, multiple commands (serially) could
trigger an error such as:
When reparsing ....bb:do_package, the basehash value changed from ... to .... The metadata is not deterministic and this needs to be fixed.
ERROR: The following commands may help:
ERROR: $ bitbake esw-conf -cdo_package -Snone
ERROR: Then:
ERROR: $ bitbake esw-conf -cdo_package -Sprintdiff
However following these commands it was not able to be reproduced. Forcing
bitbake to dump the signatures and then running bitbake-diffsigs showed
that the value of _SSTATE_EXCLUDEDEPS_SYSROOT was being set in one run, but
was blank is a different version.
Upon inspecting the code in sstate.bbclass, one usage (without the _) is
already excludes, the leading _ version is used as a cache, only if set but
is not actually required to be defined. So ignoring the value should work
properly.
(From OE-Core rev: 4ec704ed6a1cfaf0a6c20f2038e7192e361ef590)
Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "libdir" QA check tries to open every file it finds as an ELF. If
it finds a dangling symlink that looks like a library by the filename it
will try to open it and fail with FileNotFoundError error. As this
dangling symlink probably points to a real file, silently absorb the
error.
[ YOCTO #13949 ]
(From OE-Core rev: f044290f98ea66f2cecfbffd7d392dbc3d986da9)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In testing websocket hashequivalence, corrupted sstate was injected into the
autobuilder extensively. With the new release/LTS, being able to clearly
differentiate between old and new sstate is probably desireable anyway
so bump the appropriate versions.
(From OE-Core rev: 7f107c180f592be29f57f580c60a6adbbebd7714)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
meld fallback assumes that user has a gnome desktop
its better to fallback to good old diff tool which
will most distributions on build host will carry
(From OE-Core rev: d027236eac2a136b06903d4a47adbc50ccd6b7c6)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace `${HOST_PREFIX}readelf` with `${READELF}`.
When utilizing llvm for compiling packages, the invocation of GNU readelf
will consistently occur if the oe_soinstall and oe_libinstall functions,
which are defined in utils.bbclass, are called. This behavior is unfriendly
to llvm. So prefer `${READELF}` over `${HOST_PREFIX}readelf`.
(From OE-Core rev: daecdd577213da0c045e45c47e7acbd279956d41)
Signed-off-by: lixiaoyong <lixiaoyong19@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With newer glibc versions, pthread was merged into the main library so there is
no longer any need to link with -lpthread. On newer systems this means that linker
flag is dropped from places like pkgconfig .pc files. If uninative then uses this
newer library on an older system, the flag is missing and linker errors about missing
symbols occur.
Adding the linkage unconditionally to our uninative linking flags avoids that problem
at a potential cost of slight over linking.
(From OE-Core rev: 5ec2bc7ef663db4c04c85eee518297d442556481)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The llvm github repo appears to see a lot of churn on branches we're not
interested in and is in general huge and unreliable to clone.
Switch to shallow clones to improve the user experience.
(From OE-Core rev: 6cf98a4954751977fb02fb2024ef842865352138)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
postinst-useradd-* haven't been running in order of dependency.
This patch is reworked from Piotr Łobacz's patch and fixes:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15084https://bugzilla.yoctoproject.org/show_bug.cgi?id=13904
basepasswd_sysroot_postinst in base-passwd can install postinst-useradd-*
scripts with any order. Sometimes this means, for example a useradd postinst
will attempt to run without the corresponding group postinst causing errors.
This patch ensures that we first run groupadd, then useradd and then
group membership.
[RP: Tweaked to avoid removing previous fixes and for whitespace/style issues
Also ensure the scripts are changed to execute with -e to highlight errors]
(From OE-Core rev: 322ef726132a47d977d2c6ee41de5358f1e85994)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
Signed-off-by: Jan Górski <j.gorski@welotec.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit f673d3d239799fb1ab50f4aa5d44187666aa0cd7 introduced a warning for
virtual/ being used in RPROVIDES and RDEPENDS. Make it possible to
disable the warning by removing "virtual-slash from WARN_QA.
(From OE-Core rev: 968ffdb9fee5017eecce36ce878ea604c869ce95)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes use of the oe.patch.GitApplyTree.commitIgnored() function to
create commits that shall be ignored by `devtool finish`.
(From OE-Core rev: 2393dc35a93546eccee0dd313a6927c7d1512c3b)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The purpose of the change is to never sign a package not created by
the build itself.
sstate_create_package is refactored into Python and re-designed
to handle signing inside the function. Thus, the signing should never apply
to existing sstate packages. The function is therefore renamed into
sstate_create_and_sign_package.
The creation of the archive remains in a separate shellscript function.
Co-authored-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
(From OE-Core rev: ba223f8fff19ea59440d56cf3fe46200f3f71e22)
Signed-off-by: Tobias Hagelborn <tobias.hagelborn@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some packages may contain localized files not located in default path
${datadir}/locale. Add the new variable LOCALE_PATHS to allow a recipe
to define extra paths or even fully override the scanned directories.
LOCALE_PATHS is set at ${datadir}/locale by default to keep the exact
same behavior for the recipes which did not need modification.
(From OE-Core rev: 0ffc7cf01225743789ac30dd325fca05b9203be1)
Signed-off-by: Jonathan GUILLOT <jonathan@joggee.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Older meson versions save temporary scripts in /tmp.
Similarly some recipies also do that (e.g. ccan in sbsigntool).
As this can lead to unexpected build failures with no simple way
to workaround, make such setup a fatal error.
(From OE-Core rev: ee93a8e89322143252040bd5bc99259c5efff831)
Signed-off-by: Michal Sieron <michalwsieron@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should be pointing people at VIRTUAL-RUNTIME, not virtual so tweak
the warning. Try and make it clear the difference between the build
dependencies and the runtime ones.
(From OE-Core rev: 01d815aa2c0bea113fb79b51bf67c0ff90d57dd2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the new infraroot.at mirror to the MIRRORS string for https sources
since infradead.org does not support https and certain proxy configs
cause this fetch to fail.
Mirror info fetched from the following announcement:
https://lists.infradead.org/pipermail/linux-mtd/2023-August/100922.html
(From OE-Core rev: ce8f7bb81598aecafe486a461c94254b97518832)
Signed-off-by: Randolph Sapp <rs@ti.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The intention of the mirror code was to allow faster cloning of glibc.
This wasn't quite working since the full clone was needed by glibc-tests
or cross-localedef-native. I noticed this when I updated a build and it
was trying to unpack glibc but fetch cross-localedef-native.
Add entries to cover these and restore the faster cloning for glibc
and realted recipes.
(From OE-Core rev: 7891b5fb96f757274bebbb77df634e7af9054298)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* e.g. for DISTROs which define extra user-defined fields with
PACKAGE_ADD_METADATA/PACKAGE_ADD_METADATA_RPM
as undefined fields in packagedata are fatal error for rpmbuild:
"error: line 9: Unknown tag: Author: Unspecified"
as shown in:
http://errors.yoctoproject.org/Errors/Details/751706/
with "Author" field added with:
PACKAGE_CLASSES = "package_rpm"
PACKAGE_ADD_AUTHOR_METADATA = "test-author"
PACKAGE_ADD_METADATA = "Author: ${PACKAGE_ADD_AUTHOR_METADATA}"
to fix rpm build you can use:
RPMBUILD_EXTRA_PARAMS = " --define '_Author Author'"
keep in mind that this doesn't cause this Author field to be
added in .rpm, it just avoids the BUILDSPEC failure.
and for ipk build:
OPKG_MAKE_INDEX_EXTRA_PARAMS = "-f"
alternatively you can avoid additional packagedata fields ending
in the package manager (if you use them only with buildhistory
or packagedata) with:
PACKAGE_ADD_METADATA_RPM = ""
PACKAGE_ADD_METADATA_IPK = ""
PACKAGE_ADD_METADATA_DEP = ""
(From OE-Core rev: 31030e7a19a27ad424b997fad36ee4f633ce0d63)
Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since f3eaeeb734
rpm does its own parsing of /etc/passwd and /etc/group instead of relying on getpwnam() and friends.
This has an unfortunate effect of leaking build host uid/gid values for users and groups
into the cpio header inside rpm file (where previously those were always zero).
Installation of rpm packages relies on rpm header to set files ownership, and that
is a different structure that is build from .spec information, so we can avoid host
contamination by setting the paths to something bogus.
(From OE-Core rev: ac8ea64bb39a5e56599e078c7e6cd056a2aa4144)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Per 77d3529c31
rpm 4.19.1+ will not consider actual filesystem permissions and ownership, and will quietly default
to root if not expictly set otherwise in .spec file.
There's also additional diagnostics (printing what is in passwd/group)
when user/group name lookup against the sysroot fails.
That is never supposed to happen, and yet there was one report that it did:
https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/8493/steps/23/logs/stdio
Investigating that issue led to the first three commits in this patchset:
sysroot user management postinsts: run with /bin/sh -e to report errors when they happen
classes/multilib: expand PACKAGE_WRITE_DEPS in addition to DEPENDS
classes/staging: capture output of sysroot postinsts into logs
(From OE-Core rev: a9db9a56617459e8f6f6dd466f2e18a7eed5c1e3)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This particularly helps with user management postinsts as otherwise
there's no trace left of what was run, in which order, and what was the output.
Here's an example from the logs:
NOTE: Running postinst /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/lib64-recipe-sysroot/usr/bin/postinst-lib64-base-passwd, output:
b'/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/recipe-sysroot-native/usr/sbin/useradd\nRunning useradd commands...\nNOTE: lib64-ptest-runner: Performing useradd with [--root /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/lib64-recipe-sysroot --system --no-create-home --home / --user-group ptest]\n'
NOTE: Running postinst /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/lib64-recipe-sysroot/usr/bin/postinst-useradd-lib64-ptest-runner, output:
b'/srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/recipe-sysroot-native/usr/sbin/useradd\nRunning useradd commands...\nNOTE: lib64-ptest-runner: Performing useradd with [--root /srv/storage/alex/yocto/build-multilib/tmp/work/x86_64-pokymllib64-linux/lib64-quilt/0.67/lib64-recipe-sysroot --system --no-create-home --home / --user-group ptest]\nNOTE: lib64-ptest-runner: user ptest already exists, not re-creating it\n'
(From OE-Core rev: a4dc96293268804b214a02e08d266205fad428b0)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #14538]
Recipes shouldn't use "virtual/" in RPROVIDES and RDEPENDS. This was
addressed already in recipes in meta-oe and oe-core. Add a test for
this in insane.bbclass to ensure no regressions occur.
(From OE-Core rev: f673d3d239799fb1ab50f4aa5d44187666aa0cd7)
Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we use the build systems native perl, we need to check that all
perl modules required for a build are installed. For a default Fedora 39
install, autoconf-native and libxcrypt both fail to build due to missing
perl modules. After taking careful notes, this commits adds checks for
File::Compare, File::Copy, open, and FindBin.
[YOCTO #14691]
(From OE-Core rev: 3d09e759f838a7ce66fd1fb6e6a6653abce9e3a4)
Signed-off-by: Philip Balister <philip@balister.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Uses the new foreach_runtime_provider_pkgdata() API to look up all
possible runtime providers of a given dependency when resolving
file-rdeps. This allows the check to correctly handle RPROVIDES for
non-virtual dependencies
(From OE-Core rev: 018fa1b7cb5e6a362ebb45b93e52b0909a782ac9)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The vardepsexclude was subtly wrong in that it referenced STATE_MANMACH
when the actual variable name is SSTATE_MANMACH.
(From OE-Core rev: ec5054396f7fafea2a071d2695ae111fc585d6e6)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Converts IPK package generation to use zstd instead of xz. zstd has a
much larger compression/speed tradeoff range allowing users to choose
what suits them best, and fast decompression speeds. It also continues
to support parallel compression as xz did.
A new variable called ZSTD_DEFAULTS is provided to set the defaults for
places that want to use zstd for compression; the zst image conversion
command is also modified to use this.
Finally, in order for this to function properly, opkg must include zstd
support, so it is enabled all the time with no PACKAGECONFIG to turn it
off.
(From OE-Core rev: 1bc3e9bbaa670b6128c74c76b4b5264e60ce3463)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We currently use mkdir -p to create missing parent directories within SSTATE_DIR.
Reading the man page for mkdir mentions that parent directories are created with
the current umask, *not* the mode passed upon the commandline.
We could fix this by setting and resetting the umask but since we already have
decent python code able to do this, move to using that injecting a python function
into the chain of functions already present.
This should help fix the occasional sstate directory creation with the wrong
permissions.
[YOCTO #14385]
(From OE-Core rev: ae642a4b038c6946e6c8aa9778bf09099d938a31)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
