This should print the actual stack trace, for real :)
(From OE-Core rev: 9fbc7d6a2b2388e70a76cb97285e2f60bdc79624)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Autobuilder has been showing things like:
Checking sstate mirror object availability...ERROR: SState: cannot test file://universal/d9/bc/sstate:xz-native:x86_64-linux:5.2.5:r0:x86_64:8:d9bced04b194d5fc8d778eb8a0d674fa7375a42c8c50a9237e6d7672e9e7a00c_deploy_source_date_epoch.tar.zst: TimeoutError('timed out')
ERROR: SState: cannot test file://37/a0/sstate:libgcc-initial:core2-64-poky-linux:11.2.0:r0:core2-64:8:37a0a5aec105a0822df098f15ff2b67d0e7220204742b5d2b1f7958dda6fa5ce_deploy_source_date_epoch.tar.zst: TimeoutError('timed out')
ERROR: SState: cannot test file://universal/11/a4/sstate:libpciaccess-native:x86_64-linux:0.16:r0:x86_64:8:11a4d6c3a2e147ef7dd5f31c0ff2a91271dad49b561d8aa24849115081cf1842_deploy_source_date_epoch.tar.zst: TimeoutError('timed out')
done.
which is not helpful. To find out what really happened and where, the original
traceback is needed too.
(From OE-Core rev: 80a9052221fb2a12e7c652f2d1764101202fdb90)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was prompted by ltp's unreasonably long package_qa times; it has
a massive amount of executables and insane runs objdump for all of
them, serially.
This reduces the time from 4 minutes to 1m20s on my machine.
(From OE-Core rev: fac984b99fdb46949879516cb87153860f402c75)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- build some packages with icecc enabled is not supported
because of the folling that disables the icecc:
DEBUG: while parsing set_icecc_env, unable to handle non-literal command '$ICECC_CC'
- it can be replicated with:
bitbake make && bitbake make -c cleansstate && bitbake make -DD
grep ICECC_CC tmp/log/cooker/qemux86-64/console-latest.log
- bash command substitution backquote deprecated
https://www.gnu.org/software/bash/manual/html_node/Command-Substitution.htmlhttps://mywiki.wooledge.org/BashFAQ/082
(From OE-Core rev: b4e3983f38af5a5484b33de43449ae85b7e4f104)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use double quotes("") rather than single quotes('') for PS1 in
EXPORT_SDK_PS1 which will be exported in SDK environment file. Then it
could set PS1 for SDK env with some variables in host env, such as the
original PS1. The SDK PS1 could be some distinct words plus original PS1
rather than replace the whole original PS1. For example, it could set in
local.conf with
SDK_PS1 = "(oesdk) \${PS1}"
then it just prepends '(oesdk) ' before original PS1 after source the
SDK environment file.
(From OE-Core rev: e13e657dabcabd97f50794e7375777ef5ad883c8)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of the host distributions build the drivers in a way (RPATH/RUNPATH)
that tricks uninative loader into loading pieces of the host libc, if
the same pieces haven't been previously loaded by native binaries. Mixing
the two libc versions leads to failures.
This change ensures that the correct (uninative) versions are always in use.
(From OE-Core rev: 39c10816d5ec9d9c7952d786d7a3f942d25d0c27)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A superfluous package was being created, but it was never used.
(From OE-Core rev: ddfa61f691fc51c1968e8f1ff43f69dccd1632ee)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Per NIST SBoM recommendations, include the Package Supplier field for
all SPDX packages that are created. This field should generally be set
to the person or organization that is performing the build, since they
would be considered the "supplier" of the SPDX packages.
(From OE-Core rev: ca48349501e0ec93dc2448d064e1567fca390bf5)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create directory of the CVE_CHECK_MANIFEST variable before copy to it,
so that the variable can use an arbitrary directory name.
(From OE-Core rev: 9829c16301bf2dce39fa046401a984f112fa0322)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This simplifies the APIs for splitdebuginfo(), splitstaticdebuginfo()
and inject_minidebuginfo()
(From OE-Core rev: 9b00cf049032a9ed7ba3c63d3c37c7414f94edbf)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It simplifies the API to return one dict instead of seven strings.
(From OE-Core rev: 610c790a94591da5c3a6b1db938cd85e2cbdbe5a)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The split_and_strip_files() function is big and hard to follow. This
takes a small step to improve that by splitting out the part that sets
up the variables used to implement the various debug splitting methods
specified by PACKAGE_DEBUG_SPLIT_STYLE.
(From OE-Core rev: e414c6ec8563f9c448f1421f33870a908a0353b4)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the hash equivalence fix, we need to bump the sstate and hash equivalence
version numbers to ensure older task hashes aren't matched into the new namespace.
(From OE-Core rev: 3f229267e0588c747265e849ee19724033cc6a80)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Target build output, stored in do_populate_sysroot or do_package can depend
not only upon direct dependencies but also indirect ones. A good example is
linux-libc-headers. The toolchain depends on this but most target recipes do
not. There are some headers which are not used by the toolchain build and do
not change the toolchain task output, hence the task hashes can change without
changing the sysroot output of that recipe yet they can influence others.
A specific example is rtc.h which can change rtcwake.c in util-linux but is not
used in the glibc or gcc build. To account for this, we need to account for the
populate_sysroot hashes in the task output hashes.
(From OE-Core rev: 1cf62882bbac543960e4815d117ffce0e53bda07)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
g-i internally processes the output with regexes, and seems
happy with what objdump is printing. It only needs to resolve
the library name as passed to the linker to the library file name.
Also recursive resolution (that ldd is doing and objdump is not)
is not necessary.
(From OE-Core rev: 767e0880d4d729e659e859dd99c1cdb084b8ba51)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This continues where commit676757f "sstate: fix touching files inside
pseudo" and commit 29fc8599 "sstate: another fix for touching files
inside pseudo" left off.
The previous changes switched from trying to check if the sstate file is
writable before touching it, to always touching the sstate file and
ignoring any errors. However, if the sstate file is actually a symbolic
link that links to nothing, this would actually result in an empty
sstate file being created. And this in turn leads to that future
setscene tasks will fail when they try to unpack the empty file.
Change the code so that if an sstate file linking to nothing already
exists, it is overwritten with the new sstate file. Also change it so
that the temporary file that is used is always removed, even if ln
fails to link the sstate file to it.
(From OE-Core rev: 552197a0c4c9f75a9177c00b197ea91296ed9fc4)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There may be a case where we want to build an initramfs image
that doesnt inherit the same DISTRO_FEATURES (or others) from
the main image being built.
For example we may want our initramfs not to inherit a certain
conf or feature, say we want to use musl for a smaller footprint,
but if we are using TCLIBC=glibc for our DISTRO (and inherently
our main image), the initramfs image would inherit that conf and
be forced to use glibc, growing in size as a side effect, currently
avoiding this is not supported.
Allow the kernel class to create a multiconfig dependency
(mcdepends) vs depends for do_bundle_initramfs and define
our INITRAMFS_IMAGE from a separate multiconfig via two
new variables:
INITRAMFS_MULTICONFIG and INITRAMFS_DEPLOY_DIR_IMAGE which
define the multiconfig where the initramfs image should be
coming from and its deploy directory respectively, these two
keep a default definition which preserves current behavior
(do_bundle_initramfs uses depends).
Example usage:
- Create and use multiconfig initramfscfg.conf and set
TCLIBC=musl there, along with its TMPDIR.
- Add the following to our DISTRO.conf:
INITRAMFS_MULTICONFIG = "initramfscfg"
and set
INITRAMFS_DEPLOY_DIR_IMAGE to the DEPLOY_DIR_IMAGE of the
initramfscfg multiconfig (hence our main kernel will be
able to grab it from there and bundle it).
This will result in our musl based initramfs bundled in our
main kernel and our main image to be glibc based.
(From OE-Core rev: 2d317b2685211f1b0d102705a63c0000df96f45f)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids various sstate fetch errors from weird silent failures in the sstate
archive testing code caused by lack of srcrev support in the crate fetcher.
(From OE-Core rev: 8205668a339946974b4460cf1ca7e6e7a645479b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bb.fatal() case where sstate failed to find/use an archive in setcene tasks
is suboptimal. Bitbakes handling of setscene tasks will be to warn but the fatal
will turn this into an error, despite the real task being rerun.
In these failure cases other messages would usually have been printed so turn
this into a warning and raise a handled exception status so that bitbake knows
to fail the task but not print more messages.
(From OE-Core rev: 7875baeca7a3ed216b2442fb8771e51efbfa5a4f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Errors like
SState: cannot test file://universal/5f/10/sstate:m4-native:x86_64-linux:1.4.19:r0:x86_64:7:5f108b175274798b17950b5ee686bf61445e90fb204d7f795e5b879b5603e88b_deploy_source_date_epoch.tar.zst: 'GIT_SSL_CAINFO'
aren't useful, try and improve on this.
(From OE-Core rev: 1701d40536db098adb26508799bae13733f1a6fd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default rpm file color is 2 that Elf64 is preferred according to
OE-Core commit 56fa74497393a10f751d01c600c1936761e00294. For 32-bit
multilib image such as lib32-core-image-sato, Elf32 should be preferred.
Set it in multilib.bbclass.
(From OE-Core rev: 195f4a648251e53ed3070e060ad3e3f3832793e3)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Native task outputs are directly run on the build system during the build
after being built. Even if the output of a native recipe doesn't change, a
change in one of its dependencies may cause a change in the output it
generates (e.g. rpm output depends on the output of its dependent zstd
library).
This can cause poor interactions with hash equivalence, since this
recipe's output-changing dependency is "hidden" and downstream tasks only
see that this recipe has the same outhash and therefore is equivalent.
This can result in different output in different cases and issues with
reproducible builds in parcular (e.g. rpm compression changes for the same
content).
To resolve this, unhide the output-changing dependency by adding it's
unihash to this tasks outhash calculation. Unfortunately, we don't know
specifically know which dependencies are output-changing, so we have to
add all of them.
[YOCTO #14685]
(From OE-Core rev: d6c7b9f4f0e61fa6546d3644e27abe3e96f597e2)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is done by default when tar is run by the superuser, but for native
recipes the corresponding task is not run as root under pseudo, so pass
the flag explicitly.
Suggested-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(From OE-Core rev: 32f01e57e1e915e8121609fb48b0b0254e625957)
Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These were added nearly a decade ago but there are no users in OE-Core. I
checked with the likely users and they seem to have no current usage either.
Therefore remove them.
If needed for some prebuilt library somewhere, they could be implemented
in the layer using them instead but I doubt these are in use any longer.
(From OE-Core rev: 95e196babc3c18dcf0aedfb03e85493c8ae54700)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the kernel source is stored in work-shared, we need to add it
to the search path so the kernel and kernel-modules source code can
be found correctly.
(From OE-Core rev: 93a94890b9f2679945e96973937b209aa12302b7)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extend runstrip() to accept additional argument to enable
sharing it with the kernel do_strip() so that
KERNEL_IMAGE_STRIP_EXTRA_SECTIONS can be passed.
Since is_elf() understands kernel modules there is no need to keep a
seperate list for kernmodules or hardcode the values to runstrip.
(From OE-Core rev: e09a8fa931fe617afc05bd5e00dca5dd3fe386e8)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Re-use the runstrip() code from oe.packaging, for the kernel
stripping process. Since runstrip() is python the kernel do_strip()
need to be converted to Python also. The stripped kernel image
will be used for deployment in do_deploy(). This will allow the
package.bbclass to split an unstripped kernel binary for the debug
information and extended packagedata. The extended package data is
used by create-spdx.
(From OE-Core rev: e8d9caede5f08154ca615fdaba676b7a4ae05b01)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the newly added network task flag against tasks where network
access is expected. This is do_fetch, do_checkuri, do_testimage, do_testsdk
and do_testsdkext.
We can't disable networking in sstate tasks due to sstate downloads and
also so we can report hash equivalence to the server so network access
is enabled in sstate tasks.
Access within build-appliance do_image is also allowed due to the use
of pip, this is a poor example made rather obvious now and needs to be reworked.
Network access anywhere else in any other task isn't allowed.
(From OE-Core rev: 7ce1e88a3ad85bbb925bb9f7167dc0a5fd1c27f4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change a comment to reference our custom changes to the
sysconfig module, as dnf no longer uses distutils.
[YOCTO #14610]
(From OE-Core rev: b4e351c67cd6af9cabac70cddb67129ca2590222)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the comment it says we patch distutils modules, but we now are
patching sysconfig and not using distutils.
[YOCTO #14610]
(From OE-Core rev: 4f196b26d80305366d06ad51f44dd26ce81bea16)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Refactor to use bb.utils.vercmp_string_op() instead of StrictVersion
from distutils
[YOCTO #14610]
(From OE-Core rev: d4d57588eb7fcd25caf51450e0d19fc58e35a19d)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add setuptools3-base.bbclass as a re-usable starting point similar to
what used to be distutils-common-base.bbclass and disutils3-base.bbclass.
We no longer need to support python2, so no need for a
setuptools-common-base.bbclass.
Refactor setuptools3.bbclass to use setuptools3-base.bbclass instead of
the distulis*.bbclasses.
(From OE-Core rev: ca73393a36c4144662ea8570f904154188e9815a)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
distutils has been deprecated in Python 3.10 and will be removed in
Python 3.12 (predicted release date October 2023).
Add a warning now that recommends using the roughly equivalent
setuptools-* classes.
[YOCTO #14610]
(From OE-Core rev: 54b455049ee94c01c78b31b6c744c8e32b5b7737)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If IMAGE_LINK_NAME is set to "" (supported in other classes) then skip
creating symlinks for the spdx manifest files.
(From OE-Core rev: c2d5e2dbbba78d8d7036f6418360b2e84bcdc169)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We met a problem that core-image-tiny-initramfs's SDK cannot be
installed. The error message is like below.
tar: ./sysroots/core2-64-poky-linux/dev/console: Cannot mknod: Operation not permitted
In fact, the '/dev' direcotry is not needed by SDK. So remove it.
This patches uses a variable, SDK_PRUNE_SYSROOT_DIRS, to hold useless dir entries
so that it could be extended. For example, '/usr/bin' could be added if wanted.
(From OE-Core rev: 9154f71c7267e9731156c1dfd57397103e9e6a2b)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
this should prevent running into the very rare error
sqlite3.OperationalError: attempt to write a readonly database
As highlighted by https://www.sqlite.org/faq.html#q5
it is likely that the adapter won't allow use multiple exec calls
at the same time.
So it's best to prevent multiple accesses at a time, by reusing
the already in place CVE_CHECK_DB_FILE_LOCK
YOCTO #14110
(From OE-Core rev: 677f5741bd265be49d4a5bb933b3e8d8c4eec653)
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The qemu class declares functions which are architecture specific. If a user such
as meson is used in an allarch recipe, this leads to sstate which is machine
specific. To fix this, remove the architecture specific part, since there are no
binaries in allarch classes, this change shouldn't break anything.
(From OE-Core rev: 049879ba842d89f268b8e3a4e26410d13bc54158)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch was merged on the basis that it wasn't needed with recent versions
of qemu. That isn't true and has been showen to cause failures for aarch64 on
centos7 hosts. Revert the patch as we'll need a different solution.
This reverts commit 94b371e1c9.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix a couple of function name typos copy and pasted between the classes.
(From OE-Core rev: f99b98341cfb849680461cfa2992f854eebad5df)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Go has its own system for creating temporary build
sub-directories with randomized names, and setting
up debug-prefix-map on the fly to prevent those
directories leaking into target binaries. OE's own
settings were clashing with it, so this change
carefully avoids the two stepping on each other.
Additionally, the top level build directory cannot
be named 'go-something'.
(From OE-Core rev: 9985b17a30bb9b9f1bc82a44662687db5cead66e)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
go writes build-specific ids into binaries it produces
and has a custom system for calculating them from
file hashes, environment variables and other inputs
(not that dissimilar to sstate cache, actually). This can
go wrong :) in various ways (for purposes of reproducibility
in particular), so this enables useful logs to see what
happens and why.
(From OE-Core rev: a587be1d18fc55fe57d1aa5aa7c9e26af887109e)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The dependencies for do_build became a little too minimal after the
removal of recrdeptask since "bitbake go" would not package go-runtime
despite it being in DEPENDS and the resulting package having a dependency
on it.
A reasonable compromise is probably rdeptask instead of recrdeptask
which is a lot lower overhead but makes the build target more useful
and importantly, lets world builds do what you'd expect them to.
(From OE-Core rev: f3d02c328f3f182340528d11c7b10454e3f6a54b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding the dbg package allows the package bbclass to parse the
debug information which can then be used by the create_spdx bbclass
(From OE-Core rev: b35b68e4ec4a82ada20ab861d29f96cdcb21dec2)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow .bin suffix to be removed (or changed) in the various artifact
filenames. Removing this extension is useful when trying to remove
symlinks and present only unversioned image files (especially for the
FIT image).
(From OE-Core rev: cbecc3cf06eb7359fedf3c6af281cc72178cad18)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If IMAGE_LINK_NAME is empty (supported everywhere else) then do not
create the symlink for the .qemuboot file.
(From OE-Core rev: 9b19845c0506949ca61965fb92fb3f337062f377)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow setting any of the *_LINK_NAME variables to empty string in order
to disable creating symlinks for kernel artifacts, as you can already
for filesystem images with IMAGE_LINK_NAME. Additionally, for the image
type named symlinks, add a KERNEL_IMAGETYPE_SYMLINK boolean variable
which you can set to 0 to disable those symlinks as well.
(From OE-Core rev: d7341f1f22c32ff6cc95d7127f26f87d7fc9c6bd)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
