kernel.org now has a mirror of the downloads.yoctoproject.org sources
archive so include this in our mirrors list.
(From OE-Core rev: 97b819750b985f080c0e586aba0312bc0f62cdb9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f602b6c2046bbc52a95dcc68a754f1cbb2db6761)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At the last nas outage, we realised that we don't have good mirrors of the
uninative tarball if our main system can't be accessed. kernel.org mirrors
some Yocto Project data so we've ensured uninative is there. Add the appropriate
mirror url to make use of that.
(From OE-Core rev: e716c371992ea567171fb7f7bb8691ec92dfc71b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1833cb0c5841afafb468b963b74b63366b09a134)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add branch name explicitly to SRC_URI where it's not defined and switch
to using https protocol for Github projects.
The change was made using convert-srcuri script in scripts/contrib.
(From OE-Core rev: ab781d4e3fa7425d96ea770ddfd0f01f62018c5b)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch is a fixup for 676757f "sstate: fix touching files inside pseudo"
running the 'id' command inside the sstate_unpack_package
function shows that this funcion run inside the pseudo:
uid=0(root) gid=0(root) groups=0(root)
The check for [ -w ${SSTATE_PKG} ] and [ -O ${SSTATE_PKG}.siginfo ]
will always return true and the touch can fail when the real user
don't have permission or in readonly filesystem.
As the documentation refers:
- the file test operator "-w" check if the file has write permission
(for the user running the test).
- the file test operator "-O" check if you are owner of file
We can avoid this test running the touch and mask any return errors
that we have.
(From OE-Core rev: 29fc85997ade490ae46ffca37ef8e1a56957c876)
(From OE-Core rev: a5240d78f287ff5f3c5831af69e6e0ca3feaf769)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5b9210d66c)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, when shortening sstate filenames, the reserved
characters for .siginfo were not considered, when siginfo=False,
resulting in differently shortend filenames for the sstate and siginfo
files. With this change, the filenames of the truncated sstate and
siginfo files have the same basename, just as is already the case for
untruncated filenames.
Making sure that the .siginfo files always have the filename of the
corresponding sstate file plus its .siginfo suffix, also when being
truncated, makes it easier to manage the sstate cache and an sstate
mirror outside of Bitbake/Yocto.
(From OE-Core rev: 186062278e2d1e129815c4f8cf3d9003f3ac2809)
Signed-off-by: Manuel Leonhardt <mleonhardt@arri.de>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c2e0e43b7123cf5149833e0072c8edaea3629112)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
running the 'id' command inside the sstate_create_package
function shows that this funcion run inside the pseudo:
uid=0(root) gid=0(root) groups=0(root)
The check for touch files [ ! -w ${SSTATE_PKG} ]
will always return true and the touch can fail
when the real user don't have permission or
in readonly filesystem.
As the documentation refers, the file test operator "-w"
check if the file has write permission (for the user running the test).
We can avoid this test running the touch and mask any return errors
that we have.
(From OE-Core rev: 8acd5626b5b89fb48d5d7f0e4eee263793ba383f)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f6e7445c94443544e92fda97a017ce93393c5f84)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
SOURCE_DATE_EPOCH can be expanded early in the parsing process before
the class extensions are applied. This can mean the directory pointed
to for the SDE can be incorrect until later in parsing. Cache the file
name in the cached value and allow it to dynamically update.
This isn't ideal but avoding expansion of the variable likely isn't
possible and I'm not sure how else to handle this. This works around
the issue until a better solution can be found.
(From OE-Core rev: 7e3752dcce2236a4d1ad046b1df2edf8ee5619b4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 44dc97cd1223e4d2b635669627ec5f796838d42d)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With glibc 2.34, the libraries were renamed. Tweak the glob to support both
as this is needed for newer uninative versions.
[RP: tweak commit message]
(From OE-Core rev: f60da44d32a579f5ba5c95dd60f80d624de39ce4)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 98248306e4b5f023e96375293b60524574ebb686)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are situations when the user have the 'patchdir' defined
as a parameter on SRC_URI. However he doesn't know that with this
the patch is applied relatively to the receipe source dir 'S'.
- When user have 'patchdir' defined check if this directory exist.
- If the patch fails show addition info to the user:
- Import: show the striplevel
- Resolver: show the expanded 'patchdir' to the user.
The next example is from opencv in meta-oe layer, here the
patch is applied on the target directory ${WORKDIR}/git/contrib.
S = "${WORKDIR}/git"
SRCREV_FORMAT = "opencv_contrib"
SRC_URI = "git://github.com/opencv/opencv.git;name=opencv \
git://github.com/opencv/opencv_contrib.git;destsuffix=contrib;name=contrib \
file://0001-sfm-link-with-Glog_LIBS.patch;patchdir=../contrib \
"
* When the patch fail there are no message that indicates the real reason.
patchdir=../no-found-on-file-system
ERROR: opencv-4.5.2-r0 do_patch: Command Error: 'quilt --quiltrc /build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0 Output:
stdout: Applying patch 0001-sfm-link-with-Glog_LIBS.patch
can't find file to patch at input line 37
Perhaps you used the wrong -p or --strip option?
* The check of the patchdir will add a new fatal error
when the user specifies a wrong path than don't exist.
patchdir=../no-found-on-file-system
ERROR: opencv-4.5.2-r0 do_patch: Target directory '/build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/git/../no-found-on-file-system' not found, patchdir '../no-found-on-file-system' is incorrect in patch file '0001-sfm-link-with-Glog_LIBS.patch'
* When we can't aplly the patch but the patchdir exist,
show the expanded patchdir on fatal error.
patchdir=../git
ERROR: opencv-4.5.2-r0 do_patch: Applying patch '0001-sfm-link-with-Glog_LIBS.patch' on target directory '/build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/git/../git'
Command Error: 'quilt --quiltrc /build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0 Output:
stdout: Applying patch 0001-sfm-link-with-Glog_LIBS.patch
can't find file to patch at input line 37
Perhaps you used the wrong -p or --strip option?
(From OE-Core rev: 0b57a6f07a2d69f809120f1bc1988738c35c095b)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit c44bc7c0fb8b7c2e44dd93607a3bfd9733e1df80)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When reproducible builds are enabled and externalsrc is in use, the
source date epoch function is added. The conditions on the conditional
code removing the unpack task need to match the deltask function, else
the source date epoch function can end up running twice and the functions
can race with each other causing build failures or corruption.
(From OE-Core rev: 295947ac48a8c14eba8f2bedcfb7b59121a1da5f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e82095c02881410035ca23dc12692f074d8ed39b)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Externalsrc removes do_fetch, do_unpack, and do_patch. The system normally
discovers the correct reproducible date as a postfuncs of do_unpack, so this
date is never found, so it falls back to the default epoch.
Instead we can move the discovery function to a prefuncs on the epoch
deploy task. This task will run before do_configure, and since the source
is already available can run safely at anytime.
(From OE-Core rev: bc8448b55f06c75ec129fc32cee75ef945a49d3e)
Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0b7dd711a54e92ce54abe99f59fc67e683d52dfe)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the pkgconfig pclist files contain full paths which are build
host specific and the order of entries is not deterministic.
Fix both these issues so the files are deterministic.
(From OE-Core rev: 59655c76c395ffaae0342458a6a2ce0494b8e1a6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e422e29bca4af3ab4073e04490f38b05cd7c38c0)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes it possible to name files starting with a hyphen in the work
directory. Without this change rm will fail due to an unexpected option
being passed.
(From OE-Core rev: 88d67525dc70c21b5a29dbcef65d12a1c780d717)
Signed-off-by: William A. Kennington III <wak@google.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5f1a63e0de4921ef970114a16d0827fcddcdaa0e)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When signing a package, we were resolving the `PKGV` version using the
original data store (`d`). However, since that store does not have the
package name in its `OVERRIDES` list the wrong version can be returned in
recipes that produce multiple packages. One such example would be
`external-arm-toolchain.bb`.
The above issue is fixed by using `localdata` instead of `d` when
resolving the needed variables.
(From OE-Core rev: c4724cd652b5a3adc398652b48e16d1fd1a90e89)
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@verkada.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2b5f01cf5a92a873ac4c3f0ba0584cab2cc05714)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When fetching sstate, SRCPV can be expanded withuin PV which breaks the
sstate fetcher as other variables like SRC_URI are changed by the fetcher
code. Expand the variable before doing this to avoid those problems and
hence avoid setscene task failures with remote sstate storage in git
recipes.
(From OE-Core rev: 5232aa1c46380e7197098bed39fc6d5b8456caa7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0d9d7acebe0be42d4cf8a30ecbbbbcb2dfd16a4b)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add SDKPATHINSTALL which is used as the default install location of the SDK
instead of SDKPATH. This means the default install path isn't encoded into
every SDK binary, meaning if a date is used there the entire SDK doesn't
have to rebuild. Most distros can switch to only customise SDKPATHINSTALL
meaning more sstate reuse too.
[YOCTO #14100]
(From OE-Core rev: 1b4309ee196c45661ddb79d8529517a8777a8d53)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit bc4ee5453560dcefc4a4ecc5657df5cc1666e153)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the kernel sources are not fetched via git, a local git repository is
created in do_kernel_checkout. In this case we know that there will be
no remote branches and we will already be on the correct branch (since
only one branch will exist). So we can simplify things by skipping these
steps.
This also removes the assumption that the default git branch name will
be "master". Prior to this change, the final git checkout command in
do_kernel_checkout could fail if a local git repo was created and the
user had changed init.defaultBranch in their gitconfig.
(From OE-Core rev: 0c8d643d7b474a845588ad7dbf86f85a9aad5c49)
Signed-off-by: Paul Barker <paul@pbarker.dev>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit af2a9c92d4498492ca23388c7b4bbed48abdc4d7)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the mkfs.btrfs generates large images with a lot of wasted
space. This happens since OE-core updated btrfs-tools from 4.13.3 to
4.15.1 in commit 94b645aa77 ("btrfs-tools: update to 4.15.1") .
Note in mkfs.btrfs(8) manpage section -r says the following:
"
-r|--rootdir <rootdir>
...
Note This option may enlarge the image or file to ensure
it’s big enough to contain the files from rootdir. Since
version 4.14.1 the filesystem size is not minimized. Please
see option --shrink if you need that functionality.
--shrink
Shrink the filesystem to its minimal size, only works with
--rootdir option.
...
Note prior to version 4.14.1, the shrinking was done
automatically.
"
Add the --shrink option to EXTRA_IMAGECMD_btrfs to reinstate the
original behavior and un-waste the space.
(From OE-Core rev: 6a72c7c688c7985d19f0387121b3a77639ba6f60)
Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c4a99d36967302c176b62fad840b5e79486ea356)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch enables building image based SDKs for multi-libbed images
e.g. lib32-core-image-minimal and so on. Change the path to nativesdk
tools to use recipe-sysroot since thats where the nativesdk components
are installed and it will need access to qemu wrappers during build for
processing intercepts
[YOCTO #14444]
(From OE-Core rev: 3d18d1924ff0dcf8788ee856e3f056e77db8f920)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6196a785eababb040ee1dee9f33cb6d6dad77eef)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
utils.bbclass contains create_cmdline_wrapper() function that
creates wrapper script with additional arguments for any passed
"$cmd" command, and uses several calls to "dirname".
Because "dirname" is an external command, in cases of lots of
calls to wrapped "$cmd", each call of "dirname" will incur
significant overhead.
There are three same calls to "dirname": one for saving it`s
output to "realdir" variable, and other two in "exec" command.
So last two "dirname" calls can be replaced with cached value
from "realdir" variable.
(From OE-Core rev: 33ebb967a9489384c2407e7e3211f8bfca7c3916)
Signed-off-by: Oleksandr Popovych <opopovyc@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4b9cf2c80fd14386e0b88a2e6c40a9fa3f1ae0f7)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since shell_trap_code in build.py sets /bin/sh as the interpreter
we will die a silent death if our environment contains things like
"export -f bodilyfunction" and /bin/sh is really /bin/dash.
Fixes this for the case of devshell.
(From OE-Core rev: 48fe148e5ca344d2a6bad444ef48d9c5ea414448)
Signed-off-by: Joe Slater <joe.slater@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 23d296b3567aa31bad7b2a8558d4bd3e4505843b)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The commit dd555537fc
'sstate.bbclass: fix errors about read-only sstate mirrors'
adds an additional exception handler to silently mask read
only rootfs errors thrown during the touch.
The exception handler checks the error type with the python module errno
but this module needs to be imported as it don't exist.
Example of the error:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:sstate_task_postfunc(d)
0003:
File: '/home/builder/src/base/poky/meta/classes/sstate.bbclass', lineno: 778, function: sstate_task_postfunc
0774:
0775: omask = os.umask(0o002)
0776: if omask != 0o002:
0777: bb.note("Using umask 0o002 (not %0o) for sstate packaging" % omask)
*** 0778: sstate_package(shared_state, d)
0779: os.umask(omask)
0780:
0781: sstateinst = d.getVar("SSTATE_INSTDIR")
0782: d.setVar('SSTATE_FIXMEDIR', shared_state['fixmedir'])
File: '/home/builder/src/base/poky/meta/classes/sstate.bbclass', lineno: 708, function: sstate_package
0704: except PermissionError:
0705: pass
0706: except OSError as e:
0707: # Handle read-only file systems gracefully
*** 0708: if e.errno != errno.EROFS:
0709: raise e
0710:
0711: return
0712:
Exception: NameError: name 'errno' is not defined
(From OE-Core rev: 2e5cf13aa46f25d8976ece55fcb4129bfd75659f)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 15f30ad144fbe25e9a5e71bc7e42e746d2039992)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This implements various fixes in comments in cve-check.bbclass
In particular, the "whitlisted" typo is important as the "whitelisted"
word is going to be replaced in a near future.
(From OE-Core rev: 80941c3ae3c2ee8ca48462f20ef5187b26a35247)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5eecd2bf942254d08c252388594e5ec7ae330f45)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Along with the other license exclusions, we need to exclude the
top level COPYING.MIT file else when:
COPY_LIC_DIRS = "1"
COPY_LIC_MANIFEST = "1"
is set, we see eSDK failures from a pseudo abort.
[YOCTO #14366]
(From OE-Core rev: 632220213cb576941c6d4961c79286a31e9e5f42)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3eb580843de3f055e42fcce60b0f15c4190c0542)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When adding a layer which changed SSTATE_EXCLUDEDEPS_SYSROOT, the state
hashes were changing when they should not. This was caused by wider use
of setscene_depvalid which means the dependency on the variable was seen
when it was previously not.
Exclude the variable since this should be be included in the hashes.
(From OE-Core rev: 084d81058682e0c9074203c907f80d6eabe63534)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 09725a29365c69ccbd603fe3a1de72189f26d5ac)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that pseudo-native always includes statx support and we have sanity
checks on pseudo-native to ensure it always contains a minimum feature set,
we no longer need to mark pseudo-native as distro specific. This fixes
eSDK build problems.
[YOCTO #14428]
(From OE-Core rev: 4c7c9326da92ea93fdef1ad804685c21a197f1d4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3279e43fcb469edb63c7c4eb60fdc565d5751f9d)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
linux-yocto-dev tracks the latest mainline kernel, and uses
standard/* for that support.
Archived -dev versions are under v<kernel version>/standard/base.
This policy works, except that a released branch will still follow
the new kernel versions, causing potential breakage with newer
kernels than are supported in that release.
Rather than lock the SRCREVs and update branches in old releases,
we can preserve the AUTOREV nature of -dev, and allow them to
switch automatically to the archived branch based on the LINUX_VERSION
in the -dev recipe (which is unchanged in the release branch).
This is consistent with the other branch switching done for the
kernels and with the -dev workflow.
(From OE-Core rev: b3228b4140575bfc20cb4e5bade0f3737d374e86)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 84e14b6116a7d1e52051c5c80be2d8e3db67c2d7)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The sstate code tries to be careful about racing around directory creation.
In particular, the copyhardlinktree code creates the directory tree first
allowing for "already exists" errors and ignoring them, then hardlinks the
files in.
Unfortunately the sstate removal code can race against this since it
will try and remove empty directories. If there is some bad timing,
a newly created directory can be removed before it was populated, leading
to build failures.
We could try and add locking but this would damage performance, we've been
there before. It is also unclear where to actually place locks just based on
the contents of a manifest file which may cover multiple sstate install
locations for a given task.
Instead, lets disable directory removal in the problematic "shared" core
path. This could result in a few more empty directories being left on disk
but those should be harmless and better than locking hurting performance
or rare build races.
[YOCTO #13999]
[YOCTO #14379]
(From OE-Core rev: 3bea0da687d28c80f0b63e2ad81573a3601bb482)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4f94d9296394bc7ce241439f00df86eb5912875f)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When packaging is disabled using the nopackages class, ensure we don't
add to PACKAGES. This fixes builds where we have an unpackaged kernel
alongside a packaged kernel.
(From OE-Core rev: 13a1e550b0252d6f3a1164d39614f8ac847e1095)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d6b114cf5a9b22688c0b59a3afc46a07450d87b5)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a read-only sstate mirror is used in conjunction with hash equiv,
then OSError will be raised when an sstate-cache hit is achieved.
This is because sstate_task_postfunc will try to "touch" the symlinks
that point to the read-only sstate mirror when sstate_report_unihash
has changed SSTATE_PKG.
This commit adds an additional exception handler to silently mask read
only rootfs errors thrown during the touch.
The fix is also duplicated to sstate_eventhandler as the code is very
similar but it may not be needed there.
Example of the error:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:sstate_task_postfunc(d)
0003:
File: '/poky/meta/classes/sstate.bbclass', lineno: 774, function: sstate_task_postfunc
0770:
0771: omask = os.umask(0o002)
0772: if omask != 0o002:
0773: bb.note("Using umask 0o002 (not %0o) for sstate packaging" % omask)
*** 0774: sstate_package(shared_state, d)
0775: os.umask(omask)
0776:
0777: sstateinst = d.getVar("SSTATE_INSTDIR")
0778: d.setVar('SSTATE_FIXMEDIR', shared_state['fixmedir'])
File: '/poky/meta/classes/sstate.bbclass', lineno: 703, function: sstate_package
0699: if not os.path.exists(siginfo):
0700: bb.siggen.dump_this_task(siginfo, d)
0701: else:
0702: try:
*** 0703: os.utime(siginfo, None)
0704: except PermissionError:
0705: pass
0706:
0707: return
Exception: OSError: [Errno 30] Read-only file system
(From OE-Core rev: 1d86f3c8f5d47e1b3e3ca759076e24710ee7dc02)
Signed-off-by: Michael Ho <Michael.Ho@bmw.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 244b3be0358a66e0cca4016fe26144e3d7323390)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When packaging is disabled using the nopackages class, ensure we don't
add to PACKAGES. This fixes builds where we have an unpackaged kernel
alongside a packaged kernel.
(From OE-Core rev: 8068eaccf89779d374791837215895432e45e85d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2522daf22e2c27dd9c7926feda0345978217c6c3)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add SSTATETASKS to package_prepare_pkgdata[vardepsexclude] since otherwise
the task hashes vary depending upon which packaging backends are enabled
and likely other changes which add/remove unrelated sstate tasks.
(From OE-Core rev: 5663bd42a27d25be9843d9f3611ea476b553930b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4011d31d4372639fd72ee0eefae210bf59c90d13)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A "install -d" command is missing when dealing with
${UBOOT_NODTB_BINARY} in install_spl_helper function, this can lead
to invalid install error saying:
| install: cannot create .../usr/share/u-boot-nodtb-2021.04-r0.bin': No such file or directory
Let's drop all "install -d" and replace them with "install -Dm" in
install_helper/install_spl_helper functions.
(From OE-Core rev: 0fc4e10250e69a67c5250ce48bfc632d90a34097)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit db2b1da511ea3d4daef136a8b1d85b7040a46632)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The routine do_sizecheck() was historically needed by legacy
devices with limited flash memory.
The lowest extreme is probably with Zaurus collie having exactly
1024*1024 = 1048576 bytes for the kernel partition.
In the years the KERNEL_IMAGE_MAXSIZE has been converted to kilobytes
thus rounded so we have now KERNEL_IMAGE_MAXSIZE_collie = "1024".
The effect is that now the check fails because we hit curiously this
| WARNING: This kernel zImage (size=1024(K) > 1024(K)) is too big for...
even though zImage is 1047288 bytes (kernel + kexecboot-klibc-initramfs).
Fix this case using test -gt (greater) instead of -ge (greater or equal).
(From OE-Core rev: 61feebd88960eb4e074a80a0e45b36a7a1db869c)
Signed-off-by: Andrea Adami <andrea.adami@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 254ca956d63b4ce6aa294213b60bb943f9f3a9e6)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For dependencies such as "${PN}-foo", when modifying them for native
recipes, it is expected that they become "${BPN}-foo-native" rather
than "${BPN}-native-foo-native". This was previously done by removing
all occurences of "-native" from the dependency before adding
"-native" at the end. However, this fails for a recipe such as
"crate-native-tls" that happens to contain the string "-native" in the
middle of the name. Solve this by simply replacing ${PN} with ${BPN}
in the name instead before adding "-native" at the end
Also simplify adding "-native" to the end of names the recipe provides.
In this case it is not necessary to replace ${PN} with ${BPN} as the
recipes are expected to use ${BPN}-foo in the first place.
(From OE-Core rev: a71d923d0f0e966b17e915a86b5ad7222a947122)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit edaf8ff278fc96b122c4fc3266b63856e3350f4c)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
do_bootimg expects IMGDEPLOYDIR to exist, since it stores its artifacts
there. Therefore, do_bootimg should run after do_rootfs because
IMGDEPLOYDIR is created before do_rootfs runs since IMGDEPLOYDIR is
contained in do_rootfs' [cleandirs] varflag.
When do_bootimg depends on ${PN}:do_image_${LIVE_ROOTFS_TYPE},
do_bootimg is correctly ordered after do_rootfs because
do_image_${FSTYPE} tasks are added after do_image and do_image itself is
added after do_rootfs.
However, when do_bootimg doesn't depend on
${PN}:do_image_${LIVE_ROOTFS_TYPE}
(introduced by: 96f47c39f1d17f073243913d524bde84add41d8f), do_bootimg
can run before do_rootfs, thus before IMGDEPLOYDIR is created. To
avoid this situation, do_bootimg is now explicitly ordered after
do_rootfs.
(From OE-Core rev: 7b10309fa169834483589e999432caf414225b3e)
Signed-off-by: Guillaume Champagne <champagne.guillaume.c@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 73c21db8e54002b300ba4972cb49c0577acc5406)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If an existing source date epoch file was found during do_unpack, it was
deleted and a new one would be written in its place. This causes a race
with check-before-use code in get_source_date_epoch_value. Resolve the
problem by making do_unpack write the new source date epoch to a
temporary file, then do an atomic rename to ensure it's always present,
and change the check-before-use code to use a EAFP exception instead of
checking for file existence.
[YOCTO #14384]
(From OE-Core rev: 8b2fd4e5e0841b81b4f709b061b655e2266dd4da)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0b5e3b33187bf78a2d62cc886463e4b27d6bd228)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default RPM uses the number of cores as the number of threads to use,
which can result in quite antisocial memory usage.
As we control the macros for compression anyway, we can pass XZ_THREADS
to limit the number of threads if needed.
(From OE-Core rev: 959e1faa911ee67d5d84a57b932135b76cac6a53)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
What key is used to sign sstate artefacts should not affect the hash of
the object, otherwise everyone would need to use the same signing key.
(From OE-Core rev: 01a9358abe821c1da06c3243ccbcc93348042937)
Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running:
execute_pre_post_process(d, d.getVar(ROOTFS_POSTPROCESS_COMMAND))
rootfs_update_timestamp is run, which assumes that rootfs/${sysconfdir}
is already created (usually done through the do_rootfs task on linux).
This causes the build to fail if ${sysconfdir} does not exist.
This may be overlooked if debug-tweaks is enabled since some other
commands are added, one of which creates the required path
(see postinst_enable_logging).
See [1] for more info:
[1] https://github.com/aehs29/meta-freertos/issues/4
(From OE-Core rev: 179a912bf10ba02448e8d603043c454ca678ac60)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Under certain build patterns, warnings about missing manifests can appear. These
are real issues where the manifest was removed and shouldn't have been.
Martin Jansa was able to find a reproducer of:
MACHINE=qemux86 bitbake zlib-native
echo 'PR = "r1"' >> meta/recipes-core/zlib/zlib_1.2.11.bb
MACHINE=qemux86-64 bitbake zlib-native
MACHINE=qemux86 bitbake zlib-native
<the zlib-native manifest is now removed along with the sysroot-components contents>
The code maintains a per machine list of stamps but a per PACAGE_ARCH list of
stamp/manifest/workdir mappings. The latter is only appended to for speed with
the assumption that once stamps are gone, the code wouldn't trigger.
The code only ever appends to the mapping list (for speed/efficency under lock)
meaning that multiple entries can result where the stamp/workdir differs due to
version changes but the manifest remains the same.
By switching MACHINE part way through the build, the older stamp is referenced
and the manifest is incorrectly removed as it matches an now obsolete entry in
the mapping file.
There are two possible fixes, one is to rewrite the mapping file every time
which means adding regexs, iterating and generally complicating that code. The
second option is to only use the last mapping entry in the file for a given
manifest and ignore any earlier ones. This patch implments the latter.
Also drop the stale entries if we are rewriting it.
(From OE-Core rev: fe468802f697d0be41cf3407df2460e1473e35f8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This expands the error message when a network failure is detected.
It happens that some ISPs or networks block the default example.com
domain. Therefore, instead of disabling network access, it
lets the user know how to modify the test URL.
(From OE-Core rev: f54eaf65ff549a98ff98157d6b3aa48f9adc9ca5)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 62c94bb925543c1e1c5af3c751913d9f06d9597d)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
