The fix for [Yocto #13994] required the rootfs directory to be copied
(using hardlinks if possible) when modifying the fstab file under wic.
We can optimise this copy away for filesystems where we have the tools
to modify the contents of the partition image after it is created. For
ext2/3/4 filesystems we have the debugfs tool and for msdos/vfat
filesystems we have the mcopy tool. So for any of these filesystems we
skip the modification of the fstab file in the rootfs directory (and
skip the associated copy unless it is otherwise necessary) and update
the contents of fstab directly in the partition image.
(From OE-Core rev: 5fb8ae0e9159597d7eaa9307a3a8543800bf9405)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default, wic updates the /etc/fstab in the rootfs to include details
of additional partitions described in the selected wks file. If this
modification is performed in place, other tasks which create an image
file from the rootfs directory (e.g. do_image_tar and do_image_ext4)
will pick up the modified fstab file which would not be appropriate for
those images as they do not include the additional partitions described
in the wks file. wic does undo modifications to the fstab file once it
has finished creating the filesystem image, however this leaves open a
race condition if one of the other tasks reads the contents of the fstab
file from the rootfs directory between the point where wic modifies the
fstab file and the point where wic restores the files original content.
This could be solved by adding a lockfile for tasks which use the rootfs
directory to ensure that no other such task is reading the rootfs
directory while do_image_wic is running. This would serialize several
do_image_* tasks and result in slower builds, especially for large
images. Another drawback of this solution is that it is hard to
selectively optimise - adding lockfiles to do_image_* tasks would result
in these tasks always being serialized even if no fstab modification
will take place.
An alternative solution is to copy the rootfs directory when fstab needs
to be modified. The code to do this in wic already exists as it is
needed when including or excluding content in the rootfs. This still
results in an impact on build times but the copy uses hardlinks if
possible (so little data is actually copied) and we can make selective
optimisations to improve things. The rootfs copy will only take place if
fstab modification is required (or if it was already needed to include
or exclude rootfs content). We can also follow up with further
optimisations after this commit. So this second solution is chosen.
Fixes [Yocto #13994]
(From OE-Core rev: ce682a73b7447652f898ce1d1d0416a456df5416)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To exclude content from the rootfs, wic makes a copy (using hardlinks if
possible) of the rootfs directory and associated pseudo db, then removes
files & directories as needed. However if these files and directories
are removed using the python functions os.remove and shutil.rmtree, the
copied pseudo db will not be updated correctly. For files copied from
the original rootfs, if hardlinks were used successfully when copying
the rootfs this should mean that the relevant inodes can't be reused and
so the risk of pseudo aborts should be avoided. However, this logic
doesn't apply for directories (as they can't be hardlinked) or for files
added via the '--include-path' argument (as they weren't present in the
original rootfs) and so there remains some risk of inodes being reused
and the pseudo db becoming corrupted.
To fix this, use the 'rm' command under pseudo when removing files &
directories from the copied rootfs to ensure that the copied pseudo db
is updated.
(From OE-Core rev: d5db7e268947f0392c2126137571a44acd29ccd6)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default the wic working directory is placed under the output
directory. When invoking wic under bitbake, the wic output directory is
added to PSEUDO_PATHS_IGNORE to avoid issues with files being removed
from outside a pseudo environment (see oe-core commit ad8f5532ff).
However, wic will copy the rootfs directory into its working directory
if it needs to add or remove content before creating a filesystem image.
This copy of the rootfs directory must be tracked by pseudo in order to
keep the permissions correct in the resulting image. So we can't have
the wic working directory under a path in PSEUDO_PATHS_IGNORE unless
we like broken permissions.
To fix this the new '-w' argument to wic is used to move the working
directory away from the output directory.
Note that wic deletes the temporary working directory automatically
when it finishes creating an image so users won't normally see the
'tmp-wic' directory under WORKDIR.
Fixes [Yocto #14129]
(From OE-Core rev: 4d07169499c47fa9dc759e6f81843416a6dc25c5)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a path is specified for the internal wic working directory using
the -w/--workdir argument then it must not already exist. Re-using a
previous workdir could easily result in rootfs and intermediate files
from a previous build being added to the current image.
(From OE-Core rev: 2e40c8d4109024ff704c5ce40d98050ca7f34dd5)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove backported patches and remove the introspection patch as that is
no longer needed.
(From OE-Core rev: 94bc7f8c2442f6627b001f1dd40c0023260b22e8)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a race between two make invocations in 5.1. It has been
reported upstream, in the meantime add a work around patch (which
upstream may yet even apply). The autobuilder found two instances
of this race in a single build.
(From OE-Core rev: 62b8580cea567f2d9b7a900fc4302d6bc5a785b2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bash 5.1 inserts escape sequences into its output (specifically
disabling bracketed paste mode via \x1b[?2004l). I am not sure
if somehow terminal detection isn't working correctly there,
but in any case the marker is still in the output, but needs to
be checked by 'in' rather than exact equivalence.
(From OE-Core rev: c25a1b16cc50e51ecf040c2c1db128e1dfceddec)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We had reproducibility issues where the pipe size appears to vary on some build
hosts between 65535 and 4096. It should be the former on Linux in general so
standarise on that. Easiest way is to pass in CFLAGS.
(From OE-Core rev: 6de2152693c31ee1181e9eacc32b90dd63d4e0a9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[RP: Add aclocal support and patch m4 handling to adapt to OE]
(From OE-Core rev: ab4406dfdbd5e21f6fff0865228ebf5da1274505)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updates the documentation for PROVIDES so that it recommends "+="
instead of "=".
(From yocto-docs rev: 39b2ca1e27592488d396d5f0d76965f0006515a1)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit adds the description of the Initramfs bundle and boot script
new features implemented in the kernel-fitimage class.
Change-Id: Ifffa6b850308aa7ceadc4f117806cffad0137137
(From yocto-docs rev: a55c16555366c0adbf4a087b86574b07972cbc52)
Signed-off-by: Abdellatif El Khlifi <abdellatif.elkhlifi@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
poky.yaml references are only replaced in files if they are prefixed by
& and suffixed by ;.
Let's fix the missing surrounding characters.
(From yocto-docs rev: 7ee4ba7a27acd87d8c728639d1b053d2e26c6e58)
Signed-off-by: Quentin Schulz <foss@0leil.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit provides a testcase for the initramfs bundle support implemented
in kernel-fitimage.bbclass
The testcase verifies the content of the initramfs bundle node in the FIT
Image Tree Source (its).
The testcase is self-contained and the configurations are set by the test case itself.
To verify the initramfs bundle support, the testcase uses beaglebone-yocto machine.
This testcase can be run through the following command:
oe-selftest -r fitimage.FitImageTests.test_initramfs_bundle
Change-Id: I8ab8abf2c150ea515fd439784eb20c6b092bfbc5
(From OE-Core rev: 1119d577756b386507f33669fe29dafb5579a1a7)
Signed-off-by: Abdellatif El Khlifi <abdellatif.elkhlifi@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When including an initramfs bundle inside a FIT image, the fitImage is created
after the install task by do_assemble_fitimage_initramfs.
This happens after the generation of the initramfs bundle
(done by do_bundle_initramfs).
So, at the level of the install task we should not try to install the fitImage.
The fitImage is still not generated yet.
After the generation of the fitImage, the deploy task copies the fitImage from
the build directory to the deploy folder.
Change-Id: I3eaa6bba1412f388f710fa0f389f66631c1c4826
(From OE-Core rev: 1b67fd9ac74935fa41e960478c54e45422339138)
Signed-off-by: Abdellatif El Khlifi <abdellatif.elkhlifi@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit adds Initramfs bundle support to the FIT image in addition
to u-boot boot script capability.
These new features are selectable.
In case of Initramfs, the kernel is configured to be bundled with the rootfs
in the same binary (ie: zImage-initramfs-<machine>.bin). When the kernel is
copied to RAM and executed, it unpacks the Initramfs rootfs.
For more information about Initramfs please read:
https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt
For more details about the Initramfs bundle and boot script implementation
please check the kernel-fitimage.bbclass paragraph in Yocto reference
or mega manual.
Current limitations:
- Initramfs bundle FIT support has been tested on ARM 32-bit
- The kernel image type in case of ARM 32-bit is zImage
Change-Id: I901bfd899e8d733c5b9a2b6645b1d4748f4b1fda
(From OE-Core rev: 19fa415c8769a67b52babd80f71d68bf36a21db2)
Signed-off-by: Abdellatif El Khlifi <abdellatif.elkhlifi@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
apt-pkg/libapt-pkg.so.5.0.2: undefined reference to `ZSTD_endStream'
collect2: error: ld returned 1 exit status
This is because ccache-native depends on zstd-native which makes apt wronly
find it. Disable zstd for apt to fix the problem.
(From OE-Core rev: 0aa7d612b8b7e5f14b4ed38f2a32b3f7eefca31c)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since host's ccache is not reliable, so disable ccache for native recipes and
use ccache-native for other types of recipes. We need disable ccache for native
recipes is because ccache-native now depends on cmake-native which causes
circular dependencies, and it's not easy to break the circular.
(From OE-Core rev: 631bbd4896882ba2acbe5bc85bc90ab7abc794ef)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit f5b29367af4d8e5daea5771264774aa49519f9a8.
Will use ccache-native which is more reliable.
(From OE-Core rev: 1b659623430e1a6e6dd266e65bab7ca8155a7138)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
