There are several thousand files in the test directory which we don't need.
Adding these for the native and target sysroots is a crazy amount of files
to be throwing around needlessly. Delete the files from the sysroot side
of things to tidy up the sysroots and improve performance.
(From OE-Core rev: 653f8b1a8adf59d924028bfc69ead1c7437de11e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f6bced03011ad1663d68b0322a2f8aeb4d836646)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This recudes the file count from ~2850 to ~100 which is a huge win
for reducing build directory clutter, its unlikely anything uses the
terminfo data or man pages in the sysroot. This is especially helpful
as we usually end up with two copies of these sets of files.
(From OE-Core rev: cb254debc36acc085a90c665f72ecf3a28692944)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 443633dfc20177ef88a388d96745675817510c99)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove patches for CVE-2019-25013 and CVE-2020-27618 since they are
present in the branch now. Add both CVEs to CVE_CHECK_WHITELIST.
760e1d28782 gconv: Fix assertion failure in ISO-2022-JP-3 module (bug 27256)
d3cb8f6222a aarch64: fix static PIE start code for BTI [BZ #27068]
082798622d8 __vfscanf_internal: fix aliasing violation (bug 26690)
33dc30bc838 aarch64: Use mmap to add PROT_BTI instead of mprotect [BZ #26831]
46e1e64fe3e elf: Pass the fd to note processing
b6eae83717d elf: Move note processing after l_phdr is updated
c6090dcebd1 aarch64: align address for BTI protection [BZ #26988]
610e2c51504 aarch64: Fix missing BTI protection from dependencies [BZ #26926]
4c619b3eed5 x86: Check IFUNC definition in unrelocated executable [BZ #20019]
87450ecf8a8 x86: Set header.feature_1 in TCB for always-on CET [BZ #27177]
2b4f67c2b33 Update for [BZ #27130] fix
1a24bbd43e4 x86-64: Avoid rep movsb with short distance [BZ #27130]
0d9793e82a1 Fix buffer overrun in EUC-KR conversion module (bz #24973)
1d49bede4d8 tests-mcheck: New variable to run tests with MALLOC_CHECK_=3
050022910be iconv: Accept redundant shift sequences in IBM1364 [BZ #26224]
ac0a6929c5d sh: Add sh4 fpu Implies folder
3ea24955bff struct _Unwind_Exception alignment should not depend on compiler flags
5c36293f067 resolv: Serialize processing in resolv/tst-resolv-txnid-collision
2dfa659a66f resolv: Handle transaction ID collisions in parallel queries (bug 26600)
05c025abca1 support: Provide a way to clear the RA bit in DNS server responses
f688bcd83de support: Provide a way to reorder responses within the DNS test server
eba0ce60588 Remove __warndecl
5337b2af4b8 Remove __warn_memset_zero_len [BZ #25399]
c6e794640c3 aarch64: Add unwind information to _start (bug 26853)
70ee5e8b573 aarch64: Fix DT_AARCH64_VARIANT_PCS handling [BZ #26798]
8813b2682e4 x86: Optimizing memcpy for AMD Zen architecture.
e61a8fd8fad Reversing calculation of __x86_shared_non_temporal_threshold
0b9460d22e2 sysvipc: Fix IPC_INFO and SHM_INFO handling [BZ #26636]
c4aeedea598 sysvipc: Fix IPC_INFO and MSG_INFO handling [BZ #26639]
9b139b6b81a sysvipc: Fix SEM_STAT_ANY kernel argument pass [BZ #26637]
81c5484d93a AArch64: Use __memcpy_simd on Neoverse N2/V1
0f8f0ed25c1 AArch64: Improve backwards memmove performance
23482f78866 Set version.h RELEASE to "stable" (Bug 26700)
69beb5cbf85 string: Fix strerrorname_np return value [BZ #26555]
fe62c4d173f intl: Handle translation output codesets with suffixes [BZ #26383]
386543bc449 NEWS: Update for [BZ #26534] fix
cebc01cbfd6 x86-64: Fix FMA4 detection in ifunc [BZ #26534]
(From OE-Core rev: 6559f16646a0a00d06104939fa5801e8594691af)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8d05c277c5350c4d968eb488788eac7978968ef7)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python2-native executable is not available in sysroot anymore, which
causes compilation of some nodejs modules to fail. Switch to python3 as a
default python version.
(From OE-Core rev: 5cc15f53d55923815a859009a1ffe6cff145b6d7)
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d21f50ecf8e8683a92b7d234fa8225c2c1470595)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before adding a npmsw fetcher to a recipe we
should first check if the generated shrinkwrap file
contains dependencies.
(From OE-Core rev: 6bcd2e242ef85cf875a2ea1a7c208f8c30fbdffa)
Signed-off-by: Kamel Bouhara <kamel.bouhara@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ef153ad36d0299e83a03af8f207686d0d8a238b3)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some packages don't have shrinkwrap file which
means no npmsw uri is provided in the recipe.
(From OE-Core rev: 2c17be47120d388cffafe8213a4b8a92d537805c)
Signed-off-by: Kamel Bouhara <kamel.bouhara@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 47760b0d7d66b2b68ee197d359f0b7b17374d742)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The tar output seems to vary depending on the version of tar used and distro
configuration. Be explict about the output format to avoid this and be
determinstic.
(From OE-Core rev: c56f3c9febc1732aa1302524c6c4da36f16bd1f7)
(From OE-Core rev: a9b4a1e82eb45832dd641a77733737c6fd759890)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9dbe0f69f8)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This image is only buildable for x86-64, so add a COMPATIBLE assignment
to ensure it isn't attempted on others.
(From OE-Core rev: bdd8208675c8a0c0232c678804a8b62cd74f1d48)
(From OE-Core rev: 9adacc27c52af0603ada259456d7a76bee2b1307)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit db87aab8e2)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As this image isn't buildable without ptest (the packages won't exist),
depend on the ptest DISTRO_FEATURE to ensure we don't try and build it.
(From OE-Core rev: 0fe856d726c6d9c35533e32e70fbe05ef2b88b17)
(From OE-Core rev: e2222ddd8b4566d7bdac073fff801523c2cbfb0e)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 72b859aba8)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The externalsrc class deletes do_patch task which results with:
| ERROR: Task do_create_manifest in <PATH>/python3_3.8.2.bb depends upon
| non-existent task do_patch in <PATH>/python3_3.8.2.bb
Use addtask to define correct order to prevent this error, since addtask
mechanism accepts deleted tasks.
[YOCTO #14151]
(From OE-Core rev: a746d034fa7eaad4f4876fa61c5a8c3c15e211c8)
(From OE-Core rev: 782f7f4f73d7b1e1bd649db1984a63d94d2410fe)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8ed8b81af6)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a function returns any stderr it will be passed to extractPatches and
used as path to patch.
For example subprocess command output can be:
| sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
| /tmp/oepatchhuqle8fj/0001-foo.patch
| /tmp/oepatchhuqle8fj/0002-bar.patch
that will result in:
| FileNotFoundError: [Errno 2] No such file or directory: 'sh:'
To fix this I separated output, made the function return stdout and
print stderr only in case of command error.
(From OE-Core rev: 482589e2cc7c3ddeefb0a0fb98d97a9cbb18c9ec)
(From OE-Core rev: 1dadeb58e88c4d953be612654fe4a76b51ff0553)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5e2450731c)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
add CVE_VERSION_SUFFIX to indicate the version suffix type, currently
works in two value, "alphabetical" if the version string uses single
alphabetical character suffix as incremental release, blank to not
consider the unidentified suffixes. This can be expand when more suffix
pattern identified.
refactor cve_check.Version class to use functools and add parameter to
handle suffix condition.
Also update testcases to cover new changes.
(From OE-Core rev: 5dfd5ad5144708b474ef31eaa89a846c57be8ac0)
(From OE-Core rev: 73f8c25a44ee33866a8fa61824437285cea96249)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 86b42289bd)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The way distutils.version.LooseVersion compare version are tricky, it treat
all these ( "1.0-beta2", "1.0-rc1", "1.0A", "1.0p2" and "1.0pre1") as greater
version than "1.0". This might be right for "1.0A" and "1.0p1" but not for
the rest, also these version could be confusing, the "p" in "1.0p1" can be
"pre" or "patched" version or even other meaning.
Replace Looseversion with custom class, it uses regex to capture common
version format like "1.1.1" or tag format using date like "2020-12-12" as
release section, check for following known string/tags ( beta, rc, pre, dev,
alpha, preview) as pre-release section, any other trailing characters
are difficult to understand/define so ignore them. Compare release
section and pre-release section saperately.
included selftest for the version class.
[YOCTO#14127]
(From OE-Core rev: 6ced85e9ddd3569240f1e8b82130d1ac0fffbc40)
(From OE-Core rev: 02a44b507a1e49a4c460f3e1bec92832b71dfe08)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3807c6d9a7)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Pulls in:
pseudo_client: Ensure renames update open fd file paths
pseudo_client.c: Rebuild passwd paths after chroot
which should fix issues seen in apt package index creation, new
binutils and other autobuilder race issues in pseudo amongst other
issues.
(From OE-Core rev: 44d11b56001f40622c055069b0901cc4ae15c76c)
(From OE-Core rev: 0fc140e6aeba27afc5bd7c4e8d486d73fa6e3f2e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e1caf4f305)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The DEPENDS variable override ordering here was almostly certainly
incorrect and led to weird behaviour when making changes elsewhere.
Correct it.
(From OE-Core rev: c8f7e92244b3c52c275a457aced69086800351d8)
(From OE-Core rev: 3f56b2666f823dadca84f76daee829f19cce3d6f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3a59f2abe2)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An exception is fired when a BuildStarted event is sent to buildhistory bbclass
and the variable BUILDHISTORY_FEATURES is not set.
ERROR: Execution of event handler 'buildhistory_eventhandler' failed
Traceback (most recent call last):
File "<...>/meta/classes/buildhistory.bbclass", line 862, in buildhistory_eventhandler(e=<bb.event.BuildStarted object at 0x7f94c3810250>):
python buildhistory_eventhandler() {
> if e.data.getVar('BUILDHISTORY_FEATURES').strip():
reset = e.data.getVar("BUILDHISTORY_RESET")
AttributeError: 'NoneType' object has no attribute 'strip'
This can happen in a multiconfig build where the default configuration use the
buildhistory class but not the configuration in mc. It should be a rare case that
this happens and it was found in a missconfigured build.
(From OE-Core rev: a74e30a4de02c8efd3e7102ba7a4fe06df53cc34)
(From OE-Core rev: 83ee03a6f05ff60459dbcc4fcfb78f95bb0b4848)
Signed-off-by: Peter Bergin <peter@berginkonsult.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 55ead1be58)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The srctree_hash was calculated only from main source directory ignoring
changes in submodules.
[YOCTO #13748]
Use submodule--helper to determine all submodules, and calculate hash
from all git tree objects names combined.
(From OE-Core rev: 50ff9afb3990bcf60b4fa1f937506cb84028c32d)
(From OE-Core rev: 6d8a58b11117372fdbf0a86fbc6698ee509fc816)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9385670add)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a user executes the environment script instead of sourcing it,
there's an error about an empty element in PATH. This is because
bitbake isn't present in environment variable PATH. Hence, this
patch adds a sanity check to verify if bitbake is present in
PATH and if bitbake isn't present issue a warning message.
[YOCTO #13822]
(From OE-Core rev: e08799913a7f207bc63e085eb98196fd61ed57bc)
(From OE-Core rev: 59de426a624edcb02a0f7ad53ee5955d9f3b2cf3)
Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit dbd80a923c)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a user builds in a path in PSEUDO_IGNORE_PATHS, random failures
are generated. Hence this patch adds a sanity check in sanity.bbclass
to ensure that a user isn't building in PSEUDO_IGNORE_PATHS.
[YOCTO #14179]
(From OE-Core rev: 7a681525e904914e938de25df5cc64209097d15d)
(From OE-Core rev: 35f3e08c52e163e4496c0222896c49888ad5c960)
Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit bfeeaf9ff1)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Backport a patch to increase the timeout that might help with the
intermittent seek test failure.
[YOCTO #14194]
(From OE-Core rev: a7dc7a35334ad634926a1386f4a56b27aad3ce68)
(From OE-Core rev: f9f4191a6fcc1474fa2b7d96c7f82f066ae02cb4)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 7b90027aac)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test would timeout on autobuilders. This patch increases the
timeout to 60s
The test will now also exit as soon as we receive the 2 expected events
Expected runtime is around 1s if successful
Bug 14158
(From OE-Core rev: be02aa9283f805de718badd5ea12c4968da8774f)
(From OE-Core rev: c925b83cbd31345f788515ca8b642b14e5e11898)
Signed-off-by: Yi Fan Yu <yifan.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1d803b70e5)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If srcdir is under poky directory (e.g. devtool poky/build/workspace/sources)
and is not a git repository then ${@srctree_hash_files(d)} will run "git
rev-parse --git-dir" and detect poky directory as git-dir and run "'git', 'add',
'-A', '.'], cwd=s_dir" trying to add srcdir but build dir is in .gitignore and
latest git will fail with "The following paths are ignored by one of your
.gitignore files: build" which will end with "ExpansionError during parsing".
In this commit I added a check if git_dir is the same as git-dir from
TOPDIR (which will detect poky directory) and if yes, then treat srcdir
as non-git sources.
(From OE-Core rev: 95fbac8dcad6c93f4c9737e9fe13e92ab6befa09)
(From OE-Core rev: 1da57e92812a1025d975204797b761a3be3998aa)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 28bdfe0066)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
https://github.com/p11-glue/p11-kit/releases/tag/0.23.22
Release notes:
Fix memory-safety issues that affect the RPC protocol (CVE-2020-29361, CVE-2020-29362, and CVE-2020-29363), discovered and fixed by David Cook
anchor: Prefer persistent format when storing anchor [#329]
common: Fix infloop in p11_path_build [#326, #327]
proxy: C_CloseAllSessions: Make sure that calloc args are non-zero [#325]
common: Check for a NULL locale before freeing it [#321]
Build and test fixes [#313, #315, #317, #318, #319, #323, #330, #333, #334, #335, #338, #339]
c4e75e1002
patch to fix trailing newline using custom_target() caused error
with DISTRO_FEATURES api-documentation due to meson bugs, enable
manpages PACKAGECONFIG should prevent this error.
| warning: failed to load external entity "../version.xml"
| ../p11-kit-docs.xml:11: parser error : Failure to process entity version
| <releaseinfo>for p11-kit &version;</releaseinfo>
| ^
| ../p11-kit-docs.xml:11: parser error : Entity 'version' not defined
| <releaseinfo>for p11-kit &version;</releaseinfo>
| ^
| unable to parse ../p11-kit-docs.xml
(From OE-Core rev: b112ba291835061640123c13784e2b33cc73f17d)
(From OE-Core rev: f500435958fd676a00757a64572f06f5cb16c251)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 59b07a71f3)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Backport patch from
bdd3acbd48
image file in upstream patch is for test purpose only, it cause error during
do_patch so drop it:
File tests/test-images/fail/hang_114.gif: git binary diffs are not supported.
(From OE-Core rev: 7cd401b7c60dba91f6fb10395ab4a10a267cf23d)
(From OE-Core rev: c3f2e4ed9c5fe541a9aee900cc632592f9824407)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ee867453ff)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Insert an explicit pass to fetch all blobs needed by Git LFS, during the
fetch() function. This avoids the default behavior of Git LFS to wait
until 'git checkout' to begin downloading the blobs pointed to by LFS records.
Network access is not allowed at that point in the recipe's lifecycle.
[YOCTO #14191]
(Bitbake rev: c73f8f2f4a6491c6bea54839630af6994c27ad24)
Signed-off-by: Matt Hoosier <matt.hoosier@garmin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0efac66043662e7a2027192f50e92e982db2ba1c)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
None is a bad choice of return value for functions used in variables
(strings) as a failure results in concatination errors. Use a string
with a clear meaning that can be searched for instead.
(From OE-Core rev: 4e8022635fc8543d135fed3091a9f555899d1b3c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Option --noline or -L does not handled properly.
So generated code contains #line directives with
file absolute path and prevents to create
reproducible builds.
(From OE-Core rev: e02c02459b35129402b2a77b592896c787e590ff)
Signed-off-by: Oleksiy Obitotskyy <oobitots@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0bf2cb7dc2123f220accf1542c2ae4c4b4b8275a)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If do_patch task is disabled then prepare do_configure dependencies to
fetch external sources and create symlink to ${S} in devtool workspace.
[YOCTO #13036]
(From OE-Core rev: 9a1bcf47f0242215610ddf917e8bd87e86fdf419)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9e47319bfe62d289b90f7545a64dbdc1cbde7f1d)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Files could contain colons in name and we should not use
colons (':') as field separator. E.g. perl/python man
pages packages.
(From OE-Core rev: 5ea3c65c6b224a9e20cc4f41b044e598771de8a2)
Signed-off-by: Oleksiy Obitotskyy <oobitots@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 50d2ed0689f1aed6f33b4992d37e2e991c99eb07)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The fix for [Yocto #13994] required the rootfs directory to be copied
(using hardlinks if possible) when modifying the fstab file under wic.
We can optimise this copy away for filesystems where we have the tools
to modify the contents of the partition image after it is created. For
ext2/3/4 filesystems we have the debugfs tool and for msdos/vfat
filesystems we have the mcopy tool. So for any of these filesystems we
skip the modification of the fstab file in the rootfs directory (and
skip the associated copy unless it is otherwise necessary) and update
the contents of fstab directly in the partition image.
(From OE-Core rev: a56b9dbfeb410d235b5af2c5a846b3f1360d2d1d)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5fb8ae0e9159597d7eaa9307a3a8543800bf9405)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default, wic updates the /etc/fstab in the rootfs to include details
of additional partitions described in the selected wks file. If this
modification is performed in place, other tasks which create an image
file from the rootfs directory (e.g. do_image_tar and do_image_ext4)
will pick up the modified fstab file which would not be appropriate for
those images as they do not include the additional partitions described
in the wks file. wic does undo modifications to the fstab file once it
has finished creating the filesystem image, however this leaves open a
race condition if one of the other tasks reads the contents of the fstab
file from the rootfs directory between the point where wic modifies the
fstab file and the point where wic restores the files original content.
This could be solved by adding a lockfile for tasks which use the rootfs
directory to ensure that no other such task is reading the rootfs
directory while do_image_wic is running. This would serialize several
do_image_* tasks and result in slower builds, especially for large
images. Another drawback of this solution is that it is hard to
selectively optimise - adding lockfiles to do_image_* tasks would result
in these tasks always being serialized even if no fstab modification
will take place.
An alternative solution is to copy the rootfs directory when fstab needs
to be modified. The code to do this in wic already exists as it is
needed when including or excluding content in the rootfs. This still
results in an impact on build times but the copy uses hardlinks if
possible (so little data is actually copied) and we can make selective
optimisations to improve things. The rootfs copy will only take place if
fstab modification is required (or if it was already needed to include
or exclude rootfs content). We can also follow up with further
optimisations after this commit. So this second solution is chosen.
Fixes [Yocto #13994]
(From OE-Core rev: 39f98ef8c1f6b0259deb43ea902f5ac8e5b069cd)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ce682a73b7447652f898ce1d1d0416a456df5416)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To exclude content from the rootfs, wic makes a copy (using hardlinks if
possible) of the rootfs directory and associated pseudo db, then removes
files & directories as needed. However if these files and directories
are removed using the python functions os.remove and shutil.rmtree, the
copied pseudo db will not be updated correctly. For files copied from
the original rootfs, if hardlinks were used successfully when copying
the rootfs this should mean that the relevant inodes can't be reused and
so the risk of pseudo aborts should be avoided. However, this logic
doesn't apply for directories (as they can't be hardlinked) or for files
added via the '--include-path' argument (as they weren't present in the
original rootfs) and so there remains some risk of inodes being reused
and the pseudo db becoming corrupted.
To fix this, use the 'rm' command under pseudo when removing files &
directories from the copied rootfs to ensure that the copied pseudo db
is updated.
(From OE-Core rev: 9f33aa6f1dd66b1b6990b256ad822fa4cf8daa94)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d5db7e268947f0392c2126137571a44acd29ccd6)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default the wic working directory is placed under the output
directory. When invoking wic under bitbake, the wic output directory is
added to PSEUDO_PATHS_IGNORE to avoid issues with files being removed
from outside a pseudo environment (see oe-core commit ad8f5532ff).
However, wic will copy the rootfs directory into its working directory
if it needs to add or remove content before creating a filesystem image.
This copy of the rootfs directory must be tracked by pseudo in order to
keep the permissions correct in the resulting image. So we can't have
the wic working directory under a path in PSEUDO_PATHS_IGNORE unless
we like broken permissions.
To fix this the new '-w' argument to wic is used to move the working
directory away from the output directory.
Note that wic deletes the temporary working directory automatically
when it finishes creating an image so users won't normally see the
'tmp-wic' directory under WORKDIR.
Fixes [Yocto #14129]
(From OE-Core rev: 28d2e131555372a79499746057468cb18db69d69)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4d07169499c47fa9dc759e6f81843416a6dc25c5)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows programs from HOSTTOOLS (e.g. 'install', 'rm', 'mv', etc) to
be more easily executed by wic. Without this change only programs from
an actual *-native recipe built by bitbake can be executed by wic.
(From OE-Core rev: 00474d990174c3c4af6e0a6058871f925380750a)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8eb186acdecfbb3151c9a0ab148358e3fe5cce39)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a path is specified for the internal wic working directory using
the -w/--workdir argument then it must not already exist. Re-using a
previous workdir could easily result in rootfs and intermediate files
from a previous build being added to the current image.
(From OE-Core rev: 365b10a95ac0b04ca48a53e55cf5a0a330f1bc2f)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2e40c8d4109024ff704c5ce40d98050ca7f34dd5)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows the path for the temporary workdir used by wic to be set
when running wic from bitbake or directly from the command line.
(From OE-Core rev: 6136e1312fc69b1b6196aa2fb8986d99ad46e7a1)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2e7314ac3a3cab89de93c932e8efc75de0feb0a7)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upstream has merged a change to handle detection of target overrides so
that they are only applied as required. Backport this patch from
upstream and remove the OE specific patch.
(From OE-Core rev: ef87c3609ecf93158685f4d4cdc53754a6102330)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d430f425e1a5725e93833a652e5c6988576f7347)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In cmake.bbclass we set CMAKE_BUILD_PARALLEL_LEVEL using parallel_make
function and if PARALLEL_MAKE is set to empty string then this variable
is exported as "None" causing cmake to fail with:
"'CMAKE_BUILD_PARALLEL_LEVEL' environment variable
invalid number 'None' given."
(From OE-Core rev: 7ee65c50f7a38c22e3fb43b0c89e81aff24a2b7f)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2f790ded554a52ac18d1c28002142f9c62abec8b)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The toolchain-shar-extract.sh script updates the SDK relocation paths in
post-relocate-setup.sh, so avoid doing this twice. This is generally not
a problem, unless the SDK path is a subset of the SDK relocation path, in
which case the resulting path is substituted twice. To trigger the issue,
$ ./tmp/deploy/sdk/poky-glibc-x86_64-core-image-base-core2-64-qemux86-64-toolchain-3.2+snapshot.sh -y -d /home/oe/.local/opt/poky/3.2+snapshot
which generates relocation path
/home/oe/.local/home/oe/.local/opt/poky/3.2+snapshot
instead of
/home/oe/.local/opt/poky/3.2+snapshot
Fixes: 93ec145f42 ("toolchain-shar-extract: Add post-relocate scripts")
(From OE-Core rev: 9504a975c74fc00f3c27b9dbc18316143215b6a3)
Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Joshua Watt <JPEWhacker@gmail.com>
Cc: Krzysztof Zawadzki <krzysztof.zawadzki@nokia.com>
Cc: Randy Witt <randy.e.witt@linux.intel.com>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5000aabe6ac336e7b424dafa1bf76271dee6a6f1)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The $target_sdk_dir path might contain special characters, for example if
the path is /opt/poky/3.2+snapshot . Prevent grep from interpreting those
as part of the regex by using the -F parameter and multiple -e parameters
to specify which strings to filter out. Also note that the previous regex
was using asterisk as wildcard (e.g. environment-setup-*), but that should
have been regex (e.g. environment-setup-.*, with dot) to match correctly,
this is also fixed by this change.
Fixes: 9721378688 ("toolchain-shar-template.sh: Make relocation optional.")
(From OE-Core rev: b97f2901c66d6343c49d5bf90d538aa22aeae79d)
Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Joshua Watt <JPEWhacker@gmail.com>
Cc: Krzysztof Zawadzki <krzysztof.zawadzki@nokia.com>
Cc: Randy Witt <randy.e.witt@linux.intel.com>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 19d9fa7ab6c851000bc5d24281739e1b2bb8f057)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch makes gcc produce broken code. It is unclear why it is there
in the first place. Drop it.
(From OE-Core rev: c6dac9e737b27dc0e2b02a75463b0eba4b9b01d0)
Signed-off-by: Mans Rullgard <mans@mansr.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5f3cace37496fe1dc4fd045f688f7d441505c437)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
