It installs locale-base-* packages according to var IMAGE_LINGUAS.
Packages locale-base-* are split in libc-package.bbclass if variable
PACKAGE_NO_GCONV is not set. When none of ditro features libc-charsets
libc-locales and libc-locale-code is set, PACKAGE_NO_GCONV is set. Then
no locale-base-* is created and fails to create image.
Clear IMAGE_LINGUAS in such situation.
(From OE-Core rev: 85240094175a8ea726bfba19c00d4556a62862fc)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Variable PACKAGE_NO_GCONV is set in libc-package.bbclass when none of
libc-charsets, libc-locales and libc-locale-code set in DISTRO_FEATURES.
Then it causes installed-vs-shipped qa issue of glibc-locale:
| ERROR: glibc-locale-2.28-r0 do_package: QA Issue: glibc-locale:
| Files/directories were installed but not shipped in any package:
| /usr/share/i18n
| /usr/share/i18n/charmaps
| /usr/share/i18n/locales
| /usr/share/i18n/charmaps/CP737.gz
| ...
| /usr/share/i18n/locales/ru_RU
| ...
| /usr/lib64/gconv/gconv-modules
| Please set FILES such that these items are packaged. Alternatively if they
| are unneeded, avoid installing them or delete them within do_install.
| glibc-locale: 843 installed and not shipped files. [installed-vs-shipped]
So check PACKAGE_NO_GCONV during do_install and not copy those files if
PACKAGE_NO_GCONV has been set.
Simplify call of bb.utils.contains() in libc-package.bbclass as well.
(From OE-Core rev: 39840ed10af559bbcc306b378baa4723921668f5)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Waf requires that the current working directory be ${S} (the location of
the wscript) when building. Most of the time, this was true only because
B defaults to S. However, anything that changed that behavior (notably,
using externalsrc) would break the recipe. Remedy this by explicitly
changing cwd to ${S} when running waf commands. As a happy side effect,
B can be set up for "out of tree" builds to keep the source directory
clean.
(From OE-Core rev: 62dffb71ce22222c635bd90eaa47dd01f70f9c0f)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ccache disable flag was misspelled, preventing it from being
disabled.
(From OE-Core rev: 33fba601a7365aced9f4b206c1fadda997076d4c)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have a fatal error if ELF objects don't have GNU_HASH segments but it
doesn't explain what the problem is. At least give a hint to users by
suggesting that LDFLAGS wasn't passed to the compiler.
(From OE-Core rev: 5d4da6713b40e10e853eb746f700096307ffe158)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise .gir files installed in the sysroot will not be found when multilib is in use
(multilib configurations use libdir rather than datadir).
(From OE-Core rev: cfae171964ba85298d883e70635324a33b8539ac)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Skips the update alternative steps for recipes that shouldn't have them
enabled.
Fixes errors like:
nativesdk-bzip2-1.0.6-r5 do_package: bzip2: alternative target
(/opt/poky/2.5+snapshot/sysroots/i686-pokysdk-mingw32/usr/bin/bunzip2
or
/opt/poky/2.5+snapshot/sysroots/i686-pokysdk-mingw32/usr/bin/bunzip2.bzip2)
does not exist, skipping...
When building mingw SDKs
[YOCTO #12962]
(From OE-Core rev: d11576f569ee618a8e638eee3b1f17bf8a9d4264)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Several packages (eg. acl, attr, openssl) install Makefiles or other build host
configuration files for ptest so that they can run `make check`, for instance.
These build host files commonly include paths and versions of build host tools,
(eg. rpm or perl) whose presence and version depends on the build host distro.
Optionally strip lines from installed PTEST_BUILD_HOST_FILES using an
extended regex pattern defined in PTEST_BUILD_HOST_PATTERN. This has no
effect on the on-target ptest.
(From OE-Core rev: a3b657470eeea9f1954bc3a3e593a5c4907b6f17)
Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A common pattern for ptest is to install a Makefile that provides a make check
target. These generated Makefiles are normally full of build host paths, to
hosttools, and to files and scripts within the source or build tree. They also
commonly include the CFLAGS and CPPFLAGS. None of these build host paths
can possibly work on-target, so stripping them has no effect on the success of
the resulting ptests.
Stripping the HOSTTOOLS_DIR has this effect, for instance:
-MKDIR_P = /home/douglas/workspace/upstream/build/tmp/hosttools/mkdir -p
+MKDIR_P = mkdir -p
The curious WORKDIR/*= pattern avoids non-parseable -fdebug-prefix-map patterns
in the CFLAGS by adding an arbitrary ".", eg:
-fdebug-prefix-map=.=/usr/src/debug/rsyslog/8.37.0-r0
In all other cases, we simply remove the WORKDIR, eg:
CPP = arm-tait-linux-gnueabi-gcc -E --sysroot=recipe-sysroot ...
(From OE-Core rev: 137e38dbae42e9d4d413cd2f9460fb707ca77f9c)
Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a file matches the glob that is an absolute symlink then sed will try to
alter files on the host, so restrict the search to just actual files.
(From OE-Core rev: 5556b5ca9efac132754bd82af52a0f68bc4ce85a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As per the previous commit, upstream cpio has a bug which means it crashes on
append. If the image being built has already had testimage ran then cpio-native
will be in the sysroot. It's also possible that some distributions are shipping
this broken CVE patch too.
Now that our cpio-native is fixed, until we can be sure that the host cpio isn't
broken depend on cpio-native if building a cpio image.
[ YOCTO #13042 ]
(From OE-Core rev: c3b9aedcbe538d7fa74bd814644b4899769dec46)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In multilib_script.bbclass it renames script file which listed in
MULTILIB_SCRIPTS. It may mix up packages split. Take package curl as
example, ${bindir}/curl-config is packaged to curl-dev originally. But
it is renamed to curl-config-${MULTILIB_SUFFIX} and packaged to curl
when multilib is enabled.
And expand 'pkg' to fix QA warning:
| WARNING: Variable key FILES_${PN}-dev (
| ${bindir}/curl-config-${MULTILIB_SUFFIX}) replaces original key
| FILES_curl-dev (${includedir} ${FILES_SOLIBSDEV} ... ${bindir}/*-config)
Insert a necessary space to the argument 'value' of d.appendVar() as well.
(From OE-Core rev: 841bcbe429dcab54de3b89a927394750f9ccae60)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Setting them to empty before inheriting copyleft_filter
made them not getting their documented values there.
(From OE-Core rev: ee1f86ca7affc65265f219f0718f89406c9181a5)
Signed-off-by: Adrian Bunk <bunk@stusta.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A flaw was introduced by commit c5fa6034:
[ image.bbclass: use prependVarFlag for postfuncs ]
it changed to use prependVarFlag instead of appendVarFlag, then the
blank also needs change to adapt it.
(From OE-Core rev: 6085023158ffbfbaf0f3d65ef18054c003d3f463)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This can improve the performance a lot for "bitbake <recipe-native/cross/crosssdk>
-ccleansstate" when there are a lot of sstate files.
For example:
* Before
$ bitbake quilt-native -ccleansstate
- Check log.do_cleansstate:
Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*
Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package_qa.tgz*
Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package_write_rpm.tgz*
Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_packagedata.tgz*
Removing /sstate-cache/*/sstate:quilt-native::0.65:r0::3:*_populate_lic.tgz*
Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*
There are no package tasks for quilt-native, so the first 4 lines doesn't
make any sense, but the glob pattern "sstate-cache/*/*" is very time
consuming when there are no disk caches. E.g., I have more than 600,000
sstate files:
- Without disk caches
# echo 3 >/proc/sys/vm/drop_caches
$ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*")'
real 4m32.583s
user 0m5.768s
sys 0m12.892s
- With disk caches (e.g., run it in the second time)
$ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*")'
real 0m5.128s
user 0m2.772s
sys 0m2.308s
So the 4 removing *package* commands cost more than 20s or 272s in theory.
* After
$ bitbake quilt-native -ccleansstate
- Check log.do_cleansstate:
Removing /sstate-cache/*/sstate:quilt-native::0.65:r0::3:*_populate_lic.tgz*
Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*
We can see that it saved 20s or 272s in theory.
(From OE-Core rev: bb2d6349ea87f090c58001f0d4348b24c2982cde)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is a still python2 bbclass, so it has been broken since bitbake changed to
python3 which was 2 years ago. No one reported/fixed it for python3 in recent 2
years. So we can assume that no one uses it anymore.
(From OE-Core rev: 7f6da5fb54cbcf8e358e988382f45839a8b80019)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The build host umask was leaking into the thing-ptest packages
at do_install_ptest() time.
(From OE-Core rev: 891343e8ba6490ca3e1876c892269b611ddc7877)
Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some packages (eg. init-ifupdown) take their source files entirely from
openembedded-core, that is, they download no source tarball.
These recipes either don't use S at all (ie. it is empty at unpack time),
or they set S = WORKDIR (as in init-ifupdown).
Looking at the file timestamps in the WORKDIR causes a non-reproducible
SOURCE_DATE_EPOCH, as files taken from file:// URIs do not have
reproducible timestamps.
If S == WORKDIR, we are better to assume that there is no source tarball,
and to fall back to a fixed timestamp for the SOURCE_DATE_EPOCH.
This makes the init-ifupdown build reproducible.
(From OE-Core rev: d395bad0179037eb5d0fa4d921985c87ae13f3a4)
Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The glob.glob("/sstate/*/*/") is very time consuming, set
SSTATE_EXTRAPATHWILDCARD explicity to avoid that. This can save a lot of time
when there are many sstate files.
For example, I have more than 600,000 sstate files:
* Before
- Without disk caches
$ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*")'
real 4m32.583s
user 0m5.768s
sys 0m12.892s
- With disk caches
$ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*")'
real 0m4.111s
user 0m2.348s
sys 0m1.756s
* After
$ time python3 -c 'import glob; glob.glob("/sstate-cache.bak/universal/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*")'
- Without disk caches:
real 0m7.928s
user 0m0.172s
sys 0m0.124s
- With disk caches:
real 0m0.131s
user 0m0.088s
sys 0m0.044s
We can see that it saves about 3.8s with disk caches, and saves about 264s
without disk caches.
(From OE-Core rev: 8b31c919814b8bdf25b3381053656523c001ae0d)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updates the system blacklists to include packages that are known to have
problems compiling under icecream
(From OE-Core rev: fc5418e7bbdecfb27bafe595084e0fd0f991a388)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The system blacklists are moved to variables which are ignore when
hashing. This prevents changes to the blacklists from causing all
taskhashes to change (and thus rebuild).
(From OE-Core rev: f5be9f6e9180ace3362bba52c7ced3b039441d7d)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Splits the SDK test implementation into configurable Python classes. The
classes used for the normal and extensible SDKs are
${TESTSDK_CLASS_NAME} and ${TESTSDKEXT_CLASS_NAME} respectively.
This allows SDK machines to override the classes used to implement the
tests. For the traditional SDK, a common "run()" function is provided by
the class (oeqa.sdk.testsdk.TestSDK), with several hook member functions
that can be overridden in child classes, making it easier to have
consistent behavior. The extensible SDK class
(oeqa.sdkext.testsdk.TestSDKEXT) also has a common "run()" function, but
no hooks have yet been added as there is not currently a known use case
for create derived classes.
These changes should be purely organizational; no functional changes
have been made to either the standard SDK or extensible SDK tests.
[YOCTO #13020]
(From OE-Core rev: a06d53928b22d5f88276023c4d57b206db2f27f9)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Users are starting to expect OE to work under WSL which it doesn't. Add a warning to
tell them about this up front and manage expectations.
(From OE-Core rev: 62fe04c2451f612b9ee5eba469c7724b63484499)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that the hangs in httpservice are fixed we can let these tests
auto skip as appropriate.
(From OE-Core rev: 42a0d70291d551578e21f590fcb85ca72a78ccb5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use our own lsb function instead as used elsewhere by the codebase.
(From OE-Core rev: acac45a6fd604d28ef7c23d67482af3d7e8bcfe3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When calling d.appendVarFlag, a blank is needed or else it could mess
up the later appended variables.
Reported-by: Stefan Agner <stefan.agner@toradex.com>
(From OE-Core rev: 8b2df6e5874b932b356847c5bde10d5b31d67213)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This same variable was added to the make line in kernel.bbclass in
0decf1cc1c. With the addition of the
usrmerge feature, out-of-tree modules could use this same variable.
(From OE-Core rev: 1d24d1ac5b9c772d84104396438e5d1308332785)
Signed-off-by: Wes Lindauer <wesley.lindauer@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to set ownership of the license files after copying into the rootfs.
We also need to ensure we don't overwrite the rootfs files outside rootfs
generation, hence only write there when the new rootfs function parameter
is True.
[YOCTO #12961]
(From OE-Core rev: 6aa0eeaa4d31a440700f7695174de2a1dcc54866)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With this it's possible to build locale data for ARC
and not do it instead on the first boot.
(From OE-Core rev: f13c303491dc8850126ea14baedc7b63b7b5ecf4)
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now the test markup of the development tools tests is complete, this
can be further tweaked to auto run the correct tests.
(From OE-Core rev: bd4f8d12fe1f9f2643ee9e68fa2bb981134294fb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that the tests have correct markup to automatically determine which images
they can run against, clean up the default test suites logic to be simpler
and not image specific.
Some cleanup of the compiler tests still needs to be completed but this
is a good first step.
The only downside to this is more noise during testing as we now see
many skipped messages for simple images like core-image-minimal.
The auto type is being removed since it currently breaks badly due to the
socat mandatory inclusion from the meta-selftest layer which is a problem
which needs to be addressed seperately.
(From OE-Core rev: 4966bc33845752eb0aeae54b72e8ba0146a7ed52)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Display the name of the restricted licenses which caused the recipe to
be skipped. This makes it easy to determine which license or licenses
are missing and need to be checked and whitelisted.
(From OE-Core rev: b71cd1ec45e247db688b784697829c1b485ca9ca)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In newer versions of ExtUtils::MakeMaker, the location of the perl
binary to run is determined differently which results in finding
${bindir}/perl.real instead of ${bindir}/perl
(From OE-Core rev: 362bd36cbec052d2a37710498c901b699dcf5ca7)
Signed-off-by: Jens Rehsack <sno@netbsd.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that additional packages in IMAGE_INSTALL_DEBUGFS are available
before installation.
(From OE-Core rev: 06a1bbea67a7233deff80ebe3681b4e0cca6817a)
Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its been 'a while' since we used monotone, drop these revision
function remnants.
(From OE-Core rev: f1c77ab87b3c16d14deff801f48292ed348da637)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous code looks strange because of historical reason, simply return
base_get_metadata_git_revision/branch() is enough, the later functions has
handled everything like '<unknown>' and strip().
(From OE-Core rev: 8a6e70960e021e10a1d02e20e0519fdbdeac8616)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As was warned by commit 312ee68752fa [kconfig: announce removal of
oldnoconfig if used], oldnoconfig has been removed from the 4.20 kernel.
So we switch our default mode to olddefconfig.
commit fb16d8912 [kconfig: replace 'oldnoconfig' with 'olddefconfig',
and keep the old name as an alias] introduced olddefconfig in the 3.10
kernel, we shuffle oldnoconfig to the fallback target.
The fallback mode allows kernels between 3.10 and the currently listed
oldest kernel of 3.2 to continue to configure.
(From OE-Core rev: 8593dcb7e8c938530ff00ffedf7f3d02d26c3bad)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having image-buildinfo enabled causes containerimage.ContainerImageTests.test_expected_files
to fail due to the presence of an unexpected file:
['./',
'./etc/',
- './etc/build',
'./etc/default/',
'./etc/default/postinst',
Tweak the class to allow it to be disabled and disable it from the test just in
case it was enabled.
(From OE-Core rev: af67bf422a4df5b7e07894512ff73a5f493682ab)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to a bug in find [1], -ignore_readdir_race does not work correctly with
-delete. This can lead to spurious build failures when files disappear
while such a command is running; specifically this was seen in the case of
do_configure and do_populate_lic running concurrently for packages
with ${B} == ${WORKDIR}:
find: '.../sstate-build-populate_lic': No such file or directory
While the issue is fixed in the findutils git master, the find command of
the host system is called here, so we can't ensure that the used version
contains the fix. Many common distros have not updated to a recent enough
findutils version yet (Ubuntu 18.10 contains the fix, while 18.04 is still
affected).
Work around the issue by passing the output of find to 'rm -f' instead of
using -delete.
[1] https://savannah.gnu.org/bugs/?52981
(From OE-Core rev: 8079e2d62e23f7c274f46185e6dad64fa95394c1)
Signed-off-by: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In sources directory we can find patches/ and temp/.
The first one is filled with symbolic link unusable on another
machines.
The second contains yocto logs to create this archives and are
typically copied when 'S = "${WORKDIR}"'
(From OE-Core rev: 3904f98851c6a63dd9377e38f1432be6b1c0a94d)
Signed-off-by: Fabien Lahoudere <fabien.lahoudere@collabora.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Several distros are now shipping "python" as python v3 contra to the original
python guidelines. This causes users confusion/pain in trying to use our tools.
We can just force "python" to "python2" within HOSTTOOLS to avoid this issue
and hide the complexity from the user.
(From OE-Core rev: b06a6cde5c5503f456f260c773cf126085e18c8d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
