When some hosttool is a dead symlink, bitbake any recipe will
fail with error like below.
FileExistsError: [Errno 17] File exists: '/usr/bin/chrpath' -> '/path/to/builddir/tmp-glibc/hosttools/chrpath'
So we remove dead symlink under hostools/ directory to avoid
such error.
(From OE-Core rev: b03577cf85784fc052b89083054d911816f8c6e1)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure self.data isn't accessed without assignment. Also clean up old style
popen use and replace with modern/simpler subprocess.
(From OE-Core rev: 39825cba4761a6b4b2473825705975f9f421ec8b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can't just use HOST_OS, as in meta-mingw SDKs that is 'mingw32' but Meson
expects 'windows'.
(From OE-Core rev: f408bbb1cc72b84905b22d100f52c0a9f54ff3fe)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If devupstream is in use by the recipe, then that
variant has SRCPV, but the main variant may not,
in which case bitbake throws an expansion error.
(From OE-Core rev: 740e3f5e1d4a8abb07c7a4e4aedd804d679c7562)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of trying to change the RPATH in every file under the binary
directories, check that the file is an ELF first.
This means that we don't attempt to change the RPATH on the entire
Python standard library, for example.
Also return early if the directory to iterate doesn't exist.
(From OE-Core rev: 977f4baacf7c8d06d9cfe5c5e39bb8bc19f27028)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Pass --build-base so the build tree is a location we specify, which we
can ensure is empty when building.
This means we can stub out do_configure entirely and use cleandirs
instead.
When installing we need to tell it to do a build so that we can pass
--build-base, so also pass --skip-build to avoid the potential rebuild.
(From OE-Core rev: de1b57acedaaf01e10059792eb0031ceafcc8b97)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This variable isn't used by the build task, despite it's name.
(From OE-Core rev: 497ddaca42f805c119c022d77960d1eecc01436b)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These staging tasks haven't been used since ~2010, so remove them.
(From OE-Core rev: d0eba7ccf54101e708ea50afb072f1bd2cf8fb6d)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unless cachedpath is used correctly then it's just a glorified clone of
os.walk, but without any of the recent optimisations in os.walk. In
this codepath there is no point to using cachedpath.
(From OE-Core rev: 460222c54e4c65352c0687f2b6c70527cc9a2b4b)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With out these changes, a traceback displayed when a file
is listed in the SRC_URI but the file does not exist.
raise FileNotFoundError and print the patch then mark the task as failed.
(From OE-Core rev: d4926c11a4ab9148bdb640a9367c9e1891491a5b)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of reporting ${LICENSE} when a package cannot be installed
into an image because it is using an incompatible license, report the
license(s) that are actually incompatible.
(From OE-Core rev: b1863e570d4b169cd2f0ea7b4fe7c2348943cb2c)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When excluding a package from being packaged due to incompatible
licenses, report the license(s) that are actually incompatible.
(From OE-Core rev: 7d5c07e4a32a0968942ae538023c2891b59d8ab5)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of reporting ${LICENSE} when a package is identified as using
an incompatible license, report the license(s) that are actually
incompatible.
(From OE-Core rev: fb3405fac7b933a3f9d23e5faf6a3cf2d1990982)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After a number of rewrites, the code checking if a package has been
whitelisted for an incompatible license was calculating the
whitelisted packages twice (as 'whitelist' and as 'incompatwl').
(From OE-Core rev: 01dcc8c94b882d865fc72a067a90d9721ac5fea7)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes it possible to report the incompatible licenses.
(From OE-Core rev: 02a3189b4902e616f5b4277ee3d0fca10b73b4dc)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, there was SRC_DISTRIBUTE_LICENSES, an undocumented
variable that contained a static list of licenses. It was used by
expand_wildcard_licenses() to expand any wildcards used in, e.g.,
INCOMPATIBLE_LICENSE. However, since this static list of licenses has
not been kept up-to-date, many licenses were missing, with the result
that if one tried to use any of those licenses with a wildcard, no
licenses would be found, effectively ignoring that they should be
marked as incompatible.
This introduces a new (documented) variable, AVAILABLE_LICENSES, that
is automatically updated to contain all licenses found in any
directories specified by ${COMMON_LICENSE_DIR} and ${LICENSE_PATH},
and uses it instead of SRC_DISTRIBUTE_LICENSES when expanding
wildcards.
(From OE-Core rev: 8c9ef587fe499c612a878a1ab42092eb79b334ef)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some upstream projects are employing the buildtype parameter so users
can create, say, "production" versus "debug" builds. Therefore create a
configurable parameter so recipes/users can tweak it.
(From OE-Core rev: 5b12211ef7856bedafd356f6b7a03add46eca6c7)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python 3.8 has started to throw this as a warning but usage is wrong see
[1], it seems to be working by accident and not design.
Fixes
SyntaxWarning: "is" with a literal. Did you mean "=="?
[1]: https://bugs.python.org/issue34850
(From OE-Core rev: c3cb6bc21f3f84e2ed103b6438516ab85222faa9)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
goarch is used in all classes of recipes ranging from native to target,
therefore its best to contain the variables and not spill over into
recipe classes where they can adversely affect shared state reuse e.g.
go-native currently gets TUNE_FEATURES into dependency chain of
referenced variables which means go-native gets rebuilt when we change
from qemuarm to qemux86 machine types since TUNE_FEATURES is defined
with DEFAULTTUNE which would change as machines are switched
These variables are specific to arm/mips/x86 so marking them with
appropriate overrides for native recipes will be right thing here
Chose 'hardfloat' for mips which is default too, 7 for arm and sse2 for x86
somehow go-native bootstrap compiler (1.4) still needs them so feed
commonly used values or defaults.
Fixes
ERROR: go-native different signature for task do_configure.sigdata between qemux86copy and qemuarm
...
List of dependencies for variable TUNE_FEATURES changed from '{'DEFAULTTUNE', 'TUNE_FEATURES_tune-core2-32'}' to '{'DEFAULTTUNE', 'TUNE_FEATURES_tune-armv7vethf-neon'}'
changed items: {'TUNE_FEATURES_tune-core2-32', 'TUNE_FEATURES_tune-armv7vethf-neon'}
(From OE-Core rev: 83eebf5577dd0a23be937375c1a8a15e3da4fa64)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When doing reproducible builds inherit podfix to remove the possibility
of differing Perl versions causing manpages to change.
(From OE-Core rev: 5dc43edab4787906f2c9f51e51bf78d1dc966286)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Manpages generated by Pod::Man contain the version number, which isn't
reproducible if we're using the host Perl to generate manpage.
One option is to always depend on perl-native when generating manpages
but this is a heavy dependency, so instead strip out the versions in
do_install().
(From OE-Core rev: 18d8e5ac689d6eb6098f68ac785f43e9d5f5938a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids a directory not present error.
Fix a comment typo whilst here.
(From OE-Core rev: 1360d8d7d99b70a80c8cdbc1fc6d9e6752483139)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous patches meant the mkdir might no longer match the final target
directory. Fix this.
(From OE-Core rev: 0af4dae84099e8632a9ea6a4afdbea2f232bb170)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
STATE_PKG may have been changed by sstate_report_unihash so don't
cache the variable's value.
(From OE-Core rev: be29a25400c4ea285ab3f588c5831f00ba5d4f63)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its rather antisocial to overwrite SSTATE_PKG with an expanded form for the variable
and it stops the value of BB_UNIHASH being changed when the package is written out.
Instead of expanding the variable, append to it instead to avoid this rather
hard to figure out behaviour and allow the siggen code to behave as expected.
(From OE-Core rev: 62eca02024b4c44d618ab9bcf87a3166c886dadb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switch to recently released version 1.1 of NVD CVE JSON feed, as in
https://nvd.nist.gov/General/News/JSON-1-1-Vulnerability-Feed-Release
it is mentioned that
Due to changes required to support CVSS v3.1 scoring, the JSON
vulnerability feeds must be modified. This will require the consumers
of this data to update their internal processes. We will be providing
the JSON 1.1 schema on the data feeds page and the information below
to prepare for this transition.
...
The JSON 1.1 data feeds will be available on September 9th, 2019. At
that time the current JSON 1.0 data feeds will no longer available.
This change was tested briefly by issuing 'bitbake core-image-minimal'
with 'cve-check.bbclass' inherited via local.conf, and then comparing
the content between the resulting two
'DEPLOY_DIR_IMAGE/core-image-minimal-qemux86.cve' files, which did not
seem to contain any other change, except total of 167 entries like
CVSS v3 BASE SCORE: 0.0
were replaced with similar 'CVSS v3 BASE SCORE:' entries which had
scores that were greater than '0.0' (up to '9.8').
(From OE-Core rev: cc20e4d8ff2f3aa52a2658404af9a0ff358cc323)
Signed-off-by: Niko Mauno <niko.mauno@iki.fi>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Meson has support for downloading subprojects using something called
wraps. This interferes with bitbake's expectations of all downloads
being done by the fetch task. To avoid this, tell meson to not
download any wraps.
Suggested-by: Mattias Jernberg <mattias.jernberg@axis.com>
(From OE-Core rev: b547637ad84bad8f7fe27193bf636541f8588ae8)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As Alex Kanavin found, dependencies aren't always populated, particularly
with the hash equivalence server enabled locally:
'bitbake core-image-minimal' with gtk+ disabled.
can confirm with: $ tmp/work/x86_64-linux/qemu-helper-native/1.0-r1/recipe-sysroot-native/usr/bin/qemu-system-x86_64 -display gtk
qemu-system-x86_64: Display 'gtk' is not available.
Enable gtk in local.conf with: PACKAGECONFIG_append_pn-qemu-system-native = " gtk+"
'bitbake core-image-minimal', without deleting tmp/
$ tmp/work/x86_64-linux/qemu-helper-native/1.0-r1/recipe-sysroot-native/usr/bin/qemu-system-x86_64 -display gtk
qemu-system-x86_64: Display 'gtk' is not available.
This change ensures the dependencies are correctly handled as the full
sysroot is always depended upon even if things come from sstate.
(From OE-Core rev: d40853b10dd9f01d6a8dd4edcb941cfa8a544922)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This helps fix an issue where python interpreter in hg script could
overflow the BINPRM_BUF_SIZE which is 128 on most of systems, because
interpreter is hardcoded and build paths can be deep.
This patch helps, because now the absolute python interp path in hg can
be replaced with '/usr/bin/env python3' and it will ensure that python3
from native install is used instead of the one on host.
(From OE-Core rev: 3adaaab787bd105cc0049d36c7b8caeaa9b633e5)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Cc: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This ensures cmake uses the correct python binary rather than
potentially, the host system one.
(From OE-Core rev: afb8d456360d86254ab00f05866c34d947d02749)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add Zstandard (or just Zstd) compression support. This allows to
create Zstd compressed tarballs by using tar.zst as IMAGE_FSTYPES.
(From OE-Core rev: 77e0f502020085875bbc74da66815c89f19a9351)
Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test was failing to split the URL list to individual URLs, so if
SRC_URI is something like this then the test incorrectly triggers:
SRC_URI = "git://github.com/foo http://example.com/archive/foo"
Fix this by splitting the SRC_URI list and iterating through the URIs
one at time.
[ YOCTO #13660 ]
(From OE-Core rev: ddd2c5624404848ee668dabec0f61599ab5003e4)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When kernel-devicetree is in RRECOMMENDS such as via variable
MACHINE_EXTRA_RRECOMMENDS for some bsp, it shows QA warning of multilib:
| WARNING: lib32-packagegroup-base-1.0-r83 do_package: QA Issue:
| lib32-packagegroup-base package lib32-packagegroup-machine-base
| - suspicious values 'kernel-devicetree' in RRECOMMENDS [multilib]
Add kernel-devicetree to exceptions to fix the QA issue. Because there
are already 3 kernel related criteria, simplify them by judging package
names whether start with 'kernel-'. And also refactor to remove
duplicate 'not'.
(From OE-Core rev: 5e4504026c6358c7d5649843dc354247f5972558)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Find lists files in the order that the filesystem has registered them,
which can vary. Adding a sort should have minimal performance impact.
Also use the --reproducible option to cpio.
(From OE-Core rev: 930c1f69c928e21bda6bef7aad926d335195e107)
Signed-off-by: Ernst Sjöstrand <ernst.sjostrand@verisure.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In most cases, the RPATH is stripped out when the ELF file is packages,
but by then the damage is done from a reproducible perspective because
this absolute path is hashed as part of the build-id generated at link
time ([1] has a good explanation). Fortunately, newer cmake has an
option to generated relative RPATHs that use $ORIGIN to set the path, so
set it in the toolchain file.
[1]: https://gitlab.kitware.com/cmake/cmake/issues/18413
(From OE-Core rev: 44e77d3f97af4cd4ad8bc0984f093a116a830986)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Do not reset libdir and base_libdir in sysroot_strip, and just pass crude
paths as they will be reset later in strip_execs.
(From OE-Core rev: eab7f448aa537539b45ee21df4ea25de97e60a7e)
Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The opkg-utils change didn't trigger a full repackage due to its mention
in layer.conf for update-alternatives hash exclusion. Do it manually
to invalidate broken ipks.
(From OE-Core rev: 6dd59dffd73618e4c0979b284a7968055a4eb8fb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Older versions break opkg-build when reproducible builds are enabled.
Rather than trying to be selective based on which features are enabled,
lets just make this a minimum version.
(From OE-Core rev: 0fdc43da005c3c6102cf07383ad6f451d2203fa5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This code used to construct a single SQL statement that fetched the NVD data for
every CVE requested. For recipes such as the kernel where there are over 2000
CVEs to report this can hit the variable count limit and the query fails with
"sqlite3.OperationalError: too many SQL variables". The default limit is 999
variables, but some distributions such as Debian set the default to 250000.
As the NVD table has an index on the ID column, whilst requesting the data
CVE-by-CVE is five times slower when working with 2000 CVEs the absolute time
different is insignificant: 0.05s verses 0.01s on my machine.
(From OE-Core rev: 53d0cc1e9b7190fa66d7ff1c59518f91b0128d99)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove obsolete Python 2 code, and use convenience methods for neatness.
(From OE-Core rev: f19253cc9e70c974a8e21a142086c13d7cde04ff)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A previous optimisation was premature and resulted in false-negatives in the report.
Rewrite the checking algorithm to first get the list of potential CVEs by
vendor:product, then iterate through every matching CPE for that CVE to
determine if the bounds match or not. By doing this in two stages we can know
if we've checked every CPE, instead of accidentally breaking out of the scan too
early.
(From OE-Core rev: d61aff9e22704ad69df1f7ab0f8784f4e7cc0c69)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Besides checking DISTRO_FEATURES for required or conflicting features,
being able to check MACHINE_FEATURES and/or COMBINED_FEATURES may also
be useful at times.
Temporarily support the old class name with a warning about future
deprecation.
(From OE-Core rev: 5f4875b950ce199e91f99c8e945a0c709166dc14)
Signed-off-by: Denys Dmytriyenko <denys@ti.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The patch scanner works with patch files in the layer, not in the workdir, so it
doesn't need to unpack.
(From OE-Core rev: 2cba6ada970deb5156e1ba0182f4f372851e3c17)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the host system is expected to have enough capabilities that the
buildtools-tarball is not required, we don't need to bundle it.
This can save some significant space, especially when using with a minimal
eSDK.
minimal eSDK - core-image-minimal-qemux86-64
with buildtools-tarball - 34 MB installer - 281 MB installed
without buildtoools-tarball - 11 MB installer - 48 MB installed
(From OE-Core rev: e2b215054d11527a05210066b9bfd7d9a2054e01)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix parentheses placement in the message from:
Package version for package X went backwards which would break package feeds from (Y to Z)
to this one:
Package version for package X went backwards which would break package feeds (from Y to Z)
(From OE-Core rev: 3a5a61fb6b3f811bd4f7232ba902afcfd2019154)
Signed-off-by: Denys Dmytriyenko <denys@ti.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
do_ar_recipe fails on perf recipe on line:
include ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '', d)}
1. "${...}" part expands into empty string
2. bb.utils.which() takes empty string and returns first directory name from bbpath
3. shutil.copy() fails on copying directory:
Exception: IsADirectoryError: [Errno 21] Is a directory: ......
Hence, check "incfile" variable on each step.
(From OE-Core rev: 3b393da7d2d15de12e2a8a9c11591078b40b188a)
Signed-off-by: grygorii tertychnyi <gtertych@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Clean the displayed path so it clearly shows the package name and path in that
package.
(From OE-Core rev: 46bf5f0db164e88c2effdbf5437f4d0836e4aadd)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that ldconfig may get installed from a feed, use it when it's
available on the target.
(From OE-Core rev: 0f09f19dc48edf2ac50b554c18c217b7f97d4ae3)
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
