python 3.8 has started to throw this as a warning but usage is wrong see
[1], it seems to be working by accident and not design.
Fixes
SyntaxWarning: "is" with a literal. Did you mean "=="?
[1]: https://bugs.python.org/issue34850
(From OE-Core rev: c3cb6bc21f3f84e2ed103b6438516ab85222faa9)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
goarch is used in all classes of recipes ranging from native to target,
therefore its best to contain the variables and not spill over into
recipe classes where they can adversely affect shared state reuse e.g.
go-native currently gets TUNE_FEATURES into dependency chain of
referenced variables which means go-native gets rebuilt when we change
from qemuarm to qemux86 machine types since TUNE_FEATURES is defined
with DEFAULTTUNE which would change as machines are switched
These variables are specific to arm/mips/x86 so marking them with
appropriate overrides for native recipes will be right thing here
Chose 'hardfloat' for mips which is default too, 7 for arm and sse2 for x86
somehow go-native bootstrap compiler (1.4) still needs them so feed
commonly used values or defaults.
Fixes
ERROR: go-native different signature for task do_configure.sigdata between qemux86copy and qemuarm
...
List of dependencies for variable TUNE_FEATURES changed from '{'DEFAULTTUNE', 'TUNE_FEATURES_tune-core2-32'}' to '{'DEFAULTTUNE', 'TUNE_FEATURES_tune-armv7vethf-neon'}'
changed items: {'TUNE_FEATURES_tune-core2-32', 'TUNE_FEATURES_tune-armv7vethf-neon'}
(From OE-Core rev: 83eebf5577dd0a23be937375c1a8a15e3da4fa64)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When doing reproducible builds inherit podfix to remove the possibility
of differing Perl versions causing manpages to change.
(From OE-Core rev: 5dc43edab4787906f2c9f51e51bf78d1dc966286)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Manpages generated by Pod::Man contain the version number, which isn't
reproducible if we're using the host Perl to generate manpage.
One option is to always depend on perl-native when generating manpages
but this is a heavy dependency, so instead strip out the versions in
do_install().
(From OE-Core rev: 18d8e5ac689d6eb6098f68ac785f43e9d5f5938a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids a directory not present error.
Fix a comment typo whilst here.
(From OE-Core rev: 1360d8d7d99b70a80c8cdbc1fc6d9e6752483139)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous patches meant the mkdir might no longer match the final target
directory. Fix this.
(From OE-Core rev: 0af4dae84099e8632a9ea6a4afdbea2f232bb170)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
STATE_PKG may have been changed by sstate_report_unihash so don't
cache the variable's value.
(From OE-Core rev: be29a25400c4ea285ab3f588c5831f00ba5d4f63)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its rather antisocial to overwrite SSTATE_PKG with an expanded form for the variable
and it stops the value of BB_UNIHASH being changed when the package is written out.
Instead of expanding the variable, append to it instead to avoid this rather
hard to figure out behaviour and allow the siggen code to behave as expected.
(From OE-Core rev: 62eca02024b4c44d618ab9bcf87a3166c886dadb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switch to recently released version 1.1 of NVD CVE JSON feed, as in
https://nvd.nist.gov/General/News/JSON-1-1-Vulnerability-Feed-Release
it is mentioned that
Due to changes required to support CVSS v3.1 scoring, the JSON
vulnerability feeds must be modified. This will require the consumers
of this data to update their internal processes. We will be providing
the JSON 1.1 schema on the data feeds page and the information below
to prepare for this transition.
...
The JSON 1.1 data feeds will be available on September 9th, 2019. At
that time the current JSON 1.0 data feeds will no longer available.
This change was tested briefly by issuing 'bitbake core-image-minimal'
with 'cve-check.bbclass' inherited via local.conf, and then comparing
the content between the resulting two
'DEPLOY_DIR_IMAGE/core-image-minimal-qemux86.cve' files, which did not
seem to contain any other change, except total of 167 entries like
CVSS v3 BASE SCORE: 0.0
were replaced with similar 'CVSS v3 BASE SCORE:' entries which had
scores that were greater than '0.0' (up to '9.8').
(From OE-Core rev: cc20e4d8ff2f3aa52a2658404af9a0ff358cc323)
Signed-off-by: Niko Mauno <niko.mauno@iki.fi>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Meson has support for downloading subprojects using something called
wraps. This interferes with bitbake's expectations of all downloads
being done by the fetch task. To avoid this, tell meson to not
download any wraps.
Suggested-by: Mattias Jernberg <mattias.jernberg@axis.com>
(From OE-Core rev: b547637ad84bad8f7fe27193bf636541f8588ae8)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As Alex Kanavin found, dependencies aren't always populated, particularly
with the hash equivalence server enabled locally:
'bitbake core-image-minimal' with gtk+ disabled.
can confirm with: $ tmp/work/x86_64-linux/qemu-helper-native/1.0-r1/recipe-sysroot-native/usr/bin/qemu-system-x86_64 -display gtk
qemu-system-x86_64: Display 'gtk' is not available.
Enable gtk in local.conf with: PACKAGECONFIG_append_pn-qemu-system-native = " gtk+"
'bitbake core-image-minimal', without deleting tmp/
$ tmp/work/x86_64-linux/qemu-helper-native/1.0-r1/recipe-sysroot-native/usr/bin/qemu-system-x86_64 -display gtk
qemu-system-x86_64: Display 'gtk' is not available.
This change ensures the dependencies are correctly handled as the full
sysroot is always depended upon even if things come from sstate.
(From OE-Core rev: d40853b10dd9f01d6a8dd4edcb941cfa8a544922)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This helps fix an issue where python interpreter in hg script could
overflow the BINPRM_BUF_SIZE which is 128 on most of systems, because
interpreter is hardcoded and build paths can be deep.
This patch helps, because now the absolute python interp path in hg can
be replaced with '/usr/bin/env python3' and it will ensure that python3
from native install is used instead of the one on host.
(From OE-Core rev: 3adaaab787bd105cc0049d36c7b8caeaa9b633e5)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Cc: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This ensures cmake uses the correct python binary rather than
potentially, the host system one.
(From OE-Core rev: afb8d456360d86254ab00f05866c34d947d02749)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add Zstandard (or just Zstd) compression support. This allows to
create Zstd compressed tarballs by using tar.zst as IMAGE_FSTYPES.
(From OE-Core rev: 77e0f502020085875bbc74da66815c89f19a9351)
Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test was failing to split the URL list to individual URLs, so if
SRC_URI is something like this then the test incorrectly triggers:
SRC_URI = "git://github.com/foo http://example.com/archive/foo"
Fix this by splitting the SRC_URI list and iterating through the URIs
one at time.
[ YOCTO #13660 ]
(From OE-Core rev: ddd2c5624404848ee668dabec0f61599ab5003e4)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When kernel-devicetree is in RRECOMMENDS such as via variable
MACHINE_EXTRA_RRECOMMENDS for some bsp, it shows QA warning of multilib:
| WARNING: lib32-packagegroup-base-1.0-r83 do_package: QA Issue:
| lib32-packagegroup-base package lib32-packagegroup-machine-base
| - suspicious values 'kernel-devicetree' in RRECOMMENDS [multilib]
Add kernel-devicetree to exceptions to fix the QA issue. Because there
are already 3 kernel related criteria, simplify them by judging package
names whether start with 'kernel-'. And also refactor to remove
duplicate 'not'.
(From OE-Core rev: 5e4504026c6358c7d5649843dc354247f5972558)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Find lists files in the order that the filesystem has registered them,
which can vary. Adding a sort should have minimal performance impact.
Also use the --reproducible option to cpio.
(From OE-Core rev: 930c1f69c928e21bda6bef7aad926d335195e107)
Signed-off-by: Ernst Sjöstrand <ernst.sjostrand@verisure.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In most cases, the RPATH is stripped out when the ELF file is packages,
but by then the damage is done from a reproducible perspective because
this absolute path is hashed as part of the build-id generated at link
time ([1] has a good explanation). Fortunately, newer cmake has an
option to generated relative RPATHs that use $ORIGIN to set the path, so
set it in the toolchain file.
[1]: https://gitlab.kitware.com/cmake/cmake/issues/18413
(From OE-Core rev: 44e77d3f97af4cd4ad8bc0984f093a116a830986)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Do not reset libdir and base_libdir in sysroot_strip, and just pass crude
paths as they will be reset later in strip_execs.
(From OE-Core rev: eab7f448aa537539b45ee21df4ea25de97e60a7e)
Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The opkg-utils change didn't trigger a full repackage due to its mention
in layer.conf for update-alternatives hash exclusion. Do it manually
to invalidate broken ipks.
(From OE-Core rev: 6dd59dffd73618e4c0979b284a7968055a4eb8fb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Older versions break opkg-build when reproducible builds are enabled.
Rather than trying to be selective based on which features are enabled,
lets just make this a minimum version.
(From OE-Core rev: 0fdc43da005c3c6102cf07383ad6f451d2203fa5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This code used to construct a single SQL statement that fetched the NVD data for
every CVE requested. For recipes such as the kernel where there are over 2000
CVEs to report this can hit the variable count limit and the query fails with
"sqlite3.OperationalError: too many SQL variables". The default limit is 999
variables, but some distributions such as Debian set the default to 250000.
As the NVD table has an index on the ID column, whilst requesting the data
CVE-by-CVE is five times slower when working with 2000 CVEs the absolute time
different is insignificant: 0.05s verses 0.01s on my machine.
(From OE-Core rev: 53d0cc1e9b7190fa66d7ff1c59518f91b0128d99)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove obsolete Python 2 code, and use convenience methods for neatness.
(From OE-Core rev: f19253cc9e70c974a8e21a142086c13d7cde04ff)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A previous optimisation was premature and resulted in false-negatives in the report.
Rewrite the checking algorithm to first get the list of potential CVEs by
vendor:product, then iterate through every matching CPE for that CVE to
determine if the bounds match or not. By doing this in two stages we can know
if we've checked every CPE, instead of accidentally breaking out of the scan too
early.
(From OE-Core rev: d61aff9e22704ad69df1f7ab0f8784f4e7cc0c69)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Besides checking DISTRO_FEATURES for required or conflicting features,
being able to check MACHINE_FEATURES and/or COMBINED_FEATURES may also
be useful at times.
Temporarily support the old class name with a warning about future
deprecation.
(From OE-Core rev: 5f4875b950ce199e91f99c8e945a0c709166dc14)
Signed-off-by: Denys Dmytriyenko <denys@ti.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The patch scanner works with patch files in the layer, not in the workdir, so it
doesn't need to unpack.
(From OE-Core rev: 2cba6ada970deb5156e1ba0182f4f372851e3c17)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the host system is expected to have enough capabilities that the
buildtools-tarball is not required, we don't need to bundle it.
This can save some significant space, especially when using with a minimal
eSDK.
minimal eSDK - core-image-minimal-qemux86-64
with buildtools-tarball - 34 MB installer - 281 MB installed
without buildtoools-tarball - 11 MB installer - 48 MB installed
(From OE-Core rev: e2b215054d11527a05210066b9bfd7d9a2054e01)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix parentheses placement in the message from:
Package version for package X went backwards which would break package feeds from (Y to Z)
to this one:
Package version for package X went backwards which would break package feeds (from Y to Z)
(From OE-Core rev: 3a5a61fb6b3f811bd4f7232ba902afcfd2019154)
Signed-off-by: Denys Dmytriyenko <denys@ti.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
do_ar_recipe fails on perf recipe on line:
include ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '', d)}
1. "${...}" part expands into empty string
2. bb.utils.which() takes empty string and returns first directory name from bbpath
3. shutil.copy() fails on copying directory:
Exception: IsADirectoryError: [Errno 21] Is a directory: ......
Hence, check "incfile" variable on each step.
(From OE-Core rev: 3b393da7d2d15de12e2a8a9c11591078b40b188a)
Signed-off-by: grygorii tertychnyi <gtertych@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Clean the displayed path so it clearly shows the package name and path in that
package.
(From OE-Core rev: 46bf5f0db164e88c2effdbf5437f4d0836e4aadd)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that ldconfig may get installed from a feed, use it when it's
available on the target.
(From OE-Core rev: 0f09f19dc48edf2ac50b554c18c217b7f97d4ae3)
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Instead of overwriting the stamp name with 'dummy', handle
setscene promotion in the default case block
* Merge '*do_image_complete_setscene*' and '*do_image_qa_setscene*'
case handling
(From OE-Core rev: 3fe6574c93a02e2e67d16e66f24be1053af383b7)
Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its confusing to keep seeing sstate summary messages when hash equivalency is
active. This adds an option to control it. A default value is given which
maintains compatibility with different bitbake versions.
(From OE-Core rev: 038004866ff6650bcff7bb1bde36de6c0f451d29)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, if files move between recipes, do_prepare_sysroot can fail with a message like
Exception: FileExistsError: [Errno 17] File exists:
'TMPDIR/sysroots-components/core2-64/libx11/usr/include/X11/extensions/XKBgeom.h' ->
'TMPDIR/work/core2-64-poky-linux/gtk+3/3.24.8-r0/recipe-sysroot/usr/include/X11/extensions/XKBgeom.h'
This is because files are removed and then added per package. What needs to
happen is all removes need to be processed, then all additions.
This patch changes the code to process in two phases, removals first, then additions,
which avoids the problem.
(From OE-Core rev: e3e5ace6e68d5fe68e4add301a44c1a1b8607411)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
CVEs that are whitelisted or were not vulnerable when there are version
comparisons were not included in the report, so alter the logic to ensure that
all relevant CVEs are in the report for completeness.
(From OE-Core rev: 98256ff05fcfe9d5ccad360582c36eafb577c264)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change allows layers adding packages other than locales automatically to
images based on languages selected in IMAGE_LINGUAS.
E.g if a layer has recipes creating packages as:
| ${PN}-foo-en / ${PN}-foo-de / ${PN}-foo-it / ..
it would set
| IMAGE_LINGUAS_COMPLEMENTARY_append = " *-foo-%s"
in its layer.conf to enable auto-adding.
(From OE-Core rev: 3a9e9585aeb67234901b2f842113dbe20c3801b3)
Signed-off-by: Andreas Müller <schnitzeltony@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
kernel.bbclass installs non-standard kernels (where
KERNEL_PACKAGE_NAME is not "kernel") in a subdirectory of ${DEPLOYDIR}.
To achieve this kernel_do_deploy sets the deployDir shell variable to
${DEPLOYDIR} for the standard kernel or
${DEPLOYDIR}/${KERNEL_DEPLOYSUBDIR} for non-standard kernels.
kernel-devicetree.bbclass's do_deploy_append ought to do the same
and can do so by using the same shell variable.
(From OE-Core rev: db5752911fe085337b9d3d4af85f89a0c664388e)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
kernel.bbclass installs non-standard kernels (where
KERNEL_PACKAGE_NAME is not "kernel") in a subdirectory of ${DEPLOYDIR}.
To achieve this kernel_do_deploy sets the deployDir shell variable to
${DEPLOYDIR} for the standard kernel or
${DEPLOYDIR}/${KERNEL_DEPLOYSUBDIR} for non-standard kernels.
kernel-fitimage.bbclass's kernel_do_deploy_append ought to do the same
and can do so by using the same shell variable.
(From OE-Core rev: d324b22d32eaea9e4337c963c8b1a33b0ba6a2dd)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tweak the write loop slightly to avoid dict lookups that can easily be
done in the for loop.
(From OE-Core rev: 35c65b7336c52c19810e3e9e87a36a8636ac4120)
Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace copy-and-if with a filtering list comprehension.
(From OE-Core rev: eb763856be8da854d37c7d4b8e8d645ab1d3fa06)
Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By exporting ICECC_CC, ICECC_CXX, and ICECC_VERSION in a wrapper-script,
and putting this wrapper-script in the PATH, the Makefiles generated by CMake or
the autotools are able to function correctly outside of bitbake.
This provides a convenient developer workflow in which the
modify-compile-unittest cycle can happen directly in the ${B} directory.
The `rm -f $ICE_PATH/$compiler` line is transitional,
and can go at some later date (October 2020 or later, perhaps).
(From OE-Core rev: 40d74cb1d0ddce930267e49764cacb263b244091)
Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The find_program command will fail if it is used on a tool that is listed in
ASSUME_PROVIDED. This is because these tools are in the hosttools directory
which is not listed in CMAKE_FIND_ROOT_PATH so cmake will not find them.
Adding the directory HOSTTOOLS_DIR to the CMAKE_FIND_ROOT_PATH variable fixes
the initial issue of needing to search for tools in ASSUME_PROVIDED.
Note that this change alone does not fix the issue because find_program will
by default only look into the subdirectories bin and usr/bin under the paths
in CMAKE_FIND_ROOT_PATH to find the programs and the hosttools directory has
instead the symlinks directly present without these subdirectories.
Set CMAKE_PROGRAM_PATH to by default include the root directory so
find_program can search the hosttools directory without needing the prefix
directories.
(From OE-Core rev: 7847f431cd8db59fce8c9401a603c4b0678ee16d)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was used for build result reporting back in the OE-Classic days. Its
not been touched in years, isn't used as far as I know and can be removed.
(From OE-Core rev: f5d107f2ac448d6fe9fe8e4a2451e5cefdfd892a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
