Commit Graph

7759 Commits

Author SHA1 Message Date
Khem Raj
b1eea8178a go: Enable CGO and pie buildmode on rv64
go1.16 has added CGO support for riscv64 arch

(From OE-Core rev: 8e078238312948e8c7b09c66ba7a186512e995d3)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26 15:21:20 +00:00
Khem Raj
510406ddf5 goarch: Use softfloat instead of 387 for 386 goarch
387 has been removed from go 1.16 see [1]

[1] https://github.com/golang/go/issues/40255

(From OE-Core rev: fd549c139f534f28974c86a4e4c7acf6f5c40a79)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26 15:21:20 +00:00
Richard Purdie
c4baddc0d0 reproducible: Improve SOURCE_DATE_EPOCH_FALLBACK handling
Ensure the fallback value if used is written to the SDE file
and hence stored in sstate, reducing any confusion within the
code over '0' values.

Bump the HASHEQUIV_VERSION since we've had a ton of trouble
with ensuring this rolls out correctly on the autobuilder so
others may too, take a clean slate for it.

(From OE-Core rev: 53ffffa32b76330835287dfc05d4e4796841af08)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-24 00:19:38 +00:00
Jan-Simon Möller
7e9c2f33d4 reproducible_builds: SOURCE_DATE_EPOCH should not be 0
A SOURCE_DATE_EPOCH of 0 might be misinterpreted by namely rpm as no SDE.

 e.g.:
 char *srcdate = getenv(SOURCE_DATE_EPOCH);
 if (srcdate && rpmExpandNumeric(%{?clamp_mtime_to_source_date_epoch}))

Solve this by:
- providing a SOURCE_DATE_EPOCH_FALLBACK variable != 0
- changing defaults to use it
- using SOURCE_DATE_EPOCH_FALLBACK also when reading 0 out of sstate-cache

(From OE-Core rev: 9a1dde74e794362399193dc3f81c9685a83d0776)

Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-24 00:19:38 +00:00
Milan Shah
8b299a62c5 report-error.bbclass: Add layer and bitbake version info to error report
Instead of just providing local.conf info, add layer names and their
revisions with bitbake version information into error report
makes it easier to understand and reproduce failed build.

[YOCTO #9700]

(From OE-Core rev: a5afd7b1857c0012ab6e3d8bbafa67a96ff9e9e0)

Signed-off-by: Milan Shah <mshah@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-23 22:35:01 +00:00
Klaus Heinrich Kiwi
cfc0e21b10 kernel-fitimage: Don't use unit addresses on FIT
Das U-Boot 2021.4-rc1 has the following commit:

    commit 3f04db891a353f4b127ed57279279f851c6b4917
    Author: Simon Glass <sjg@chromium.org>
    Date:   Mon Feb 15 17:08:12 2021 -0700

        image: Check for unit addresses in FITs

        Using unit addresses in a FIT is a security risk. Add a check for
        this and disallow it.

        CVE-2021-27138

Adjust the kernel-fitimage.bbclass accordingly to not use unit
addresses. This changte is required before we can bump U-Boot to 2021.4.

(From OE-Core rev: 6047be9f8f0f5d616fda11d83b682c1b8aeaa0ae)

Signed-off-by: Klaus Heinrich Kiwi <klaus@linux.vnet.ibm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-23 22:35:01 +00:00
Khem Raj
47a2fffca4 goarch.bbclass: Fix ppc64le detection
ppc64le should go above ppc64 in checks otherwise it gets subsumed
wrongly with ppc64 check

(From OE-Core rev: 126ecfe0d2ce65b2d2cf5598612e60bad7b3302e)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-23 22:35:01 +00:00
Khem Raj
1963444d58 linuxloader: Deal with little-endian ppc64 ldso name
On ppc64le 'le' should be appended to ldso name

(From OE-Core rev: dd221cdc8630a5adba1f87ff7a237021e9a95548)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-23 22:35:00 +00:00
Richard Purdie
3266d1ee1c license_image: Don't canonicalise INCOMPATIBLE_LICENSE
The code internally correctly handles canonicalisation of these license
fields, we shouldn't call it manually. The issue is that the fields can
contain wildcards and GPLv3* means something quite different to GPLv3-only*.

(From OE-Core rev: 957f48ce78ad1865653e8b8f25909a65ba8fc535)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-21 22:01:28 +00:00
Richard Purdie
55a3da9c0d licenses: Update INCOMPATIBLE_LICENSE for 'or-later' handling
Where a user adds "GPLv3" to INCOMPATIBLE_LICENSE they almost certainly
mean both GPLv3-only and GPLv3-or-later. Update the code to handle this
correctly.

(From OE-Core rev: 08cbf17485b6443a6118acfac8200eb6c61445a3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-21 22:01:28 +00:00
Richard Purdie
6638a7289e licenses: Fix canonical license for 'or-later' handling
GPLv2 and GPLv2+ are two difference licenses with different meanings
and we can't just pretend they're the same thing. Change the code
to treat them separately.

(From OE-Core rev: d1baf74ac92fe0c8c32dff101fd77d77f70fd583)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-21 22:01:28 +00:00
Richard Purdie
87ed5dab07 recipes: Update common-licenses references to match new names
The licenses were renamed to match their SPDX names, fix the
references in LIC_FILES_CHKSUM in OE-Core.

(From OE-Core rev: 9af48917cfe583d2db9e1e088c7e396fcf638949)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-21 22:01:28 +00:00
Jan-Simon Möller
0ccf53d8e1 package_rpm: Enable use_source_date_epoch_as_buildtime in package_rpm class
Adding --define 'use_source_date_epoch_as_buildtime 1' to rpmbuild
ensure that the rpm header does have a consistent BUILDTIME tag.

This allows the rpms to be built reproducibly.

(From OE-Core rev: f16721df6b3d0d3e681d9a1b3393417bd62b6275)

Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-19 11:17:11 +00:00
Konrad Weihmann
c640057eda cmake: set CMAKE_EXPORT_NO_PACKAGE_REGISTRY
if a cmake file uses export(PACKAGE) command it creates a
folder ~/.cmake/package/<name> in the current user's
home-dir.
fix this host contermination by setting CMAKE_EXPORT_NO_PACKAGE_REGISTRY
to ON by default, which makes the export() command do nothing

(From OE-Core rev: fc7bd81cfe52c35fe26d146c1def8a7493bd2e25)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-19 11:17:11 +00:00
Anthony Bagwell
7cf4daf53e kernel-fitimage: fix dtbo support for fit images
8a2f4e143 added support for u-boot boot script but missed adding the
extra parameter to fitimage_emit_section_config on the dtbo branch

(From OE-Core rev: 22bac8aea0d5d28cc5a3bf20edf638225cce2f88)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-19 11:17:11 +00:00
Alejandro Hernandez Samaniego
cc0e18a7f7 sstate.bbclass: Split sstate summary into locally and network found artifacts
Modify sstate class to add extra output, differentiating
between artifacts found locally and those found over the network.

(From OE-Core rev: f017a2f677fa289c36b0e56ce480f69d7674ca11)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-19 11:17:11 +00:00
Ross Burton
27e9c6871a rootfs_deb: handle aarch64 SDK_ARCH
This class maps from OE architectures to dpkg architectures, but wasn't
handling aarch64 correctly.

(From OE-Core rev: 315fe8ab3b5bb8a709be98cd8998d4525fbe8725)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-18 23:40:01 +00:00
Richard Purdie
2c0511b2a1 distutils3-base: Fix after native packaging changes
Now that native recipes have PACKAGES, this DEPENDS construct doesn't
work. It applies to target and nativesdk recipes so adjust accordingly.

(From OE-Core rev: 5d459ba1c13c89b246a0f8d743027e1de93da910)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-17 11:01:17 +00:00
Richard Purdie
522607e704 classes: Drop now unneeded umask flags
Now that 022 is the default in BB_DEFAULT_UMASK in bitbake.conf, we
don't need any of these task flags, clean up.

(From OE-Core rev: 816fca781943a7dbf40391d9db34c7bf12711962)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-16 11:28:17 +00:00
Yoann Congal
f13c22358a npm.bbclass: avoid building target nodejs for native npm recipes
The current recipe unconditionally RDEPENDS on nodejs (the target one).
When building on the "-native recipe" of "BBCLASSEXTEND native" recipe,
the target nodejs is unnecessarily built.

This patch fixes this by only RDEPENDS on nodejs when building for the target.

(From OE-Core rev: 92a9a86df9e3bcffb13d2f8b5dcbe7822170f734)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-12 23:32:16 +00:00
akuster
73befa8f41 cve-check: add include/exclude layers
There are times when exluding or including a layer
may be desired. This provide the framwork for that via
two variables. The default is all layers in bblayers.

CVE_CHECK_LAYER_INCLUDELIST
CVE_CHECK_LAYER_EXCLUDELIST

(From OE-Core rev: 5fdde65ef58b4c1048839e4f9462b34bab36fc22)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-09 08:56:10 +00:00
akuster
29e280f7ee cve-check.bbclass: add layer to cve log
Lets include whcih layer a package belongs to and
add it to the cve logs

(From OE-Core rev: 00d965bb42dc427749a4c3985af56ceffff80457)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-09 08:56:10 +00:00
Ross Burton
20f926f086 autotools: no need to depend on gnu-config
autoconf 2.70 onwards installs its own copies of config.guess/config.sub
which we keep up to date when autoconf builds, so there's no need to
depend on gnu-config for those files.

(From OE-Core rev: 332145c34b4aac2e74a713070af25414e1fd8c9c)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-07 22:26:42 +00:00
Ross Burton
bea9c8b988 autotools: remove intltoolize logic
autoconf 2.70 now invokes intltoolize, so there's no need to do it again
in this class.

(From OE-Core rev: e24ac6605aeaae42475d3f753dc9452093af5a14)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-07 22:26:42 +00:00
Ross Burton
66dab7e17b autotools: disable gtkdocize for now
This breaks kmod, so for now we can continue to do it ourselves.

(From OE-Core rev: 628e0263e3bb768ea771d0e0260fdb18e16c871e)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-07 22:26:42 +00:00
Ross Burton
1dbef3ce98 autotools: don't warn about obsolete usage
New autoconf warns about obsolete macro usage, but there is quite
a lot of obsolete usage in the wild which isn't really in our problem.

(From OE-Core rev: a152b5a37aec247b0540b82ad6c9bdc20c532d21)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-07 22:26:42 +00:00
Mike Looijmans
2fcbd0f115 license_image.bbclass: Don't attempt to symlink to the same file
Sometimes (that is, in all my builds) the lic_manifest_dir and
lic_manifest_symlink_dir end up pointing to the same file, resulting
in an error like this:
  Exception: FileExistsError: [Errno 17] File exists: '/.../tmp-glibc/deploy/licenses/my-image-tdkz15' -> '/.../tmp-glibc/deploy/licenses/my-image-tdkz15'

First check to see if this is the case before attempting to create
the link.

(From OE-Core rev: 50f83fb542065eaf7a20ac07b63ae06441ada180)

Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-06 09:03:00 +00:00
Martin Jansa
c0ec68956c image_types.bbclass: tar: use posix format instead of gnu
* gnu isn't compatible with  --xattrs used e.g. here:
d3a832f66e/classes/image_types_ostree.bbclass (L16)
causing do_image_tar failing with:

| tar: --xattrs can be used only on POSIX archives
| Try 'tar --help' or 'tar --usage' for more information.

* https://www.gnu.org/software/tar/manual/html_chapter/tar_8.html
  says about posix format:

  This is the most flexible and feature-rich format.
  It does not impose any restrictions on file sizes or file name lengths.
  This format is quite recent, so not all tar implementations are able to handle it properly.
  However, this format is designed in such a way that any tar implementation able to read `ustar'
  archives will be able to read most `posix' archives as well, with the only exception that any
  additional information (such as long file names etc.) will in such case be extracted as plain
  text files along with the files it refers to.

  This archive format will be the default format for future versions of GNU tar.

  and:

  The default format for GNU tar is defined at compilation time.
  You may check it by running tar --help, and examining the last lines of its output.
  Usually, GNU tar is configured to create archives in `gnu' format, however, future version will switch to `posix'.

* I've compared tar on centos7 and ubuntu-18.04:

bash-4.2$ cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)

bash-4.2$ tar --version
tar (GNU tar) 1.26
...

bash-4.2$ tar --help | tail -n 5
*This* tar defaults to:
--format=gnu -f- -b20 --quoting-style=escape --rmt-command=/etc/rmt
--rsh-command=/usr/bin/ssh
...

bitbake@e0ee76f81c2f:/$ grep VERSION /etc/os-release
VERSION="18.04.5 LTS (Bionic Beaver)"
VERSION_ID="18.04"
VERSION_CODENAME=bionic

bitbake@e0ee76f81c2f:/$ tar --version
tar (GNU tar) 1.29
...

bitbake@e0ee76f81c2f:/$ tar --help | tail -n 5
...
*This* tar defaults to:
--format=gnu -f- -b20 --quoting-style=escape --rmt-command=/usr/lib/tar/rmt
--rsh-command=/usr/bin/rsh

Both support posix format (as pax POSIX 1003.1-2001). But centos7 version is
already too old anyway, because it doesn't support --sort=name used since:
https://git.openembedded.org/openembedded-core/commit/?id=4fa68626bbcfd9795577e1426c27d00f4d9d1c17
and
https://git.openembedded.org/openembedded-core/commit/?id=f19e43dec63a86c200e04ba14393583588550380
says that 1.28 is the minium version now and
https://git.openembedded.org/openembedded-core/commit/?id=7a66434cf11b7f051699b774e4fccd6738351368
recommends to use install-buildtools for hosts with tar < 1.28

On the other side latest tumbleweed from:
https://hub.docker.com/r/opensuse/tumbleweed
with tar-1.33 alredy defaults to posix format:

b99dbb3d86dd:/ # head -n 3 /etc/os-release
NAME="openSUSE Tumbleweed"
ID="opensuse-tumbleweed"

b99dbb3d86dd:/ # tar --version
tar (GNU tar) 1.33
...

b99dbb3d86dd:/ # tar --help | tail -n 3
*This* tar defaults to:
--format=posix -f- -b20 --quoting-style=escape --rmt-command=/usr/bin/rmt
--rsh-command=/usr/bin/ssh

I've packaged some sample rootfs directory with both tars and the result is
identical (with --format=gnu as well as --format=posix).

with ubuntu:
tar --sort=name --format=gnu --numeric-owner -cf rootfs.ubuntu.gnu.tar -C rootfs .
tar --xattrs --xattrs-include=* --sort=name --format=posix --numeric-owner -cf rootfs.ubuntu.posix.tar -C rootfs .
tumbleweed:
tar --sort=name --format=gnu --numeric-owner -cf rootfs.tumbleweed.gnu.tar -C rootfs .
tar --xattrs --xattrs-include=* --sort=name --format=posix --numeric-owner -cf rootfs.tumbleweed.posix.tar -C rootfs .
centos7 (without --sort=name):
tar --format=gnu --numeric-owner -cf rootfs.centos7.gnu.tar -C rootfs .
tar --xattrs --xattrs-include=* --format=posix --numeric-owner -cf rootfs.centos7.posix.tar -C rootfs .

size is identical:
-rw-r--r-- 1 mjansa mjansa 2487480320 Feb  5 09:19 rootfs.ubuntu.gnu.tar
-rw-r--r-- 1 mjansa mjansa 2487480320 Feb  5 10:17 rootfs.centos7.gnu.tar
-rw-r--r-- 1 mjansa mjansa 2487480320 Feb  5 10:26 rootfs.tumbleweed.gnu.tar
-rw-r--r-- 1 mjansa mjansa 2579875840 Feb  5 10:15 rootfs.ubuntu.posix.tar
-rw-r--r-- 1 mjansa mjansa 2579875840 Feb  5 10:16 rootfs.centos7.posix.tar
-rw-r--r-- 1 mjansa mjansa 2579875840 Feb  5 10:26 rootfs.tumbleweed.posix.tar

but md5s aren't:
5e3880283379dd773ac054e20562fdea  rootfs.centos7.gnu.tar
abeaf992c780aa780a27be01365d26f5  rootfs.centos7.posix.tar
0c6ee59d87ab56583293262de110bca4  rootfs.tumbleweed.gnu.tar
1555bc7276eaba924bf82a13a010fd6d  rootfs.tumbleweed.posix.tar
553d802bba351e273191bd5b2a621b66  rootfs.ubuntu.gnu.tar
b6d7b43b30174686f6625ba3c7aefdc6  rootfs.ubuntu.posix.tar

diffoscope shows some differences when using gnu format:

$ diffoscope rootfs.tumbleweed.gnu.tar rootfs.ubuntu.gnu.tar
...
-00239890: 3030 3000 3030 3737 3637 0020 4b00 0000  000.007767. K...
+00239890: 3030 3000 3031 3135 3737 0020 4b00 0000  000.011577. K...
...
-00239900: 0075 7374 6172 2020 0000 0000 0000 0000  .ustar  ........
+00239900: 0075 7374 6172 2020 0072 6f6f 7400 0000  .ustar  .root...
...
-00239920: 0000 0000 0000 0000 0000 0000 0000 0000  ................
+00239920: 0000 0000 0000 0000 0072 6f6f 7400 0000  .........root...

with posix format there are also some differences shown by diffoscope:

$ diffoscope rootfs.tumbleweed.posix.tar rootfs.ubuntu.posix.tar
 016a4c00: 2e2f 7573 722f 6269 6e2f 5061 7848 6561  ./usr/bin/PaxHea
-016a4c10: 6465 7273 2f63 6861 7474 722e 6532 6673  ders/chattr.e2fs
-016a4c20: 7072 6f67 7300 0000 0000 0000 0000 0000  progs...........
+016a4c10: 6465 7273 2e32 322f 6368 6174 7472 2e65  ders.22/chattr.e
+016a4c20: 3266 7370 726f 6773 0000 0000 0000 0000  2fsprogs........
...
 03937000: 2e2f 7573 722f 6269 6e2f 5061 7848 6561  ./usr/bin/PaxHea
-03937010: 6465 7273 2f63 6f6e 7461 696e 6572 642d  ders/containerd-
-03937020: 6374 7200 0000 0000 0000 0000 0000 0000  ctr.............
+03937010: 6465 7273 2e32 322f 636f 6e74 6169 6e65  ders.22/containe
+03937020: 7264 2d63 7472 0000 0000 0000 0000 0000  rd-ctr..........

so cannot really say which format is better for reproducible tar
archives from different distros, but posix at least supports xattrs
and it's the format for future.

(From OE-Core rev: 3ecea58f2a3382d9f4b410d6ad7089111334cb6f)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-06 09:03:00 +00:00
Richard Purdie
c2f2963623 package: Ensure do_packagedata is cleaned correctly
In an earlier commit, libprocps was split into a separate package leaving
no shlibs in the main package. A bug was seen where igt-gpu-tools wouldn't
build correctly in some cases as it thought the librbary was still in the
main package, throwing qa errors as a result.

The issue was due to an extra file being left in the sstate output of
the do_packagedata task in the shlibs2/ folder which contained the bad
shlibs information.

The reason for this was that the temporary directory used in this
task wasn't being cleaned so files which were deleted were not handled
correctly. Add a missing cleandirs entry to fix this.

(From OE-Core rev: 50f17d0a655a3a2556f9fcad67259101c2814a36)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-03 21:45:49 +00:00
Dorinda
785b5489ab sanity.bbclass: Check if PSEUDO_IGNORE_PATHS and paths under pseudo control overlap
Added a sanity check for when PSEUDO_IGNORE_PATHS and paths under pseudo control overlap to avoid random failures generated.

[YOCTO #14193]

(From OE-Core rev: 6e4bd8cabcdedf4b52345ef5eb421f71d0f19b1d)

Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-03 21:45:49 +00:00
Richard Purdie
9dbe0f69f8 image_types: Ensure tar archives are reproducible
The tar output seems to vary depending on the version of tar used and distro
configuration. Be explict about the output format to avoid this and be
determinstic.

(From OE-Core rev: c56f3c9febc1732aa1302524c6c4da36f16bd1f7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-31 10:37:16 +00:00
Richard Purdie
87cb7577db Revert "sanity.bbclass: check if PSEUDO_IGNORE_PATHS and ${S} overlap"
This reverts commit c5a6cc4146402620851e2a1f2b01d69989150ba2.

This was accientally added and shouldn't have been, it has a v2 and was then
determined not to be required.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-30 10:55:58 +00:00
Awais Belal
44ecc935b2 kernel.bbclass: fix deployment for initramfs images
The do_bundle_initramfs() only processes kernel image
types that are found in KERNEL_IMAGETYPE_FOR_MAKE whereas
the build system can generate other types that are not
directly supported by the kernel build system. In which
case when we come to the deploy phase not all the images
mentioned in KERNEL_IMAGETYPES would have a respective
initramfs bundled image. An example is using vmlinux.gz
in KERNEL_IMAGETYPES and enabling initramfs and then we
see

install: cannot stat 'arch/arm64/boot/vmlinux.gz.initramfs': No such file or directory

So we align the deploy phase with bundle initramfs phase
and pick up relevant initramfs bundled images using
KERNEL_IMAGETYPE_FOR_MAKE instead of KERNEL_IMAGETYPES.

(From OE-Core rev: 526bdd88ccd758204452579333ba188e29270bde)

Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-30 10:41:04 +00:00
Lee Chee Yang
86b42289bd cve_check: add CVE_VERSION_SUFFIX to indicate suffix in versioning
add CVE_VERSION_SUFFIX to indicate the version suffix type, currently
works in two value, "alphabetical" if the version string uses single
alphabetical character suffix as incremental release, blank to not
consider the unidentified suffixes. This can be expand when more suffix
pattern identified.

refactor cve_check.Version class to use functools and add parameter to
handle suffix condition.

Also update testcases to cover new changes.

(From OE-Core rev: 5dfd5ad5144708b474ef31eaa89a846c57be8ac0)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-30 10:41:04 +00:00
Dorinda
ef208aaf0f sanity.bbclass: check if PSEUDO_IGNORE_PATHS and ${S} overlap
added a sanity check for when PSEUDO_IGNORE_PATHS and ${S} overlap to avoid random failures generated.

[YOCTO #14193]

(From OE-Core rev: c5a6cc4146402620851e2a1f2b01d69989150ba2)

Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-30 10:41:04 +00:00
Vyacheslav Yurkov
4bb1a003de npm.bbclass: use python3 for npm config
python2-native executable is not available in sysroot anymore, which
causes compilation of some nodejs modules to fail. Switch to python3 as a
default python version.

(From OE-Core rev: d21f50ecf8e8683a92b7d234fa8225c2c1470595)

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-30 10:41:03 +00:00
Richard Purdie
dcaed52dde gtk-doc: Disable dependencies in native case
Ensure that dependencies like qemu-native aren't added in the native
case since we don't want docs generate for native recipes.

(From OE-Core rev: c5ee8ac6929d91f39ebf6dc6786440dc3dbd02ff)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-29 21:21:53 +00:00
Richard Purdie
c4257ed8b1 native: Stop clearing PACKAGES
Native recipes have been special and they don't have packages generated
from them. The RDEPENDS/RPROVIDES and other runtime package specific
variables can contain important data about dependencies recipes need
though and currently it is required to write this information explicitly
in the native case.

We now delete the packaging tasks for native recipes which removes the
need to clear PACKAGES. The next step to improve the metadata is to
stop clearing it and ensure any entries in these variables are remapped
appropriately. The R* variables were already being processed by the class
extension code but the implementation was suboptimal.

This patch stops clearing PACKAGES and PACKAGES_DYNAMIC and fixes the places
where that caused issues in OE-Core, for example PACKAGES additions in anonymous
python without the "-native" suffix and a case where the included classes
caused a self reference in DEPENDS which would once have been removed by
the previous code.

The implementation uses datastore/parser parameters to ensure that the
variable overrides are not overwritten when calling setVar which is appropriate
for a function as close to the core as this one is.

Some now unneeded code in python3-setuptools is dropped, there are further
changes like this which can follow.

This change was verified with OE-Core by comparing task-depends.dot generated
by "bitbake world -g" before and after the change, the files were identical.

(From OE-Core rev: fd6a007efa7cb45101a66f294af81d9d33bb3fab)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-29 21:21:53 +00:00
Richard Purdie
b033409243 package_rpm: Clean up unset runtime package variable handling
If the R* runtime package variables are unset it could cause trackbacks. There
were some fallbacks already, clean this up to handle consistently.

The code was expecting strings but setting defaults of empty lists
which silently were converted to strings by the "or" statements which
was a nightmare to understand or alter.

(From OE-Core rev: 06bf68012302330c2b14f59541f78f48c7389c37)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-29 21:21:53 +00:00
Peter Bergin
55ead1be58 buildhistory.bbclass: avoid exception for empty BUILDHISTORY_FEATURES variable
An exception is fired when a BuildStarted event is sent to buildhistory bbclass
and the variable BUILDHISTORY_FEATURES is not set.

    ERROR: Execution of event handler 'buildhistory_eventhandler' failed
    Traceback (most recent call last):
      File "<...>/meta/classes/buildhistory.bbclass", line 862, in buildhistory_eventhandler(e=<bb.event.BuildStarted object at 0x7f94c3810250>):
         python buildhistory_eventhandler() {
        >    if e.data.getVar('BUILDHISTORY_FEATURES').strip():
                 reset = e.data.getVar("BUILDHISTORY_RESET")
    AttributeError: 'NoneType' object has no attribute 'strip'

This can happen in a multiconfig build where the default configuration use the
buildhistory class but not the configuration in mc. It should be a rare case that
this happens and it was found in a missconfigured build.

(From OE-Core rev: a74e30a4de02c8efd3e7102ba7a4fe06df53cc34)

Signed-off-by: Peter Bergin <peter@berginkonsult.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-29 21:21:53 +00:00
Tomasz Dziendzielski
24e95b9a0b insane: native-last: Only print classes inherited after native/nativesdk
For usability, only print the classes inherited after native/nativesdk
which makes it easier for the user to see where the problem is.

Realted to [YOCTO #5729].

(From OE-Core rev: 78f7cf59783faab6ef8d4f4fde774754db946519)

Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-28 23:12:05 +00:00
Tomasz Dziendzielski
eb604952d5 insane: Add missing INSANE_SKIP mechanism for native-last QA check
Ensure that the native-last QA check can be controlled by the INSANE_SKIP
variable (realted to [YOCTO #5729]).

(From OE-Core rev: 2d95aee64766341bf81f610386bac222e329f1bb)

Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-28 23:12:05 +00:00
Richard Purdie
a89f77b92d staging: Clean up files installed into the sysroot
There are a variety of files being installed into $datadir which we
don't need. Pick the top "offenders" which amount of thousands of files
and simply don't install them. These include things like test data,
terminfo data, locale data for native tools and so on. This saves
copying these files into native and target sysroots and should improve
performance (smaller sstate, fewer files to copy around).

With this and the python recipe change, alsa-tools went from:

recipe-sysroot: 18357
recipe-sysroot-native: 14129

to

recipe-sysroot: 10809
recipe-sysroot-native: 8079

which is a decent improvement.

(From OE-Core rev: 366c72941fe1c24d0b1d96df46e13cb9eb4e79d6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-28 23:12:05 +00:00
Tomasz Dziendzielski
9385670add externalsrc: Detect code changes in submodules
The srctree_hash was calculated only from main source directory ignoring
changes in submodules.

[YOCTO #13748]

Use submodule--helper to determine all submodules, and calculate hash
from all git tree objects names combined.

(From OE-Core rev: 50ff9afb3990bcf60b4fa1f937506cb84028c32d)

Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-28 23:12:05 +00:00
Dorinda
dbd80a923c sanity.bbclass: sanity check for if bitbake is present in PATH
If a user executes the environment script instead of sourcing it,
there's an error about an empty element in PATH. This is because
bitbake isn't present in environment variable PATH. Hence, this
patch adds a sanity check to verify if bitbake is present in
PATH and if bitbake isn't present issue a warning message.

[YOCTO #13822]

(From OE-Core rev: e08799913a7f207bc63e085eb98196fd61ed57bc)

Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-27 10:17:45 +00:00
Dorinda
bfeeaf9ff1 sanity: Verify that user isn't building in PSEUDO_IGNORE_PATHS
If a user builds in a path in PSEUDO_IGNORE_PATHS, random failures
are generated. Hence this patch adds a sanity check in sanity.bbclass
to ensure that a user isn't building in PSEUDO_IGNORE_PATHS.

[YOCTO #14179]

(From OE-Core rev: 7a681525e904914e938de25df5cc64209097d15d)

Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-27 10:17:45 +00:00
Tomasz Dziendzielski
0f0c0df633 insane: Add test for native/nativesdk inherit order
Classes native/nativesdk should be inherited last to prevent unexpected
behaviour.

[YOCTO #5729]

(From OE-Core rev: 55a0197fe62577fd51d41d87822e6d64d85c7680)

Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-27 10:17:45 +00:00
Lee Chee Yang
3807c6d9a7 cve-check: replace Looseversion with custom version class
The way distutils.version.LooseVersion compare version are tricky, it treat
all these ( "1.0-beta2", "1.0-rc1", "1.0A", "1.0p2" and "1.0pre1") as greater
version than "1.0". This might be right for "1.0A" and "1.0p1" but not for
the rest, also these version could be confusing, the "p" in "1.0p1" can be
"pre" or "patched" version or even other meaning.

Replace Looseversion with custom class, it uses regex to capture common
version format like "1.1.1" or tag format using date like "2020-12-12" as
release section, check for following known string/tags ( beta, rc, pre, dev,
alpha, preview) as pre-release section, any other trailing characters
are difficult to understand/define so ignore them. Compare release
section and pre-release section saperately.

included selftest for the version class.

[YOCTO#14127]

(From OE-Core rev: 6ced85e9ddd3569240f1e8b82130d1ac0fffbc40)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-23 17:08:54 +00:00
Ross Burton
02360c2e53 base: use URI instead of decodeurl when detecting unpack dependencies
decodeurl() has limitations, primarily that it doesn't handle query
parameters at all. If a SRC_URI looks like this:

  http://example.com/download.tar.gz?something

Then the returned path attribute is download.tar.gz?something.  This means
the filename extension detection fails and required tools are not added
to the dependencies.

(From OE-Core rev: 5cd396c099730b765fc6cd82e2d7748f99de7157)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-23 17:08:54 +00:00
Tomasz Dziendzielski
28bdfe0066 externalsrc: Fix parsing error with devtool non-git sources
If srcdir is under poky directory (e.g. devtool poky/build/workspace/sources)
and is not a git repository then ${@srctree_hash_files(d)} will run "git
rev-parse --git-dir" and detect poky directory as git-dir and run "'git', 'add',
'-A', '.'], cwd=s_dir" trying to add srcdir but build dir is in .gitignore and
latest git will fail with "The following paths are ignored by one of your
.gitignore files: build" which will end with "ExpansionError during parsing".

In this commit I added a check if git_dir is the same as git-dir from
TOPDIR (which will detect poky directory) and if yes, then treat srcdir
as non-git sources.

(From OE-Core rev: 95fbac8dcad6c93f4c9737e9fe13e92ab6befa09)

Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-23 17:08:53 +00:00