Commit Graph

7759 Commits

Author SHA1 Message Date
Richard Purdie
dd6b55d70c package/scripts: Fix FILES_INFO handling
There is a long standing bug where FILES_INFO isn't written into pkgdata
with a package suffix. This means if the files are read into the datastore
as intended, the last one "wins".

Fix this to work as intended. Most of the call sites using the data need
to be updated to handle this and the overrides change correctly.

Also fix some other problematic references noticed along the way.

(From OE-Core rev: a1190903e0a61a12c9854c96af918ae8d12c6327)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-17 09:53:04 +01:00
Tim Orling
ddd4b8c9a1 package.bbclass: FILER* override syntax
pkgdata no longer includes FILERPROVIDES* nor
FILERDEPENDS*. Fix with new override syntax.

Also apply to FILERPROVIDESLIST and FILERDEPENDSLIST.

(From OE-Core rev: 8f96a2d13bbae8fb70ed7feafdcff26544e3710d)

Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-13 22:43:56 +01:00
Joe Slater
49ae4a23d8 terminal.bbclass: force bash for devshell
Since shell_trap_code in build.py sets /bin/sh as the interpreter
we will die a silent death if our environment contains things like
"export -f bodilyfunction" and /bin/sh is really /bin/dash.

Fixes this for the case of devshell.

(From OE-Core rev: 23d296b3567aa31bad7b2a8558d4bd3e4505843b)

Signed-off-by: Joe Slater <joe.slater@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-13 22:43:56 +01:00
Joshua Watt
fa6c07bc1a classes/cve-check: Move get_patches_cves to library
Moving the function will allow other classes to capture which CVEs have
been patched, in particular SBoM generation.

Also add a function to capture the CPE ID from the CVE Product and
Version

(From OE-Core rev: 75d34259a715120be1d023e4fd7b6b4b125f2443)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-13 14:44:06 +01:00
Quentin Schulz
3ae80177fb insane.bbclass: fix new override syntax migration
A few variables and messages were not migrated over to the new override
syntax (_ to :).

Let's fix that.

(From OE-Core rev: c595d6040d2e0ef94d7da043b41226e90dddf318)

Signed-off-by: Quentin Schulz <foss@0leil.net>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-13 14:44:06 +01:00
Thomas Perrot
50d8801d72 kernel-fitimage: images should not be signed with the same keys as the configurations
Otherwise the "required" property, from UBOOT_DTB_BINARY, will be set to "conf"
and no error will be raised in case of error.

(From OE-Core rev: 080e0dfed710035b2e40187d9d639ecf5ab84be2)

Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-13 14:44:06 +01:00
Paul Barker
a428960183 pypi: Allow override of PyPI archive name
Some packages on PyPI don't follow the usual expectations for archive
naming. For example, the archive for asyncio-mqtt 0.10.0 is named
asyncio_mqtt-0.10.0.tar.gz (with an underscore instead of the dash used
in the package name).

To handle these edge cases a new PYPI_ARCHIVE_NAME variable is
introduced. By default this is set to the expected archive name based on
the PyPI package name, version and extension but it can be set to a
different value if needed in a recipe which inherits the pypi class.

(From OE-Core rev: 9659f5a51a2d094b45b52136feac4402d501b4f2)

Signed-off-by: Paul Barker <paul@pbarker.dev>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-13 14:44:05 +01:00
Alejandro Hernandez Samaniego
650b720f19 baremetal-helloworld: Enable RISC-V 32 port
$ runqemu nographic
runqemu - INFO - Running bitbake -e ...
KERNEL: [tmp/deploy/images/qemuriscv32/baremetal-helloworld-image-qemuriscv32.bin]
MACHINE: [qemuriscv32]
runqemu - INFO - Running tmp/work/x86_64-linux/qemu-helper-native/1.0-r1/recipe-sysroot-native/usr/bin/qemu-system-riscv32

Hello OpenEmbedded on RISC-V 32!

(From OE-Core rev: d4cca7471f2167b56347fa7b1364bb84a200b1f5)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-12 06:26:16 +01:00
Chen Qi
8ae0ee62ca multilib.bbclass: fix new override syntax for virtclass-multilib
the 'virtclass-multilib-xxx' is an override, so use ':' instead of
'_' for TARGET_VENDOR and DEFAULTTUNE.

(From OE-Core rev: a522972821339f42dcdddb334e843e21584bfbea)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-12 06:26:16 +01:00
Jose Quaresma
33ba016f4c sstate.bbclass: fix error handling when sstate mirrors is ro
The commit dd555537fc
'sstate.bbclass: fix errors about read-only sstate mirrors'
adds an additional exception handler to silently mask read
only rootfs errors thrown during the touch.

The exception handler checks the error type with the python module errno
but this module needs to be imported as it don't exist.

Example of the error:

File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:sstate_task_postfunc(d)
     0003:
File: '/home/builder/src/base/poky/meta/classes/sstate.bbclass', lineno: 778, function: sstate_task_postfunc
     0774:
     0775:    omask = os.umask(0o002)
     0776:    if omask != 0o002:
     0777:       bb.note("Using umask 0o002 (not %0o) for sstate packaging" % omask)
 *** 0778:    sstate_package(shared_state, d)
     0779:    os.umask(omask)
     0780:
     0781:    sstateinst = d.getVar("SSTATE_INSTDIR")
     0782:    d.setVar('SSTATE_FIXMEDIR', shared_state['fixmedir'])
File: '/home/builder/src/base/poky/meta/classes/sstate.bbclass', lineno: 708, function: sstate_package
     0704:        except PermissionError:
     0705:            pass
     0706:        except OSError as e:
     0707:            # Handle read-only file systems gracefully
 *** 0708:            if e.errno != errno.EROFS:
     0709:                raise e
     0710:
     0711:    return
     0712:
Exception: NameError: name 'errno' is not defined

(From OE-Core rev: 15f30ad144fbe25e9a5e71bc7e42e746d2039992)

Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-12 06:26:15 +01:00
Vyacheslav Yurkov
550511e446 rootfs-postcommands: add QA check for overlayfs
The check is conditional and only enabled when overlayfs is set in
DISTRO_FEATURES

(From OE-Core rev: 4734799bacf0a5d2487e1cde3ae1c00223b032b2)

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-12 06:26:15 +01:00
Vyacheslav Yurkov
d6e1f08ee3 overlayfs.bbclass: generate overlayfs mount units
It's often desired in Embedded System design to have a read-only rootfs.
But a lot of different applications might want to have a read-write access
to some parts of a filesystem. It can be especially useful when your update
mechanism overwrites the whole rootfs, but you want your application data
to be preserved between updates. This class provides a way to achieve that
by means of overlayfs and at the same time keeping the base rootfs read-only.

(From OE-Core rev: 18377d6f09fc8855c71f2e5c097cbbbccf5632ce)

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-12 06:26:15 +01:00
Michael Opdenacker
a99473976d cve-check: remove deprecated CVE_CHECK_CVE_WHITELIST
This variable has been deprecated since Yocto Project version 3.0.

(From OE-Core rev: f8ac58568b2dceef54a743369460019b3a3eeccd)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-12 06:26:15 +01:00
Richard Purdie
492205ea83 packagedata: Fix after override syntax change
Fix a reference that should have been part of the override syntax change
causing packages to be written out incorrectly.

(From OE-Core rev: 0f978b4f03e71267ad0a8a5054141e7727f2944f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-06 09:25:25 +01:00
Michael Opdenacker
ae55dea68d cve-check: improve comment about CVE patch file names
(From OE-Core rev: 8aa613480663e11ecc62278d8c57ca719eb23899)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-06 06:34:58 +01:00
Michael Opdenacker
4c28645c04 cve-check: update link to NVD website for CVE details
The old URL schema
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-ID
now redirects to
https://nvd.nist.gov/vuln/detail/CVE-ID

(From OE-Core rev: 57adb57a9d9b08c08ab606ec7b561792e4f4ff2d)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-06 06:34:58 +01:00
Michael Opdenacker
fce14b90e3 cve-check: fix comments
This implements various fixes in comments in cve-check.bbclass
In particular, the "whitlisted" typo is important as the "whitelisted"
word is going to be replaced in a near future.

(From OE-Core rev: 5eecd2bf942254d08c252388594e5ec7ae330f45)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-06 06:34:58 +01:00
Khem Raj
d935d5db42 distrooverrides.bbclass: Correct override syntax
(From OE-Core rev: b95d50f6ed6bf21d48c4cd22ffe9e8edc1480135)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-05 08:53:41 +01:00
Richard Purdie
1900cb43f1 devupstream: Allow support of native class extensions
It is useful to be able to use the class with recipes using BBCLASSEXTEND
for native extensions. This adds the magic required to do that.

[YOCTO #11449]

(From OE-Core rev: 17bab13b0f2431757d8ddd66489bb720c13a0320)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-04 20:45:41 +01:00
Richard Purdie
174ceef844 image: Drop COMPRESS_CMD
This was replaced by CONVERSION_CMD a long time ago and is no longer referenced
in core. Remove the references to it.

(From OE-Core rev: 576d52cdaca047d290c3b10b26aa2244da230dbb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-04 20:45:41 +01:00
Richard Purdie
34cc76ff05 image/image_types: Convert CONVERSION_CMD/COMPRESS_CMD to new override syntax
For consistency, use override syntax for these variables as well since
it is more consistent with the rest of the image code. We may be able to use
these as proper overrides in due course.

(From OE-Core rev: 52674c4b1fdf79829095031b2e342d44fb0dc181)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-04 20:45:41 +01:00
Richard Purdie
ab0cba3411 uboot-extlinux-config: Fix missing override conversion
(From OE-Core rev: 988e6c5a6add25ccd1d880f4d4f8c257afde4e47)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-04 20:45:41 +01:00
Richard Purdie
ea984415f6 meta: Convert IMAGE_TYPEDEP to use override syntax
The IMAGE_TYPEDEP variable would make more sense to match the form of the
other image override variables, convert it to use the overrides format.

(From OE-Core rev: 8573f6b2a7af9867da0b21936ffd2cd2a417de1d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-04 20:45:41 +01:00
Denys Dmytriyenko
fdf57744b4 meta: convert nested overrides leftovers to new syntax
Those were missed in previous rounds of automated and manual conversion.

(From OE-Core rev: 22f9c7268b542baf6cd8aa0e34c8fb7aa1579e08)

Signed-off-by: Denys Dmytriyenko <denis@denix.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-04 20:45:41 +01:00
Chen Qi
3c492c59c0 insane.bbclass: fix the file-rdeps QA message for the new override syntax
Instead of replacing '_<pkg>', we should be replacing ':<pkg>'.
Otherwise, when we meet a 'file-rdeps' QA error, the error message would be
like below.

  QA Issue: /usr/lib64/libatopology.so.2.0.0:libatopology contained in package libatopology ...

The ':libatopology' should not be in the error message.

(From OE-Core rev: d65541c13b1346468dce749523b1dc04cfb7f70e)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-04 20:45:41 +01:00
Richard Purdie
c568dd9799 license: Exclude COPYING.MIT from pseudo
Along with the other license exclusions, we need to exclude the
top level COPYING.MIT file else when:

COPY_LIC_DIRS     = "1"
COPY_LIC_MANIFEST = "1"

is set, we see eSDK failures from a pseudo abort.

[YOCTO #14366]

(From OE-Core rev: 3eb580843de3f055e42fcce60b0f15c4190c0542)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-04 20:45:41 +01:00
Paul Barker
a185e98371 kernel-yocto: Simplify no git repo case in do_kernel_checkout
If the kernel sources are not fetched via git, a local git repository is
created in do_kernel_checkout. In this case we know that there will be
no remote branches and we will already be on the correct branch (since
only one branch will exist). So we can simplify things by skipping these
steps.

This also removes the assumption that the default git branch name will
be "master". Prior to this change, the final git checkout command in
do_kernel_checkout could fail if a local git repo was created and the
user had changed init.defaultBranch in their gitconfig.

(From OE-Core rev: af2a9c92d4498492ca23388c7b4bbed48abdc4d7)

Signed-off-by: Paul Barker <paul@pbarker.dev>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-04 10:38:26 +01:00
Richard Purdie
825043ec77 populate_sdk_ext: Fix handling of TOOLCHAIN_HOST_TASK in the eSDK case
The current way TOOLCHAIN_HOST_TASK is handled within eSDK builds is
causing much confusion, even to people who should understand the code.

For a normal SDK, some layers append to TOOLCHAIN_HOST_TASK unconditionally
which is fine, until the eSDK tries to override the variable to it's own
values. It does not support or use packages installed using this variable
and would use native recipes instead, it is a very different approach.

In the referenced bug, binaries are added but not relocated leading to
confusing errors.

To fix this, add a new variables for the eSDK TOOLCHAIN_HOST_TASK
to be explict and force the eSDK code to use this instead. The setVar
in non-parsing context will clear out any appends resolving the current
issues. The patch also gets rid of some dubious task override use.

[YCOTO #14047]

(From OE-Core rev: 30912ba084aa8600156edddbe4f2db0b85e869d0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02 15:44:11 +01:00
Richard Purdie
55a37651ac sstate: Fix rebuilds when changing layer config
When adding a layer which changed SSTATE_EXCLUDEDEPS_SYSROOT, the state
hashes were changing when they should not. This was caused by wider use
of setscene_depvalid which means the dependency on the variable was seen
when it was previously not.

Exclude the variable since this should be be included in the hashes.

(From OE-Core rev: 09725a29365c69ccbd603fe3a1de72189f26d5ac)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02 15:44:11 +01:00
Richard Purdie
f62f684de8 sdk: Decouple default install path from built in path
Add SDKPATHINSTALL which is used as the default install location of the SDK
instead of SDKPATH. This means the default install path isn't encoded into
every SDK binary, meaning if a date is used there the entire SDK doesn't
have to rebuild. Most distros can switch to only customise SDKPATHINSTALL
meaning more sstate reuse too.

[YOCTO #14100]

(From OE-Core rev: bc4ee5453560dcefc4a4ecc5657df5cc1666e153)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02 15:44:11 +01:00
Richard Purdie
65133015b1 meta: Manual override fixes
The automated conversion of OE-Core to use the new override sytax isn't
perfect. This patches some mis-converted lines and some lines which were missed
by the automation.

(From OE-Core rev: 4e9a06b64b43131b731fb59a0305f78a98e27fbd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02 15:44:10 +01:00
Richard Purdie
bb6ddc3691 Convert to new override syntax
This is the result of automated script conversion:

scripts/contrib/convert-overrides.py <oe-core directory>

converting the metadata to use ":" as the override character instead of "_".

(From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02 15:44:10 +01:00
Richard Purdie
2f2875f73c autotools/base/icecc: Remove prepend from function names
Using prepend as part of a function name is a poor choice. Whilst we're
about to make the syntax explict, improve the names anyway making the
conversion easier and the intent clear that this isn't an override.

(From OE-Core rev: 9d002acae720b0a8e96a6734424a142b86880461)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-30 00:11:51 +01:00
Changqing Li
4a1381d350 archiver.bbclass: fix do_ar_configured failure for kernel
(From OE-Core rev: d7776a23cbea836ddb8ac5ec77012af2449ab875)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-28 23:47:00 +01:00
Marek Vasut
7668153777 kernel-uboot: Handle gzip and lzo compression options
Since 5c72105e29 ("kernel-uboot: allow compression option to be configurable")
it is possible to select kernel compression method, however the resulting
image is always compressed with gzip, so selecting any other method than
gzip results in unbootable images. Add support for lzo for starters, since
that is fast to decompress and useful in low boot time scenarios.

Note that we should likely add some check for unsupported compression
methods. We should also add dependency on lzop-native I think.

(From OE-Core rev: f1257d61b76d027d4ece734439993b6bf4e48907)

Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Sinan Kaya <okaya@kernel.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-28 23:47:00 +01:00
Marek Vasut
e2802c2186 image_types: Restore pre-btrfs-tools 4.14.1 mkfs.btrfs shrink behavior
Currently the mkfs.btrfs generates large images with a lot of wasted
space. This happens since OE-core updated btrfs-tools from 4.13.3 to
4.15.1 in commit 94b645aa77 ("btrfs-tools: update to 4.15.1") .

Note in mkfs.btrfs(8) manpage section -r says the following:
"
  -r|--rootdir <rootdir>
...
       Note This option may enlarge the image or file to ensure
       it’s big enough to contain the files from rootdir. Since
       version 4.14.1 the filesystem size is not minimized. Please
       see option --shrink if you need that functionality.

  --shrink
     Shrink the filesystem to its minimal size, only works with
     --rootdir option.
...
       Note prior to version 4.14.1, the shrinking was done
       automatically.
"

Add the --shrink option to EXTRA_IMAGECMD_btrfs to reinstate the
original behavior and un-waste the space.

(From OE-Core rev: c4a99d36967302c176b62fad840b5e79486ea356)

Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-28 23:47:00 +01:00
leimaohui
34159787df archiver.bbclass: Fix patch error for recipes that inherit dos2unix.
do_unpack_and_patch error happens for these recipes inherit dos2unix.

(From OE-Core rev: 2ceda7c90c0087f52693c54d5ccab143b27f4d21)

Signed-off-by: Lei Maohui <leimaohui@fujitsu.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-24 16:33:46 +01:00
Adam Romanek
28d3fe177c own-mirrors: Add support for s3:// scheme in SOURCE_MIRROR_URL
(From OE-Core rev: 5185ddb19cae2afb948d0ce83ec1ee356c81965e)

Signed-off-by: Adam Romanek <romanek.adam@gmail.com>
Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-19 13:34:26 +01:00
Mike Crowe
72e4483f12 licence_image: Add lic-pkgs IMAGE_FEATURE
Installing license packages is similar to installing -dev or -dbg
packages, so let's invent a "lic-pkgs" IMAGE_FEATURE that does so and
document it in core-image.bbclass.

This image feature only works if LICENSE_CREATE_PACKAGE is set, so
refuse to generate an image if the lic-pkgs feature is enabled without
LICENSE_CREATE_PACKAGE.

(From OE-Core rev: eee8179f5b920d3f8907db23cbc061ed6770a02a)

Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-13 14:27:53 +01:00
Richard Purdie
7f9f9a0c37 sstate: Drop pseudo exclusion
Now that pseudo-native always includes statx support and we have sanity
checks on pseudo-native to ensure it always contains a minimum feature set,
we no longer need to mark pseudo-native as distro specific. This fixes
eSDK build problems.

[YOCTO #14428]

(From OE-Core rev: 3279e43fcb469edb63c7c4eb60fdc565d5751f9d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-11 23:00:52 +01:00
Richard Purdie
0c39b42cb1 report-error: Drop pointless inherit
The base class is always inherited, drop the unneeded inherit.

(From OE-Core rev: 837d62c066589e5a12a0bf894ae7b24e8b048665)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-10 22:43:41 +01:00
Sakib Sajal
8ce9e8b268 buildstats.bbclass: log host data on failure to task specific file
host data, for both interval and failure, was previously
logged into the same file which was difficult to read
as the files file were usually large.

host data is now logged into separate files, for each
type of logging (failure and interval) and also for each
failed task making it easier to read/parse.

(From OE-Core rev: 1a0fb3c0794f4e66086e567a297b4d9379c6b8f3)

Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-10 22:42:22 +01:00
Richard Purdie
6491203eba license: Drop adding RRECOMMENDS for license packages
This changes behaviour when LICENSE_CREATE_PACKAGE is in use. Packages
no longer have RRECOMMENDS adding to them.

It was highlighted that this doesn't apply to PACKAGES_DYNAMIC, nor can
it easily be made to do so. There is also a much easier way to handle this
which is:

IMAGE_INSTALL_COMPLEMENTARY += "*-lic"

which works on a per image basis and doesn't change the underlying
package dependencies. I propose we switch to this instead.

(From OE-Core rev: 5348ffce46d6706b7bb10e41a59e0f6cf16c62b0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-09 23:44:53 +01:00
Saul Wold
6a9449f0c1 testimage.bbclass: Add dump-guest-memory cmd
This command takes a set of command arguments, one of which requires
a filename so use %s which the MonitorDumper will translate to a
unique filename in the dated qmp directory.

CMD Before: {paging:false,protocol:file:%s.img}
CMD After: {paging:false,protocol:file:/yocto/poky/build/tmp/log/runtime-hostdump/qmp_00_dump-guest-memory.img}

(From OE-Core rev: 563e78491be3763f53f9ea53241853b099f5747b)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-02 23:14:33 +01:00
Mark Hatle
c1dcadb7f8 populate_sdk_ext: Error if trying to generate an eSDK from a mulitconfig
Running a build such as:

  bitbake mc:my_config:core-image-minimal -c populate_sdk-ext

will result in an error like:

ERROR: Task base-files.do_fetch attempted to execute unexpectedly
Task .../poky/build-mc/tmp-my_config-glibc/work/qemuarm64-poky-linux/core-image-minimal/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta/recipes-core/base-files/base-files_3.0.14.bb:do_packagedata, unihash ec5ba0e6b31561daba005fb49c5239c8e46913465b51166b5905f3e5ffcf2741, taskhash ec5ba0e6b31561daba005fb49c5239c8e46913465b51166b5905f3e5ffcf2741
Task .../poky/build-mc/tmp-my_config-glibc/work/qemuarm64-poky-linux/core-image-minimal/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta/recipes-core/base-files/base-files_3.0.14.bb:do_package_write_rpm, unihash 1c7d7509c2ff6dcf11009fbec444726826214795d60474ec8d3262d89c40a955, taskhash 1c7d7509c2ff6dcf11009fbec444726826214795d60474ec8d3262d89c40a955
Task .../poky/build-mc/tmp-my_config-glibc/work/qemuarm64-poky-linux/core-image-minimal/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta/recipes-core/base-files/base-files_3.0.14.bb:do_populate_sysroot, unihash 9cc3672f4fa62491f545b15cf617a64cd77d15a2cfd432b57d4b936bc415f40d, taskhash 9cc3672f4fa62491f545b15cf617a64cd77d15a2cfd432b57d4b936bc415f40d
Task .../poky/build-mc/tmp-my_config-glibc/work/qemuarm64-poky-linux/core-image-minimal/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta/recipes-core/base-files/base-files_3.0.14.bb:do_package_qa, unihash 8ada5f62092c971df8dda1d71c728e42994e1dcf2bbdab419de43867d77b64cc, taskhash 8ada5f62092c971df8dda1d71c728e42994e1dcf2bbdab419de43867d77b64cc
Task .../poky/build-mc/tmp-my_config-glibc/work/qemuarm64-poky-linux/core-image-minimal/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_qa, unihash 16656a339389e407a5fdca5d64983af845288f3b3cc5582398e5247efb393257, taskhash 16656a339389e407a5fdca5d64983af845288f3b3cc5582398e5247efb393257
Task .../poky/build-mc/tmp-my_config-glibc/work/qemuarm64-poky-linux/core-image-minimal/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_complete, unihash ef88c74a9f4ae4d252c421eb4e399773aa50cea7c51ffbeed9011e5198a16abb, taskhash ef88c74a9f4ae4d252c421eb4e399773aa50cea7c51ffbeed9011e5198a16abb
This is usually due to missing setscene tasks. Those missing in this build were: {'.../poky/build-mc/tmp-my_config-glibc/work/qemuarm64-poky-linux/core-image-minimal/1.0-r0/sdk-ext/image/tmp-renamed-sdk/layers/poky/meta/recipes-core/base-files/base-files_3.0.14.bb:do_package_qa',

Instead of letting the system error, we simply tell the user this is not supported.

As long as the eSDK is constructed based on the primary library, it works fine.

(From OE-Core rev: b359c60071585fa323124fc6febe652fe0128b52)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-02 23:14:32 +01:00
Richard Purdie
2b3ced714a sstate/staging: Handle directory creation race issue
The sstate code tries to be careful about racing around directory creation.
In particular, the copyhardlinktree code creates the directory tree first
allowing for "already exists" errors and ignoring them, then hardlinks the
files in.

Unfortunately the sstate removal code can race against this since it
will try and remove empty directories. If there is some bad timing,
a newly created directory can be removed before it was populated, leading
to build failures.

We could try and add locking but this would damage performance, we've been
there before. It is also unclear where to actually place locks just based on
the contents of a manifest file which may cover multiple sstate install
locations for a given task.

Instead, lets disable directory removal in the problematic "shared" core
path. This could result in a few more empty directories being left on disk
but those should be harmless and better than locking hurting performance
or rare build races.

[YOCTO #13999]
[YOCTO #14379]

(From OE-Core rev: 4f94d9296394bc7ce241439f00df86eb5912875f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-02 07:24:21 +01:00
Michael Ho
dd555537fc sstate.bbclass: fix errors about read-only sstate mirrors
If a read-only sstate mirror is used in conjunction with hash equiv,
then OSError will be raised when an sstate-cache hit is achieved.

This is because sstate_task_postfunc will try to "touch" the symlinks
that point to the read-only sstate mirror when sstate_report_unihash
has changed SSTATE_PKG.

This commit adds an additional exception handler to silently mask read
only rootfs errors thrown during the touch.

The fix is also duplicated to sstate_eventhandler as the code is very
similar but it may not be needed there.

Example of the error:

File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:sstate_task_postfunc(d)
     0003:
File: '/poky/meta/classes/sstate.bbclass', lineno: 774, function: sstate_task_postfunc
     0770:
     0771:    omask = os.umask(0o002)
     0772:    if omask != 0o002:
     0773:       bb.note("Using umask 0o002 (not %0o) for sstate packaging" % omask)
 *** 0774:    sstate_package(shared_state, d)
     0775:    os.umask(omask)
     0776:
     0777:    sstateinst = d.getVar("SSTATE_INSTDIR")
     0778:    d.setVar('SSTATE_FIXMEDIR', shared_state['fixmedir'])
File: '/poky/meta/classes/sstate.bbclass', lineno: 703, function: sstate_package
     0699:    if not os.path.exists(siginfo):
     0700:        bb.siggen.dump_this_task(siginfo, d)
     0701:    else:
     0702:        try:
 *** 0703:            os.utime(siginfo, None)
     0704:        except PermissionError:
     0705:            pass
     0706:
     0707:    return
Exception: OSError: [Errno 30] Read-only file system

(From OE-Core rev: 244b3be0358a66e0cca4016fe26144e3d7323390)

Signed-off-by: Michael Ho <Michael.Ho@bmw.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-29 22:18:57 +01:00
Richard Purdie
1d1afe6a2b kernel-devicetree: Fix interaction when packaging disabled
When packaging is disabled using the nopackages class, ensure we don't
add to PACKAGES. This fixes builds where we have an unpackaged kernel
alongside a packaged kernel.

(From OE-Core rev: d6b114cf5a9b22688c0b59a3afc46a07450d87b5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-27 22:51:05 +01:00
Richard Purdie
10b39fb94b kernel: Fix interaction when packaging disabled
When packaging is disabled using the nopackages class, ensure we don't
add to PACKAGES. This fixes builds where we have an unpackaged kernel
alongside a packaged kernel.

(From OE-Core rev: 2522daf22e2c27dd9c7926feda0345978217c6c3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-27 22:51:05 +01:00
Richard Purdie
525b85660e package_pkgdata: Avoid task hash mismatches for generic task changes
Add SSTATETASKS to package_prepare_pkgdata[vardepsexclude] since otherwise
the task hashes vary depending upon which packaging backends are enabled
and likely other changes which add/remove unrelated sstate tasks.

(From OE-Core rev: 4011d31d4372639fd72ee0eefae210bf59c90d13)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-26 11:27:39 +01:00