Commit Graph

7759 Commits

Author SHA1 Message Date
Li Wang
df1e1dc92e linux-modules.tgz: fix file permissions to root
the files of linux-modules should be same permissions with rootfs.tar.bz2,
because it is a part of rootfs when used to install.

(From OE-Core rev: 71e141906069c6f754199512741f6e3d5b72fee7)

Signed-off-by: Li Wang <li.wang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-22 17:07:44 +01:00
Quentin Schulz
fa7b29e22b base/insane: Check pkgs lics are subset of recipe lics only once
Move logic checking that all packages licenses are only a subset of
recipe licenses from base.bbclass to the insane.bbclass so that it's
evaluated only once, during do_package_qa.

As explained in the linked bugzilla entry, if a package license is not
part of the recipe license, the warning message gets shown an
unreasonable amount of time because it's evaluated every time a recipe
is parsed.

[YOCTO #10130]

This also makes it possible to silence this error with INSANE_SKIP.

(From OE-Core rev: 852408ed4be1f64c57e196688728b7ed223d3493)

Signed-off-by: Quentin Schulz <quentin.schulz@streamunlimited.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-21 10:52:07 +01:00
Ricardo Ribalda
81e481b017 classes/image_types_wic: Reorder do_flush_pseudodb
When IMAGE_FSTYPES contains more types than wic, it can happen than the
pseudo database is not flushed properly.

This can be solved by changing the order of when do_flush_pseudodb is
launched.

Yocto Bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13898
Fixes: dde90a5dd2b2 ("wic: Fix multi images .wks with bitbake")
(From OE-Core rev: 445b0a9544b55735496bbb23dbff3399b3b9e9a4)

Signed-off-by: Ricardo Ribalda <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-19 22:57:27 +01:00
Paul Barker
af728be16f archiver.bbclass: Make do_deploy_archives a recursive dependency
To ensure that archives are captured for all dependencies of a typical
bitbake build we add do_deploy_archives to the list of recursive
dependencies of do_build. Without this, archives may be missed for
recipes such as gcc-source which do not create packages or populate a
sysroot.

do_deploy_archives is also added to the recursive dependencies of
do_populate_sdk so that all sources required for an SDK can be captured.

(From OE-Core rev: cd3f766ae1875dfa1f63b68e2ae5b6a950ce67ae)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-18 06:33:36 +01:00
Diego
106acf899e image_types: declare support for wic.zst
Declare images in wic.zst format as supported, as bmaptool now supports
zstd:
1b8437d584
Given the very fast decompression speed of zstd over, for example, gzip,
writing images with bmaptool can provide big write speed improvements
when decompression speed is the limiting factor (especially with very
sparse images).

(From OE-Core rev: 704b01f33196cfdeaa57e2f2602ff2d52b25ab93)

Signed-off-by: Diego Rondini <diego.rondini@kynetics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-18 06:33:36 +01:00
Bruce Ashfield
68ad047228 kernel/reproducibility: kernel modules need SOURCE_DATE_EPOCH export
If CONFIG_IKHEADERS is set to =m, then reproducibility issues creep
into the modules build, since the variables we are setting for the
main kernel build are not present.

Since the source code must be available for a possibly git query
on the timestamp, there didn't seem to be an easy way to move the
environment variable setting to a common routine. As such, we
duplicate the block of code that exports the required variables for
reproducible builds. There is a maintenance risk to this, but any
issues should be easy enough to catch.

(From OE-Core rev: 82cdfcdccfedd320ebc0cdc778c7d4966198b96f)

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-15 16:31:45 +01:00
Joshua Watt
54865c7c1e classes/kernel: Generate reproducible kernel module tarball
If reproducible binaries are requested, generate the kernel modules
tarball in a reproducible way. Namely, sort the tarball contents, clamp
the mtime to the source date epoch, compress the archive with a
consistent compression level (9), and don't include the name and
timestamp in the compressed file.

[YOCTO #12719]

(From OE-Core rev: 5c6a472c82a9b3c122fc54d3ee7171508c643a8b)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-14 14:15:11 +01:00
wenlin.kang@windriver.com
f63e51a165 populate_sdk_base.bbclass: fix warning: name not matched
When do populate_sdk with SDK_ARCHIVE_TYPE = "zip", have below warning
in log file, this patch fixes it.

zip warning: name not matched: sysroots/core2-64-poky-linux/etc/tmpfiles.d/etc.conf
zip warning: name not matched: sysroots/core2-64-poky-linux/etc/tmpfiles.d/home.conf
zip warning: name not matched: sysroots/core2-64-poky-linux/etc/resolv-conf.systemd
zip warning: name not matched: sysroots/core2-64-poky-linux/etc/mtab
zip warning: name not matched: sysroots/core2-64-poky-linux/etc/resolv.conf
zip warning: name not matched: sysroots/core2-64-poky-linux/var/lock
zip warning: name not matched: sysroots/core2-64-poky-linux/usr/lib/environment.d/99-environment.conf
zip warning: name not matched: sysroots/core2-64-poky-linux/sbin/klogd
zip warning: name not matched: sysroots/core2-64-poky-linux/sbin/syslogd
zip warning: name not matched: sysroots/x86_64-pokysdk-linux/usr/bin/chfn
zip warning: name not matched: sysroots/x86_64-pokysdk-linux/usr/bin/chsh
zip warning: name not matched: sysroots/x86_64-pokysdk-linux/usr/bin/passwd
zip warning: name not matched: sysroots/x86_64-pokysdk-linux/usr/bin/chpasswd
zip warning: name not matched: sysroots/x86_64-pokysdk-linux/sbin/vipw
zip warning: name not matched: sysroots/x86_64-pokysdk-linux/sbin/vigr
zip warning: name not matched: sysroots/x86_64-pokysdk-linux/sbin/nologin

(From OE-Core rev: 5f7df08aca81ac269490876fe84c7fc5b1d5739c)

Signed-off-by: Wenlin Kang <wenlin.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-14 14:15:11 +01:00
Anton Eliasson
fbb08a1561 meson.bbclass: Close the log file after reading
This fixes warnings like:

    WARNING: package-name-0.0.1-r0 do_configure: <string>:164: ResourceWarning:
    unclosed file <_io.TextIOWrapper
    name='/source_directory/build/tmp/work/arch/package-name/0.0.1-r0/package-name-0.0.1//meson-logs/meson-log.txt'
    mode='r' encoding='UTF-8'>

(From OE-Core rev: 4fe8bb4e3f15a3f4e153605d57096c565c5fac21)

Signed-off-by: Anton Eliasson <anton.eliasson@axis.com>
Reviewed-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Anton Eliasson <anton.eliasson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-12 15:20:42 +01:00
Tim Orling
e063a3924a pypi.bbclass: use new pypi UPSTREAM_CHECK_URI
Upstream https://pypi.python.org/pypi/${PYPI_PACKAGE}/
redirects to https://pypi.org/project/${PYPI_PACKAGE}/

(From OE-Core rev: 308206a1d2dad33ba70a1461e7e101eaea0e1675)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-12 15:20:42 +01:00
Konrad Weihmann
158f5db602 pypi.bbclass: mind package suffix on version check
Some pypi packages do have suffixes like dev, or a0 or b1.
When doing a version check on these, the version will get falsely
identified as major release versions.
Add a terminating slash to rule out those false positives

(From OE-Core rev: 21e3b2226597fce9ae9328cf4bdbb4cdba3aa711)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-12 15:20:42 +01:00
Peter Kjellerstedt
1f1650e5d8 sstate.bbclass: Do not fail if files cannot be touched
It may be that a file is not allowed to be touched, e.g., if it is a
symbolic link into a global sstate cache served over NFS.

(From OE-Core rev: 462e3f505217c9b9e0d2c1cbdafb0c8910bb6a59)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-09 18:57:21 +01:00
Richard Purdie
643c26562f insane: Promote warnings into errors
There are several warnings the build can emit which show real problems
in the way recipes are being built. Some distros like poky and the Yocto
Project autobuilder have been fixing the issues these show for some time.

OE-Core has therefore been clean of these errors and warnings for some time.
Promote warnings into errors to match the defaults in poky to encourage
people to fix these real issue and reduce confusion when people switch
distros or customise.

(From OE-Core rev: a6a3a4a3739df3ab867fbe7e5a8cb5cac5ee124a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-07 12:16:00 +01:00
Richard Purdie
e568c8ec2a utils: Drop is_machine_specific()/machine_paths()
There appear to be no users of these in OE-Core and their functionality
is questionable at best too. Probably safest to remove them entirely
at this point.

(From OE-Core rev: 03c5cfd7611c88ce82d16619a49e3ea9fa7385d8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-05 13:19:55 +01:00
Richard Purdie
2457c6c886 utils: Drop FILESPATHPKG usage
Nothing in OE-Core uses this and its been deprecated for some time. Remove
the last reference to it.

(From OE-Core rev: 5b27c2f58fa45a55199c08ef219e3b41a09d2fe4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-05 13:19:55 +01:00
Richard Purdie
b55f765ae4 abi_version/staging: Bump versions to force rebuild after sstate corruption
A "broken" buildtools-extended-tarball has been released into the wild
where it is optimising binaries for the host processsor. This is fine in
local usage but in a non-homogeneous cluster like our autobuilder, this
results in SIGILL on other machines when the sstate is shared amongst them
and is painful to debug.

The buildtools tarball has been fixed but we need to invalidate the hash
equivalence and sstate data. This change does that. Adding to OE-Core
rather than autobuilder local changes as its good to illustrate how to
do this and the issue is potentially wider than just Yocto Project
infrastructure.

(From OE-Core rev: 23cdfa9eb23146a69edb77a83ef03719a62ddaae)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-02 09:08:52 +01:00
Richard Purdie
f7b8747c52 sanity: Require gcc 6 or later
New versions of rpm have issues with the libgomp support in gcc 5.x
so raise the minimum to 6 or later. This mainly affects Ubuntu 16.04.

(From OE-Core rev: aca2e5816203b54e0955eaa99fc980d010052d5d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-02 09:08:52 +01:00
Richard Purdie
a98d9c5b71 image/packagegroup: Remove PACKAGE_GROUP_*, long since deprecated
This was deprecated in 2014 so we can safely remove the old code now.

(From OE-Core rev: fe1b79188cbe8159a0950f0c02d7f476a6694a04)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-29 15:02:19 +01:00
Ovidiu Panait
2aa51e2e31 populate_sdk_base: create_sdk_files: Use _nativesdk variants for libdir and base_libdir
Use the _nativesdk variants for libdir and base_libdir to fix the resulting
ld.so.conf paths in the SDK for systems with libdir=/lib64.

(From OE-Core rev: a6077e87f4eeb89b0617883c9c01e08219ae44dc)

Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-29 15:02:19 +01:00
Ovidiu Panait
a3b21540c3 populate_sdk_base: Add ld.so.conf for nativesdk-binutils
Extend the functionality provided by commit [1] to the SDK as well. This way we
can make sure that nativesdk-binutils finds SDK libraries first rather than
host ones.

This is useful for example when trying to build the linux kernel using
nativesdk-gcc. This scenario currently fails because it tries to link to host
libraries rather than SDK host ones:

make x86_64_defconfig
make bzImage
...
error: Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel
Makefile:1101: recipe for target 'prepare-objtool' failed
make: *** [prepare-objtool] Error 1
....
/../../../../x86_64-wrlinuxsdk-linux/bin/ld: /lib/x86_64-linux-gnu/libpthread.so.0: undefined reference to `__libc_vfork@GLIBC_PRIVATE'
...

[1] 15049c610b [buildtools-tarball: Add an ld.so.conf for nativesdk-binutils]

(From OE-Core rev: 7ec84a463ad4c45aee9cd2cbc75b43e5aab5cd18)

Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-29 15:02:19 +01:00
Ricardo Ribalda Delgado
c58711f0ea wic: Fix multi images .wks with bitbake
In order to support .wks files with multiple images inside bitbake we
need to explicitly set the pseudo database in use.

Eg: If we try this .mks:
part / --source rootfs --ondisk sda --fstype=ext4
part /export --source rootfs --rootfs=core-image-minimal-mtdutils --fstype=ext4

The username for all the files under /export will be set to the runner
of bitbake (usually UID 1000).

Before we run wic, we need to make sure that the pseudo database will be
flushed, and contains all the data needed.

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: dde90a5dd2b22a539095d1bac82acc15c6380ac8)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Richard Purdie
3e9cdd8788 staging: Fix overlapping file failures
If there are different providers of a file and they are swiched when the
recipe isn't machine specific, we can get tracebacks due to the overlapping
files. The issue is that the previous provider isn't uninstalled since
the system can't tell whether some later task needs them.

By tracking which tasks we depend upon, the code can now choose to
uninstall more things since a later task can reinstall if/as needed.

The code here was to protect against code with two different tasks
running in parallel which is still protected agaisnt.

[YOCTO #13702]

(From OE-Core rev: 9d8e6daa866d2f19b2a6324072b984a866715426)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Yann Dirson
d645fe38d3 package: get_package_mapping: avoid dependency mapping if renamed package provides original name
Packages with a runtime dependency on a target package whose name is
changed by the PKG_* mechanism must rebuild when that mapping changes,
but we have no way of tracking this today, so
eg. packagegroup-machine-base ends up with a relationship on a
versioned kernel-image, and does not get rebuilt when that version
changes, leading to unsatisfiable dependency and reproducibility
issue.

OTOH there is no reason for the dependency to get rewritten if the
renamed package already has a RPROVIDES on the non-rewritten package
name, and if the dependency relationship is an unversionned one.  This
is what this patch prevents.

Note that this may not cover all cases of rewritten package names.

Notably I had to let the rewrite be done in the case of versionned
dependencies, as package managers usually can follow "Provides" in
such case; this includes many dependencies against shared-lib packages
renamed to their soname, and those at least are OK, since the
dependent recipe should explicitly depend on the target recipe.

(From OE-Core rev: 920beaaeef62b558e046f32c8ef0332250969ef1)

Signed-off-by: Yann Dirson <yann@blade-group.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Dan Callaghan
6efa038c01 package.bbclass: inject "minidebuginfo" into packaged binaries
"Mini debuginfo" is a special section in ELF executables containing
minimal compressed debuginfo for non-exported symbols:

https://sourceware.org/gdb/onlinedocs/gdb/MiniDebugInfo.html

It lets debugging tools produce better stack traces, including local
function names, without incurring the space overhead of full debuginfo.
The feature was originally developed for Fedora:

https://fedoraproject.org/wiki/Features/MiniDebugInfo

but nowadays it is widely supported in the ecosystem, including in gdb
and elfutils (and therefore also in tools which use elfutils, such as
systemd-coredump).

This patch adds an optional extra step in package.bbclass to inject
minidebuginfo while stripping and splitting out debuginfo. It can be
enabled by setting PACKAGE_MINIDEBUGINFO=1. In my testing, this
increases the size of resulting binaries by roughly 5%.

The code for producing and re-injecting the minidebuginfo is my own
Python implementation but corresponds directly to the shell
implementation that RPM uses for doing the same:

https://github.com/rpm-software-management/rpm/blob/rpm-4.15.1-release/scripts/find-debuginfo.sh#L261

(From OE-Core rev: 4df992ce50c2d12e356b6d9fe7b23a6320c8b4df)

Signed-off-by: Dan Callaghan <dan.callaghan@opengear.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Alexander Kanavin
c7ce37d3dd testimage: add an overall timeout setting
This is useful when tests misbehave and get stuck, or when
a significant increase in testing time is undesirable and
needs to be caught automatically.

(From OE-Core rev: d77546e910ad9048f0057f4465716d417b810065)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Alexander Kanavin
4ae71d667d testimage.bbclass: correctly process SIGTERM
Python's unittest will not propagate exceptions upside
of itself, but rather will just catch and print them.

The working way to make it stop is to send a SIGINT
(e.g. simulate a ctrl-c press), which will make it exit
with a KeyboardInterrupt exception.

This also makes pressing ctrl-c twice from bitbake work
again (previously hanging instances of bitbake and qemu were
left around, and bitbake would no longer start until they
were killed manually).

(From OE-Core rev: 06568a06ca22ee279d1829d26d4c38738233c06a)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Konrad Weihmann
da0ab97f61 testimage: enable ovmf support
Add support for running wic images with EFI as testimage.
Introduces a variable called QEMU_USE_OVMF for configuration.

(From OE-Core rev: 3af8aaff68ed332d812ea7dc184d392700ad7882)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-24 14:10:08 +01:00
Domarys Correa
5c90ec2d80 insane.bbclass: Add test for shebang line length
Shebang lines longer than 128 characters can give an error
depending on the operating system.
This implements a test that signals an error when locating a
faulty shebang.

YOCTO: #11053

(From OE-Core rev: e694b09cacf8eb0b7e189f449816d3d6a337cb37)

Signed-off-by: Domarys Correa <domarys.correa@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-24 14:10:08 +01:00
Konrad Weihmann
8361b47d7f runqemu: add QB_ROOTFS_EXTRA_OPT parameter
Content of the optional parameter will be appended to the rootfs-device
in the qemu configuration, in case QB_ROOTFS_OPT is not specified.
By default this is empty.

Example use cases are:

Defining 'readonly' when using squashfs, so multiple instances of qemu
can share the same base image, something that cannot be done by just
specifying 'snapshot'.

Defining 'bootindex=0' which helps to get past the EFI shell in
ovmf-binary. This also enables the use case of running WIC images with
EFI bootloader through the testimage.bbclass.

(From OE-Core rev: e9b8c194636cb5505774a2a71bf54450580dd5b8)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-24 14:10:08 +01:00
Bartosz Golaszewski
0cc4c008d6 qemuboot.bbclass: don't redefine IMGDEPLOYDIR
This variable is already defined in image.bbclass and there's not need
to redefine it here.

(From OE-Core rev: 3ab3b10d14798d27c8e5a096daab36d1bad64e80)

Signed-off-by: Bartosz Golaszewski <bgolaszewski@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-24 14:10:07 +01:00
Paul Barker
b72dbb2e4b kernel-yocto.bbclass: Fix deps when externalsrc is used
do_kernel_configme was recently removed from SRCTREECOVEREDTASKS so this
task still runs when externalsrc is used. This task normally runs after
do_patch but when externalsrc is used, do_patch is removed and this ordering
restriction does nothing. This allows bitbake to execute do_kernel_configme
too early, causing races with do_unpack.

This is fixed by adding in a dependency on do_unpack when externalsrc is
used.

(From OE-Core rev: 75d31beb03d9369448d8d77e12321aa2d91bebf0)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-24 14:10:07 +01:00
Alejandro Hernandez Samaniego
26da846034 Windows: Enable Windows builds under WSLv2 and warn accordingly
Due to the architectural changes between Windows Subsystem for Linux v2,
and WSL v1 it should now be possible to run bitbake on the several distros
offered through the Microsoft Store.

WSLv2 is available on Windows 10 build number > 18917

The current build number may be checked by opening a cmd prompt on Windows
and running:

C:\Users\myuser>ver

Microsoft Windows [Version 10.0.19041.113]

If a distro has already been installed via the Microsoft Store, then we can
check which WSL version its using by opening a Windows Powershell (notice this
is a powershell and not a cmd prompt):

C:\WINDOWS\system32> wsl -l -v
  NAME      STATE           VERSION
  * Ubuntu    Running         2
    Debian    Stopped         1

In this case it shows two distros installed, Ubuntu running WSLv2 and
Debian running WSLv1

To change the version of WSL being used by a certain distro run:

C:\WINDOWS\system32> wsl --set-version <Distro> 2

e.g

C:\WINDOWS\system32> wsl --set-version Debian 2

For more information on installing WSLv2 please look at:

  https://docs.microsoft.com/en-us/windows/wsl/wsl2-install

There are some caveats related to the way storage is handled by WSLv2 though,
and at this point these have to be managed by the user manually, the storage
space used by WSL is not reflected immediately and since bitbake heavily uses
storage, after several builds this can prove to be a bit of an issue.

WSLv2 uses a VHDX file for storage, this issue can be easily avoided by
optimizing this file every now and then, this can be done via the following:

1.- Find the location of your VHDX file:
 - Get the distro app package directory.
 - Open Windows Powershell as Administrator and run:
  Get-AppxPackage -Name "*<DISTRO>*" | Select PackageFamilyName

 e.g.:
  PS C:\WINDOWS\system32> Get-AppxPackage -Name "*Ubuntu*" | Select PackageFamilyName

 PackageFamilyName
  -----------------
   CanonicalGroupLimited.UbuntuonWindows_79abcdefgh

 Replace the PackageFamilyName (and your user) on the following path:

 C:\Users\<user>\AppData\Local\Packages\<PackageFamilyName>\LocalState\

 e.g.

 ls C:\Users\<user>\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\

 Mode                 LastWriteTime         Length Name
  -a----         3/14/2020   9:52 PM    57418973184 ext4.vhdx

 The VHDX file path is:
  C:\Users\<user>\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx

2.- Optimize your VHDX file (Also on Powershell):
 - Make sure WSL is shutdown
  wsl --shutdown

 - Optimize it
  optimize-vhd -Path C:\Users\<user>\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx -Mode full

 A progress bar should be shown while optimizing the VHDX file.

 As an example, after building core-image-sato, removing the TMPDIR did not reflect
 any changes on Windows Explorer for storage space being used, after optimizing the
 VHDX file, 14 extra GB were shown as free.

 So, as long as the the user optimizes its storage, the builds should run smoothly.

This patch warns the user that is running bitbake under WSLv2, that they should
optimize the VHDX file eventually to avoid storage issues.

The same check previoulsy used for WSLv1 works for WSLv2, checking for the kernel
version:
WSLv1:
Linux version 4.4.0-19041-Microsoft (Microsoft@Microsoft.com)

WSLv2:
Linux version 4.19.84-microsoft-standard (oe-user@oe-host)

Builds have been tested under Ubuntu and Debian distros offered and installed through
the Microsoft Store, and other distros should be able to run builds just as fine.

Performance wise, using the same hardware, and same configuration a comparison between
builds using native Linux vs WSLv2 for the following targets has been performed:
- core-image-minimal
- core-image-sato
- core-image-sato-sdk
- meta-toolchain

No real evidence of any performance changes could be found, with WSLv2 builds running even
faster in some cases.

Running a recently built image can be done just as smoothly, if using "nographic" as
argument for runqemu, or if its a graphical image, installing an X server and running
runqemu runs just as fine.

Happy bitbaking.

(From OE-Core rev: c42cec0c1c57c4e67dc7cdb07c5e4aba14a847d3)

Signed-off-by: Alejandro Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Alejandro Hernandez Samaniego <alhe@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-06 14:25:52 +01:00
Paul Gortmaker
45157a4e82 sanity.bbclass: echo current SDK_VENDOR if it is invalid
It can be frustrating if this sanity check triggers, but you
don't know why; you haven't explicitly set any SDK vars, or
similar.

At least echo out the offending value, so the end user has
a bit more information to go on.

Before:
  SDK_VENDOR should be of the form '-foosdk' with a single dash
After:
  SDK_VENDOR should be of the form '-foosdk' with a single dash; found '-overc-sdk'

Cc: Ross Burton <ross.burton@intel.com>
(From OE-Core rev: e238fa177bd72bc5d165fbe4f640132267a1d3fd)

Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-05 11:46:38 +01:00
Martin Kelly
1b15cc6e65 cmake: fix Python executable with cmake
Currently, CMake has two ways of finding Python executables:
FindPythonInterp and FindPython/FindPython3. FindPythonInterp is
deprecated and may be removed at some point. Currently, python3native
sets PYTHON_EXECUTABLE, which FindPythonInterp uses. This is a problem
for a few reasons:

- Setting PYTHON_EXECUTABLE as an environment variable doesn't work, as
  CMake needs it to be set as an explicit CMake option via -D.
- Projects using the newer FindPython/FindPython3 don't pickup the right
  Python, as the newer routines use Python_EXECUTABLE and
  Python3_EXECUTABLE.

Fix this by setting PYTHON_EXECUTABLE, Python_EXECUTABLE, and
Python3_EXECUTABLE using -D options to EXTRA_OECMAKE.

The CMake routines are documented below:
https://cmake.org/cmake/help/latest/module/FindPythonInterp.html
https://cmake.org/cmake/help/latest/module/FindPython.html
https://cmake.org/cmake/help/latest/module/FindPython3.html

(From OE-Core rev: a22200a646eaf42cd4902a2fe3358d29717ac129)

Signed-off-by: Martin Kelly <mkelly@xevo.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-05 11:46:38 +01:00
Richard Purdie
4c9cfe61a5 sanity: Require gcc 5.0 minimum version
After polling various develoeprs, its clear that many layers are struggling
with gcc 4.8 and its better for the project to adopt 5.0 as a minimum
version at this point in time. We should have technology like
buildtools-extended-tarball available to ensure things still work on Centos 7
and Debian 8.

(From OE-Core rev: abc741af16311cb473b7e3185ae34265b243d804)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-05 11:46:37 +01:00
Tim Orling
3783313700 sanity.bbclass: add test for gcc < 4.8
It is known that old versions of gcc prior to 4.8 causes builds to fail.

Add a test for BUILD_CC == 'gcc' and gcc < 4.8 and recommend using
scripts/install-buildtools or user built buildtools-extended-tarball.

Use the new get_host_compiler_version function from lib/oe/utils.py

NOTE: another solution is to install devtoolset-6+ from scl [1], but
this is a rather large install (> 1 Gb) and fairly invasive.

[1] https://www.softwarecollections.org/en/scls/rhscl/devtoolset-6/

Adding this code means we can increase the minimum version easily
in the future too (which will soon be needed).

RP: Change minimum version from 5.0 to 4.8 for initial patch
(From OE-Core rev: 3bb3b9cbad82b2f09386153226d1d4e769b7347b)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-01 11:44:24 +01:00
Tim Orling
8c1aeb60c8 sanity.bbclass: recommend using install-buildtools
For old tar version (< 1.28), recommend using
scripts/install-buildtools

Drop check for tar version 1.24. Dubious extra value.

(From OE-Core rev: 7a66434cf11b7f051699b774e4fccd6738351368)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-01 11:44:23 +01:00
Lee Chee Yang
7c37e5c7cd cve-check: CPE version '-' as all version
CPE version could be '-' to mean no version info.
Current cve_check treat it as not valid and does not report these
CVE but some of these could be a valid vulnerabilities.

Since non-valid CVE can be whitelisted, so treat '-' as all version
and report all these CVE to capture possible vulnerabilities.

Non-valid CVE to be whitelisted separately.

[YOCTO #13617]

(From OE-Core rev: c69ee3594079589d27c10db32bc288566ebde9ef)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-01 11:44:23 +01:00
rpjday@crashcourse.ca
06aa1aa9fe core-image.bbclass: add mention of allow-root-login, stateless-rootfs
Add mention/description of missing IMAGE_FEATURES.

(From OE-Core rev: 9f126a97d61bed481b64351fdcf27beaf6365a52)

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-30 12:33:29 +01:00
Changqing Li
b4cd7ef48e vala.bbclass: add libdir to XDG_DATA_DIRS
Otherwise .gir files installed in the sysroot will not be found when
multilib is in use.

for vapigen, it need to find .vapi and .gir files under XDG_DATA_DIRS,
for gobject-introspection, multilib configurations use libdir rather
than datadir

(From OE-Core rev: b88f0590b4606d8adc8d728086a22325c1ae56b1)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-29 12:05:31 +01:00
Peter Kjellerstedt
c2b67f880f archiver.bbclass: Correct a typo
Also add a missing space in a warning message.

(From OE-Core rev: 6d1d0dccafeaeb971ad18a8bb4ebdd2860d407d5)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-22 10:48:44 +00:00
Mike Looijmans
27ff81bfd8 classes/populate_sdk_base: Implement xz compression options
Building an SDK on a machine with 8GB RAM resulted in excessive swapping
due to the xz compressor using ~20GB of memory. This is because xz is
being called with "-T 0 -9".

To allow tuning the compression versus memory usage, introduce a variable
named SDK_XZ_OPTIONS that defaults to a more sane default:
SDK_XZ_OPTIONS ?= "${XZ_DEFAULTS} ${SDK_XZ_COMPRESSION_LEVEL}"
The use of XZ_DEFAULTS fixes the excessive memory usage.
The SDK_XZ_COMPRESSION_LEVEL variable allows overriding the speed vs
compression. In an office or development environment the extra time
spent on compressing a few percent more is just not worth it.

(From OE-Core rev: 25533a48a19e3e6eb9f96be51b1373830b5bb9f0)

Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-22 10:48:43 +00:00
Khem Raj
9b7daa6650 linuxloader: Add get_musl_loader_arch function
get_musl_loader_arch returns the arch part of ldso for musl, this is
used in get_musl_loader() as well as independently usable, which is
needed for multilib support in musl. Musl stores all ldso in /lib
be it multilib or not, therefore do not use base_libdir instead directly
use /lib

[YOCTO #11971]

(From OE-Core rev: 0b694592c417c1d21913905f5032d300c9635866)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-21 22:39:00 +00:00
Adrian Freihofer
2f6a47c4d4 runqemu: support multiple NICs
Emulating more than one network interface with runqemu is a bit tricky,
but possible. For example, the following leads to an emulated device with
eth0 and eth1:

QB_NETWORK_DEVICE_prepend = " \
    -device virtio-net-device,mac=52:54:00:12:34:03 \
"

or

QB_NETWORK_DEVICE_append = " \
    -device virtio-net-pci,mac=52:54:00:12:34:03 \
"

When booting Qemu with two NICs, the kernel does not know which
interface the specified ip=192.168.7.... command line argument
should be applied. This delays the boot process for a very long
time and a guest wihtout IP configuration.

This add two new configuraton parameters to runqemu:
QB_CMDLINE_IP_SLIRP and QB_CMDLINE_IP_TAP to explicitely specify the ip=
kernel command line arguments for tap and slirp mode.

Note: Simply adding "::eth0" broke some builds on the Yocto autobuilder.

(From OE-Core rev: 59bfdc331c1494c05ab38804b281878a1f571f6d)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-21 22:39:00 +00:00
Kai Kang
48f7290a8e base.bbclass: extend PACKAGECONFIG for conflict package configs
There are mutually exclusive PACKAGECONFIGs in recipes. Though it
declares that package configs are exclusive, it can't prevent users to
set them at same time. Extend PACKAGECONFIG to support specifying
conflicted package configs.

(From OE-Core rev: 734475b3f86d88a548bc9eb91d836bd1b9335e9f)

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-12 12:46:08 +00:00
Rich Persaud
b3ee2f9fc5 grub-efi-cfg: enable per-label APPEND override
For legacy bios boot configurations, syslinux supports multiple
labels with per-label APPEND definitions.  grub-efi-cfg supports
multiple labels, but only a single APPEND definition.

Enable optional per-label APPEND definitions for grub EFI, with
variable names prefixed by "grub_" to isolate grub definitions from
syslinux defintions.

Example use from an ISO image recipe that inherits grub-efi-cfg:

 LABELS_LIVE="foo bar"
 APPEND_grub_foo = "linuxcmdline"

No change in behavior for those using APPEND without overrides.

(From OE-Core rev: 20ad7705d1060300d53304c259b1f07d04b86eba)

Signed-off-by: Rich Persaud <rp@stacktrust.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-11 11:44:29 +00:00
Christopher Clark
59e64ae58c image-prelink: remove assumption of sysconfdir presence
If sysconfdir is not present in the image filesystem then the temporary
creation of a prelink.conf will fail. Fix this by creating sysconfdir
temporarily if needed beforehand and then remove any directories that
were created afterwards.

fixes: OpenXT OXT-1751

(From OE-Core rev: 6bf3e3db355643c4b2e20346230c09e45e519035)

Signed-off-by: Christopher Clark <christopher.w.clark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-11 11:44:29 +00:00
Christopher Clark
2ac00852cd kernel.bbclass: fix SOURCE_DATE_EPOCH for non-git kernel builds
The source directory is not always a git repository, so when querying
git for data to set SOURCE_DATE_EPOCH, specify ${S}/.git as the git
directory to prevent retrieving incorrect data from any parent directory.

Fixes the following errors with the prior logic when building a kernel
that is not obtained from a git repository:

1. With TMPDIR set to a directory outside any git repository on a
mounted filesystem, reproducible builds fail in do_compile with this git
error:

  fatal: not a git repository (or any parent up to mount point <abspath>)
  Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

aborting before the error handling logic.

2. With TMPDIR located within a subdirectory of a git repository, the
SOURCE_DATE_EPOCH timestamp would be that of said repository rather than
that of the kernel.

(From OE-Core rev: 270ae94fe345b9ce98d822034cbfad7e24c5f393)

Signed-off-by: Christopher Clark <christopher.w.clark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-10 23:20:33 +00:00
Chee Yang Lee
e753b71b39 cve-check: fix ValueError
fix below error for whitelisted recipe and recipe skip cve check.

Error:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_cve_check(d)
     0003:
File: '/poky-master/meta/classes/cve-check.bbclass', lineno: 59, function: do_cve_check
     0055:        try:
     0056:            patched_cves = get_patches_cves(d)
     0057:        except FileNotFoundError:
     0058:            bb.fatal("Failure in searching patches")
 *** 0059:        whitelisted, patched, unpatched = check_cves(d, patched_cves)
     0060:        if patched or unpatched:
     0061:            cve_data = get_cve_info(d, patched + unpatched)
     0062:            cve_write_data(d, patched, unpatched, whitelisted, cve_data)
     0063:    else:
Exception: ValueError: not enough values to unpack (expected 3, got 2)

(From OE-Core rev: 64a362bd2dd0b4f3165d5162adbc600826af66f8)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-10 23:20:33 +00:00
Paul Barker
3abf367f48 kernel-yocto.bbclass: Support config fragments with externalsrc
The merging of config fragments is performend in the do_kernel_configme
task and so config fragments will not be supported when this task is
removed from the dependency tree.

kernel-yocto adds additional tasks which may modify the source directory
to SRCTREECOVEREDTASKS so that they are removed when using externalsrc.
However, do_kernel_configme should be safe to use, the only modification
to the source tree is the potential creation of the '.kernel-meta'
directory and the '.metadir' file.

(From OE-Core rev: 44f04c039a4d61dd18666e42b9b9865cbc3ada9e)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-10 23:20:33 +00:00