Commit Graph

6620 Commits

Author SHA1 Message Date
Changqing Li
b4cd7ef48e vala.bbclass: add libdir to XDG_DATA_DIRS
Otherwise .gir files installed in the sysroot will not be found when
multilib is in use.

for vapigen, it need to find .vapi and .gir files under XDG_DATA_DIRS,
for gobject-introspection, multilib configurations use libdir rather
than datadir

(From OE-Core rev: b88f0590b4606d8adc8d728086a22325c1ae56b1)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-29 12:05:31 +01:00
Peter Kjellerstedt
c2b67f880f archiver.bbclass: Correct a typo
Also add a missing space in a warning message.

(From OE-Core rev: 6d1d0dccafeaeb971ad18a8bb4ebdd2860d407d5)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-22 10:48:44 +00:00
Mike Looijmans
27ff81bfd8 classes/populate_sdk_base: Implement xz compression options
Building an SDK on a machine with 8GB RAM resulted in excessive swapping
due to the xz compressor using ~20GB of memory. This is because xz is
being called with "-T 0 -9".

To allow tuning the compression versus memory usage, introduce a variable
named SDK_XZ_OPTIONS that defaults to a more sane default:
SDK_XZ_OPTIONS ?= "${XZ_DEFAULTS} ${SDK_XZ_COMPRESSION_LEVEL}"
The use of XZ_DEFAULTS fixes the excessive memory usage.
The SDK_XZ_COMPRESSION_LEVEL variable allows overriding the speed vs
compression. In an office or development environment the extra time
spent on compressing a few percent more is just not worth it.

(From OE-Core rev: 25533a48a19e3e6eb9f96be51b1373830b5bb9f0)

Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-22 10:48:43 +00:00
Khem Raj
9b7daa6650 linuxloader: Add get_musl_loader_arch function
get_musl_loader_arch returns the arch part of ldso for musl, this is
used in get_musl_loader() as well as independently usable, which is
needed for multilib support in musl. Musl stores all ldso in /lib
be it multilib or not, therefore do not use base_libdir instead directly
use /lib

[YOCTO #11971]

(From OE-Core rev: 0b694592c417c1d21913905f5032d300c9635866)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-21 22:39:00 +00:00
Adrian Freihofer
2f6a47c4d4 runqemu: support multiple NICs
Emulating more than one network interface with runqemu is a bit tricky,
but possible. For example, the following leads to an emulated device with
eth0 and eth1:

QB_NETWORK_DEVICE_prepend = " \
    -device virtio-net-device,mac=52:54:00:12:34:03 \
"

or

QB_NETWORK_DEVICE_append = " \
    -device virtio-net-pci,mac=52:54:00:12:34:03 \
"

When booting Qemu with two NICs, the kernel does not know which
interface the specified ip=192.168.7.... command line argument
should be applied. This delays the boot process for a very long
time and a guest wihtout IP configuration.

This add two new configuraton parameters to runqemu:
QB_CMDLINE_IP_SLIRP and QB_CMDLINE_IP_TAP to explicitely specify the ip=
kernel command line arguments for tap and slirp mode.

Note: Simply adding "::eth0" broke some builds on the Yocto autobuilder.

(From OE-Core rev: 59bfdc331c1494c05ab38804b281878a1f571f6d)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-21 22:39:00 +00:00
Kai Kang
48f7290a8e base.bbclass: extend PACKAGECONFIG for conflict package configs
There are mutually exclusive PACKAGECONFIGs in recipes. Though it
declares that package configs are exclusive, it can't prevent users to
set them at same time. Extend PACKAGECONFIG to support specifying
conflicted package configs.

(From OE-Core rev: 734475b3f86d88a548bc9eb91d836bd1b9335e9f)

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-12 12:46:08 +00:00
Rich Persaud
b3ee2f9fc5 grub-efi-cfg: enable per-label APPEND override
For legacy bios boot configurations, syslinux supports multiple
labels with per-label APPEND definitions.  grub-efi-cfg supports
multiple labels, but only a single APPEND definition.

Enable optional per-label APPEND definitions for grub EFI, with
variable names prefixed by "grub_" to isolate grub definitions from
syslinux defintions.

Example use from an ISO image recipe that inherits grub-efi-cfg:

 LABELS_LIVE="foo bar"
 APPEND_grub_foo = "linuxcmdline"

No change in behavior for those using APPEND without overrides.

(From OE-Core rev: 20ad7705d1060300d53304c259b1f07d04b86eba)

Signed-off-by: Rich Persaud <rp@stacktrust.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-11 11:44:29 +00:00
Christopher Clark
59e64ae58c image-prelink: remove assumption of sysconfdir presence
If sysconfdir is not present in the image filesystem then the temporary
creation of a prelink.conf will fail. Fix this by creating sysconfdir
temporarily if needed beforehand and then remove any directories that
were created afterwards.

fixes: OpenXT OXT-1751

(From OE-Core rev: 6bf3e3db355643c4b2e20346230c09e45e519035)

Signed-off-by: Christopher Clark <christopher.w.clark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-11 11:44:29 +00:00
Christopher Clark
2ac00852cd kernel.bbclass: fix SOURCE_DATE_EPOCH for non-git kernel builds
The source directory is not always a git repository, so when querying
git for data to set SOURCE_DATE_EPOCH, specify ${S}/.git as the git
directory to prevent retrieving incorrect data from any parent directory.

Fixes the following errors with the prior logic when building a kernel
that is not obtained from a git repository:

1. With TMPDIR set to a directory outside any git repository on a
mounted filesystem, reproducible builds fail in do_compile with this git
error:

  fatal: not a git repository (or any parent up to mount point <abspath>)
  Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

aborting before the error handling logic.

2. With TMPDIR located within a subdirectory of a git repository, the
SOURCE_DATE_EPOCH timestamp would be that of said repository rather than
that of the kernel.

(From OE-Core rev: 270ae94fe345b9ce98d822034cbfad7e24c5f393)

Signed-off-by: Christopher Clark <christopher.w.clark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-10 23:20:33 +00:00
Chee Yang Lee
e753b71b39 cve-check: fix ValueError
fix below error for whitelisted recipe and recipe skip cve check.

Error:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_cve_check(d)
     0003:
File: '/poky-master/meta/classes/cve-check.bbclass', lineno: 59, function: do_cve_check
     0055:        try:
     0056:            patched_cves = get_patches_cves(d)
     0057:        except FileNotFoundError:
     0058:            bb.fatal("Failure in searching patches")
 *** 0059:        whitelisted, patched, unpatched = check_cves(d, patched_cves)
     0060:        if patched or unpatched:
     0061:            cve_data = get_cve_info(d, patched + unpatched)
     0062:            cve_write_data(d, patched, unpatched, whitelisted, cve_data)
     0063:    else:
Exception: ValueError: not enough values to unpack (expected 3, got 2)

(From OE-Core rev: 64a362bd2dd0b4f3165d5162adbc600826af66f8)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-10 23:20:33 +00:00
Paul Barker
3abf367f48 kernel-yocto.bbclass: Support config fragments with externalsrc
The merging of config fragments is performend in the do_kernel_configme
task and so config fragments will not be supported when this task is
removed from the dependency tree.

kernel-yocto adds additional tasks which may modify the source directory
to SRCTREECOVEREDTASKS so that they are removed when using externalsrc.
However, do_kernel_configme should be safe to use, the only modification
to the source tree is the potential creation of the '.kernel-meta'
directory and the '.metadir' file.

(From OE-Core rev: 44f04c039a4d61dd18666e42b9b9865cbc3ada9e)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-10 23:20:33 +00:00
Paul Barker
f5930bfb8f kernelsrc.bbclass: Fix externalsrc support
When the externalsrc class is used the tasks listed in
SRCTREECOVEREDTASKS are deleted to prevent them being executed. If
externalsrc is used for the kernel then this will include
virtual/kernel:do_patch.

We can depend on do_shared_workdir instead as this will survive when
externalsrc is used.

(From OE-Core rev: 2c17d35cc7b9c5e01fd5829858d2f0234e7ac8d6)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-10 23:20:33 +00:00
Richard Purdie
995f7985f6 reproducibile_build: Fix SDE file generation when unpack reruns
Currently, if an existing TMPDIR is rebuilt, do_fetch/do_unpack can rerun
but SDE would remain unchanged. This leads to different results compared
to a fresh build. An example change which triggered this is:
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=cb4e69e6346a9fbeebf83a5d5397cacbd41d48b5

Instead, delete any existing SDE and recalculate if we're reunning.
Also rename and drop the do_ prefix since these are for tasks,
not functions.

(From OE-Core rev: 5d579fc2fe71637fc6e071aa66542befa39ac8bb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-08 08:21:11 +00:00
Chee Yang Lee
932ac04cb8 cve-check: show whitelisted status
change whitelisted CVE status from "Patched" to "Whitelisted".

[Yocto #13687]

(From OE-Core rev: 181bdd670492525f9488d52c3ebb9a1b142e35ea)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-07 10:06:17 +00:00
Andrej Valek
75bebb3546 populate_sdk_ext: enable custom templateconf.cfg
Do not always override templateconf.cfg content. Add option to use
already existing file.

(From OE-Core rev: e524e49ef22fd69882d5d2d01cd84db790e9cb88)

Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-07 10:06:16 +00:00
Richard Purdie
67511eda71 sstate: Drop obsolete check in hash validation
Now this function has a summary parameter we can drop this check.

It could well be why the mysterious "locked sigs" selftest fails
intermittently if this function were called with a single hash
to check.

[YOCTO #13605] (with luck)

(From OE-Core rev: 02aabe0e59f73bf206d9bada1e7089832ceed254)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-06 17:14:04 +00:00
Ming Liu
c412d3440c kernel-yocto.bbclass: fix a wrong inter-task dependency
do_kernel_checkout and do_symlink_kernsrc are both modifying ${S}, they
could conflict with eacher other, move do_kernel_checkout after
do_symlink_kernsrc does fix that.

(From OE-Core rev: 965090f42bc0576e938a0575b7938a1ff60b0018)

Signed-off-by: Ming Liu <liu.ming@toradex.com>
Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-06 08:19:18 +00:00
Joshua Watt
fea1d99036 classes/kernel.bbclass: Fix parsing errors
legitimize_package_name wants the actual value of KERNEL_REVISION, so
use d.getVar() to fetch it as is done elsewhere in the file. Failing to
do so can result it weird errors at parsing time.

(From OE-Core rev: 9bff4d14693c1890fc181ec68c9f883dc4e4accf)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-06 08:19:18 +00:00
Andreas Müller
5e1f0d593e mime-xdg.bbclass: Fix typo in comment
(From OE-Core rev: d6fc0081f53ebc5b12ffff917cf3c04c270a69f1)

Signed-off-by: Andreas Müller <schnitzeltony@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-01 17:50:30 +00:00
Richard Purdie
6c43f76248 package_ipk: Add missing variables to the task hash
Several variables used by the task are not included in the hash, as highlighted
by recent changes to MAINTAINERS not causing rebuilds. Fix this.

(From OE-Core rev: 0bfb2e984062e2a00f8989d26aebb89b112d81d2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-23 23:33:48 +00:00
Richard Purdie
5281e3f101 package: Correct variable dependecies
Ensure that the plain variable X is also added to the dependencies
as well as X_<pkg>.

Allow the funciton to be called with a different variable list too.

(From OE-Core rev: c392401e10ff43d10b7e57c9d552522a02c91878)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-23 23:33:48 +00:00
Peter Kjellerstedt
b645630090 Revert "rootfs: Make BUILDNAME a weak default in reproducible_build_simple"
This reverts commit 8380df6566.

Giving BUILDNAME a static default in reproducible_build_simple.bbclass
to have /etc/version be generated with that name lead to other users
of BUILDNAME no longer working as expected. E.g., buildstats.bcclass
would now write its information in a single directory, overwriting the
statistics for each build.

(From OE-Core rev: dcd1fbe153369634c73370cbe22e1dbdac080146)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-21 17:48:09 +00:00
Khem Raj
f9e0c99568 insane.bbclass: Correct typo inhert->inherit
(From OE-Core rev: 8fdceff7a9581173726c2a2636a5815accca73e2)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18 23:53:54 +00:00
Robert P. J. Day
e1a2a5a206 fix various, innocuous typos
Corrections:

  - environment
  - accommodate
  - conversion
  - compatible

(From OE-Core rev: 9797d3b45b4e1b9d77f0f2ee299c17b48d8d3cf6)

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18 23:53:54 +00:00
Alexander Kanavin
ae6ec0b809 weston: add a basic runtime test
The test is checking that weston is able to start.

(From OE-Core rev: 57700767f90eea8f2b78187c42581aca42d50bbf)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18 23:53:54 +00:00
Kevin Hao
dd372f09fc wic: Honor APPEND
APPEND is used to add additional parameters to kernel by features, such
as read-only-rootfs. So we should honor this variable when we compose
the kernel parameter in wic. I know we also can resolve this kind of
issue by using the .wks.in template introduced by commit 42e870c5ed
("image_types.bbclass: support template .wks.in files for wic"), but the
APPEND is needed by all the wks, it would seems pretty ridiculous to me
to change all the .wks to .wks.in and then foist the APPEND into them.
So the APPEND is definitely deserved to export to the wic directly.

[Yocto #12809]

(From OE-Core rev: 18981b8a457104391dfd94938c247eac04e4ed50)

Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-15 22:42:14 +00:00
Khem Raj
4d6ace4011 image.bbclass, package-index.bb: Delete do_populate_lic task
These are meta packages

(From OE-Core rev: e6ea95ae85763670aef35f7cb025eea693138d07)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-15 22:42:14 +00:00
Richard Purdie
f73bf58f05 populate_sdk_ext: We now require python3, not python
We no longer expect a "python" binary in PATH so update the eSDK's
expectations to match. This was the only failure on autobuilder test
systems with python missing.

(From OE-Core rev: 946ce21b10dcad506edcaadb4e4242c049e4c316)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-15 10:26:42 +00:00
Richard Purdie
a819c88b87 staging: Handle races between binaries and their libs
There is a long standing issue where a binary could be installed into the
sysroot before its library dependencies. We've always argued nothing should
use the binary until it has been installed by a dependency but there are issues
around binaries which conflict with the host system, for example patch,
python3, gzip and more.

With the recent patch changes we've seen issues like:
ERROR: gdb-cross-canadian-powerpc-8.3.1-r0 do_patch: Command Error: 'quilt --quiltrc /home/pokybuild/yocto-worker/qemuppc/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/gdb-cross-canadian-powerpc/8.3.1-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0  Output:
Applying patch 0009-Change-order-of-CFLAGS.patch
patch: /lib64/libattr.so.1: version `ATTR_1.3' not found (required by patch)
Patch 0009-Change-order-of-CFLAGS.patch does not apply (enforce with -f)

which is a symptom of this issue (libattr-native is a dependency of patch-native).

There are other ways to fix this such as disabling libattr in patch, installing
patch to a subdirectory and requiring PATH manipulation and so on.

We can simply fix the staging code to handle /bin/ after everything else so
do that and avoid all these other complications.

(From OE-Core rev: 29d17fe23265bf0c7defa14ffc0f677af15c6818)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-15 10:26:42 +00:00
Alex Kiernan
2fd5bda2b3 devicetree.bbclass: include symbols in base DT
When processing overlays, the base device tree must be compiled with
symbols, otherwise attempting to apply overlays in U-Boot will fail
with:

  failed on fdt_overlay_apply(): FDT_ERR_NOTFOUND
  base fdt does did not have a /__symbols__ node
  make sure you've compiled with -@

(From OE-Core rev: d075e39c05ace6dad2c66a5e8c4b1e75aa751b6a)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-14 13:07:23 +00:00
Alex Kiernan
36c879ca38 kernel-fitimage: fix devicetree reproducibility
Ensure that the order of dtb/dtbo files in the generated fitimage is
reproducible.

Fixes: 71bfa9838cbf ("kernel-fitimage: Handle overlays in EXTERNAL_KERNEL_DEVICETREE")
(From OE-Core rev: 9e4a91b63dd8e0c1708da2ac7de461b35fb0b011)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-14 13:07:23 +00:00
Robert P. J. Day
16c9a82c6c icecc.bbclass: replace superfluous "+=" with "="
(From OE-Core rev: 8cd1150a813ba7aa395f3f62c6b91a3561e81c92)

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-14 13:07:23 +00:00
Khem Raj
64e1bf5486 crosssdk: Set nativesdk specific distro features
Currently, normal distro features e.g. ld-is-gold is impacting
crosssdk recipes, which actually should not be the case, since
that feature is essentially intended for target packages and not
nativesdk packages

(From OE-Core rev: aec9f9bd9549938a6ed42e9879f3a2fdcc89463d)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-14 13:07:23 +00:00
Richard Purdie
5fd3eb4bda package_ipk: Minor import tweak + ensure packages regenerated
The recent opkg-utils change didn't update all the packages since its
marked as ABISAFE (and has to be due to update-alternatives).

Fix a minor import issue to avoid multiple imports of glob which
also causes packages to re-generate.

(From OE-Core rev: ef24a545d11febb96d1c0f02c60d9701295ef592)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-13 14:09:17 +00:00
Mark Hatle
35d68aea95 populate_sdk_base: if the SDKIMAGE_FEATURES changes, refresh the SDK
Since the features are processed by a python fragment, we need to explicitly
list the variables that should affect the resulting hash, and thus sstate
re-use.

(From OE-Core rev: c7c12efba87c803d94a7c9c2f8c98b66aeba6d43)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-13 12:19:14 +00:00
Alex Kiernan
c9ab7b0d22 kernel-fitimage: Handle overlays in EXTERNAL_KERNEL_DEVICETREE
When using EXTERNAL_KERNEL_DEVICETREE, collect DTB overlays too (*.dtbo)
as well as iterating down into sub-directories so using the behaviour
for naming which matches KERNEL_DEVICETREE.

(From OE-Core rev: 169ebd59f11845a3a5a7157719217ccf0844e448)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-13 12:19:14 +00:00
Alejandro Hernandez Samaniego
a3416a5933 testimage: Extend runtime testing infrastructure to allow unconventional booting processes to be tested
The current runtime infrastructure contains hardcoded values which Ill refer to
as patterns, these patterns are either searched through or sent via the serial
terminal to communicate between HOST and TARGET.

These patterns are required since they allow us to check when a device has
finished booting, to log in, and to check whether a command sent from our tests
has returned, this way we are able to check both the status of the commands that
were sent along with its output.

The testing process goes somewhat as follows:
1. Launch QEMU and start booting.
2. Check when the device has booted by looking for the pattern login:.
3. Log in as the root user (default for our images).
4. Check that we were able to log in succesfully.
5. Start running the runtime test cases defined by TEST_SUITES.
6. One of such test cases could send a command to the QEMU target.
7. Check whether that command returned.
8. Check its output and status, return whether the test case passed or failed.

This patch allows this set of patterns to be defined instead of being hardcoded,
but it also automatically sets the defaults that we have been using in the past
if they have not been manually defined, for this reason, the patch is less
invasive and should not affect in any way how tests are currently being run.

Cases that can be enabled with this patch:
- A customized image that does not use the root user (or maybe we want to check
what happens if we dont use the root user).
- An image where the PS1 env variable has been modified, and the prompt pattern
wouldnt match the default.
- Baremetal applications, which do not follow the conventional way of booting
Linux and would probably not show a prompt for a user to log in, same applies
for testing bootloaders.
- poky-tiny: Using DISTRO=poky-tiny and an image such as the core-image-tiny
from meta-intel, which boots directly to RAM, and does not show a log in prompt
since it does not contain a conventional init process.

The code itself contains comments that should be self explanatory but here is an
example on how these patterns can be defined in a hypothetical case where we
want to run test cases as the webserver user instead:

TESTIMAGE_BOOT_PATTERNS = "send_login_user search_login_succeeded"
TESTIMAGE_BOOT_PATTERNS[send_login_user] = "webserver\n"
TESTIMAGE_BOOT_PATTERNS[search_login_succeeded] = "webserver@[a-zA-Z0-9\-]+:~#"

The variable TESTIMAGE_BOOT_PATTERNS defines which patterns to override when
used to communicate with the target when booting, anyone familiar with the
PACKAGECONFIG syntax should have no trouble setting these.

Other patterns would still be set up as default, e.g.
search_reached_prompt would still be login:

The accepted flags for TESTIMAGE_BOOT_PATTERNS are the following:
search_reached_prompt, send_login_user, search_login_succeeded,
search_cmd_finished.

They are prefixed with either search/send, to differentiate if the pattern is
meant to be sent or searched to/from the target terminal.

A working example of this code that falls under the baremetal case mentioned
above along with a test case is present on the meta-freertos layer, which tests
an RTOS image built with OpenEmbedded and automatically runs a test case on it
after booting such image:

As usual, INHERIT += "testimage" needs to be present on local.conf
$ bitbake freertos-demo -c testimage

RESULTS:
RESULTS - freertos_echo.FreeRTOSTest.test_freertos_echo: PASSED (2.00s)
SUMMARY:
freertos-demo () - Ran 1 test in 2.006s
freertos-demo - OK - All required tests passed (successes=1, skipped=0,
failures=0, errors=0)

(From OE-Core rev: 3ab2cbfeff371e8791b031a2852eeef80101a831)

Signed-off-by: Alejandro Hernandez Samaniego <aehs29@gmail.com>
Signed-off-by: Alejandro Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-13 12:19:14 +00:00
Alejandro Hernandez Samaniego
e597809307 testimage: Allow testing on QEMU machines with a single serial port
commmit 6cde67d0a8 enables the use of qemurunner on machines that
only have a single serial port, but still sets the default value
as serial_ports=2 if not provided.

The testimage class does not call qemurunner with a serial_ports
argument, hence always defaulting to two.

Pass the serial_ports argument from the testimage class to allow
tests to run on QEMU machines with a single serial port.

(From OE-Core rev: a50d0163770f0b405a8de8a8a9cccd48c1de4112)

Signed-off-by: Alejandro Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-13 12:19:14 +00:00
Jaewon Lee
ed4bdd0f91 sstate.bbclass: fix issue while handling long sstate filenames
When moving to python3, divison using '/' now returns float instead of
an integer. In upstream commit b8025e9720
sstate filenames longer than the limit are changed to just include
necessary info + 3 fields just for information. The space left over
after the necessary info is divided into 3 for each of the fields.
Using '//' instead to do the division to solve the following error
message:

avail = (254 - len(hash + "_" + taskname + extension) -
len(components[0]) - len(components[1]) - len(components[5]) -
len(components[6]) - 7) / 3
    >        components[2] = components[2][:avail]
             components[3] = components[3][:avail]
TypeError: slice indices must be integers or None or have an __index__
method

(From OE-Core rev: 2acfee61a062c6520a413b2a797544d968bb0c76)

Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Mark Hatle
38a754214a package.bbclass: Support stripping and debug copy of static libraries
By default, we won't copy and strip static libraries.  However, this
functionality can be useful in some cases where people are doing
development on the target, and don't generally want the larger debug
capable static libraries.  To enable the new functionality set:

    PACKAGE_DEBUG_STATIC_SPLIT = '1'

Add a new function splitstaticdebuginfo.  Thus function will copy the
unmodified static library into the specific debug directory location.
By keeping an unmodified version, it is possible for a user trying
to debug something to use -L /usr/lib/.debug-static and their existing
build commands to switch from stripped to full debug versions.

The PACKAGE_DEBUG_SPLIT_STYLE will select between two different
approaches, /usr/lib/debug-static or <path>/.debug-static.

Additionally you can now choose to strip static libraries to conserve
space.  If either 'PACKAGE_DEBUG_STATIC_SPLIT' or 'PACKAGE_STRIP_STATIC'
is set to 1, the static library will be stripped.  (This is not on by
default, as it could make diagnosing static library usage difficult in
some cases.)

Add to insane.bbclass a skip to the staticdev warning for the specific
-dbg package versions.

(From OE-Core rev: 17fa66c8199d73f0b59b2b3e609075933bf1e74b)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Mark Hatle
dae03ec7e8 package.bbclass: Allow INHIBIT_PACKAGE_STRIP_FILES to skip .ko and static libs
Change the order of the skip processing to happen before any .ko and static
library processing.  This will allow these types of files to be individually
skipped if necessary.

(From OE-Core rev: c50fcd54ffe60b63d042d05e6cf538a593dc410f)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Mark Hatle
6a543159a3 package.bbclass: Fix debug source processing for static libraries
Format of the sources list is the [ (file, [source, ...]), ... ] before
this change, the static libraries were processed but the items were
included incorrectly causing no sources for static libraries to be
included.

(From OE-Core rev: fa356b23c2f4599681693bba50d36659b07a8125)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Bruce Ashfield
6113de76c8 kern-tools: fix merge_config when LD contains parameters
To ensure that the kernel linker is used when allno/mod/yes config
merge_config steps were executed, the call to make was tweaked to
explicitly pass LD.

But since the variable wasn't quoted, any parameters to LD (like
the sysroot) were mistakenly passed to make, and hence could trigger
an error on some architectures.

We also tweak the logging to hightlight errors like this in the
future and avoid losing it in the noise of merge configs sometimes
overly verbose output.

(From OE-Core rev: a60c4c116efecd7a6ee5a11b1d366bb00b9d23ce)

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Gavin Li
cc71bb7b73 kernel-yocto: fix defconfig detection in find_sccs()
The current code would cause a file like "config.bin" to added to the
config sources list. I am sure the intention was to add any files with
defconfig in its name and not the other way around.

(From OE-Core rev: 06577d49c50b7e9f1f40fe5b52ec88d1bdc1430c)

Signed-off-by: Gavin Li <gavinli@thegavinli.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Martin Jansa
7ba241a697 kernel-yocto.bbclass: do_kernel_configme: don't use +errexit, merge_config_build.log and fail when /.config wasn't created
* for whatever reason, instead of silently continuing to build default kernel config

(From OE-Core rev: ec281a948a23365bd4536139dac077fb1a5eae6d)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Martin Jansa
040ec2d8ab kernel-yocto.bbclass: set KERNEL_LD also for merge_config.sh
* when ld-is-gold is used, merge_config.sh silently fails and doesn't generate anything
  useful in .config (and also include directory isn't created):

  tmp-glibc-bfd-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build:
  total 164K
  drwxr-xr-x 4 martin martin 4.0K Feb  5 02:27 .
  drwxrwxr-x 7 martin martin 4.0K Feb  5 02:27 ..
  -rw-rw-r-- 1 martin martin 133K Feb  5 02:28 .config
  -rw-rw-r-- 1 martin martin   39 Feb  5 02:27 .gitignore
  drwxr-xr-x 4 martin martin 4.0K Feb  5 02:27 include
  -rw-rw-r-- 1 martin martin  201 Feb  5 02:27 Makefile
  drwxrwxr-x 4 martin martin 4.0K Feb  5 02:27 scripts
  lrwxrwxrwx 1 martin martin   61 Feb  5 02:27 source -> /OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source

  tmp-glibc-gold-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build:
  total 28K
  drwxr-xr-x 3 martin martin 4.0K Feb  5 02:26 .
  drwxrwxr-x 7 martin martin 4.0K Feb  5 02:26 ..
  -rw-rw-r-- 1 martin martin   74 Feb  5 02:26 .config
  -rw-rw-r-- 1 martin martin   39 Feb  5 02:26 .gitignore
  -rw-rw-r-- 1 martin martin  201 Feb  5 02:26 Makefile
  drwxrwxr-x 4 martin martin 4.0K Feb  5 02:26 scripts
  lrwxrwxrwx 1 martin martin   61 Feb  5 02:26 source -> /OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source

  $ cat tmp-glibc-gold-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/.config
  CONFIG_LOCALVERSION="-yocto-standard"

* and because the failure is silent, it will happily start building
  kernel with default defconfig, which even builds OK for qemux86-64
  but on qemux86 I've noticed this issue, because incorrectly
  configured kernel build fails with:

  $ tail -n 20 tmp-glibc-gold-qemux86/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/temp/log.do_compile
    LD       /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/libsubcmd-in.o
    AR       /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/libsubcmd.a
    HOSTCC  scripts/mod/mk_elfconfig
    CC      scripts/mod/devicetable-offsets.s
    CC      scripts/mod/empty.o
  cc1: error: code model 'kernel' not supported in the 32 bit mode
  cc1: sorry, unimplemented: 64-bit mode not compiled in
  make[2]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/scripts/Makefile.build:99: scripts/mod/devicetable-offsets.s] Error 1
  make[2]: *** Waiting for unfinished jobs....
  cc1: error: code model 'kernel' not supported in the 32 bit mode
  cc1: sorry, unimplemented: 64-bit mode not compiled in
  make[2]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/scripts/Makefile.build:266: scripts/mod/empty.o] Error 1
  make[1]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/Makefile:1111: prepare0] Error 2
  make[1]: *** Waiting for unfinished jobs....
    LD       /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/objtool-in.o
  /OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/tools/objtool
    LINK     /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/objtool
  make: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/Makefile:179: sub-make] Error 2
  WARNING: exit code 1 from a shell command.

* the issue happens in log.do_kernel_configme, but the log is completely useless:

  cat tmp-glibc-gold-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/temp/log.do_kernel_configme
  DEBUG: Executing python function extend_recipe_sysroot
  NOTE: Direct dependencies are ['virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-devtools/bison/bison_3.5.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb:do_populate_sysroot', 'virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-extended/bc/bc_1.07.1.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/binutils/binutils-cross_2.33.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/quilt/quilt-native_0.66.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/gcc/gcc-cross_9.2.bb:do_populate_sysroot']
  NOTE: Installed into sysroot: ['bison-native', 'bc-native', 'binutils-cross-i686', 'gcc-cross-i686', 'autoconf-native', 'xz-native', 'flex-native', 'gnu-config-native', 'automake-native', 'libtool-native', 'texinfo-dummy-native', 'gettext-minimal-native', 'readline-native', 'zlib-native', 'gmp-native', 'libmpc-native', 'mpfr-native', 'linux-libc-headers', 'm4-native', 'ncurses-native', 'pkgconfig-native']
  NOTE: Skipping as already exists in sysroot: ['kern-tools-native', 'quilt-native']
  DEBUG: sed -e 's:^[^/]*/:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/recipe-sysroot-native/:g' /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/bison-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/gcc-cross-i686/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/autoconf-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/gnu-config-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/automake-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/libtool-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/gmp-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/ncurses-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/pkgconfig-native/fixmepath | xargs sed -i -e 's:FIXMESTAGINGDIRTARGET:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/recipe-sysroot:g; s:FIXMESTAGINGDIRHOST:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/recipe-sysroot-native:g' -e 's:FIXME_PSEUDO_SYSROOT:/OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/pseudo-native:g' -e 's:FIXME_HOSTTOOLS_DIR:/OE/build/oe-core/tmp-glibc/hosttools:g' -e 's:FIXME_PKGDATA_DIR:/OE/build/oe-core/tmp-glibc/pkgdata/qemux86:g' -e 's:FIXME_PSEUDO_LOCALSTATEDIR:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/pseudo/:g' -e 's:FIXME_LOGFIFO:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/temp/fifo.10476:g'
  DEBUG: Python function extend_recipe_sysroot finished
  DEBUG: Executing shell function do_kernel_configme
  DEBUG: Shell function do_kernel_configme finished

  ]because merge_config.sh is redirected to work-shared/qemux86/kernel-source/.kernel-meta/cfg/merge_config_build.log
  and bbfatal_log called in do_kernel_configme only when merge_config.sh fails which it should in this case
  but doesn't.

  The merge_config_build.log shows 5.4 specific error:
    scripts/Kconfig.include:39:  gold linker 'i686-oe-linux-ld' not supported
  but then it happily continues reporting what isn't applied in not created
  .config file

  --- tmp-glibc-bfd-qemux86-do_kernel_configme/work-shared/qemux86/kernel-source/.kernel-meta/cfg/merge_config_build.log  2020-02-05 02:28:01.455520207 +0100
  +++ tmp-glibc-gold-qemux86-do_kernel_configme/work-shared/qemux86/kernel-source/.kernel-meta/cfg/merge_config_build.log 2020-02-05 02:26:30.656470909 +0100
  @@ -171,586 +171,5113 @@
     HOSTCC  scripts/kconfig/symbol.o
     HOSTLD  scripts/kconfig/conf
   scripts/kconfig/conf  --alldefconfig Kconfig
  -./.tmp.config.jDjIEYi9Yq:2208:warning: unexpected data:  #
  -./.tmp.config.jDjIEYi9Yq:2209:warning: unexpected data:  # Generic Driver Options
  -./.tmp.config.jDjIEYi9Yq:2210:warning: unexpected data:  #
  -./.tmp.config.jDjIEYi9Yq:3102:warning: symbol value 'm' invalid for SAMPLE_SECCOMP
  -./.tmp.config.jDjIEYi9Yq:3119:warning: symbol value 'm' invalid for NF_CT_PROTO_GRE
  -./.tmp.config.jDjIEYi9Yq:3120:warning: symbol value 'm' invalid for NF_CT_PROTO_SCTP
  -./.tmp.config.jDjIEYi9Yq:3121:warning: symbol value 'm' invalid for NF_CT_PROTO_UDPLITE
  -#
  -# configuration written to .config
  -#
  +scripts/Kconfig.include:39:  gold linker 'i686-oe-linux-ld' not supported
  +make[2]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/scripts/kconfig/Makefile:73: alldefconfig] Error 1
  +make[1]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/Makefile:567: alldefconfig] Error 2
   make[1]: Leaving directory '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build'
  +make: *** [Makefile:179: sub-make] Error 2
  +grep: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/.config: No such file or directory
  +Value requested for CONFIG_LOCALVERSION not in final .config
  +Requested value:  CONFIG_LOCALVERSION=""
  +Actual value:
  +
  +grep: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/.config: No such file or directory
  +Value requested for CONFIG_LOCALVERSION_AUTO not in final .config
  +Requested value:  # CONFIG_LOCALVERSION_AUTO is not set
  +Actual value:

* I don't know why merge_config.sh uses separate more difficult to find merge_config_build.log
  instead of leaving the output to end in log.do_kernel_configme, I'll send it as separate
  commit so that it can be discussed there.

(From OE-Core rev: 36fe2efc91fec5a52374a2dc4814a69076a2d28e)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Paul Barker
83c1977475 kernel-yocto: Only override CONFIG_LOCALVERSION if LINUX_VERSION_EXTENSION is set
CONFIG_LOCALVERSION may already be set in a defconfig or config fragment
and this should not be unconditionally overridden.

(From OE-Core rev: 613c43f910cbdf5e03ef3d5bbf92de69082a30c9)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Paul Barker
f88946d48d kernel-yocto: Drop setting of unused variables in do_kernel_metadata
The machine_branch and machine_srcrev variables were set but not used in
do_kernel_metadata.

(From OE-Core rev: ee358e03b1ad6eb0624cee0306fef807de0116a6)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Paul Barker
de2ef01204 kernel-yocto: Move defaults and tasks from linux-yocto.inc into bbclass
This allows the kernel-yocto bbclass to be inherited in a recipe without
needing to include linux-yocto.inc. The bbclass should stand on its own
and linux-yocto.inc does a few things which may not be desired in other
kernel recipes (such as modifying KERNEL_FEATURES).

The LINUX_VERSION_EXTENSION default is not moved as other kernel recipes
may not want this setting in place.

(From OE-Core rev: 48d8f97975c0b167c0f4f32145813f00b8c8a7a8)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Mark Hatle
cdcacf26ec microblaze: Adjust Linux items from microblazeeb to microblaze
Due to recent changes to the tune, in order to match config.guess, the name
of the big-endian microblaze architecture was changes to 'microblaze'.

(From OE-Core rev: 6f6a6bbac684ead3fe6d070d61f17c2f611a2c87)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00