Commit Graph

3193 Commits

Author SHA1 Message Date
Alexander Kanavin
9facffec78 selftest/bblayers: adjust the revision for the layer setup test
Using a tag is not actually robust enough, e.g. poky-contrib
checkouts do not come with any tags. So let's use a revision
matching yocto-4.0, that ought to be present.

(From OE-Core rev: 1246aa8e4c9e6fce2f7700cd8e8ad9566a21d6e3)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-01 10:07:02 +01:00
Alexander Kanavin
fe7eb16537 selftest/bblayers: add a test for creating a layer setup and using it to restore the layers
This does a basic run-through of the bitbake-layers plugin, and the resulting json layer config
and the layer setup script that uses it. Only poky is actually fetched by the script.

(From OE-Core rev: 84ddd6fc6effbb74499409da7e85c09c8a1ff9ea)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-01 10:07:02 +01:00
Alexander Kanavin
e561fc1cbe bitbake-layers: add ability to save current layer repository configuration into a file
This addresses a long standing gap in the core offering:
there is no tooling to capture the currently configured layers
with their revisions, or restore the layers from a configuration
file (without using external tools, some of which aren't particularly
suitable for the task). This plugin addresses the 'capture' part.

Note that the actual writing is performed by a sub-plugin; one such
sub-plugin is provided (for the json + python script format), but
more can be added (e.g. kas, repo, etc.).

How to save a layer configuration:

a) Running with default choices:

$ bitbake-layers create-layers-setup /srv/work/alex/meta-alex/
NOTE: Starting bitbake server...
NOTE: Created /srv/work/alex/meta-alex/setup-layers.json
NOTE: Created /srv/work/alex/meta-alex/setup-layers

b) Command line options:

NOTE: Starting bitbake server...
usage: bitbake-layers create-layers-setup [-h] [--output-prefix OUTPUT_PREFIX] [--writer {oe-setup-layers}] [--json-only] destdir

 Writes out a configuration file and/or a script that replicate the directory structure and revisions of the layers in a current build.

positional arguments:
  destdir               Directory where to write the output
                        (if it is inside one of the layers, the layer becomes a bootstrap repository and thus will be excluded from fetching).

optional arguments:
  -h, --help            show this help message and exit
  --output-prefix OUTPUT_PREFIX, -o OUTPUT_PREFIX
                        File name prefix for the output files, if the default (setup-layers) is undesirable.
  --writer {oe-setup-layers}, -w {oe-setup-layers}
                        Choose the output format (defaults to oe-setup-layers).

                        Currently supported options are:
                        oe-setup-layers - a self-contained python script and a json config for it.

  --json-only           When using the oe-setup-layers writer, write only the layer configuruation in json format. Otherwise, also a copy of scripts/oe-setup-layers (from oe-core or poky) is provided, which is a self contained python script that fetches all the needed layers and sets them to correct revisions using the data from the json.

(From OE-Core rev: 5606d1a123a3816ab45e49ee7707ed84c9c23c5c)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-01 10:07:02 +01:00
Joshua Watt
f5d6792d68 meta/files: add layer setup JSON schema and example
Defines a common schema for layer setup that can be consumed by tools to
know how to fetch and assemble layers for end users. Also includes an
example of the layer setup that constructs poky/meta-intel/imaginary product layer
for reference.

The schema can be used to validate a layer setup file with the commands:

 $ python3 -m pip install jsonschema
 $ jsonschema -i meta/files/layers.example.json meta/files/layers.schema.json

(From OE-Core rev: 72740b5dd635579e373b4bfe6ccacfe6a02aa998)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>

Alex: I made the following modifications to Joshua's original commit:

- moved the files from meta/lib to meta/files

- the example json showcases a multi-repo, multi-layer setup
instead of just poky - closer to a typical product

- added oe-selftest that validates the example json against the schema using python3-jsonschema-native

- the schema is modified so that:

-- all lists (sources, layers, remotes) are replaced by objects keyed by 'name' properties of the list items.
This allows using them as dicts inside Python, and makes the json more compact and readable.

-- added 'contains_this_file' property to source object

-- replaced 'remote' property with a 'oneOf' definition for git with a specific
'git-remote' property. 'oneOf' is problematic when schema validation fails:
the diagnostic is only that none of oneOf variants matched, which is too non-specific.

-- added 'describe' property to 'git-remote' object.

-- removed description property for a layer source: it is not clear how to add that
when auto-generating the json

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-01 10:07:02 +01:00
Alexander Kanavin
3f9e3feb22 bitbake-layers: add a command to save the active build configuration as a template into a layer
This is the reverse of setting up a build by pointing TEMPLATECONF to a directory
with a template and running '. oe-init-build-env': this takes the config files from build/conf,
replaces site-specific paths in bblayers.conf with ##OECORE##-relative paths, and copies
the config files into a specified layer under a specified template name.

In many or perhaps most cases such static prefabricated configurations (that require no
further editing) are just enough, and I believe they should be offered by the
official configuration management. On the other hand, generating build configurations with a
sufficiently versatile tool is a far more complex problem, and one we should try to tackle
once we see where and how static configs fall short.

Tooling to discover and select these templates when setting up a build will be provided later on.

How to use:

alex@Zen2:/srv/work/alex/poky/build-layersetup$ bitbake-layers save-build-conf ../../meta-alex/ test-1
NOTE: Starting bitbake server...
NOTE: Configuration template placed into /srv/work/alex/meta-alex/conf/templates/test-1
Please review the files in there, and particularly provide a configuration description in /srv/work/alex/meta-alex/conf/templates/test-1/conf-notes.txt
You can try out the configuration with
TEMPLATECONF=/srv/work/alex/meta-alex/conf/templates/test-1 . /srv/work/alex/poky/oe-init-build-env build-try-test-1
alex@Zen2:/srv/work/alex/poky/build-layersetup$

(From OE-Core rev: f319534dc8fc68dfe120d129154a509f0cd6a3b0)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-01 10:07:02 +01:00
Ross Burton
f941dad9d7 oeqa/selftest: add test for debuginfod
Add a new selftest to exercise the debuginfod support, by starting a
debuginfod on DEPLOY_DIR and verifying that an image can fetch the
symbols for a binary.

(From OE-Core rev: d035fd394fd2747ab4b75867af6123f3efb1990f)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-30 10:33:35 +01:00
Andrei Gherzan
50f150e7e3 selftest: Add module for testing rootfs postcommands
The initial implementation adds tests for 'tidy_shadowutils_files'.

(From OE-Core rev: 9640ce00c986626573a748859129b6e2eeeafa35)

Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-25 11:08:06 +01:00
Andrei Gherzan
fe796c2a17 rootfspostcommands.py: Cleanup subid backup files generated by shadow-utils
When creating users, shadow-utils might create backup files for
subordinate ID files (subid, subgid). Make sure we clean them up
similarly to the other backup files shadow-utils creates.

(From OE-Core rev: 4e4ea5adea8a00b4a78ffbe7cc60931deb74c161)

Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-25 11:08:06 +01:00
Andrei Gherzan
29bb84b82b rootfspostcommands.py: Restructure sort_passwd and related functions
This change proposes a restructure of the functions in
rootfspostcommandstests.py to clarify the purpose of each function and
also, make it scalable for other use cases (for example adding support
for removing subid backup files).

The main function of interest here is 'tidy_shadowutils_files' which
brings in the functionality of the old 'sort_passwd' making it clear
that it doesn't only sort the passwd file:
- delete backup files
- it sorts passwd, group and the associated shadow files

The other functions are also renamed for consistency and clarity and
more documentation was added.

(From OE-Core rev: 81a0a4dbfb0313b967a7a98eb7fd1e13edb1a9be)

Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-25 11:08:06 +01:00
Pavel Zhukov
f237f6e36c parselogs: Ignore xf86OpenConsole error
[Yocto #13854]

If VT argument was not specified Xorg server tries to bind to VT1, then
VT2 and so on. In some cases (runqemu with nographics or serial options
for example) VT1 can be taken by systemd getty service which generates
error message. Do not fail on this message if Xorg is running. (covered
by test_xorg_running test)

(From OE-Core rev: d047f493e0c7f341dd307a4d8dd0db08a22824f1)

Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-25 11:08:06 +01:00
Ross Burton
238660fcca oeqa/selftest: rewrite gdbserver test
The gdbserver test case didn't actually work and doesn't follow the
documentation for how to use gdbserver in Yocto.  Rewrite the test case
to follow the documented process so if that breaks then we're aware.

(From OE-Core rev: a8eddb71b16a2b958cde54d0dbd35f7a9467ddd2)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:24:11 +01:00
Ross Burton
e015c6cf32 oeqa/commands: add support for running cross tools to runCmd
If native_sysroot is passed, also support the caller passing in the
target_sys and add that to the path if so.  This allows runCmd() to be
used to invoke the cross tools.

(From OE-Core rev: afa3d3ba00b40fd29e9852eeaa2c2c9b68f18659)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:24:11 +01:00
Ross Burton
31dc02701d oeqa/qemurunner: add run_serial() comment
Add a comment explaining the non-obvious return codes.

(From OE-Core rev: 6572baffa02ba6b8a686490d55af17cacb528920)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:24:11 +01:00
Pavel Zhukov
077bf3c410 selftest: Add regression test for rpm filesnames
Escaping globs and quoting in rpm spec files is tricky and requires a
bit of dancing. In addition to that it changes from time to time.
Adding (simple) regression test for different types of filename
patterns. Cover brackets and parentheses in first iteration

[Yocto #13746]

(From OE-Core rev: 142432217c152970249884fad240f7441cb1a2ad)

Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-21 22:51:42 +01:00
Alexander Kanavin
f2fb3c54a3 devtool: do not leave behind source trees in workspace/sources
These are typically auto-extracted with modify/upgrade from recipes
and can be easily recreated. On the rare occasions where they need
to be reused, they are still available under workspace/attic (which
is already used for old recipes and appends), so nothing gets lost.

This avoids the annoyance of devtool refusing to proceed because
there is a previous source tree in workspace/sources.

For independent source trees behave as before: do nothing.

Adjust the test that previously deleted those trees by hand.

(From OE-Core rev: 9bfb95d070d68d5ab5adfe0ea096f5fbf9cad8b0)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-21 22:51:42 +01:00
Jon Mason
7bde98700f oeqa/parselogs: add qemuarmv5 arm-charlcd masking
On qemuarmv5, arm-charlcd is logging an error because the device isn't
present on the virtual machine.  Mask it off, as that device could be
present on the physical hardware (and we want to use the same kernel
config as the real hardware).

(From OE-Core rev: c03c33a4032f995a288f7287e79f43fcd3140aa1)

Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-21 22:51:41 +01:00
Ernst Sjöstrand
c80405aa9d cve-check: Don't use f-strings
Since we're keeping cve-check aligned between the active branches,
and dunfell is supported on Python 3.5, we can't use f-strings.

(From OE-Core rev: 1821cf7464cbba521b55a9c128fe8812c0cc5eca)

Signed-off-by: Ernst Sjöstrand <ernstp@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-14 08:13:32 +01:00
Richard Purdie
ce08cf4825 lib: Add copyright statements to files without one
Where there isn't a copyright statement, add one to make it explicit.
Also add license identifiers as MIT if there isn't one.

(From OE-Core rev: bb731d1f3d2a1d50ec0aed864dbca54cf795b040)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-12 12:00:43 +01:00
Richard Purdie
0f6817f2a4 selftest/runtime_test: Use testexport in IMAGE_CLASSES, not globally
testexport doesn't make sense outside the scope of SDKs and images so
use via IMAGE_CLASSES instead of in the global scope.

(From OE-Core rev: ffa7556ae58dd4d806bf1881f5e208d16a64b833)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-12 11:46:26 +01:00
Richard Purdie
a74792b15d selftest/runtime_test/incompatible_lic: Use IMAGE_CLASSES for testimage
testimage should be included via IMAGE_CLASSES, not globally with INHERIT.

(From OE-Core rev: 4cdb29c7342b16a6c9294268a674a1414eed88e5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-12 11:46:26 +01:00
Yogesh Tyagi
b9a57cad4d gdbserver : add selftest
The test runs gdbserver on qemu and connects the gdb client from host over TCP.

It builds a cross gdb on the host and compiles the program to be debugged on the target,
launches the gdbserver and tries to connect cross gdb to it.

[YOCTO #13996]

(From OE-Core rev: 37164f7e39eea3a1e594d8306d2569868438ba93)

Signed-off-by: Yogesh Tyagi <yogesh.tyagi@intel.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-10 08:25:28 +01:00
Richard Purdie
35183c1b33 selftest/wic: Tweak test case to not depend on kernel size
The current test assumes the kernel size leaves a certain amount of whitespace
in the output. Improve this constraint so a slightly larger kernel doesn't fail
the test.

(From OE-Core rev: bd60c44bef4a1b5d3c8fe77a9e6d3a8f43b0dea4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-08 15:44:21 +01:00
Thomas Roos
c8e4c239f2 oeqa devtool: Add tests to cover devtool handling of various git URL styles
Add two test cases for git URL styles that trigger reformat_git_url.

[YOCTO #11394]

(From OE-Core rev: 5593439a5efbb53fc46099650ae86943751b0c4e)

Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-08 15:44:21 +01:00
Richard Purdie
0246dd5f50 selftest/reproducible: Exclude rust/rust-dbg for now until we can fix
There looks to be a reproducibility issue left in one of the rust
libraries. It doesn't appear to be a string issue but some binary
problem. Disable rust from the reproducibility testing until we can
get to the bottom of the issue (allowing wider testing of all the
other improvements).

(From OE-Core rev: a261333f6591ea94afc567dee04a2e3c6d5059cf)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-08 15:44:20 +01:00
Richard Purdie
2ead8dc18b oeqa/selftest/sstate: Ensure tests are deterministic
glob.glob() depends on the order of files on disk and selecting [0]
is race prone. We should cover all the nativesdk files so rework
the function to do this.

Spotted as some oe-selftests failed, some passed and it raised a question
of why!

(From OE-Core rev: 8818478420a5c73b1dc1710774545f7e984307da)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-07 13:18:56 +01:00
Richard Purdie
2ef7affa31 oeqa/sdk/rust: Fix file deletion for multilib SDKs
We need to use shutil.rmtree here since removedirs() only covers
directories. Make the exception for specific too to make errors
easier to catch.

(From OE-Core rev: 9d2a661e46123a2292f7887658e6fa54923dbcc0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-01 14:18:16 +01:00
Mihai Lindner
403dfe91ea create-spdx: Fix supplier field
The correct field name is "supplier" according to SPDX schema.
The "supplier" field translates to "PackageSupplier", but that's for
tag-value format.

(From OE-Core rev: ca8db0e0a2860ac1e3f537471fa71b43c3be0a58)

Signed-off-by: Mihai Lindner <mihai.lindner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-28 14:43:21 +01:00
Alexander Kanavin
9b3fcb0d91 selftest/meta_ide: add a test for running SDK tests directly in a yocto build
There's been a recent discussion about how we can make the Yocto SDK
experience better [1]. One of the ideas was to eliminate the SDK
as a separate artefact altogether and simply provide everything
that the SDK and eSDKs do directly in a yocto build. This does not
mean that people have to 'learn Yocto', but rather that the integrators
should provide a well-functioning sstate cache infrastructure (same as
with minimal eSDK, really), and a few wrapper scripts for setting up the build
and the SDK environment that run layer setup and bitbake behind the scenes.

[1] https://lists.openembedded.org/g/openembedded-architecture/topic/thoughts_on_the_esdk/90990557

So without further ado, here's how you get a 'SDK' without building one:

1. Set up all the needed layers and a yocto build directory.

2. Run:
$ bitbake meta-ide-support
$ bitbake -c populate_sysroot gtk+3
(or any other target or native item that the application developer would need)
$ bitbake populate-sysroots

3. Set up the SDK environment:
. tmp/deploy/images/qemux86-64/environment-setup-core2-64-poky-linux
(adjust accordingly)

Et voila! The Unix environment is now set up to use the cross-toolchain from
Yocto, exactly as in the SDK. And devtool/bitbake are available to extend it,
exactly as in the eSDK.

Theare are numerous benefits here: no need to produce, test, distribute and maintain
separate SDK artifacts. No two separate environments for the yocto build and the SDK.
Less code paths where things can go wrong. Less awkward, gigantic tarballs. Less
SDK update headaches: 'updating the SDK' simply means updating the yocto layers with
git fetch or layer management tooling. Built-in SDK extensibility: just run bitbake
again to add more things to the sysroot, or add layers if even more things are required.

How is this tested?

Exactly same as the regular SDK:
$ bitbake -c testsdk meta-ide-support

This runs the same toolchain tests from meta/lib/oeqa/sdk/cases as the regular
sdk testing does.

(From OE-Core rev: 5c845d7f4ea6ae7ba18ed43180dad28775cace31)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-28 11:50:17 +01:00
Alexander Kanavin
638f988602 oeqa/sdk: allow epoxy/galculator tests to run in esdk and direct yocto builds
Other tests already have similar tweaks.

(From OE-Core rev: 3134d19ba15bb783389c40617d5e2b568c7cd81c)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-28 11:50:17 +01:00
Alexander Kanavin
3afd79df9e oeqa/sdk: add a test class for running SDK tests directly in a Yocto build
This is a simpler version of the same class in testsdk.py, as it does not
need to unpack and set up the SDK, and can proceed to the tests straight away.

(From OE-Core rev: be21c62e5bd96164aab9e01168b7a43c6de44c17)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-28 11:50:17 +01:00
Alexander Kanavin
29c52d7b26 toolchain-scripts.bbclass: adjust toolchain_create_tree_env_script to better replicate (e)SDK
Specifically:
1. Place the environment setup file into $B and not into $TMPDIR,
so that the recipe using the class can itself better decide what to do with the file.

2. Use global, unified sysroots (provided through build-sysroots recipe)
and not recipe-specific ones, as this allows flexible on-the-fly management of what
libraries are available to build applications, without having to modify any
recipes, similar to eSDK 'extensible' part.

This also requires adjustment of the sstate sametune_samegsigs test, as meta-ide-support
becomes dependent on $MACHINE (unified sysroots have it in their paths)
and needs to be excluded from the test.

3. Add a few missing settings that have been added to SDK environment files.

4. Add a snippet to the environment setup file that also runs the relocation scripts.
In regular SDKs this is executed by the SDK installer, in direct SDK we can do it when
setting up the environment.

(From OE-Core rev: db5dfd78ae441201778b1175f4fb9a3eba994899)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-28 11:50:17 +01:00
Otavio Salvador
1e3e20e92e oeqa/sdk: Add basic rust cargo test
Add a QA test to the SDK to test that a basic cargo build works.

[RP: Tweaked to work for multilibs and updated to match toolchain changes]

(From OE-Core rev: d0cfe587bc897e79ef01805cc9a42fbca28c883c)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-28 11:07:33 +01:00
Ross Burton
a87284d4ab oeqa/gotoolchain: set CGO_ENABLED=1
In cross-compiles CGO_ENABLED=1 needs to be set explicitly, as otherwise
Go refuses to use it even if CC is already set.

This fixes the selftest on setups where the host and the SDK target
don't have matching architectures.

[ YOCTO #14859 ]

(From OE-Core rev: 19be072619d39267df44f23c4c8b64f3808f6148)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-25 22:59:01 +01:00
Ross Burton
e857abbfcb oeqa/gotoolchain: put writable files in the Go module cache
By default 'go mod' creates read-only files, but that just complicates
things.  Add -modcacherw to make the cache read/write, so it can be
cleaned up without needing to chmod.

(From OE-Core rev: 7ff30e0d9fe8527cbc2f8ca84e0300fdc84663b6)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-25 22:59:01 +01:00
Ross Burton
e49831d1f8 oeqa/selftest: rename git.py to intercept.py
By naming this test class git.py, any attempt to import GitPython (as
needed by oelib.buildhistory) failed.

As this class exercises the intercepts, rename it to intercept.py.

(From OE-Core rev: d557cbbf86767bc2ebf2beb3d70af3b3ca5e0529)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-25 22:59:00 +01:00
Ross Burton
454a0ea700 zlib: remove historical movement of libz.so to /lib
Historically there's been a split between /lib for early boot and
/usr/lib for everything else, but with modern systems this split is
meaningless and incomplete. If a minimal system for early boot is
needed, it should be a full minimal system in a initramfs.

[RP: Fixed up selftest to match]
(From OE-Core rev: 990073dfc167354b4af41db83ac46c18b1aa99d5)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-25 22:59:00 +01:00
Shruthi Ravichandran
17e2eaed03 package_manager/ipk: do not pipe stderr to stdout
Some opkg commands print an error during cleanup when the tmp_dir
does not exist and an attempt is made to delete it. The error messages
are harmless and the opkg commands eventually succeed.
When these commands are run and stderr is piped to stdout, the error
messages may clobber the stdout and cause unexpected results while
parsing the output of the command. Therefore, when parsing the output
of a command, do not pipe stderr to stdout. Instead, capture stderr
and stdout separately, and upon success, send stderr to bb.note().

(From OE-Core rev: f2167ae80258253eb47a5b148546b265320284cc)

Signed-off-by: Shruthi Ravichandran <shruthi.ravichandran@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-25 22:59:00 +01:00
Richard Purdie
b890bca84d base/reproducible: Change Source Date Epoch generation methods
In bc, we found that there are files newer than Changelog (e.g. scan.l)
which means after install runs, the timestamp is clamped to SDE which means
"bitbake bc -C compile" would then try and regenerate generated files and
complain flex was missing.

Rather than assuming Changelog/NEWS are magic for dates, drop that scan
method and scan all unpacked files. This shouldn't be that slow as they'd
be in the disk cache already after an unpack.

For bc, this changes SDE by about two minutes and avoids the problems
rebuilding.

Add a version comment to the task definition as changes in python library
code won't trigger a rebuild and we need one here.

(From OE-Core rev: 32dda0ad91a9e7946351c897578b4c97ae142341)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-23 13:55:26 +01:00
Ross Burton
33ff9b7d13 oeqa/runtime: add test that the kernel has CONFIG_PREEMPT_RT enabled
This is the absolute bare minimum for testing the RT patches, but it
does mean we if we build and boot a RT kernel we can verify that it is
what we expect.

(From OE-Core rev: 0301d5845115d09299f87683b3efa46f3b4c7be9)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-20 14:27:06 +01:00
Joshua Watt
f414763fa0 sstatesig: Include all dependencies in SPDX task signatures
SDPX generation involves looking through BB_TASKDEPDATA for
dependencies, then linking to the generated documents for those
dependencies. These document links use a checksum to validate the
document, which means that if a upstream document changes, all
downstream documents must be regenerated to get the new checksum,
otherwise the compendium of documents produced by the build will have
broken links; therefore all dependent task should be included in the
signature (even from "ABI safe" recipes).

(From OE-Core rev: 5fe543b9ceec971cf0297ff0ae3b0ccc4703cece)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-19 10:54:20 +01:00
Jose Quaresma
4161dbbbd6 oe/utils: remove the ThreadedPool
The ThreadedPool in OE-core is mainly because python2
doesn't have threaded pools but python2 is dead for
some time now and python3 have a ThreadPoolExecutor.

The only local in OE-core where this ThreadedPool
is in use is on the sstate.bbclass that is ported
to the python3 ThreadPoolExecutor.

(From OE-Core rev: aa8fd5e7c2a1125895accfd55ce9320819a10959)

Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-15 17:51:34 +01:00
Richard Purdie
150f5ee03c selftest/runtime_test/virgl: Disable for all almalinux
We can't support vgem on RHEL derived distros so disable this test for
all almalinux hosts rather than specific versions.

(From OE-Core rev: e921f3c1b917072e4c5a110c7dfeeadd2e571bde)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-15 16:59:09 +01:00
Ming Liu
e116ad7c69 meta: introduce UBOOT_MKIMAGE_KERNEL_TYPE
Sometimes an end user might want to choose another kernel type argument
for uboot-mkimage other than "kernel", for instance: "kernel_noload".

Let's introduce a variable UBOOT_MKIMAGE_KERNEL_TYPE to support that,
and it could be used by BSP layers as well.

(From OE-Core rev: e288686e97de1265eeeaf452141e1473867efb1b)

(From OE-Core rev: 4eb7bbcc2f08b25387a15b7e4a89ef199783c973)

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-15 12:33:21 +01:00
Richard Purdie
b051ca1289 oeqa/selftest/sstatetests: Update test to work with bitbake changes
Bitbake is dropping the DL_DIR fallback for local file urls and ensuring
local urls are fetchable. This test fails as it copies the meta directory
of COREBASE but not scripts and nativesdk-qemu-helper references runqemu
from there which doesn't exist in the copied data.

Tweak to symlink scripts into position in the copied metadata which
avoids the now fatal parsing error.

(From OE-Core rev: 5a5199943de5df9a4d44277d07f4313642c34b3a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-15 12:22:08 +01:00
Paulo Neves
de85c2f47b oeqa: test_invalid_recipe_src_uri expect parse time error
Before this commit the test expected a runtime error but the logic of
the fetcher changed so that files not found for checksumming cause a
fatal interruption of the parse, thuse never reaching the runtime error
this test expected. This commit aligns with the bitbake changes to the
fetcher.

(From OE-Core rev: 8b506bfbdd18dfdb411080f69ef5c0d416b3f2e0)

Signed-off-by: Paulo Neves <ptsneves@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-15 12:22:08 +01:00
Paul Eggleton
74b25399de oe-selftest: devtool: test modify git recipe building from a subdir
Add a test that verifies that devtool modify + devtool finish do the
right thing on a recipe that fetches from git and sets S to point to
a subdirectory of the source tree. We have a few examples among the core
recipes, dos2unix is a convenient one so let's use that. (The test first
verifies that that is still true in case the recipe is changed in
future.)

(From OE-Core rev: a84d9ed14173b0bf467ea78dff4f0f7bae0bc082)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-14 23:22:10 +01:00
Paul Eggleton
da71c928cd devtool: finish: handle patching when S points to subdir of a git repo
If devtool finish needs to create a patch and have it applied to the
sources for a recipe where S points to a subdirectory of the sources,
then the patch needs to be applied at the root of the repo i.e. we need
to add a patchdir= parameter to the SRC_URI entry.

(From OE-Core rev: ad3736d9ca14cac14a7da22c1cfdeda219665e6f)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-14 23:22:09 +01:00
Paul Eggleton
3a2a584737 patch: handle if S points to a subdirectory of a git repo
If PATCHTOOL = "git", SRC_URI fetches from a git repo and S points to
a subdirectory of the checked out sources, then we were erroneously
initialising the subdirectory as its own git repo. Check if the returned
top-level repo directory is a subdirectory of WORKDIR and do not
run initialise the source directory if that is the case.

(This was a regression introduced with OE-Core revision
6184b56a7a0fc6f5d19fdfb81e7453667f7da940, however we didn't have a test
that verified the behaviour.)

(From OE-Core rev: 9cca53a2bcbf6809615ce5626c86c6ee481a7a76)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-14 23:22:09 +01:00
Alexandre Belloni
120fec3303 oeqa/selftest/bbtests: Update message lookup for test_git_unpack_nonetwork_fail
bitbake's output changed, update the test

(From OE-Core rev: 7e64b63f96dd1d71e263e7bbbe6591e51e98395a)

Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-08 00:09:42 +01:00
Paulo Neves
2ef273f424 oeqa/sysroot.py: Check bitbake return status
bitbake ran but we incorrectly did not assert the exit status needs to
be non 0. Now all sysroot tests commands expected to fail are verified
to do so.

(From OE-Core rev: 5fe8c14f50d414e768588cef0675d8ef296ced77)

Signed-off-by: Paulo Neves <ptsneves@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-08 00:09:42 +01:00