Commit Graph

7759 Commits

Author SHA1 Message Date
Wonmin Jung
60a0a2fb1e kernel: Set proper LD in KERNEL_KCONFIG_COMMAND
With 'ld-is-gold' and linux kernel 5.4 or later, menuconfig
task for kernel recipes will fail with:

$ bitbake -c menuconfig virtual/kernel
...
scripts/kconfig/mconf  Kconfig
scripts/Kconfig.include:43:  gold linker 'x86_64-poky-linux-ld' not supported
/OE/build/tmp/work-shared/qemux86-64/kernel-source/scripts/kconfig/Makefile:29:
 recipe for target 'menuconfig' failed
make[2]: *** [menuconfig] Error 1
/OE/build/tmp/work-shared/qemux86-64/kernel-source/Makefile:606:
 recipe for target 'menuconfig' failed
make[1]: *** [menuconfig] Error 2
/OE/build/tmp/work-shared/qemux86-64/kernel-source/Makefile:185:
 recipe for target '__sub-make' failed
make: *** [__sub-make] Error 2
Command failed.

This is because that the KERNEL_LD variable already set in
kernel-arch.bbclass isn't used by do_menuconfig function of
cml1.bbclass.

To fix this issue specify LD variable while calling the kernel
menuconfig command through KERNEL_KCONFIG_COMMAND.

(From OE-Core rev: 1faf66ce0b1f8f5165277161e07e25e672370c3f)

Signed-off-by: Wonmin Jung <wonmin82@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-17 21:53:13 +00:00
Alexander Kanavin
acfdab8daa meta: drop _PYTHON_SYSCONFIGDATA_NAME hacks
(From OE-Core rev: d3a81dd0e72a3495bfc7cc969c2bb806b666023d)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-16 14:27:46 +00:00
Alexander Kanavin
3ec7f79ad2 distutils3-base.bbclass: use python3targetconfig
(From OE-Core rev: 9c8f666097802cb594a759989edcf01603a22df3)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-16 14:27:46 +00:00
Alexander Kanavin
c9f7ed88a9 python3: split python target configuration into own class
Setting _PYTHON_SYSCONFIGDATA_NAME in python3native class globally was
problematic as it was leaking into host python environment, which
was causing tracebacks depending on host distro and action
(typically anything involving importing sysconfig module).

The new class sets the variable only in specific tasks where it is needed,
and should be inherited explicitly:
- use python3native to run scripts with native python
- use python3targetconfig to run scripts with native python
if those scripts need to access target config data (such
as correct installation directories). This also adds a dependency
on target python, so should be used carefully to avoid lengthening builds.

(From OE-Core rev: 5a118d4e7985fa88f04c3611f8db813f0dafce75)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-16 14:27:46 +00:00
Bruce Ashfield
92eceb5f1d kernel: provide module.lds for out of tree builds in v5.10+
The upstream commit 596b0474d3d [kbuild: preprocess module linker
script], adds a dependency on module.lds for external module
building.

Since module.lds is generated as part of 'modules_prepare', we
must make it available with the other kernel artifacts in the
kernel shared workdir, otherwise out of tree builds fail.

This fixes errors like:

    | make[4]: *** No rule to make target 'scripts/module.lds', needed by
        'build/tmp/work/qemuarm64-poky-linux/cryptodev-module/1.11-r0/git/cryptodev.ko'.
        Stop.
    | make[4]: *** Waiting for unfinished jobs....

We also ensure that kernel-devsrc has a copy to support on
target module builds that are often prepared with 'make scripts
prepare'. Those targets won't regenerate it, so the build fails.
If 'make modules_prepare' is used, the file will be regenerated
and overwrite our copy (as expected).

(From OE-Core rev: 0fc66a0b64953aae38d0124b57615fffaec8de52)

Signed-off-by: Pan, Kris <kris.pan@intel.com>
Signed-off-by: Lili Li <lili.li@intel.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-16 14:27:46 +00:00
Konrad Weihmann
e8fd6bb2ec testimage: print results for interrupted runs
When a run is ended by overall timeout, print the already executed
testcases, to provide some hints which testcase might made the
test suite reach global timeout.
Nonetheless make the testrun exit with an error

(From OE-Core rev: 2bcc643195a3b3c66d698fac8b7af037c08545ac)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-11 10:08:12 +00:00
Joseph Reynolds
834ef343b3 add new extrausers command passwd-expire
This enhances extrausers with a new passwd-expire command that causes
a local user's password to be expired as if the `passwd --expire`
command was run, so the password needs to be changed on initial login.

Example: EXTRA_USERS_PARAMS += " useradd ... USER; passwd-expire USER;"

Tested: on useradd accounts
When configured with Linux-PAM, console login prompts for and can
successfully change the password.  OpenSSH server works.  Dropbear
SSH server notes the password must be changed but does not offer a
password change dialog and rejects the login request.

(From OE-Core rev: 1bdcfa4b0d378947a6759fb91872a4edc9a42622)

Signed-off-by: Joseph Reynolds <joseph-reynolds@charter.net>
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-11 10:08:12 +00:00
Nathan Rossi
ce447d70df cml1.bbclass: Handle ncurses-native being available via pkg-config
The linux kernel will by default use pkg-config to get ncurses(w) paths,
falling back to absolute path checks otherwise. If the build host does
not have ncurses installed this will fail as pkg-config will not search
the native sysroot for ncurses.

To more all kernel/kconfig sources, inject the equivalent native
pkg-config variables similar to what is done by the pkg-config-native
script. This only affects the menuconfig python task itself and the
oe_terminal call inside it.

(From OE-Core rev: abb95c421bb67d452691819e3f63dabd02e2ba37)

Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-08 14:03:20 +00:00
Andrej Valek
fad3498a79 autotools: CONFIG_SHELL defaults
Do not hard-code default shell to /bin/bash even if CONFIG_SHELL is already
set to other shell, but keep /bin/bash as a default. This will fix a shadow
issue, where CONFIG_SHELL is exported to /bin/sh, but /bin/bash is used
even if it's not installed.

(From OE-Core rev: 019d9128af813cb87b702ae10aa630c79fc24c00)

Signed-off-by: Andrej Kozemcak <andrej.kozemcak@siemens.com>
Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-08 14:03:20 +00:00
Gratian Crisan
da85f7389e kernel-module-split.bbclass: identify kernel modconf files as configuration files
Currently the modconf fragments representing the configuration for
kernel modules are written out to appropriate .conf files and added to
the FILES variable. However they are not identified as 'configuration
files' and installing a new version of a kernel module results in a
conflict and a failed installed because the respective .conf file is
already in place from a previous install.

Add the generated .conf files to the CONFFILES variable denoting their
true nature.

(From OE-Core rev: 1a70a92d1f1006be115429a4262259c9084f484d)

Signed-off-by: Gratian Crisan <gratian.crisan@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-08 14:03:20 +00:00
Andrey Zhizhikin
8978115d3b insane: add GitLab /archive/ tests
Archives produced by GitLab should be avoided in the same way as those
produced by GitHub.

Extend SRC_URI check to include GitLab and inform user that recipe
should be converted to use git protocol.

Link: https://www.mail-archive.com/openembedded-devel@lists.openembedded.org/msg73109.html
Link: https://lists.openembedded.org/g/openembedded-core/message/144035
Cc: Martin Jansa <martin.jansa@gmail.com>
(From OE-Core rev: d76b33c6a8489378a1f5500554367127199ae19d)

Signed-off-by: Andrey Zhizhikin <andrey.zhizhikin@leica-geosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-02 11:35:10 +00:00
Alexander Kanavin
354f96bf78 python: update 3.8.5 -> 3.9.0
Drop 0001-Do-not-hardcode-lib-as-location-for-site-packages-an.patch,
0001-configure.ac-fix-LIBPL.patch and 0001-python3-Do-not-hardcode-lib-for-distutils.patch
as they are all replaced by the new --platlibdir option to ./configure

Rename 0001-Lib-sysconfig.py-fix-another-place-where-lib-is-hard.patch to
0001-Lib-sysconfig.py-use-libdir-values-from-configuratio.patch
and describe the changes better.

License-Update: documentation now dual license under PSF & BSD
(not relevant for the recipe.
(From OE-Core rev: 7347556b18b45c5f9afc2cade565a75c95876914)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-30 13:22:49 +00:00
Khem Raj
aea9a37ae3 qemuboot.bbclass: Fix a typo
(From OE-Core rev: 2b5fb66344432390aa0cc199ad3f9ec2a4da26bb)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-30 13:22:48 +00:00
Richard Purdie
14335a18dd Revert "classes/buildhistory: also save recipe info for native recipes"
This reverts commit d123606c4bef85c2436b40f51e47b602b7600c0b.

This change contains races as it will start poking into do_package task
directories from do_populate_sysroot. If we want to do this for native
recipes, we need to add guards around the package code and only make
this happen for native in populate_sysroot, not target in
populate_sysroot too. Backtrace from an example problem below:

ERROR: openssl-1.1.1g-r0 do_populate_sysroot: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:buildhistory_emit_pkghistory(d)
     0003:
File: '/home/ross/Yocto/poky/meta/classes/buildhistory.bbclass', lineno: 319, function: buildhistory_emit_pkghistory
     0315:
     0316:        write_pkghistory(pkginfo, d)
     0317:
     0318:    # Create files-in-<package-name>.txt files containing a list of files of each recipe's package
 *** 0319:    bb.build.exec_func("buildhistory_list_pkg_files", d)
     0320:}
     0321:
     0322:python buildhistory_emit_outputsigs() {
     0323:    if not "task" in (d.getVar('BUILDHISTORY_FEATURES') or "").split():
File: '/home/ross/Yocto/poky/bitbake/lib/bb/build.py', lineno: 256, function: exec_func
     0252:    with bb.utils.fileslocked(lockfiles):
     0253:        if ispython:
     0254:            exec_func_python(func, d, runfile, cwd=adir)
     0255:        else:
 *** 0256:            exec_func_shell(func, d, runfile, cwd=adir)
     0257:
     0258:    try:
     0259:        curcwd = os.getcwd()
     0260:    except:
File: '/home/ross/Yocto/poky/bitbake/lib/bb/build.py', lineno: 503, function: exec_func_shell
     0499:    with open(fifopath, 'r+b', buffering=0) as fifo:
     0500:        try:
     0501:            bb.debug(2, "Executing shell function %s" % func)
     0502:            with open(os.devnull, 'r+') as stdin, logfile:
 *** 0503:                bb.process.run(cmd, shell=False, stdin=stdin, log=logfile, extrafiles=[(fifo,readfifo)])
     0504:        except bb.process.ExecutionError as exe:
     0505:            # Find the backtrace that the shell trap generated
     0506:            backtrace_marker_regex = re.compile(r"WARNING: Backtrace \(BB generated script\)")
     0507:            stdout_lines = (exe.stdout or "").split("\n")
File: '/home/ross/Yocto/poky/bitbake/lib/bb/process.py', lineno: 184, function: run
     0180:        if not stderr is None:
     0181:            stderr = stderr.decode("utf-8")
     0182:
     0183:    if pipe.returncode != 0:
 *** 0184:        raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
     0185:    return stdout, stderr
Exception: bb.process.ExecutionError: Execution of '/yocto/ross/build/tmp/work/neoversen1-poky-linux/openssl/1.1.1g-r0/temp/run.buildhistory_list_pkg_files.4158804' failed with exit code 2:
/yocto/ross/build/tmp/work/neoversen1-poky-linux/openssl/1.1.1g-r0/temp/run.buildhistory_list_pkg_files.4158804: 183: cd: can't cd to /yocto/ross/build/tmp/work/neoversen1-poky-linux/openssl/1.1.1g-r0/packages-split/openssl-engines

(From OE-Core rev: 59aac6e134289d657d80bfb1d6f25b388d539818)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-26 22:08:36 +00:00
Paul Eggleton
1932b04a1e classes/buildhistory: also save recipe info for native recipes
If we want to also collect SRC_URI for native recipes we need to ensure
that the code that writes out all of the recipe info is called - there
isn't a do_packagedata for native recipes so we need to piggyback on
do_populate_sysroot instead.

(From OE-Core rev: d123606c4bef85c2436b40f51e47b602b7600c0b)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-20 11:11:46 +01:00
Paul Eggleton
71191e8e7a classes/buildhistory: record SRC_URI
It can be useful to record SRC_URI into buildhistory for the purposes of
tracking exactly which sources got built (we already have SRCREV) as
well as getting an indication when changes to the SRC_URI relate to
changes in the output.

(From OE-Core rev: 70714795aafc98a6df1df7f944867093eb8cafc7)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-20 11:11:46 +01:00
Martin Jansa
52276e385d siteinfo: Recognize bigendian sh3be and sh4be
* seems to be broken for many years, does someone still use sh3/sh4?

  scripts/tune/log.fake-sh3.sh3eb:
  Parsing recipes...
  ERROR: /OE/build/oe-core/openembedded-core/meta/recipes-devtools/vala/vala_0.48.9.bb: Unable to determine endianness for architecture 'sh3eb'
  ERROR: /OE/build/oe-core/openembedded-core/meta/recipes-devtools/vala/vala_0.48.9.bb: Please add your architecture to siteinfo.bbclass

  scripts/tune/log.fake-sh4.sh4aeb
  scripts/tune/log.fake-sh4.sh4eb:
  Parsing recipes...
  ERROR: /OE/build/oe-core/openembedded-core/meta/recipes-support/vim/vim-tiny_8.2.bb: Unable to determine endianness for architecture 'sh4eb'
  ERROR: /OE/build/oe-core/openembedded-core/meta/recipes-support/vim/vim-tiny_8.2.bb: Please add your architecture to siteinfo.bbclass

(From OE-Core rev: 1f26495884b8f567aecadc5936651846dfeed3f5)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-17 23:22:19 +01:00
Martin Jansa
acd3354e93 siteinfo: Recognize 32bit PPC LE
* since this commit, all powerpc little-endians (both 32bit and 64bit) use "le" suffix:
  commit b6ac40f1cbabb20896bf113568f7735a462ed1a6
  Author: Khem Raj <raj.khem@gmail.com>
  Date:   Sun Dec 29 10:44:03 2019 -0800

    powerpc, powerpc64: Append little-endianness to tune arch

* 64bit variants were already fixed in:
  commit e62cdb9b88b575b5cfcdd65ca558edc237c43b2a
  Author: Khem Raj <raj.khem@gmail.com>
  Date:   Sun Dec 29 10:44:02 2019 -0800

    siteinfo: Recognize 64bit PPC LE

* but 32bit are still failing:
  scripts/tune/log.fake-power5.powerpcle
  scripts/tune/log.fake-power5.powerpcle-nf
  scripts/tune/log.fake-power6.powerpcle
  scripts/tune/log.fake-power6.powerpcle-nf
  scripts/tune/log.fake-power7.powerpcle
  scripts/tune/log.fake-power7.powerpcle-nf
  scripts/tune/log.fake-power9.powerpcle
  scripts/tune/log.fake-power9.powerpcle-nf
  scripts/tune/log.fake-power9.ppcp9le
  scripts/tune/log.fake-ppc476.powerpcle
  scripts/tune/log.fake-ppc476.powerpcle-nf
  scripts/tune/log.fake-ppc603e.powerpcle
  scripts/tune/log.fake-ppc603e.powerpcle-nf
  scripts/tune/log.fake-ppc7400.powerpcle
  scripts/tune/log.fake-ppc7400.powerpcle-nf
  scripts/tune/log.fake-ppce300c2.powerpcle
  scripts/tune/log.fake-ppce300c2.powerpcle-nf
  scripts/tune/log.fake-ppce300c3.powerpcle
  scripts/tune/log.fake-ppce300c3.powerpcle-nf
  scripts/tune/log.fake-ppce500.powerpcle
  scripts/tune/log.fake-ppce500.powerpcle-nf
  scripts/tune/log.fake-ppce500mc.powerpcle
  scripts/tune/log.fake-ppce500mc.powerpcle-nf
  scripts/tune/log.fake-ppce500v2.powerpcle
  scripts/tune/log.fake-ppce500v2.powerpcle-nf
  scripts/tune/log.fake-ppce5500.powerpcle
  scripts/tune/log.fake-ppce5500.powerpcle-nf
  scripts/tune/log.fake-ppce6500.powerpcle
  scripts/tune/log.fake-ppce6500.powerpcle-nf

  Parsing recipes...
  ERROR: /OE/build/oe-core/openembedded-core/meta/recipes-support/debianutils/debianutils_4.11.1.bb: Unable to determine endianness for architecture 'powerpcle'
  ERROR: /OE/build/oe-core/openembedded-core/meta/recipes-support/debianutils/debianutils_4.11.1.bb: Please add your architecture to siteinfo.bbclass
  ...

(From OE-Core rev: a31436f38da27883b42810adf3066ea498ca4a5d)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-17 23:22:19 +01:00
Ross Burton
b6e1e3150e waf: add ${B} to do_configure[cleandirs]
As waf is always out-of-tree, we can delete ${B} before every build.

(From OE-Core rev: 92bf649f16926f8e3b2f7b8aab9cd7f31bbc5082)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-17 23:22:19 +01:00
Ross Burton
85b6301c61 waf: don't assume the waf intepretter is good
Waf typically uses `python` as the intepretter but inside a task this
does not exist.  Typically this is solved by patching waf (see the
glmark2 recipe) but not all versionf of Waf support Python 3 so we can't
assume a specific interpretter.

Instead, create a new variable WAF_PYTHON for the correct interpretter,
and default this to `python3`.  If the user has a recipe that needs
Python 2 then this can be changed in the recipe.

(From OE-Core rev: 802e80d35e6374b9b80f89068d00b84fe2d04ca1)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-17 23:22:19 +01:00
Khem Raj
b6d0627f25 autotools.bbclass: Order CONFIG_SHELL before CACHED_CONFIGUREVARS
This helps in overriding CACHED_CONFIGUREVARS with wrappers to run
configure under e.g. for static analysers like scan-build from clang,
while it should not change the functionality in normal case.

Since CONFIG_SHELL was introduced, it silently broke this use case and
failed running static analyser on autotool based recipes

(From OE-Core rev: 14c3454db0108ff78b73eecfae179a69241d9f5c)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-17 12:34:29 +01:00
zangrc
400359f1d2 classes: Fixed the problem of undefined variables when compiling meta-toolchain.
The product name when compiling meta-toolchain is:
poky-glibc-x86_64-${IMAGE_BASENAME}-cortexa57-qemuarm64-toolchain-3.1+snapshot.host.manifest
poky-glibc-x86_64-${IMAGE_BASENAME}-cortexa57-qemuarm64-toolchain-3.1+snapshot.target.manifest
poky-glibc-x86_64-${IMAGE_BASENAME}-cortexa57-qemuarm64-toolchain-3.1+snapshot.testdata.json
poky-glibc-x86_64--cortexa57-qemuarm64-toolchain-3.1+snapshot.sh

inherit image-artifact-names to solve this problem.

(From OE-Core rev: 1a566f1685258bdd6da50c8d2183ff21bed41bb3)

Signed-off-by: Zang Ruochen <zangrc.fnst@cn.fujitsu.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-17 12:34:29 +01:00
Douglas Royds
7e38f67127 externalsrc: No single-task lock if S != B
Allow different recipes to build things from the one external source tree in
parallel, but only if the build is happening outside the source tree.

(From OE-Core rev: 8c6b12683c3160a5b7c62d1fe00a9f848e062df0)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-17 12:34:29 +01:00
Michael Thalmeier
acc22a2753 IMAGE_LOCALES_ARCHIVE: add option to prevent locale archive creation
Under some circumstances it is not desirable to create a combined locale
archive (/usr/lib/locale/locale-archive).
The new variable IMAGE_LOCALES_ARCHIVE defaults to '1', so the default
behaviour is not changed.

(From OE-Core rev: 8d78b819c2ec33fce3a34254fa90864ee5fa7617)

Signed-off-by: Michael Thalmeier <michael.thalmeier@hale.at>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-13 09:42:08 +01:00
Naoki Hayama
a1f503df6e uninative: Fix typo in error message
Fix typo in an error message.
s/verson/version/

(From OE-Core rev: bc96db2e0b5b8a9cc2c909ea70df290e03a50b94)

Signed-off-by: Naoki Hayama <naoki.hayama@lineo.co.jp>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-13 09:42:08 +01:00
Joshua Watt
633652284b classes/waf: Add build and install arguments
Adds variables that can be used to allow a recipe to pass extra
arguments to `waf build` and `waf install`. In most cases, you want to
pass the same arguments to `build` and `install` (since install is a
superset of `build`), so by default setting EXTRA_OEWAF_BUILD also
affects `waf install`, but this can be overridded.

(From OE-Core rev: 493e17a2f5cbbbe3b1e435dadb281b007bca2cbf)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-13 09:42:08 +01:00
Martin Jansa
f5b4cd065f base.bbclass: warn when there is trailing slash in S or B variables
* to make sure we won't hit such corner cases in future add a warning
  to prevent trailing slashes and duplicated slashes even when they
  in most cases don't cause harm

* only a few cases were found in layers included in my world builds:
  oe-core: 1
  meta-oe: 7
  meta-python2: 1
  meta-qt5: 1
  meta-aws: 1

  will send patches for these once this warning is approved for oe-core

(From OE-Core rev: 8a4c473c07cba159cf88ed775b9f073c6adf31d4)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-10 13:37:56 +01:00
Martin Jansa
a5ab8d428f base.bbclass: use os.path.normpath instead of just comparing WORKDIR and S as strings
* cannot use os.path.samefile, because S/B might not exist at this time yet

* there is issue with PSEUDO_IGNORE_PATHS when some recipe sets e.g.
  S = "${WORKDIR}/"
  whole WORKDIR gets added to PSEUDO_IGNORE_PATHS and then the build
  can fail with various strange errors, in my case do_package was
  failing when do_package calls:
  fix_perms(.../1.0-r0/package/etc, 755, 0, 0, /etc)
  and fails with "[Errno 1] Operation not permitted:"

(From OE-Core rev: 50b11a61ab29acb8ec990668353e0b7305114628)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-10 13:37:56 +01:00
Richard Purdie
49bc76ec33 abi_version,sanity: Tell users TMPDIR must be clean after pseudo changes
After the recent path ignore changes, there are invalid entries in exiting
pseudo databases which will trip up users with the new abort() (proving
how common the pseudo potential corrpution is!). Inform them a clean TMPDIR
is needed.

(From OE-Core rev: 016ee90e210c9b15b80e8370d83f41a14867a413)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-09 12:01:24 +01:00
Ross Burton
4a35bcc9d1 devtool: fix modify with patches in override directories
If a recipe applies patches which are in machine-specific override
directories, devtool will fail to fetch the patches that don't match the
default configuration.  For example where there are patches at
qemux86/x86.patch and qemuarm/arm.patch:

SRC_URI = "file://source"
SRC_URI_append_qemuarm = " file://arm.patch"
SRC_URI_append_qemux86 = " file://x86.patch"

The patch apply phase sets OVERRIDES but does not set FILESOVERRIDES, so
it cannot find the patch files as the search path isn't correct.  Fix
this by setting FILESOVERRIDES too.

Also when iterating through the overrides we need to be sure that other
overrides that are used are not enabled, so extend no_overrides instead of
simply appending the current override.

Fixes most but not all of [ YOCTO #14060 ].

(From OE-Core rev: a372cdf8e175423c47faeecc98ad076ee26bbec8)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-08 08:08:32 +01:00
Richard Purdie
c194e5fac6 wic: Handle new PSEUDO_IGNORE_PATHS variable
Adjust wic to correctly handle the new PSEUDO_IGNORE_PATH variable and avoid
inode corruption issues.

(From OE-Core rev: 13500f5234361385c365c7c35e83f99435500481)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-07 11:18:38 +01:00
Richard Purdie
a2db4fa127 base/bitbake.conf: Enable pseudo path filtering
This is a pretty big change to the way pseudo operates when used in OpenEmbedded.
Normally, pseudo monitors and logs (adds to its database) any file created or
modified whilst in a fakeroot environment. There are large numbers of files
we simply don't care about the permissions of whilst in that fakeroot context,
for example ${S}, ${B}, ${T}, ${SSTATE_DIR}, the central sstate control
directories,

This change uses new functionality in pseudo to ignore these directory trees,
resulting in a cleaner database with less chance of "stray" mismatches if files
are modified outside pseudo context. It also should reduce some overhead from
pseudo as the interprocess round trip to the server is avoided.

There is a possible complication where some existing recipe may break, for
example, we found a recipe which was writing to "${B}/install" for
"make install" in do_install and since we listed ${B} as not to be tracked,
there were errors trying to chown root for files in this location.

This patch fixes a few corner cases in OE-Core when used with this new
ignore list:

* The archiver directory matched a "${WORKDIR}/deploy*" pattern so was renamed
  to something else since that directory does need its root permissions
* The ${S} and ${B} ignoring is conditional on them being different to ${WORKDIR}
* package_write_* task output (the debs/rpms/ipks) are now owned by the build
  user so we don't want the file ownership information in the hashequiv outhash
  calculation even if they are built under pseudo.
* The fontcache postinstall intercept is run under qemu outside of pseudo context
  so delete files it may delete up front where pseudo can see this.
* SSTATE_DIR is in PSEUDO_PATHS_IGNORE, which is in FAKEROOTENV which is cached
  by bitbake. We therefore need to trigger reparsing if this changes, which means
  SSTATE_DIR can be in BB_HASHBASE_WHITELIST but not BB_HASHCONFIG_WHITELIST.
  Rework the variables to handle this. This otherwise breaks some of our sstate
  tests in oe-selftest.
* Ignore the temp directory wic uses for rebuilding rootfs.

(From OE-Core rev: ad8f5532ffaead9a5ad13e1034fe9e5e1b7979f4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-07 11:18:38 +01:00
Joshua Watt
b534c8f338 classes/sanity: Bump minimum python version to 3.5
Bumps the minimum python version to 3.5 to match bitbake and the test
matrix

(From OE-Core rev: df13c0f2348898023fb7ee1b229e9b5ccc893609)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-07 10:18:55 +01:00
Usama Arif
e65267d376 kernel-fitimage: generate openssl RSA keys for signing fitimage
The keys are only generated if they dont exist. The key
generation can be turned off by setting FIT_GENERATE_KEYS to "0".
The default key length for private keys is 2048 and the default
format for public key certificate is x.509.

(From OE-Core rev: 8dfaf5cd4eb5c8e352e7833ec47db1a14ea58b47)

Signed-off-by: Usama Arif <usama.arif@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-06 23:14:24 +01:00
Chen Qi
23bb87bff7 populate_sdk_ext.bbclass: add ESDK_MANIFEST_EXCLUDES
Add ESDK_MANIFEST_EXCLUDES to enable excluding items in sdk-conf-manifest.

By default, files under conf/ are all added to sdk-conf-manifest, as the
manifest file is set to 'conf/*'.

However, there are situations where some configuration files under conf/
directory are not intended to be added to sdk-conf-manifest, thus adding
ESDK_MANIFEST_EXCLUDES to enable users to do this.

This variable takes the form of glob matching.
e.g.
ESDK_MANIFEST_EXCLUDES = "conf/autogen*"
This would exclude all files under conf/ starting with 'autogen' from
sdk-conf-manifest.

(From OE-Core rev: 2d71e427b530ec4ea5524efa951b6a87f21b8b22)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30 15:01:51 +01:00
Chris Laplante
3d24ecf540 cve-check: add CVE_CHECK_REPORT_PATCHED variable to suppress reporting of patched CVEs
Default behavior is not changed. To suppress patched CVEs, set:

        CVE_CHECK_REPORT_PATCHED = ""

(From OE-Core rev: 05bd9f1f006cf94cf5324f96df29cd5862abaf45)

Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30 15:01:51 +01:00
Chris Laplante
47a35a3843 cve-check: introduce CVE_CHECK_RECIPE_FILE variable to allow changing of per-recipe check file
The addition of this variable also makes it possible to change the
output suffix of the check files, e.g. in local.conf:

CVE_CHECK_MANIFEST_append = ".txt"
CVE_CHECK_RECIPE_FILE_append = ".txt"

(From OE-Core rev: 0d40f1482c6d87785ae47c46c2305e1df46f459a)

Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30 15:01:51 +01:00
Saul Wold
406857b019 testimage: Add testimage_dump_target to kwargs
This passes the list of commands to run on the OEQemuTarget when
the TargetDumper needs to run in a test context due to a failure
on the target.

This is added here as a kwargs because the 'd' dictionary is not
available in the staticmethod getTarget in the
OERuntimeTestContextExecutor class. The OEQemuTarget is different
from the QemuTarget which already uses the list of commands from
testimage_dump_target from 'd'. The create_dir() is needed to
initialize the TargetDumper's dump_dir variable.

(From OE-Core rev: a63675fab4d9f638570912b15a07932f549cc4d1)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30 15:01:51 +01:00
Jon Mason
ceefa75ac0 Space-comma Cleanups
Multiple files have " ," instead of ", " in expressions.  This changes
them to conform to the way the rest of them are done.

Found and corrected via:
git ls-files | xargs sed --follow-symlinks -i 's/ ,d/, d/g'

(From OE-Core rev: 36c3afd2dd8bded02ea8f255e89a09ebd75c795b)

Signed-off-by: Jon Mason <jon.mason@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30 15:01:51 +01:00
Ross Burton
20d9825ed6 sstate: set mode explicitly when creating directories in sstate-cache
When creating directories in the sstate-cache, explicitly set the mode
passed to mkdir to 0775 so that the directories are group writable, as
otherwise they cannot be shared with other users.

(From OE-Core rev: 1b0624dffe1b8496533c86dfed873112c8c0a01b)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30 15:01:51 +01:00
Richard Purdie
1dd31a8309 staging: Ensure cleaned dependencies are added
Most recipe-sysroot dependencies are handled by these prefuncs. "configure"
is special with a decidated task, prepare_recipe_sysroot which runs
beforehand.

do_prepare_recipe_sysroot does not have to be run before/after
fetch/unpack/patch, they're independent tasks. If fetch/unpack/patch
have sysroot dependencies and those tasks rerun, stale items from the
sysroot could be uninstalled and since prepare_recipe_sysroot doesn't
re-run, they could be missing by the time configure runs.

Fix this by adding the prefunc for do_configure which would ensure
the sysroot is in a good state.

This issue can be reproduced by modifying do_unpack and watching
binutils-native and vulkan-headers failing due to missing makeinfo
or ninja.

(From OE-Core rev: 5de83bbd7c55a63cbee88a3fb983c0889036a248)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30 15:01:51 +01:00
Peter A. Bigot
fd903cda43 go-mod.bbclass: use append to add modcacherw
This class provided default build flags but includes the base go class
which also provides default flags.  Use a different mechanism to
extend the default flags without discarding `-trimpath` from the base
class.

(From OE-Core rev: a9d0a79e8d320174737263ce099a259d85a92ceb)

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-28 14:15:48 +01:00
Rasmus Villemoes
cbf566ec43 kernel.bbclass: ensure symlink_kernsrc task gets run even with externalsrc
Commit c5dfc2586b41 (kernel.bbclass: run do_symlink_kernsrc before
do_patch) fixed a race between do_symlink_kernsrc and
do_populate_lic. However, I missed the fact that when
externalsrc.bbclass is in use, the do_patch task doesn't exist,
meaning that do_symlink_kernsrc now doesn't get run at all, breaking
the build.

We need this task to run regardless of whether do_patch exists or not,
so reinstate the configure->symlink_kernsrc dependency explicitly.

Fixes: c5dfc2586b41 (kernel.bbclass: run do_symlink_kernsrc before
do_patch)
Reported-by: Chanho Park <parkch98@gmail.com>

(From OE-Core rev: a2b50b74d609ce079ab36b82d4c7c3539fb56485)

Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-28 14:15:48 +01:00
Norman Stetter
0679d0b859 sstate.bbclass: Check file ownership before doing 'touch -a'
In contrast to 'touching' a file without parameters 'touch -a' can only be
performed by the file owner. In case of a shared sstate-cache served
locally (e.g. over NFS),  where the user running bitbake is not the
owner of sstate-cache files, even if he has write access on group level,
the sstate_unpack_package task will fail.
Checking if the file is owned by the user before attempting to run
'touch -a' on it solves this.

(From OE-Core rev: f1554e8bd40c640fd58daac18ecf7b458e754109)

Signed-off-by: Norman Stetter <norman.stetter@garz-fricke.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-28 14:15:48 +01:00
Bruce Ashfield
3250e3311f kernel-yocto: add KBUILD_DEFCONFIG search location to failure message
It was pointed out that since the check for the existence of
KBUILD_DEFCONFIG knows where it is looking, that should be part of
the error/log message.

We only look in one location for the defconfigs, so add where we
checked to the message.

(From OE-Core rev: 1d4d80be41d273abd1fb6416976603d924457973)

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-25 10:54:30 +01:00
Khem Raj
b6b985a223 qemuboot: Add QB_RNG variable
RNG passthru has been enabled on all qemu machines but its being added
to each one of them, with this patch its turned into QB variables which
defaults to host passthru, yet it can be overridden if needed via
machine or config metadata if needed.

(From OE-Core rev: 26dd24506ef36088e17f999ce5489dc4b72194e8)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-24 22:32:49 +01:00
Konrad Weihmann
cf9fd0e770 testexport: rename create_tarball method
as otherwise when using it in combination with archiver.bbclass we would
have 2 methods of the same name but with different signatures
leading to various hard to understand exceptions

(From OE-Core rev: ce0c1cb2fc67881d2edcbe269ea934917ccfb48a)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-21 23:54:39 +01:00
Jens Rehsack
0a54b771f1 image-artifact-names: make variables overridable
Since the newly introduced image-artifact-names class is inherited at later
processing stage, individual IMAGE_NAME settings are overridden (hopefully
by accident instead on purpose).

Allow derived distributions define their own name schema by setting defaults
iwth question mark.

(From OE-Core rev: d0f18376dca5649aa80aff2536378763740ecda9)

Signed-off-by: Jens Rehsack <sno@netbsd.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-20 12:48:11 +01:00
Martin Jansa
a00ca02fbd kernel.bbclass: use camelCase notation for bash variables in do_deploy
* to match other variables there like deployDir imageType

[YOCTO #12937]

(From OE-Core rev: 13c5d1b686255d385d200cbd06b7000080809fc7)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-17 09:47:45 +01:00
Martin Jansa
d1f5f2391e kernel.bbclass: eliminate (initramfs_)symlink_name variables
* they are used only once, we can use the value directly
* notice that .bin extension isn't part of the variable values

[YOCTO #12937]

(From OE-Core rev: 22874b43de45e86418f683eb9d4394d7a8a08033)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-17 09:47:45 +01:00