Commit Graph

5820 Commits

Author SHA1 Message Date
Ross Burton
7b10c902cb classes: sanity-check LIC_FILES_CHKSUM
We assume that LIC_FILES_CHKSUM is a file: URI but don't actually verify this,
which can lead to problems if you have a URI that resolves to a path of / as
Bitbake will then dutifully checksum / recursively.

[ YOCTO #12883 ]

(From OE-Core rev: e2b8a3d5a10868f9c0dec8d7b9f5f89fdd100fc8)

(From OE-Core rev: a5db618986746bf9082c4b3cbdac4e523328432e)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-29 15:23:50 +01:00
Joshua Watt
50072546a5 classes/sanity: Clean up getstatusoutput usage
Replace usage of oe.utils.getstatusoutput() with direct subprocess
calls.

(From OE-Core rev: 140ecb4af80c44680278f98153353f2900e7fa98)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-06 16:24:02 +01:00
Martin Jansa
86aaecf948 gio-module-cache.bbclass: disable update_gio_module_cache postinst script for nativesdk
* it fails to execute native binary inside the qemu usermode for target
  arch as shown e.g. for qemuarm and raspberrypi3 build on x86_64
  builder:

  qemuarm-webos-linux-gnueabi/webos-ndk-basic/1.0.0-1-r3/temp/log.do_populate_sdk:
    NOTE: > Executing update_gio_module_cache-nativesdk intercept ...
    WARNING: The postinstall intercept hook 'update_gio_module_cache-nativesdk' failed, details in log.do_populate_sdk
    qemuarm-webos-linux-gnueabi/webos-ndk-basic/1.0.0-1-r3/sdk/image/opt/webos-sdk-x86_64/7.0~s14/sysroots/x86_64-webossdk-linux/usr/libexec/nativesdk-gio-querymodules: Invalid ELF image for this architecture

  qemuarm-webos-linux-gnueabi/my-sdk/1.0.0-1-r3/sdk/image/opt/webos-sdk-x86_64/7.0~s14/sysroots/armv5te-webos-linux-gnueabi/usr/libexec/gio-querymodules:
    ELF 32-bit LSB  executable, ARM, EABI5 version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 3.2.0, BuildID[sha1]=5b7f0c77e8ee9587f4e02eaf1d54a1e230e539bd, stripped
  qemuarm-webos-linux-gnueabi/my-sdk/1.0.0-1-r3/sdk/image/opt/webos-sdk-x86_64/7.0~s14/sysroots/x86_64-webossdk-linux/usr/libexec/nativesdk-gio-querymodules:
    ELF 64-bit LSB  executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 3.2.0, BuildID[sha1]=eeded124aa53c7ac997dd6326e5d9b75e8d9c43d, stripped

  qemuarm-webos-linux-gnueabi/webos-ndk-basic/1.0.0-1-r3/intercept_scripts-ac629c4abfb418548877d2a412f7e552bd21e66f0b645b8875dc56ed9f0df40d/update_gio_module_cache-nativesdk
bindir=/opt/webos-sdk-x86_64/7.0~s14/sysroots/x86_64-webossdk-linux/usr/bin
base_libdir=/opt/webos-sdk-x86_64/7.0~s14/sysroots/x86_64-webossdk-linux/lib
libexecdir=/opt/webos-sdk-x86_64/7.0~s14/sysroots/x86_64-webossdk-linux/usr/libexec
libdir=/opt/webos-sdk-x86_64/7.0~s14/sysroots/x86_64-webossdk-linux/usr/lib
binprefix=nativesdk-

set -e

PSEUDO_UNLOAD=1 qemuwrapper -L $D -E
LD_LIBRARY_PATH=$D${libdir}:$D${base_libdir} \
        $D${libexecdir}/${binprefix}gio-querymodules
$D${libdir}/gio/modules/

[ ! -e $D${libdir}/gio/modules/giomodule.cache ] ||
        chown root:root $D${libdir}/gio/modules/giomodule.cache

  raspberrypi3-webos-linux-gnueabi/webos-ndk-basic/1.0.0-1-r3/temp/log.do_populate_sdk:
    NOTE: > Executing update_gio_module_cache-nativesdk intercept ...
    WARNING: The postinstall intercept hook 'update_gio_module_cache-nativesdk' failed, details in log.do_populate_sdk
    raspberrypi3-webos-linux-gnueabi/webos-ndk-basic/1.0.0-1-r3/sdk/image/opt/webos-sdk-x86_64/7.0~s14/sysroots/x86_64-webossdk-linux/usr/libexec/nativesdk-gio-querymodules: Invalid ELF image for this architecture

  raspberrypi3-webos-linux-gnueabi/my-sdk/1.0.0-1-r3/sdk/image/opt/webos-sdk-x86_64/7.0~s14/sysroots/cortexa7t2hf-neon-vfpv4-webos-linux-gnueabi/usr/libexec/gio-querymodules:
    ELF 32-bit LSB  executable, ARM, EABI5 version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 3.2.0, BuildID[sha1]=5267f1e542b014522af5ab54443d768ba6b47351, stripped
  raspberrypi3-webos-linux-gnueabi/my-sdk/1.0.0-1-r3/sdk/image/opt/webos-sdk-x86_64/7.0~s14/sysroots/x86_64-webossdk-linux/usr/libexec/nativesdk-gio-querymodules:
    ELF 64-bit LSB  executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 3.2.0, BuildID[sha1]=eeded124aa53c7ac997dd6326e5d9b75e8d9c43d, stripped

  raspberrypi3-webos-linux-gnueabi/webos-ndk-basic/1.0.0-1-r3/intercept_scripts-a4270d1427cca0a9d172dbcd8dc262957c8e081c657e1123cc9ad551d65f22ea/update_gio_module_cache-nativesdk
bindir=/opt/webos-sdk-x86_64/7.0~s14/sysroots/x86_64-webossdk-linux/usr/bin
base_libdir=/opt/webos-sdk-x86_64/7.0~s14/sysroots/x86_64-webossdk-linux/lib
libexecdir=/opt/webos-sdk-x86_64/7.0~s14/sysroots/x86_64-webossdk-linux/usr/libexec
libdir=/opt/webos-sdk-x86_64/7.0~s14/sysroots/x86_64-webossdk-linux/usr/lib
binprefix=nativesdk-

set -e

PSEUDO_UNLOAD=1 qemuwrapper -L $D -E
LD_LIBRARY_PATH=$D${libdir}:$D${base_libdir} \
        $D${libexecdir}/${binprefix}gio-querymodules
$D${libdir}/gio/modules/

[ ! -e $D${libdir}/gio/modules/giomodule.cache ] ||
        chown root:root $D${libdir}/gio/modules/giomodule.cache

* it isn't needed in master, because nativesdk postinst were fixed by:
  commit d10fd6ae3fe46290c6e3a5250878966d9f12ca3f
  Author: Alexander Kanavin <alexander.kanavin@linux.intel.com>
  Date:   Mon Jun 11 16:38:20 2018 +0300
  Subject: qemuwrapper-cross: enable multilib and nativesdk variants of the script
* which depends on:
  commit d4f5b8e26acaadffac6df10f9a9d9ebfb3045f5f
  Author: Alexander Kanavin <alexander.kanavin@linux.intel.com>
  Date:   Mon Jun 11 16:38:17 2018 +0300
  Subject: gtk-immodules-cache.bbclass: convert cache creation to postinst_intercept mechanism

* backporting just these 2 isn't enough, we would need to backport
  something else as well, otherwise it fails with:
  webos-ndk-basic/1.0.0-1-r3/intercept_scripts-a4270d1427cca0a9d172dbcd8dc262957c8e081c657e1123cc9ad551d65f22ea/update_gio_module_cache-nativesdk:
  nativesdk-qemuwrapper: not found

  and at this point I would rather safely disable it for nativesdk
  in sumo instead of backporting bunch more commits to stable branch

(From OE-Core rev: 11487d960e8a10ba9f33cffaa631e941b8874fa6)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-30 00:02:53 +01:00
Ross Burton
884b883562 insane: optimise buildpath search
Instead of decoding every file we open as UTF-8 (with many errors as machine
code isn't UTF-8), convert the build path to the UTF-8 byte representation and
search for that instead.

(From OE-Core rev: ffb52d383bfe413cf31fef13663fe9937a146c76)

(From OE-Core rev: e0e366731116e62857fa3bdec9e3897aafcc8137)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-30 00:02:52 +01:00
Richard Purdie
20a5843837 staging: Always use the default sysroot for allarch recipes
Without this, recipes can't find allarch data files like autoconf-archive.

(From OE-Core rev: 8ae70703f68853a8714a4fb8fa5d959b5e21a02d)

(From OE-Core rev: 9e7aaabc456e186274912edcf6b7f5bb2be1333d)

(From OE-Core rev: a46011097e76813a440a5497d97e56b83541bcb5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-30 00:02:52 +01:00
Richard Purdie
f55516fef5 staging: Improve fixup processing code
With the fixes to other parts of multilib, it was found that the fixup code's
assumptions about the recipe sysroot were incorrect. We need to use the value
calculated earlier in the function.

It turns out there is a rather neat way to do this which cleans up the code
as an added bonus.

(From OE-Core rev: 2c1978fe1a5b72167c49010fbdd39a9e2eefdef8)

(From OE-Core rev: ef7e612e54d7629518a13ed3b1c2967bf70347a2)

(From OE-Core rev: c1455f3b052fc589fad628e97e35bcc9296726e1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-30 00:02:52 +01:00
Richard Purdie
842ad94cad staging/image: Fix multilib recipe sysroot issues
Currently if you enable multilib, then build an image, the multilib
recipe sysroot is build in the wrong WORKDIR. If you then clean and
rebuild the image you see "file exists" errors.

This patch ensures the real WORKDIR is used consistently and then
cleans/rebuilds also work correctly.

(From OE-Core rev: c013ae59a158378d06ecf8eb123df0a10bf986b4)

(From OE-Core rev: 7631301b5fc27ab9dda00fcf3d4a0faf685c26dd)

(From OE-Core rev: e167e99621009f92d677ff6dac303d09e42361b5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-30 00:02:52 +01:00
Ross Burton
dbd9f6867e mirrors: add Debian snapshot mirror for 2018
Add a Debian snapshot mirror from 2018 (specifically, 10th March, the date 9.4
was released) to DEBIAN_MIRRORS.

(From OE-Core rev: f3f394913b4e4a7c601ad1158faaf8b9d493e1c7)

(From OE-Core rev: 3d969e8f2979c7b4e7dff268f68c53b782bfa5e4)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-19 10:53:52 +01:00
Andre McCurdy
9f3445c9e5 dropbear: drop run time detection of read-only rootfs
Previously, when dropbear was started via its init script, relocation
of DROPBEAR_RSAKEY_DIR to support read-only rootfs was handled at
run time from within the init script.

Update the init script to take advantage of the read-only rootfs
config setup by read_only_rootfs_hook() and therefore be consistent
with startup under systemd (where relocation of DROPBEAR_RSAKEY_DIR
is handled by the read_only_rootfs_hook() at build time).

(From OE-Core rev: 4990f87b2f6a8b30c8d1c767636e7f5527f595ba)

(From OE-Core rev: b1f3fac327da43ccc079a03d3554f1365b298608)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02 11:41:25 +01:00
Chen Qi
2ab2828c71 testimage.bbclass: move codes into testimage_main
testimage-auto is expected to run testimage task's codes automatically.
But in fact, it's currently missing some codes, including testimage_sanity
and create_rpm_index.

This leads to the problem of unexpected runtime failure of test_dnf_makecache.
The error message is as below.

  RESULTS - dnf.DnfRepoTest.test_dnf_makecache - Testcase 1744: ERROR

This error is caused by the fact that create_rpm_index is not executed
before running the tests.

There's no reason why such codes should not be in testimage_main, so
move them into it.

(From OE-Core rev: fa7ba486ded13907f63f9300f66350ba2835a3f7)

(From OE-Core rev: d55013db85db28bb061b40976cdda7022fa1eb22)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02 11:41:25 +01:00
Chen Qi
e8f0fbad1c testimage.bbclass: also check 'auto' to create rpm index
Having 'auto' in TEST_SUITES will also run the 'dnf' test cases,
so also check it to determine whether to create rpm index or not.

This is to fix the following error when TEST_SUITES = "auto".

  RESULTS - dnf.DnfRepoTest.test_dnf_makecache - Testcase 1744: ERROR

(From OE-Core rev: c1f4177848d25a9121f2a85da655ee414cd424b1)

(From OE-Core rev: 43f62481f40fdd1399e468d7fbeb93793e19f9a9)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02 11:41:25 +01:00
Chen Qi
1066af1468 testimage.bbclass: fix behavior of empty TEST_SUITES
The current behaviour of TEST_SUITES is very confusing.

setting: TEST_SUITES = ""
result:  Execute all test cases.

setting: TEST_SUITES = "some_case_not_exist"
result:  Error out with 'Empty test suite' message.

The expected behaviour of TEST_SUITES should be:
1. when 'auto' is in it, execute as many test cases as possible
2. when no valid test case is specified, error out and give user message

The first one is implemented by a previous patch.
The second one is fixed in this patch.

Also add debug message to show test cases to be executed. This is
for easier debugging.

(From OE-Core rev: 909568821fbad8a6a7034b10a2dc349a210fdfc6)

(From OE-Core rev: 57f08e7ace4cc9486f86aafd1ade6ab5ad6a94d7)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02 11:41:25 +01:00
Ming Liu
d85fcec563 image_types_wic: add do_image_wic before do_image_complete
We have some tasks depending on image's do_image_complete task, and we
are also using WKS files to generate partitioned images, but now there
is lacking a inter dependency between do_image_wic and
do_image_complete, so we have to depend on both of them.

Fixed by adding the dependency.

(From OE-Core rev: e3a25f06f2cde701415f4130a43c9b3895d42f10)

(From OE-Core rev: a2f8f3d3ed92898c71c68dbfe27523e77e604af9)

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-15 17:56:57 +01:00
Richard Purdie
6c5e0625eb toolchain-scripts/meta-ide-support: Handle dash shells correctly
Where /bin/sh is dash, the recent toolchain scripts change fails as the $(pwd)
usage in oe-init-build-env doesn't function correctly. Fix this by saving
and restoring the cwd and calling the script within its own directory.

This fixes meta-ide-support on dash based systems.

(From OE-Core rev: dceca6d34071b4cbef9e28bbf19dc12f5d925525)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-04 13:35:02 +01:00
Chin Huat Ang
d7d6ef7df7 toolchain-scripts: preserve host path in environment setup script
The environment setup script generated in the build directory sets the PATH
variable by expanding ${PATH} which would have host paths filtered. Sourcing
this script to run runqemu will not work as it complains host stty (/bin/stty)
cannot be found.

To resolve this, the script no longer expands ${PATH} during generation time,
instead it will now source oe-init-build-env to initialize the build
environment so that all host paths will be preserved. Also be sure to prepend
STAGING_BINDIR_TOOLCHAIN to the PATH variable so that the toolchain from the
build directory can be found.

[YOCTO #12695]

(From OE-Core rev: a64a144096c0637387244b89ed22f4b5352b2522)

Signed-off-by: Chin Huat Ang <chin.huat.ang@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-04 13:35:02 +01:00
Ola x Nilsson
df08ba32a6 package.bbclass: Make staticlib problems non-fatal
Allow debugsource listing using dwarfsourcefiles to fail for static
libraries when the archive content is not as expected.

(From OE-Core rev: e2235b7567a9aba474cda4cdc20cc9bfffc63711)

Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-23 17:26:05 +01:00
Joshua Watt
d331af97a0 Revert "icecc.bbclass: Improve error reporting"
This reverts commit b28114cf1e.

The "-E" option for flock is not ubiquitously supported, so don't use
it.

(From OE-Core rev: 802a2877a78a44c17f3e142f7d12017a08d09dad)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-23 17:26:04 +01:00
Richard Purdie
a67ad9544a package: Fix PACKAGELOCK handling
PACKAGELOCK is there to protect readers of PKGDATA_DIR from writes and files
changing whilst they're being read. With various changes to the codebase,
the lock code has become confused as the files are now written by the sstate
code in do_packagedata, not in do_package directly any longer.

This change cleans up the code so read sites take the shared lock (anything in
do_package), write sites take the full lock (do_packagedata sstate).

The lock from do_package sstate is no longer needed since it doesn't write
outside WORKDIR.

(From OE-Core rev: d46cadbbb42aa71f9436d640891d6ccc8f8e3618)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-23 17:26:04 +01:00
Richard Purdie
2d93f1c351 package.bbclass: Only try and process static lib debug symbols on targetos != mingw*
"package.bbclass: Include dbgsrc for static libs" introduced a regression
on mingw, fix this by excluding on that TARGETOS.

(From OE-Core rev: 305dda730738a8fb3789047b06fcc45d10212aa3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-20 08:12:17 +01:00
Ola x Nilsson
47651bd6aa package.bbclass: Include dbgsrc for static libs
The debugsource must be added from the package providing the static
lib, because any package using that lib does not have access to the
source code.

Fixes [YOCTO #12558]

(From OE-Core rev: eefa5ba35663fabe1f3f8cf7f1ff126d51240613)

Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-20 08:12:17 +01:00
Richard Purdie
1b2f8e6de6 uninative: Set the dynamic linker to use at compile time
Its possible some dynamic runtime library in the dependency chain may
come from sstate and link to libraries which need the libc from
uninative. If we don't do this and binaries are run at do_install time
they would fail to find the symbols from the later libc. Examples:

cmake-native do_install:
bin/cmake: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by TOPDIR/tmp/work/x86_64-linux/cmake-native/3.10.3-r0/recipe-sysroot-native/usr/lib/libexpat.so.1)

dbus-native do_install:
tmp/work/x86_64-linux/dbus-native/1.12.2-r0/build/bus/.libs/lt-dbus-daemon: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x32/build/build/tmp/work/x86_64-linux/dbus-native/1.12.2-r0/recipe-sysroot-native/usr/lib/libexpat.so.1)

This issue is resolved when the interpreter is changed at sstate unpack
time but this isn't soon enough to avoid issues at compile/install time.

By specifing which dynamic linker/loader to use at compile time, this
race window is removed entirely.

(From OE-Core rev: 35867ee035030ab76fc9ccdb0eb1c3f80126301c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-18 18:57:06 +01:00
Richard Purdie
fa742af158 uninative: Add allow-shlib-undefined to BUILD_LDFLAGS and drop other workarounds
We have a problem when for example, a glibc 2.27 based system builds some
library like libpopt-native and puts it into sstate then it is reused
on a pre glibc-2.27 system to build something which depends on popt like
rpm-native. This results in an error like:

recipe-sysroot-native/usr/lib/libpopt.so: undefined reference to `glob@GLIBC_2.27'

In the past we've had this problem with new symbols like getrandom and
getentropy, here its with a more complex symbol where there is an old
version and a newer version.

We've looked into various options, basically we cannot link against our
uninative libc/ld.so since we don't have the right headers or compiler
link libraries. The compiler doesn't allow you to switch in a new set
either, even if we did want to ship them. Shipping a complete compiler,
dev headers and libs also isn't an option.

On the other hand if we follow the ld man page, it does say:

"""
The reasons for allowing undefined symbol references in shared libraries
specified at link time are that:

- A shared library specified at link time may not be the same as the one
  that is available at load time, so the symbol might actually be
  resolvable at load time.
"""

which is exactly this case. By the time the binary runs, it will use
our uninative loader and libc and the symbol will be available.

Therefore we basically have a choice, we get weird intermittent bugs,
we drop uninative entirely, or we pass this option.

If we pass the option, we can drop the other workarounds too.

(From OE-Core rev: 75a62ede393bf6b4972390ef5290d50add19341a)

(From OE-Core rev: d18bf7fa8e80d6cfaf3fdbe1ab06eec84b954432)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-18 18:57:06 +01:00
Richard Purdie
606ff37690 bitbake.conf: Set and export TZ envvar to UTC
We just ran into an issue where tar failed to build on one server setup
but built everywhere else just fine.

It was running makeinfo to regenerate some docs files and makeinfo was too
old for the host it was running on. There was no dependency on makeinfo-native
as it was not meant to be regenerating the docs.

It was being regenerated as a date from a timestamp used in the docs
was different in Asian timezones than in the other timezones our builds
were being tested in.

I added an entry to https://wiki.yoctoproject.org/wiki/TipsAndTricks/
about how this was debugged.

As such, lets default to setting and exporting TZ to 'UTC' as was already
pioneered by the reproducibile builds work. This makes the builds
deterministic.

[YOCTO #12665]

(From OE-Core rev: 2a90ae7a3286724ff9e3615c4dbf56038f703810)

(From OE-Core rev: e31f31f81efe4b60938b724bece2a03c7c74a68d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-18 18:57:06 +01:00
Mark Hatle
49bfeb0a9c package.bbclass: Add '-b' option to file call in isELF
The isELF function works by running:

   result = file <pathname>
   if 'ELF' in result

By default 'file' will prepend the result with the path name of the file
that is being checked.  This usually works fine, such as:

$ file /home/foo/openembedded-core/meta/classes/package.bbclass
/home/foo/openembedded-core/meta/classes/package.bbclass: Python script, ASCII text executable, with very long lines

However, if the path includes 'ELF', ELF will end up in the result, and then
the check will return positive.

$ file /home/ELF/openembedded-core/meta/classes/package.bbclass
/home/ELF/openembedded-core/meta/classes/package.bbclass: Python script, ASCII text executable, with very long lines

This will then result in the isELF coming back true, and possibly causing the
checks that use isELF, such as the 'is it already stripped' check, to do the
incorrect thing.

Adding the '-b' option to file will result in the path being omitted in the
result:

$ file /home/ELF/openembedded-core/meta/classes/package.bbclass
Python script, ASCII text executable, with very long lines

(From OE-Core rev: 5a324e9b2cf6378f8eaa4e394f9cb36d4e2680ac)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-18 18:57:06 +01:00
Juro Bystricky
b8ce7d0800 rootfs-postcommands.bbclass: improve binary reproducibility
Conditionally support binary reproducibility of rootfs images.
If REPRODUCIBLE_TIMESTAMP_ROOTFS is specified then:

1. set /etc/timestamp to a reproducible value
2. set /etc/version to a reproducible value
3. set /etc/gconf: set mtime in all %gconf.xml to reproducible values

The reproducible value is taken from the variable REPRODUCIBLE_TIMESTAMP_ROOTFS.

[YOCTO #11176]
[YOCTO #12422]

(From OE-Core rev: 11e45082ad00b9c172e59bf6b2a76dd613773f5a)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-18 18:57:06 +01:00
Ross Burton
fb9e636538 testimage: run systemtap test on SDK images
(From OE-Core rev: f7f26bd0fe7085515bdebf23107ed8647a0c98fe)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-18 18:57:06 +01:00
Joshua Watt
59e51d2ac8 icecc.bbclass: Bump version number
Bump the version number for force remotes to use a newly generated
environment, since the old one potentially had a few bugs

(From OE-Core rev: 6c3b2ac41f3412ebce8364df637d64e34daab8a6)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-13 16:58:07 +01:00
Joshua Watt
b28114cf1e icecc.bbclass: Improve error reporting
Improve reporting when the icecream environment cannot be created by
assigning the flock call a specific error number when the lock fails so
it can be distinguished from environment creation errors.

(From OE-Core rev: 563448a7a3ca87cc07528c18f8047913e0468e7a)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-13 16:58:07 +01:00
Joshua Watt
089cd3bf55 icecc.bbclass: Add ICECC_ENV_DEBUG variable
The ICECC_ENV_DEBUG variable can be set in local.conf to pass additional
debugging options to the Icecream toolchain creation script.

(From OE-Core rev: be913284bb34ebf4a71770646044603a2a27d01b)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-13 16:58:07 +01:00
Ross Burton
8b50dfcecf populate_sdk_base: use xz -T instead of pixz
xz has native support for threaded compression now and SDK creation was the only
part of oe-core which is using pixz instead of xz.

Not only does this remove pixz-native from the SDK dependencies, but in my
limited testing xz -T0 is slightly faster and produces smaller archives than
pixz for the same input.

(From OE-Core rev: ce1cfa57d9828c0898f52e736f671ce8db534031)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-13 16:58:06 +01:00
Ross Burton
a559ffab30 image_types: use pigz to create .gz files
Since pigz is no longer a drop-in replacement for gzip (oe-core 1624b7b) the
image creation has been using gzip instead of pigz, despite still depending on
pigz-native.  Fix this by invoking pigz explicitly.

(From OE-Core rev: 214fa7fe3b162162d2fa8b31eec28bedd86fcc7d)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-13 16:58:06 +01:00
Martin Jansa
0a0d01addd license.bbclass: be a bit more strict when searching ${PN}-${LICENSE_PACKAGE_SUFFIX} in packages
* linux-firmware contains ${PN}-license package since this commit:
  commit 1ee083da0730408fffdbbf5f29abc299c0e61be9
  Author: Jackie Huang <jackie.huang@windriver.com>
  Date:   Mon Apr 13 10:17:21 2015 +0800

    linux-firmware: fix the mess of licenses
* LICENSE_CREATE_PACKAGE functionality in license.bbclass when enabled
  adds new package with suffix:
  LICENSE_PACKAGE_SUFFIX ??= "-lic"
  but then it checks if ${PN}-${LICENSE_PACKAGE_SUFFIX} is included
  in PACKAGES before adding it and when found it shows:
  WARNING: linux-firmware-1_0.0+gitAUTOINC+4c0bf113a5-r0 do_package: linux-firmware-lic package already existed in linux-firmware.
  and doesn't add the ${PN}-lic to PACKAGES and causes another warning:
  WARNING: linux-firmware-1_0.0+gitAUTOINC+4c0bf113a5-r0 do_package: QA Issue: linux-firmware: Files/directories were installed but not shipped in any package:
  /usr
  /usr/share
  /usr/share/licenses
  /usr/share/licenses/linux-firmware

  that's because it was searching ${PN}-lic in PACKAGES as a string
  so it found ${PN}-lic as a substring of ${PN}-license, add a split
  to search in an list

(From OE-Core rev: 9b9897fc034819385a9d4ce591cc79dd458f3f24)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-10 09:10:36 +01:00
Peter Kjellerstedt
91782cf429 license.bbclass: Minor simplification of get_deployed_dependencies()
Since ${SSTATE_ARCHS} now contains ${PACKAGE_EXTRA_ARCHS} there is no
longer any need to add those extra architectures to the list of
architectures handled in get_deployed_dependencies().

(From OE-Core rev: e55e6df4f1434458cdfa0e2d3610b48119e5a782)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-10 09:10:36 +01:00
Peter Kjellerstedt
4ea831fb7f sstate.bbclass: Add ${PACKAGE_EXTRA_ARCHS} to SSTATE_ARCHS
This makes sure files provided by packages that use any of the extra
architectures defined using ${PACKAGE_EXTRA_ARCHS} are cleaned from
tmp/sysroot-components when sstate_eventhandler2() executes.

Without this, changing a package from using one of the extra
architectures to another architecture would lead to files being
leftover in tmp/sysroot-components, which could later be picked up
when creating the RSS for another package rather than the files that
belonged to the updated package.

(From OE-Core rev: b959b3bd92aa2939d407800fc4167b0535fd4674)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-10 09:10:36 +01:00
Peter Kjellerstedt
144faeb10c classes: Use ${MACHINE_ARCH} instead of ${MACHINE} for stamp-extra-info task flag
Without this change, there will be two sstate index files in
tmp/sstate-control for any machine that contains a dash in the
name.

(From OE-Core rev: 29e7799bdb3773c40492e01448e0c614ed44583d)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-10 09:10:36 +01:00
Victor Kamensky
a35416cafd image: add image-combined-dbg helper class
There is IMAGE_GEN_DEBUGFS="1" variable that enables build of
additional rootfs-dbg and additional archive that contains
complimentary symbols files for a given image. But the issue
with this resulting directory and tarball that before use it
has to be combined with original image content. It is required
since all cross debugging tools like gdb, perf, and systemtap
need file system that contains both target executables/libraries
and their symbols. Those tools need to find executable/library
first and through it debuglink note find corresponding symbols
file.

image-combined-dbg when added to USER_CLASSES just copies
final resulting rootfs back into rootfs-dbg creating combined
target and debug symbols rootfs that can be used for debugging
directly.

(From OE-Core rev: 7ccb077f51c9232de70e5c6f9897fd9986e3be9d)

Signed-off-by: Victor Kamensky <kamensky@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-07 11:44:50 +01:00
Tom Hochstein
a6567a4c21 archiver: Fix archive for linked kernel source
When archiving a kernel, if S is ${WORKDIR}/git, then
${WORKDIR}/git is in fact a soft link into work-shared,
and the archive contains just the soft link. Fix by
archiving the real directory.

(From OE-Core rev: 564204dd6b73697ef4b6c17243067892876cc323)

Signed-off-by: Tom Hochstein <tom.hochstein@nxp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 14:49:07 +01:00
Tom Hochstein
c2eec907cb archiver: Fix build errors for kernels that don't use kernel-yocto.bbclass
Special handling for kernel source in work-shared was
incorrectly limited to kernel-yocto.bbclass instead
of kernel.bbclass.

(From OE-Core rev: b8c5680a67888d43540366a40bd0c501e926ac31)

Signed-off-by: Tom Hochstein <tom.hochstein@nxp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 14:49:07 +01:00
Alexander Kanavin
243fd5a74d gio-module-cache.bbclass: pass in ${libexecdir}
When it was something else than /usr/libexec (e.g. when
installing native SDK packages), things broke down.

(From OE-Core rev: d99e819a6cbde6d1116c434ddba4c5f8eca7e6d8)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 14:49:07 +01:00
Alexander Kanavin
e4b8c2bd20 sdk.py: run postinst intercepts
Previously this wasn't done, and so any packages installed from
populate_sdk would not have the postinsts fully executed
(particularly generation of various caches via running nativesdk or target
binaries with qemu wasn't working).

[YOCTO #12630]

(From OE-Core rev: a484ff072eea3f47de2c3348048201249cefa46b)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 14:49:07 +01:00
Enrico Jorns
42a513489d base.bbclass: fix do_unpack[cleandirs] varflag handling
As introduced by a56fb90dc3 ('base.bbclass
wipe ${S} before unpacking source') the base.bbclass uses a python
anonymous function to set the 'do_unpack' varflag 'cleandirs' to either
'${S}' or '${S}/patches' depending on equality of '${S}' and '${WORKDIR}'.

Not that this only differs from the way almost all other recipes set or
modify a tasks 'cleandirs' flag, it also has a significant impact on the
kernel.bbclass (and possibly further ones) and causes incorrect
behavior for rebuilds triggered by source modification, e.g. by a change
of the defconfig file for a kernel build.

The kernel.bbclass tries to extend do_unpack[cleandirs]:

| do_unpack[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} ${STAGING_KERNEL_BUILDDIR}"

As python anonymous functions are evaluated at the very end of recipe
parsing, the d.setVarFlag('do_unpack', 'cleandirs', '${S}') statement in
base.bbclass will overwrite every modification to cleandirs that is done
as shown for the kernel class above.

As a result of this, a change to a kernels 'defconfig' will lead to an
updated defconfig file in ${WORKDIR}, but as ${B} never gets cleaned and
${B}/.config still exists, it will not be copied to ${B}/.config and
thus not find its way in the build kernel.

This is a severe issue for the kernel development and build process!

This patch changes setting of the cleandirs varflag in base.bbclass to
a simple variable assignment as almost all other recipes do it. This now
again allows overwriting or appending the varflag with common methods
such as done in kernel.bbclass.

This issue affects morty, pyro, rocko and master.

(From OE-Core rev: 20901b9783220aa6e7adae4951c531919c20859b)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 14:49:07 +01:00
Paul Eggleton
5e6ffb2adb classes/externalsrc: ensure cleandirs code handles non-absolute paths
It's possible that a trailing or extra slash somewhere in the external
source path could result in the directory not being removed from
cleandirs; it's also possible that a cleandirs entry is somewhere
underneath the source tree and that tree should never have parts of it
deleted by the build system. Use oe.path.is_path_parent() (which makes
paths absolute before checking them) to find out if any path in
cleandirs is anywhere underneath the external source path, and drop it
if it is.

(From OE-Core rev: 87ec0fb470274d980f8553a85f778809c269c5d7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 14:49:07 +01:00
Paul Eggleton
eb37cdd890 classes/externalsrc: handle if cleandirs contains python expressions
Use the existing oe.recipeutils.split_var_value() function to split the
unexpanded value of the cleandirs varflag, in case it contains python
expressions - we don't want to split the expression itself as the chunks
will not expand properly individually and we can miss something that
expands to the source tree (and thus it can get deleted, the avoidance
of which is the whole point of this code).

(From OE-Core rev: 611cee69d7300a4746edd752c9557af60e7beecc)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 14:49:06 +01:00
Cuero Bugot
f5cd91aae0 uninative: add variables to the whitelist so that it does not re-triger recipe parsing
When uninative is activated (poky's default) internal datastore variables are modified (NATIVELSBSTRING and SSTATEPOSTUNPACKFUNCS) to enable uninative
support. This is happening after parsing is done at the beginning of the build. On the next bitbake call the recipe would be parsed if the two
variables above were not added to the parsing whitelist BB_HASHCONFIG_WHITELIST.

The fix is to add these two variables to the recipe parsing whitelist BB_HASHCONFIG_WHITELIST, this is done at recipe parsing time, only when
uninative.bbclass is used.

(From OE-Core rev: 75bb95ada98ef129d2fa48568f27dddb078c852c)

Signed-off-by: Cuero Bugot <cbugot@sierrawireless.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-03 23:53:20 +01:00
Jonathan Liu
4ca75e4226 kernel.bbclass: improve reproducibility
Set KBUILD_BUILD_VERSION=1 to avoid build version being incremented on
every build. It is visible in the output of "cat /proc/version" after
the hash character.

(From OE-Core rev: 7fa32ee42c3320e9e2b24ef747e43484b719f617)

Signed-off-by: Jonathan Liu <net147@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-03 23:53:19 +01:00
California Sullivan
dfd5dc1f55 default-distrovars: set default KERNEL_IMAGETYPE(S)
With the change from assuming kernels will be named "vmlinuz"
everywhere, to instead using KERNEL_IMAGETYPE, we require that
KERNEL_IMAGETYPE is set to something. Instead of setting the default in
multiple individual files, set it in default-distrovars.inc.

x86(-64) arches get bzImage as the default. Others get zImage as per
meta/conf/documentation.conf.

Also set KERNEL_IMAGETYPES, as we will eventually be switching away from
KERNEL_IMAGETYPE.

Thanks to Andre McCurdy <armccurdy@gmail.com> for the arch defaults
part.

(From OE-Core rev: a57d784211a39587538094425ee0246e9ddfbf9d)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-31 09:48:42 +01:00
Richard Purdie
080a2138db package_rpm: set _builddir to B not S
Recipes which use a shared workdir (e.g. gcc-runtine and libgcc) can
race over temporary files causing interesting build failures.

Using B instead of S avoids this problem.

[YOCTO #12605]

(From OE-Core rev: d6c13a5ff441f7076eb327c0d0b747bd7603db0f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-30 00:31:18 +01:00
Richard Purdie
6b96d9ff73 multilib: Don't extend make-mod-scripts as a multilib version doesn't make any sense
The multilib version would race against then non-ml version leading
to all kinds of odd build failures.

(From OE-Core rev: 6bb70bd3857edb8cb6cc1317f57b899a89be2653)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-30 00:31:18 +01:00
Mikko Rapeli
f6cbf2eadd mirrors.bbclass: change Debian anonscm to salsa
Debian anonscm service in Alioth is shutdown and thus
fetching sources fails.

https://wiki.debian.org/Alioth

"Alioth is broken, and there is nobody around to fix it. Don't ask the remaining people who give it life support to implement fixes and changes. It is being replaced by a cocktail of ?GitLab (see Salsa), read-only repos and keep-alive mechanisms. See below for more information."

https://wiki.debian.org/Salsa

"What is Salsa?

Salsa is the name of a collaborative development server for Debian based on the gitlab software. Salsa is supposed to provide the necessary tools for package maintainers, packaging teams and other Debian related individuals and groups for collaborative development.

What is the status of Salsa?

After various discussions about the future of Alioth, the Alioth Sprint in August 2017 gave birth to the initial setup of the the upcoming Salsa service. The productive weekend resulted in a working prototype and was launched as a beta in December 2017. It left its beta status in January 2018."

(From OE-Core rev: 08ff7b42d8b7d06ef61255185c95e900ada8769b)

Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-28 12:09:52 +01:00
Andre McCurdy
e092acd524 package.bbclass: use single quotes for path passed to file in isELF()
Apparently there are recipes in the wild which generate files with
filenames containing '$' characters - which cause errors during
packaging.

Instead of adding another special case to escape '$' characters when
constructing the command passed to oe.utils.getstatusoutput(), switch
to using single quotes to quote the path - and therefore make isELF()
consistent with the way filenames and paths are quoted by every other
caller of oe.utils.getstatusoutput() in oe-core.

(From OE-Core rev: 7877761534b0c2492da6289e9f2269d41b6ed464)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-28 12:09:52 +01:00