Commit Graph

7759 Commits

Author SHA1 Message Date
Richard Purdie
29482de968 package: Drop subshell usage for dwarfsrcfile generation.
The command for running dwarfsrcfiles is simple and does not need a subshell
for each execution. By expanding out this function to use check_output()
from subprocess and a list of arguments, the shell overhead can be dropped.

For recipes with lots of files this gives a significant saving.

(From OE-Core rev: 6334129dfbe266602fab70ce445641053a05be6c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24 11:52:27 +01:00
Richard Purdie
3fa835d9bc package_ipk|deb: Use oe.utils.multiprocess_launch
The current code had broken exception handling due to the use of a
"traceback" variable as well as an import. Use the new library code
for this instead which reduces code duplication and has fixed/improved
exception handling.

The chdir code can be dropped since any directory changes are in other
processes now so there is no need for it here and the code no longer
changes directory.

(From OE-Core rev: bcd47389f4b1fc69d2bb4da01933bfa1fdcae092)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24 11:52:27 +01:00
Richard Purdie
2245a94bb8 package: Rework PACKAGELOCK based upon sstate for do_packagedata
I think this lock dates from before we had sstate for do_packagedata.

Since WORKDIR is recipe specific and we write into WORKDIR, we no longer need
any write locks in the do_packagedata code itself, its handled by the sstate
task lock for the final copy in at the end. The final write lock can be simply
removed.

The only time we need read locking is when actually reading data from the
shared directory. We can therefore reduce the window the lock is held
significantly as well, hence improving the speed of packagedata tasks running
in parallel.

(From OE-Core rev: f7106cdf2190d9ec59132a1cb2bb431d653cd9c5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24 11:52:27 +01:00
Ruslan Bilovol
bf5b43495a relocate_sdk.py: remove hardcoded SDK path
This patch removes hardcodes added to relocate_sdk.py
during SDK build, making it flexible and reusable.
Now default SDK path is passed to the script as
parameter rather then harcoded inside it.

This allows to reuse this script for multiple
relocations, and adds possibility to relocate
SDK multiple times

(From OE-Core rev: 6671a4d980c8bef8f402780a308f6c43a25044aa)

Signed-off-by: Ruslan Bilovol <rbilovol@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24 11:52:27 +01:00
Oleksii Konoplitskyi
1d8c7fa369 package.bbclass: exclude packages as shlib providers
Some packages containing shared libraries might be registered
as shlib providers when they shouldn't (for example, the lib is for
their private use and must not generate any dependency).

EXCLUDE_FROM_SHLIBS is targeted at that, but it could be set
for entire recipe only.

This patch allows to set list of packages that are not candidates
for shlib providers.

Variable EXCLUDE_PACKAGES_FROM_SHLIBS is used, e.g.:
EXCLUDE_PACKAGES_FROM_SHLIBS = "glibc-ptest glibc-mtest"

Cc: Andrii Bordunov <aborduno@cisco.com>
(From OE-Core rev: 43be374a6deceeaf105794f034957c65bb6bec72)

Signed-off-by: Oleksii Konoplitskyi <okonopli@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24 11:52:26 +01:00
Khem Raj
b297c1aa4b recipes: Enforce ARM ISA just for arm arches <= armv5
armv7+ used thumb2 ISA and it compiles fine with thumb2
issues are only when using thumb1 ISA

(From OE-Core rev: c0ef8a91f671f30acd92e2734144f7ddf1acda53)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:42 +01:00
Olof Johansson
a24e8944e7 spdx.bbclass: Encode strings before passing to hashlib
In python3, passing a unicode object to hashlib will result in an
exception that encourages you to encode it first.

(From OE-Core rev: b06a44f1081ea422a365e80bc79b2aeb2783d23f)

Signed-off-by: Olof Johansson <olofjn@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:42 +01:00
Olof Johansson
83b3e1463a spdx.bbclass: Make use of bb.utils' sha1_file()
The same functionality already exists within bitbake, so avoid
duplicating.

(From OE-Core rev: 978f5a8f16bf5942aad73d761df2a00aeb36339d)

Signed-off-by: Olof Johansson <olofjn@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:42 +01:00
Olof Johansson
ae7d14ab44 spdx.bbclass: Fix undefined variable error
The path variable is used in an error message a few lines later, but was
never defined.

(From OE-Core rev: 863ff90b788f66241860e27e1fd3a791b00984cc)

Signed-off-by: Olof Johansson <olofjn@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:42 +01:00
Olof Johansson
a60a3c11f4 spdx.bbclass: Replace deprecated string.replace with str.replace
The string.replace function is removed in python3. Instead, the str
method "replace" should be used instead.

(From OE-Core rev: 8538aabf62d866f36764b4b136ee8575308df690)

Signed-off-by: Olof Johansson <olofjn@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:42 +01:00
Ross Burton
8df7480d1c insane: optimise buildpath search
Instead of decoding every file we open as UTF-8 (with many errors as machine
code isn't UTF-8), convert the build path to the UTF-8 byte representation and
search for that instead.

(From OE-Core rev: ffb52d383bfe413cf31fef13663fe9937a146c76)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:42 +01:00
Paul Eggleton
8274ccae0d classes/buildhistory: properly process escaped chars from pkgdata
All values written out to pkgdata are escaped (see write_if_exists() in
package.bbclass). In practice there tend not to be characters that need
escaping except in the scriptlets (pkg_preinst, pkg_postinst, pkg_prerm
and pkg_postrm) where currently we still see the escape codes in the
corresponding files within buildhistory (e.g. \n and \t) and thus also
in the output of buildhistory-diff, hindering proper diffing of changes.
To fix this, when we read values from pkgdata and write them out to
buildhistory, we need to interpret the escape codes by doing the exact
reverse of what we do in package.bbclass.

(From OE-Core rev: c258379181a438cb01728d223b3d05e0ab205941)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:41 +01:00
Paul Eggleton
2e8787593f classes/buildhistory: handle packaged files with names containing spaces
The FILELIST field of the package info file in the buildhistory
repository is a space-separated list of all of the files in the package.
If a name of a file packaged by a recipe contains a space character then
of course the result was that we didn't handle its name properly. To fix
that, use quotes around any filename containing spaces and at the other
end use these quotes to extract the proper entries.

Fixes [YOCTO #12742].

(From OE-Core rev: 801b705957dc683030d11393f43407d0b3506b6a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:41 +01:00
Richard Purdie
dd31a246aa populate_sdk_base: Use default value assignment for SDK_TITLE/SDKEXTPATH
This allows the user mode flexibility about centrally overriding these
rather than needing to do it on a per recipe basis.

(From OE-Core rev: b0fb5bbd5f62857c32b2e071bcac841f856b1f1b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:41 +01:00
Richard Purdie
da596a2ad6 testsdk: Enable multiprocess execution
This uses the new concurrenttest code to enable parallel test execution
if specified.

(From OE-Core rev: 07d19fb3adab7a8d83ba83d9a16395f70d7b7a47)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:41 +01:00
Richard Purdie
4e4958cba2 oeqa/core/threaded: Remove in favour of using concurrenttests
We have several options for parallel processing in oeqa, parallel
execution of modules, threading and mulitple processes for the runners.

After much experimentation is appears the most scalable and least
invasive approach is multiple processes using concurrenttestsuite
from testtools. This means we can drop the current threading code
which is only used by the sdk test execution.

oeqa/decorator/depends: Remove threading code

Revert "oeqa/sdk: Enable usage of OEQA thread mode"
This reverts commit adc434c063.

Revert "oeqa/core/tests: Add tests of OEQA Threaded mode"
This reverts commit a4eef558c9.

Revert "oeqa/core/decorator/oetimeout: Add support for OEQA threaded mode"
This reverts commit d3d4ba902d.

(From OE-Core rev: a98ab5e560e73b6988512fbae5cefe9e42ceed53)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:41 +01:00
Anuj Mittal
ebbd1e4a94 image-live: fix check for rootfs size with hddimg images
Currently we check if the rootfs.img size is greater than 4 GB and use
mcopy to copy it over to a hddimg, a FAT32 volume.

This results in problems when IMAGE_ROOTFS_SIZE is set to be exact 4 GB
(4294967296 bytes). mcopy uses the max offset/file size to be 4294967295
and as a result, it errors out without any warning to the user.

Change the 4 GB check to '-ge' to catch these cases. Since we allow
IMAGE_ROOTFS_SIZE to be in KBs, the maximum allowed is 4194303 KB. Also,
tweaked the error message to refer to wic too.

Fixes [YOCTO #12776]

(From OE-Core rev: 16956dacd4be2c64e5816ccb2b222b5b128838c9)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-15 16:16:16 +01:00
Ross Burton
6fbddd24f4 distutils3: pass build arguments when doing a clean
We should pass the build arguments to setup.py when doing a clean, because
sometimes the arguments are required for setup.py get started.

(From OE-Core rev: f9324af88a99eca28b160fa31aa4516fd397e44b)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-10 17:33:00 +01:00
Ross Burton
7af5e3bad3 meson: improve code style
Use elif for consistency.

(From OE-Core rev: 2ad89f12c9819326b29588a8a6c642aaae990f18)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-10 17:33:00 +01:00
Martin Jansa
d0389bf1cd kernel.bbclass: move variables for kernel artifacts names to separate bbclass
* this makes it easier to access these variables from some other bbclass
  e.g. sdcard_image-rpi.bbclass in meta-raspberry where we need to know
  how some files in deploy are named, but we cannot inherit kernel.bbclass
  as it's used in image recipe not kernel recipe
* alternatively we can move these to bitbake.conf like similar image variables are:
  meta/conf/bitbake.conf:IMAGE_BASENAME = "${PN}"
  meta/conf/bitbake.conf:IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}-${DATETIME}"
  meta/conf/bitbake.conf:IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"

(From OE-Core rev: 7d0ef0eaa1bfe97015a774c26f5791622e7e8b12)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-10 17:33:00 +01:00
Martin Jansa
c159b6249a kernel.bbclass: use the consistent naming schema for initramfs
* use INITRAMFS_BASE_NAME and INITRAMFS_SYMLINK_NAME variables, like
  other kernel artifacts are using
* use "${PKGE}-${PKGV}-${PKGR}-${MACHINE}-${DATETIME}" instead of
  "${PV}-${PR}-${MACHINE}-${DATETIME}" to be consistent with other files
* allow to modify default symlink name with INITRAMFS_SYMLINK_NAME
  instead of currently used:
  initramfs_symlink_name=${type}-initramfs-${MACHINE}

(From OE-Core rev: 935b9d5a2bd12effad65f69a631ecff96b8bb553)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-10 17:33:00 +01:00
Martin Jansa
c5b70d7155 kernel-devicetree.bbclass: Fix and simplify instalation of DTB files
* add 2 new variables:
  KERNEL_DTB_BASE_NAME
  KERNEL_DTB_SYMLINK_NAME
  instead of reusing KERNEL_IMAGE_SYMLINK_NAME and than expecting that
  default value ${MACHINE} was being used in e.g.:
  DTB_SYMLINK_NAME=`echo ${symlink_name} | sed "s/${MACHINE}/${DTB_BASE_NAME}/g"`

* install normal DTB files only once even if there is multiple entries
  in KERNEL_IMAGETYPE_FOR_MAKE and don't prefix them with the type of
  the kernel image, use the KERNEL_IMAGETYPE_FOR_MAKE as a prefix only
  when installing them bundled with kernel or initramfs image.

* deploy the files from ${D}/${KERNEL_IMAGEDEST}/ instead of kernel
  build directory, so that we don't need to call
  DTB_PATH=`get_real_dtb_path_in_kernel "${DTB}"`
  again in do_deploy

* create all links in do_deploy task, because default KERNEL_DTB_BASE_NAME
  like KERNEL_IMAGE_BASE_NAME contains PKGR and PKGR is different in
  do_install and do_deploy, because kernel.bbclass calls
  meta/classes/kernel.bbclass:do_install[prefuncs] += "package_get_auto_pr"
  meta/classes/kernel.bbclass:do_deploy[prefuncs] += "package_get_auto_pr"

* the filenames are a bit different, but with separate variable it
  should be easier for other bbclasses which use these DTB files to
  find them correctly, just use either the cannonical name
  $dtb_base_name.$dtb_ext or $dtb_base_name-${KERNEL_DTB_SYMLINK_NAME}.$dtb_ext
  because PKGR (and other PKG* variables) might be different in your
  task and kernel.do_deploy task.

* fix DTB files being deployed with incorrect filenames when
  KERNEL_IMAGE_SYMLINK_NAME isn't set to ${MACHINE}, e.g. instead of
  the default:
-rw-r--r-- 2 bitbake bitbake 1.4K Nov 20 07:41 deploy/images/raspberrypi3-64/Image-1-4.9.59+git0+e7976b2aff-r0.2-lirc-rpi-20171120043031.dtbo
lrwxrwxrwx 2 bitbake bitbake   64 Nov 20 07:41 deploy/images/raspberrypi3-64/Image-lirc-rpi.dtbo -> Image-1-4.9.59+git0+e7976b2aff-r0.2-lirc-rpi-20171120043031.dtbo
lrwxrwxrwx 2 bitbake bitbake   64 Nov 20 07:41 deploy/images/raspberrypi3-64/lirc-rpi.dtbo -> Image-1-4.9.59+git0+e7976b2aff-r0.2-lirc-rpi-20171120043031.dtbo
  I was getting:
-rw-r--r-- 2 bitbake bitbake 1348 Nov 20 10:28 deploy/images/raspberrypi3-64/Image-linux-raspberrypi-lirc-rpi.dtbo
lrwxrwxrwx 2 bitbake bitbake   37 Nov 20 10:28 deploy/images/raspberrypi3-64/Image-linux-raspberrypi-lirc-rpi-master-20171120102653.dtbo -> Image-linux-raspberrypi-lirc-rpi.dtbo
lrwxrwxrwx 2 bitbake bitbake   37 Nov 20 10:28 deploy/images/raspberrypi3-64/lirc-rpi.dtbo -> Image-linux-raspberrypi-lirc-rpi.dtbo
  and e.g. sdcard_image-rpi.bbclass from meta-raspberrypi:
  37e4e18f4a/classes/sdcard_image-rpi.bbclass
  was failing in:
  mcopy -i ${WORKDIR}/boot.img -s ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE}-${DTB_BASE_NAME}.dtb ::${DTB_BASE_NAME}.dtb
  because ${KERNEL_IMAGETYPE}-${DTB_BASE_NAME}.dtb doesn't exist in my
  build, due to
  DTB_SYMLINK_NAME=`echo ${symlink_name} | sed "s/${MACHINE}/${DTB_BASE_NAME}/g"`
  not replacing whole "${KERNEL_IMAGE_SYMLINK_NAME}" (read ${MACHINE})
  with just ${DTB_BASE_NAME}

* with this change applied the deploy dir looks like this:
-rw-r--r-- 2 bitbake bitbake 1.4K Nov 20 15:49 deploy/images/raspberrypi3-64/lirc-rpi-1-4.9.59+git0+e7976b2aff-r0.8-raspberrypi3-64-20171120154716.dtbo
lrwxrwxrwx 2 bitbake bitbake   74 Nov 20 15:49 deploy/images/raspberrypi3-64/lirc-rpi.dtbo -> lirc-rpi-1-4.9.59+git0+e7976b2aff-r0.8-raspberrypi3-64-20171120154716.dtbo
lrwxrwxrwx 2 bitbake bitbake   74 Nov 20 15:49 deploy/images/raspberrypi3-64/lirc-rpi-raspberrypi3-64.dtbo -> lirc-rpi-1-4.9.59+git0+e7976b2aff-r0.8-raspberrypi3-64-20171120154716.dtbo
  and works correctly even with DISTRO using different naming scheme

* the sdcard_image-rpi.bbclass still needs to be modified, I've provided
  updated version here:
  https://github.com/agherzan/meta-raspberrypi/pull/159
* mpc8315e-rdb.conf MACHINE in meta-yocto-bsp also needs small fix:
  https://lists.yoctoproject.org/pipermail/poky/2018-July/011436.html

(From OE-Core rev: 1860d9d3c62e2e94cd68a809385873ffd8270b6d)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-10 17:32:59 +01:00
Martin Jansa
85d1f532f3 kernel-devicetree.bbclass: Use lowercase names for shell variables
* just to make it more clear what is local shell variable and what is
  replaced by bitbake from the metadata and also to prevent the variable
  to be incorrectly expanded by bitbake if someone happens to define
  e.g. DTB_BASE_NAME

(From OE-Core rev: 98b0f13f0650d970aac7441e7fcfc1089570785f)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-10 17:32:59 +01:00
Jackie Huang
7c1adcc6d7 rootfs-postcommands: split ssh_allow_empty_password
"allow root login" should not be bundled in ssh_allow_empty_password,
because some distro may want only one of "allow root login" and "allow
empty password", so split it out into ssh_allow_root_login and add new
imagefeature allow-root-login so they can be controlled separately,
debug-tweaks will still include both of them.

(From OE-Core rev: 1ab494f06a12548a902298afabd0a842161ef10d)

Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-09 22:48:38 +01:00
Joshua Watt
a4c1720eb3 classes/sstate: Remove unused argument
Removes an unused argument to pstaging_fetch()

(From OE-Core rev: ab0eb2bf0db6e7da7c9b9bb4d46621dbf76dcc2a)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 22:59:33 +01:00
Joshua Watt
660374adb6 classes/sstate: Remove trailing whitespace
(From OE-Core rev: dc7c181f231db1290ea7d7d5e42476e70a488d73)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 22:59:33 +01:00
Ross Burton
b11fc28a1c setuptools.bbclass: depend on python-setuptools
python-distribute is obsolete and is now simply a PROVIDES of python-setuptools,
so use the right name.

The identical change for setuptools3.bbclass was done in 2015 in 8922e60.

(From OE-Core rev: 87e38f0306400609aeac92bc13fd6f41d61e6271)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 22:55:02 +01:00
Ross Burton
1276f1755b meson: map mipsel TARGET_ARCH to mips for the cross file
Meson uses 'mips' for both big- and little-endian MIPS machines, so map mipsel
to mips.

(From OE-Core rev: 23734432a24da77aa838ad4bdcbcc294cde08348)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 22:55:02 +01:00
Chen Qi
ef98f64b71 testsdk.bbclass: set PREMIRRORS for kernel to speed up test
Currently if we do a testsdkext task for the image, it would take
very long time to finish. The time is mostly spent fetching kernel
source via network.

We have done some configuration in auto.conf, trying to make use
of own-mirrors.bbclass to avoid fetching kernel via network.

However, the solution normally does not work. Below is some log
from log.do_fetch.

  DEBUG: Fetcher failure: Unable to find file \
  file:///path/to/downloads/git2_git.yoctoproject.org.linux-yocto.git.tar.gz \
  anywhere.

The tar.gz file is not available. It is generated only if
BB_GENERATE_MIRROR_TARBALLS is set to "1". The default value of
BB_GENERATE_MIRROR_TARBALLS is "0", and according to the manual, users
need choose to set it to "1" only if they are trying to make a source
mirror. So generally, this var's value is "0".

Anyway, we do need to avoid fetching kernel source from network when
doing testsdkext. So set PREMIRRORS in auto.conf to achieve this.

After this change, the time reduces from 4209.131s to 1399.436s on
my local machine.

[YOCTO #12729]

(From OE-Core rev: fd18ddb0664f69d77681590774b0123251a98728)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 22:55:02 +01:00
Pablo Saavedra
303fb675f2 patch: Jail patch_task_patch_prefunc in classes/patch into the workdir
With PATCHTOOL=git patches the changes in the patch_task_postfunc
of the classes/patch. This works OK when the S dir is a Git repo
but doesn't if the source is a tarball.

The while condition in the patch_task_patch_prefunc must be
jailed into the WORKDIR. In the opposite, when you are executing
the recipe out of a Git subtree the function simply fails but when
your recipes are into a Git repo the patch_task_postfunc execute a
commit over your BSP local Git repo adding the changes in an
arbitrary Git repo found in the path from the SOURCE directory to
the '/'. This situation is highly probable in  cases like ~home
directories under the control of a .git repo or Yocto BSP which
manage the meta layers as git submodules.

This patch fix the changes introduced in

  classes/patch: when PATCHTOOL = "git" double-check the repository
  commit: 86ab56b551
  Author: Paul Eggleton <paul.eggleton@linux.intel.com>
  Date:   Tue Dec 5 14:36:58 2017 +1300

  classes/patch: when PATCHTOOL = "git" double-check the repository

  If a bug is present or the user has set PATCHTOOL = "git" on a source
  tree that isn't git, if we try to perform git operations (such as
  committing or changing branches) when extracting source, then we might
  in fact be running those operations on the metadata repository if the
  build directory is underneath, say, poky or OE-Core, and that could
  make a mess. Check if the source tree is a git repository and refuse
  to continue if it isn't.

(From OE-Core rev: db6ce9d8838b1f9064604654ab579da3e237f361)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 22:55:02 +01:00
Ross Burton
2d28e8370e distutils: clean the build tree in do_configure
base_do_configure() tries to do "make clean" if there is a Makefile present.
For most recipes using distutils there is not a Makefile, but we do know that
"setup.py clean" will work so call that instead.

(From OE-Core rev: b805cefb24566772a2beb5d02036266e45370913)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 22:55:02 +01:00
Martin Jansa
8b92648246 Revert "kernel-devicetree: Corrected normalize_dtb"
This reverts commit 2e7f3b2b9318d1e5395ad58131eafb873f614326.

It was breaking quite common use case that the dtb files are in
some subdirectory and then kernel build fails to build them.

As reported by khem:
http://lists.openembedded.org/pipermail/openembedded-core/2018-July/152578.html
me:
http://lists.openembedded.org/pipermail/openembedded-core/2018-July/152579.html
on raspberrypi3 build:

make[3]: *** No rule to make target 'arch/arm/boot/dts/dwc2.dtbo'.  Stop.
arch/arm/Makefile:345: recipe for target 'dwc2.dtbo' failed
make[2]: *** [dwc2.dtbo] Error 2
Makefile:146: recipe for target 'sub-make' failed

and trevor on the IRC:
20:35:49 < tlwoerner> the recent 2e7f3b2b9318d1e5395ad58131eafb873f614326 commit in oe-core seems to cause dragonboard-410c's kernel to fail to build
20:36:26 < tlwoerner> for the dragonboard-410c, KERNEL_DEVICETREE is set to "qcom/apq8016-sbc.dtb" but the build failure is:
20:36:37 < tlwoerner> *** No rule to make target 'arch/arm64/boot/dts/dts/qcom/apq8016-sbc.dtb'.  Stop.
20:36:44 < tlwoerner> i.e. the "qcom/" is getting removed
20:37:08 < tlwoerner> oops!!
20:37:33 < tlwoerner> wrong copy&paste, the actual error is:
20:37:36 < tlwoerner> *** No rule to make target 'arch/arm64/boot/dts/apq8016-sbc.dtb'.  Stop.
20:37:53 < tlwoerner> i.e., the "qcom/" is being stripped out

(From OE-Core rev: 0d725c76c113dec441a7319a6ee997e4ae8c4c88)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-05 11:39:06 +01:00
Chunrong Guo
b6bb7e1345 kernel-fitimage: add support for ext2.gz initramfs files
(From OE-Core rev: e2e0c91c2c8229d9ed0958de98b5a9b14c6805d1)

Signed-off-by: Chunrong Guo <chunrong.guo@nxp.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-05 11:39:06 +01:00
Ross Burton
5193aedb45 meson: map architecture to correct values in cross file
The cross file specifies the host/target cpu_family, which should be one of a
defined set of values[1] but if it isn't Meson won't complain and instead
recipes may behave unexpectedly.

[1] http://mesonbuild.com/Reference-tables.html#cpu-families

(From OE-Core rev: e33b902a1dc4294dac148715f4d3ca5b0a6ee1b7)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-05 00:22:08 +01:00
Ross Burton
c2ba7177f1 meson: various class improvements
- Ensure that the PACKAGECONFIG arguments are always in EXTRA_OEMESON

- Log the arguments that are being passed in do_configure.

- Do verbose builds so the compile logs are useful for debugging build problems

(From OE-Core rev: 3112ff268d095a65ecb893dd6ca88a85b0f70446)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-05 00:22:08 +01:00
Richard Purdie
0118f01787 multilib: Tweak previous cross-canadian multilib fix
As well as setting RECIPE_SYSROOT we also need to set STAGING_DIR_HOST/TARGET.

(From OE-Core rev: 59a0a05235d80c86251cf45d7142bfc57f2e70d2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-04 00:02:17 +01:00
Richard Purdie
f755066a00 staging: Always use the default sysroot for allarch recipes
Without this, recipes can't find allarch data files like autoconf-archive.

(From OE-Core rev: 8ae70703f68853a8714a4fb8fa5d959b5e21a02d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-04 00:02:17 +01:00
Paulo Neves
396e269c5c kernel-devicetree: Corrected normalize_dtb
The normalize_dtb function was buggy because
it only converted from .dts suffix to .dtb
suffix if the user passed a full source path to
KERNEL_DEVICETREE containing the /dts/ path.
The problem is that if the user did that there
would be a warning.

On the othet hand if user just set the variable
KERNEL_DEVICETREE="file.dts" the bbclass translation
to the respective .dtb target did not occur and
make would fail saying it has no rule to make target
file.dts

This patch decouples the logic of having /dts/ in the
path from the target translation.

(From OE-Core rev: 2e7f3b2b9318d1e5395ad58131eafb873f614326)

Signed-off-by: Paulo Neves <ptsneves@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-04 00:02:17 +01:00
Alex Kiernan
bba7c19eeb kernel-fitimage: Make DTB key insertion optional
If UBOOT_DTB_BINARY is empty, then don't try inserting the U-Boot
signing keys into the DTB. In this configuration the keys are expected
to be already present in U-Boot's DTB.

(From OE-Core rev: a0d74767f7bd18c853df6b0be162363076d8f965)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-04 00:02:16 +01:00
Richard Purdie
9fff2b5a75 multilib: Fix issues with some cross-canadian toolchain sysroots
MACHINE = "qemumips64"
MULTILIBS = "multilib:lib64 multilib:lib32"
DEFAULTTUNE = "mips64-n32"
DEFAULTTUNE_virtclass-multilib-lib64 = "mips64"
DEFAULTTUNE_virtclass-multilib-lib32 = "mips32r2"

bitbake core-image-minimal -c populate_sdk

Results in gcc-cross-canadian-mips failing to build due to the use
of an incorrect sysroot, fix this. All nativesdk pieces should be in
the same sysroot (unprefixed).

(From OE-Core rev: ae48ee6627e6c1c4f1fcc4ead40edc968e64f7fe)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02 21:37:47 +01:00
Richard Purdie
6ca693f284 staging: Improve fixup processing code
With the fixes to other parts of multilib, it was found that the fixup code's
assumptions about the recipe sysroot were incorrect. We need to use the value
calculated earlier in the function.

It turns out there is a rather neat way to do this which cleans up the code
as an added bonus.

(From OE-Core rev: 2c1978fe1a5b72167c49010fbdd39a9e2eefdef8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02 21:37:47 +01:00
Richard Purdie
29353297b5 classes/utils: Improve all_multilib_tune_values
Currently there is duplication in the code, we can clean this up
by extending the multilib variants list.

This code also currently fails its its called from an existing multilib
context since its assumes the data store passed in is the non-multilib
case. When building an image, say lib32-core-image-sato, this leads to
incorrect PATH values.

To fix this, we also request a data store for the "" variant allowing
this corner case to be fixed and for the function to correctly return
values for each multilib even when a multilib is already selected.

(From OE-Core rev: cc1c481560b81c223170230430308c7e2f08c812)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02 21:37:47 +01:00
Richard Purdie
f1079cd193 staging/image: Fix multilib recipe sysroot issues
Currently if you enable multilib, then build an image, the multilib
recipe sysroot is build in the wrong WORKDIR. If you then clean and
rebuild the image you see "file exists" errors.

This patch ensures the real WORKDIR is used consistently and then
cleans/rebuilds also work correctly.

(From OE-Core rev: c013ae59a158378d06ecf8eb123df0a10bf986b4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02 21:37:47 +01:00
Alp Özmert
42b3761274 license: Fix and extend recommendations for license packages.
Changed package recommendations so that the license package of a
recipe is recommended for all packages of a recipe instead of for one
package given by the recipe name.

Pre-patch behaviour results in a missing recommendation when a recipe
does not have a package with the same name.

(From OE-Core rev: 07343ff6282dd18432ecee5d9b80ad1fb86217f1)

Signed-off-by: Alp Özmert <info@ib-oezmert.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02 21:37:47 +01:00
Andre McCurdy
3fcec2cfa2 bitbake.conf: handle cmake -dev files packaging with default rules
Move packaging rules for cmake -dev files from cmake.bbclass into
bitbake.conf to handle recipes (e.g. harfbuzz 1.8.1) which build with
autotools but also install cmake -dev files.

(From OE-Core rev: 543e39ad5e2baa0f1ece013a89483783e6b15dd9)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-28 12:26:54 +01:00
Olof Johansson
16f060c56f insane.bbclass: Don't let warnings make previous errors non-fatal
package_qa_handle_error() returns True on non-fatal issues and False on
fatal issues. But the current usage has been to do

  sane = package_qa_handle_error(...)

which would always reset sanity status to be that of the last issue
identified. This change the assignments to use the &= operator instead:

  sane &= package_qa_handle_error(...)

As far as I can tell, this is not a real problem in practice, because
warnings of different levels (WARN_QA, ERROR_QA) does not seem to have
been mixed in a way that triggered this issue.

(From OE-Core rev: 21d015f6c9927598d64c48c925638619b25cf232)

Signed-off-by: Olof Johansson <olofjn@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-28 09:22:35 +01:00
Olof Johansson
fed27b0945 insane.bbclass: Make missing license file fatal
If a license file referenced from LIC_FILES_CHKSUM doesn't exist,
insane.bbclass would output an error message, but would continue the
build. This change makes this error fatal (as I suspect has been the
intention).

(From OE-Core rev: da29440633706fb7a346391d97894d6f2cbb0d01)

Signed-off-by: Olof Johansson <olofjn@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-28 09:22:35 +01:00
Alex Kiernan
8d33b69ed0 kernel: Set SOURCE_DATE_EPOCH to kernel git timestamp if not set
If SOURCE_DATE_EPOCH is unset (in addition to the existing "0" behaviour)
parse out the top most commit timestamp from the kernel tree to use as the
timestamp.

(From OE-Core rev: 2f0dd67a5a8d4269f5155004d532d8fa972b3223)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-28 09:22:35 +01:00
Alex Kiernan
3c46673100 image: Set COREBASE as the git directory for timestamp
When REPRODUCIBLE_TIMESTAMP_ROOTFS is unset and we want to parse one
from git, use COREBASE as the base for the git command so we have a
known repository which we're using. Without this the build may fail
if the current directory is not part of a git repository.

(From OE-Core rev: 1c2197f96d69547e10b74dc722d9a569d9a2b2b6)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-28 09:22:35 +01:00
Damien Riegel
cd928d3a98 systemd: escape paths passed to shell
Systemd mount configuration file must have a name that match the mount
point directory they control. So for instance, if a mount file contains

    [Mount]
    ...
    Where=/mnt/my-data

The file must be named `mnt-my\x2ddata.mount`, or systemd will refuse to
honour it.

If this config file contains an [Install] section, it will silently fail
because the unit file is not escaped properly when systemctl is called.
To fix that, make sure paths are escaped through `shlex.quote`.

(From OE-Core rev: bbd9524256461f1bcafd4103edd575e668de76f8)

Signed-off-by: Damien Riegel <damien.riegel@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-27 13:55:22 +01:00