Commit Graph

6229 Commits

Author SHA1 Message Date
Ross Burton
24b9bcf043 insane: improve license checksumming logic
Instead of opening files as bytes and battling decoding to UTF-8 which can throw
exceptions, open directly as strings and replace invalid codepoints.  This
handles licenses in encodings which are not UTF-8 but are based on ASCII much
better.

Also instead of extracting the license lines, writing them to a file, and then
hashing the file, hash the lines directly.

(From OE-Core rev: 63ef9d342277c4ba541b78cbb45ef181f071f495)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-24 17:00:52 +00:00
Alistair Francis
2187d06166 goarch: Add riscv64
Although RISC-V 64-bit doesn't have official golang support there are
forks that now exist with at least some support and work is ongoing in
the upstream tree. In order to be able to use the goarch class add
support for RISC-V.

For more details see here:
https://github.com/golang/go/issues/27532

(From OE-Core rev: 91e3b2a762124bf5cf923654ef3a7c871b84c82f)

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-21 16:19:23 +00:00
Steven Hung (洪于玉)
49ffd2da21 class/sanity: strip the output of get_filesystem_id()
A previous commit 2f44b9b replace oe.utils.getstatusoutput() to
subprocess.check_output(). check_output() don't remove a trailling newline. Add strip()

(From OE-Core rev: 172c3e85c601a61f3c668f83b75f4c1eb31dbd4d)

Signed-off-by: Steven Hung (洪于玉) <Steven.Hung@mediatek.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-18 11:12:26 +00:00
Douglas Royds
7d9b03567d cmake: Reduce verbosity for make invocation
Since the dawn of time, we have set CMAKE_VERBOSE_MAKEFILE=1 in cmake.bbclass.
Back in 2016, we also explicitly set VERBOSE=1 in cmake_do_compile(),
to ensure that make (and ninja) output were verbose in log.do_compile.

Turning off CMAKE_VERBOSE_MAKEFILE=1 means that make (or ninja)
invocations from the command-line are non-verbose,
giving CMake's default human-readable output on the terminal instead.
The user can still invoke VERBOSE=1 make if they do want verbose output.
This has no effect on the verbose output that goes into the logs.

(From OE-Core rev: 75c4b61513d8b089e835fb8d5923d8749fed7880)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-12 11:57:41 -07:00
Bruce Ashfield
40c065800f linux-yocto: fix SRC_URI patching with unified repos
As reported by Paul Barker, my attempt to allow the patching
of kernel meta data broke other repository configurations, since
the meta data patch routing was matching too broadly and the
same repo would end up being patched twice.

Using his suggested fix, we are up and running with both types
of repos again.

(From OE-Core rev: 0fd09633e3c2fb4668c7700516d288f8c5356ff6)

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-12 11:57:41 -07:00
Khem Raj
e53aa46f43 go: Add recipes for 1.12 release and make it default
- Refresh patches
- Enable GOCACHE required as of Go 1.12

(From OE-Core rev: f559fd6df2978f9093672794420eada2b7452987)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-09 14:43:11 +00:00
Douglas Royds
f1376858b3 icecc: Slightly more verbose log messages in the success case
The path containing the symlinks to icecc itself.
The path to the build env tarball.

(From OE-Core rev: 4b1f421ab5c7e5a3ee7bad5744149d331c6e0511)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-09 14:43:11 +00:00
Ming Liu
f52f818cf0 rm_work: sort the value of do_build dependencies
This fixes some 'basehash changed' errors when rm_work is being
inherited.

(From OE-Core rev: e74158b6cc1d683ab14ef5d47ec531f986fc2259)

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-09 14:43:11 +00:00
Jaewon Lee
58dbd3bfb9 device-tree.bbclass: Add support to compile overlays separately
Currently only dts files are considered when looping through files to
compile. Modifying the loop to compile other files that are overlays.
Also surrounding this check with a try block as the function to find
overlays parses the file for a '/plugin/' tag, and there may be files in
the DT_FILES_PATH directory that are not parseable.

(From OE-Core rev: bb1629820443bfedc72378a7c88f0656a2f3f7f1)

Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Manjukumar Matha <manjukumar.harthikote-matha@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-09 14:43:10 +00:00
Robert Yang
09af407145 insane.bbclass: Add configure-unsafe and configure-gettext
So that the errors can be controlled by ERROR_QA or WARN_QA, and make them work
with INSANE_SKIP.

(From OE-Core rev: 4b86ebb6a87f75421a1d12d9d5dde303938c2e9f)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-07 21:55:29 +00:00
Robert Yang
b25608d491 insane.bbclass: Make INSANE_SKIP work for qa pkgconfig and la
The INSANE_SKIP = "pkgconfig" or "la" didn't work, this patch fixes the problem.

[YOCTO #13087]

(From OE-Core rev: 756d6419f10d388c5daa63b7607e580d8b5079f1)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-07 21:55:29 +00:00
Achille Fouilleul
739c9a69ca utils: Fix create_wrapper
Wrappers generated by create_wrapper do not work properly:
python, for example, looks at argv[0] to know if it is
running from a virtualenv. Because the wrapper replaces
argv[0] with python's canonical path, python cannot tell
whether it has been invoked directly or from a virtualenv
symlink.

This change makes the wrapper pass the original argv[0] down
to the real command.

(From OE-Core rev: 791e5bfcb449024f295a4f8f507c1012cff76f9d)

Signed-off-by: Achille Fouilleul <achille.fouilleul@gadz.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-06 23:12:27 +00:00
Joshua Watt
ccc413fd93 Add source package to ${PACKAGES}
Permanently adds the -src source package to ${PACKAGES} instead of
relying on creating it dynamically at packaging time. The source package
is now defined in bitbake.conf, just like -dev and -dbg packages.

For compatibility, the -src package is still added dynamically if it was
missing, since some recipes have a tendency to completely override
PACKAGES and do so without manually adding back the -src package.

This allows RDEPENDS on the -src packages, which wasn't previously
possible.

[YOCTO #13203]

(From OE-Core rev: b25e1edf0204fc2f64aa8d66e09b8e2d67b90e17)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-06 23:12:27 +00:00
Bruce Ashfield
04bfce60c8 kernel-yocto: allow kernel-meta repositories to be patched
For testing purposes, it is often easier to patch a fragment
in a kernel-meta repository versus needing to make a copy or
modify the source repository.

We can allow this sort of patching when a patchdir of kernel-meta
is passed (to indicate the nested kernel-meta repository).

Also note that we must patch the meta data before they are
processed/gathered, since migrated copies to the kernel source
directory will be used later.

(From OE-Core rev: 324f9c8181150c23e3ce1ace4c45794cc8894167)

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-06 16:44:37 +00:00
Jaewon Lee
206ca252c6 devtool: Support kmeta directory usage with devtool modify/finish
When using Kmeta directories, devtool finish will add every single file
in the directory to the bbappend. This is because in the current
implementation, the get_recipe_local_files function treats the kmeta
directory like a file. Modifying the function to loop through the
provided directories and return all included files instead of just the
top level directory. This will enable correct file to file comparison
when determing which files are new/changed and need to be added to the
bbappend.

Adding an extra check in devtool-source.bbclass to not copy the cfg file
if its already included somewhere in the kmeta directory

Also during 'modify', when moving necessary files in the kmeta directory
from the workdir to oe-local-files, the dangling parent directories are
left behind.  This in itself is not an issue as the temporary devtool
workspace is automatically deleted, but this causes an incorrect include
directory to be added in kernel-yocto.bbclass.  Changing the order of
the if statements to catch the correct conditional. This is safe to do
as when not in the devtool context, there will be no oe-local-files
directory.

(From OE-Core rev: 4671011b5b02d0989aa0fdcb50e18cb385a0e95e)

Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Acked-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-06 10:39:25 +00:00
Matt Madison
d967e6d6f2 uboot-extlinux-config.bbclass: add missing variable dependencies
do_create_extlinux_config performs its own override processing
for several variables, so we have to explicitly add the label-
suffixed variable names to its vardeps to make sure that changes
get detected.

(From OE-Core rev: ad792edf61157f6cd63a2c6aa8e53edc134301d0)

Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-04 14:27:06 +00:00
Richard Purdie
d22b6e03a5 mcextend: Add helper class useful for multiconfig
This class allows users to easily create multiconfig BBCLASSEXTEND recipes
and is generally useful. It will be used by new tests so add as a general
feature.

(From OE-Core rev: 9ad19eb12c24d1ddc967215af0ebafd2cd2cb665)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-03 15:38:13 +00:00
Justin Bronder
a2938316e3 kernel: add objtool to shared workdir when CONFIG_UNWINDER_ORC is set
Based on: https://patchwork.openembedded.org/patch/148047/

When x86_64 enables CONFIG_UNWINDER_ORC [ORC (Oops Rewind Capability)
unwinder for unwinding kernel stack traces], objtool is required to
generate the required information.

If we don't copy objtool to the shared workdir, out-of-tree modules
cannot generate object files.

For instance, meta-skeleton/hello-mod fails with:
    | make[3]: *** No rule to make target 'tools/objtool/objtool', needed by ...

(From OE-Core rev: 7d4fdd3b95c949277f77df90bb2e8120b8d167ae)

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-02 16:49:07 +00:00
Alex Kiernan
55b0f515bf kernel-fitimage: Replace tabs with spaces in python
Fix:

  WARNING: python should use 4 spaces indentation, but found tabs in kernel-fitimage.bbclass, line 33

(From OE-Core rev: 00716a795f10e223150b0f9b214d185d654f4cc1)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-28 17:47:10 +00:00
Khem Raj
9d973d3b4e default-distrovars: Drop DISTRO_FEATURES_LIBC
After eglibc was merged into glibc, Kconfig support was also dropped so
these libc features therefore are not effective anymore and can be
removed

(From OE-Core rev: c62b1cc06613a4cdddf53290e6203559f43fc62d)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-28 13:21:54 +00:00
Alexander Kanavin
48522906a2 qemu: split the native version into usermode and system parts
The rationale is to streamline the overall build.

The system parts are only needed to run target images, and so can be
built towards the end of the build process. At the same time, the
system parts may need gtk+-native and mesa-native which add significantly
to the build time.

On the other hand, the usermode parts have almost no dependencies
and can be built quickly. They are needed at recipes build time to
run target binaries, and so are required quite early in the typical
build process.

(From OE-Core rev: 4a558a5f2db68538e0edad798ddf48eb9510a7d6)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-28 13:21:54 +00:00
Manjukumar Matha
a55784451e kernel-fitimage.bbclass: Introduce a way to provide external dtb
Currently kernel-fitimage gets the dtb from KERNEL_DEVICETREE, however
there are instances when the DTB file can be generated using other
recipes, provide a way to include the DTB instead of inkernel device
tree.

Add support for external device tree during the creation of fitimage.  If
virtual/dtb is set using PREFERRED_PROVIDER_virtual/dtb = "devicetree", which
inherits the devicetree.bbclass then use the path provided in
EXTERNAL_KERNEL_DEVICETREE else use KERNEL_DEVICETREE during fitimage process

(From OE-Core rev: 084f4de4dbaf9821516fc0254d35f4fb04311d27)

Signed-off-by: Manjukumar Matha <manjukumar.harthikote-matha@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-26 23:04:31 +00:00
Alexander Kanavin
1525641392 gtk-doc.bbclass: unset LD_LIBRARY_PATH
meson has been found to set it erroneously.

(From OE-Core rev: 920824125eb87928a329a072fa6c5a839efc932b)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25 12:35:59 +00:00
Richard Purdie
e51d27b86e staging: Fix staging_populate_sysroot_dir native corruption
bitbake gdk-pixbuf gdk-pixbuf-native
bitbake build-sysroots -c build_target_sysroot

can lead to tracebacks as gdk-pixbuf-native is being installed into the
target sysroot. The issue is that the x86_64 (common BUILD_ARCH) sysroot
components directory can contain a mix of native and target artefacts.

Differentiate by the "-native" in the recipe names. Should also trim
down the size of the sysroot used in eSDK.

(From OE-Core rev: 57f32836060bba0eaa6b36f53146dd6fd07b77ed)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25 10:43:06 +00:00
Manjukumar Matha
5397a7acd8 populate_sdk_ext.bbclass: Fix the derivate sdk build error from an published eSDK
When you install the eSDK in publish mode and then try to build a
derivative sdk using devtool build-sdk, the following error happens

| NOTE: Generating sstate task list...
| NOTE: Generating sstate-cache...
| NOTE: Generating sstate-cache...
| DEBUG: Python function copy_buildsystem finished
| NOTE: Executing install_tools ...
| DEBUG: Executing shell function install_tools
| install: cannot stat /core/meta/files/ext-sdk-prepare.py':
No such file or directory

This patch will fix the error by installing ext-sdk-prepare.py in
publish mode, so that derivative sdk can be generated.

(From OE-Core rev: aab4089a57a655ca5a7792f05eb5d8dcdb934ae1)

Signed-off-by: Manjukumar Matha <manjukumar.harthikote-matha@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25 10:43:06 +00:00
Manjukumar Matha
7bf275b54e devicetree.bbclass: Add virtual provider for dtb
Add virtual provider for dtb. This class can be used to generate dtb
from static devicetree, for reference see meta-xilinx layer
meta-xilinx/meta-xilinx-bsp/recipes-bsp/device-tree/device-tree.bb

By providing a virtual provider we can use it to differentiate between
in-kernel dtb or an external one. This can be set in local.conf as
PREFERRED_PROVIDER_virtual/dtb = "devicetree"

(From OE-Core rev: a7d52dbffd0da95de82d910c595a6c27d9b2ad0d)

Signed-off-by: Manjukumar Matha <manjukumar.harthikote-matha@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25 10:43:06 +00:00
Richard Purdie
a5b7c86f23 kernel: Ensure an initramfs is added if configured
If ${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE_NAME}.cpio does not exist,
nor any of the compressed variants, nothing is copied to kernel build's
./usr directory.

The code does not fail, but silently proceeds without a bundled initramfs.

Change to fail and tell the user something is wrong.

Also, if an initramfs is found, contrary to the comments, it does not stop
at the first uncompressed/compressed cpio image found. Instead it keeps
processing all so the last is used. Fix this to behave as per the comments.

[YOCTO #12909]

(Patch by Leon Woestenberg)

(From OE-Core rev: 5b5604e288af755eb5553a97d26533445b2cf94b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25 10:43:06 +00:00
Peter Kjellerstedt
b44bd9fc6b libc-package.bbclass: Add a progress meter for the package task
The package task for glibc-locale takes a very long time to execute,
especially if using qemu. In that case, a progress meter helps a lot to
show the progress of the task.

(From OE-Core rev: bd2f9b2785779ef65c0c3664dae03b01ced59f6d)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-19 16:14:57 +00:00
Peter Kjellerstedt
4c56903824 libc-package.bbclass: Only extract wanted files in prep_locale_tree()
prep_locale_tree() predates the usrmerge DISTRO_FEATURE, which meant it
was not prepared for the case when ${base_libdir} == ${libdir}. This
lead to it extracting files and directories where it shouldn't.

(From OE-Core rev: 0bfc4dd0a9acc305b78fe9d5c2a7fb9afe708684)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-19 16:14:57 +00:00
Peter Kjellerstedt
dc9abea5f0 libc-package.bbclass: Correct a few paths for when usrmerge is used
(From OE-Core rev: a880377cd67b65101bf95e8dbad6221f498e38ef)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-19 16:14:57 +00:00
Alexander Kanavin
85a4d9f53b clutter-gtk-1.0: convert to meson
This required adding an option to control gobject introspection
via a custom patch, and tweaking clutter.bbclass to not enforce
autotools.

(From OE-Core rev: 9b9c12268bf9365101da4a985f70c06740dfeb8f)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-19 16:14:57 +00:00
Manjukumar Matha
c71dc290b0 kernel-fitimage.bbclass: Fix the dependency issue while generating fitimage_initramfs
When building fitimage_initramfs, the correct depedency is to build
after do_bundle_initramfs. We can run into the following dependency
issue

DEBUG: Python function extend_recipe_sysroot finished
DEBUG: Executing shell function do_assemble_fitimage_initramfs
aarch64-xilinx-linux-objcopy: 'vmlinux': No such file

This happens because initramfs renames vmlinux to vmlinux.bak while
generating vmlinux.initramfs, there is a chance that fitimage_initramfs
can also start during this process and create the above issue.

This patch resolve the dependency issue by running fitimage_initramfs
task after do_bundle_initramfs

(From OE-Core rev: 8f0bece39a634fce5bd882cbd9e289ea905a0b17)

Signed-off-by: Varalaxmi Bingi<varalaxm@xilinx.com>
Signed-off-by: Manjukumar Matha <manjukumar.harthikote-matha@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-18 10:33:13 +00:00
Richard Purdie
eea86c4f0a native: Enable RDEPENDS handling
Native recipes don't currently honour their RDEPENDS. In the case of
some python scripts this has started causing problems since whilst they're
not needed at build time (DEPENDS), they are needed at runtime.

We put off making this change due to circular dependency issues. I believe
the three such problems in OE-Core are now fixed, as is the dependency loop
identfication code in bitbake so its time to improve this situation.

[YOCTO #10113]

(From OE-Core rev: c62520b63284927e177831c351fafa4d2768cb1f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-17 10:19:59 +00:00
Richard Purdie
050aab7f45 package: Rework debug source file handling
Currently we parallel process the files we install running dwarfsrcfiles over
each one in parallel threads but requiring a lock to write the results to one
file. This is not ideal for performance and means we can't then use per file
data for other purposes such as source code license processing.

Rework the code so that the list of source files is generated per installed
file and is reusable.

The code still generates a null separated debugsources.list file since this
is used by a shell pipeline but it no longer needs locking.

(From OE-Core rev: 95de93988eb725c14102f642ebabff3920ae194f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-17 10:19:59 +00:00
Anuj Mittal
bd5c9ca7c8 meson.bbclass: point to llvm-config
Allow packages using llvm-config to find it using meson's dependency
interface.

(From OE-Core rev: e2e733c813e1eabaaff795a30b82e703dfeecfe7)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-15 16:05:37 +00:00
Joshua Watt
8d47356ad2 classes/license_image.bbclass: Fix rootfs license file permissions
Fixes up the permissions on the license files when they are put on the
target file system so that they are readable by everyone. Previously,
they would have inherited whatever permissions the file had in the
recipe, which may not have been appropriate.

[YOCTO #13175]

(From OE-Core rev: 8190d192fceb9b0969385507d3d4bca7be75c810)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-15 08:17:49 +00:00
Robert Yang
407a02b057 base.bbclass: Check BuildStarted for HOSTTOOLS
There might be no bb.event.ConfigParsed event if bitbake server is running, so
check bb.event.BuildStarted too to make sure HOSTTOOLS_DIR exists.

Fixed:
$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native
$ rm -fr tmp
$ bitbake quilt-native
ERROR: Error running gcc  --version: /bin/sh: gcc: command not found

This error is caused by enable_uninative(), it runs twice (ConfigParsed and
BuildStarted), the error would happen when there is no ConfigParsed event
(no hosttools is created), but BuildStarted. This patch can fix the problem.

[YOCTO #13022]

(From OE-Core rev: da798db0a48282e3d4f58890a7aec42c3deff0b8)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-15 08:17:49 +00:00
Khem Raj
c6a243dbb2 image_types.bbclass: Set memory usage limit and CPU threads for xz
when building with opkg backend and huge packages e.g. chromium/llvm all
going in parallel, memory pressure causes xz to catapult with

do_package_write_ipk: Failed to create package, opkg-build failed with: xz: (stdin): Cannot allocate memory

since there are many tasks going on in parallel, xz adds to memory pressure
and it wants it all, put an upper limit for memory xz can use

We add a variable XZ_MAXRAM with 30% of RAM limit and can be customized
if builders have more memory one can set it like

XZ_DEFAULTS = "-M 0 -T 0"

(From OE-Core rev: 28b277a93a34bba033d9d0d9f3227c9453efd384)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-14 11:20:55 +00:00
Silvio Fricke
1689d590e5 image_types: add base64 conversion
Sometimes it is useful to have a base64 representation of an image.

(From OE-Core rev: 9d00b146c4125b195fac5a0743fe7d8e59edef7f)

Signed-off-by: Silvio Fricke <silvio.fricke@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-12 14:04:31 +00:00
Alexander Kanavin
e2c3247c23 python3: upgrade to 3.7.2
I took the same approach as the recent perl upgrade: write recipe from scratch,
taking the pieces from the old recipe only when they were proven to be necessary.

The pgo, manifest and ptest features are all preserved.

New features:

- native and target recipes are now unified into one recipe

- check_build_completeness.py runs right after do_compile() and verifies that
all optional modules have been built (a notorious source of regressions)

- a new approach to sysconfig.py and distutils/sysconfig.py returning values
appropriate for native or target builds: we copy the configuration file to a
separate folder, add that folder to sys.path (through environment variable
that differs between native and target builds), and point python to the file
through another environment variable.

There were a few other patches where it was difficult to decide if the patch
is still relevant, and how to test that it works correctly; please add those
as-needed by testing the new python.

(From OE-Core rev: 02714c105426b0d687620913c1a7401b386428b6)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-08 10:57:19 +00:00
Richard Purdie
251ded90d4 archiver/package_rpm: Fix the worst src.rpm generation race
The package_rpm code is writing outside the task's sstate directory into
the sstate of do_deploy_archives. This is "out of spec" since if the
task is installed from sstate, the files are not restored. This means
the files may appear/disappear, things are not deterministic and there are
races.

Extend the do_package_write_rpm code to handle writing the src.rpm into
place to avoid these issues. There are other problems but this avoids races
around this file.

(From OE-Core rev: c6e151ba7fe0f14044537cf0ab2cac436f1496e3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-06 08:29:07 +00:00
André Draszik
d05823086d update-alternatives: correctly escape PATHs when updating FILES_${PN}
The recently added support for updating FILES based on the file renames
that are happening here is using a regex replace, but failed to
properly escape the search pattern (the full path). This manifests itself
in FILES not being updated as soon as the full path contains any
character that has a special meaning, e.g. '+'.

In other words an original path (alt_target in the code) like
    /opt/poky/2.6+snapshot/sysroots/i686-pokysdk-linux/sbin/losetup
can't be matched, and hence we fail to update FILES with the new value,
causing packaging errors.

Fix by using re.escape() on the original path before passing into re.sub()

Fixes: 5c23fe378732 ("update-alternatives: try to update FILES_${PN} when
renaming a file"), or bcb3e7b7f8 in poky.git

[YOCTO #13058]

(From OE-Core rev: 126743162397e4145902b3f127f2dafd80a8a49b)

Signed-off-by: André Draszik <andre.draszik@jci.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-06 08:29:06 +00:00
Richard Purdie
52faf80b7c package_rpm/archiver: Apply bandaid to src.rpm creation
| error: create archive failed on file /media/build1/poky/build/tmp/work/all-poky-linux/xcursor-transparent-theme/0.1.1+gitAUTOINC+23c8af5ba4-r0/deploy-sources/allarch-poky-linux/xcursor-transparent-theme-0.1.1+gitAUTOINC+23c8af5ba4-r0/xcursor-transparent-theme-0.1.1+git0+23c8af5ba4-r0.src.rpm: cpio: read failed - No such file or directory
| Building target platforms: noarch-poky-linux
| Building for target noarch-poky-linux

This is caused by:

$ cat log.task_order

do_cleansstate (24289): log.do_cleansstate.24289
do_deploy_archives_setscene (24395): log.do_deploy_archives_setscene.24395
do_fetch (24407): log.do_fetch.24407
[..]
do_package_write_rpm (25448): log.do_package_write_rpm.25448
do_package_qa (25451): log.do_package_qa.25451

So do_deploy_archives can run from sstate, created a .src.rpm in WORKDIR/deploy-sources,
then it was removed when rpm was running. This leads to a broken Source line in the
spec file as the original file was found by the os.listdir().

This fix is just a bandaid over much more fundamental problems sadly.

(From OE-Core rev: a10020ace4c3cd863c782760f7cbecea557ec6e7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-05 13:53:48 +00:00
André Draszik
bcb3e7b7f8 update-alternatives: try to update FILES_${PN} when renaming a file
When using update-alternatives, FILES_${PN} must be
referencing the new name after update-alternatives has
renamed files.

This is more or less OK when having static lists of files to
be packaged into a package, but makes it quite hard to
dynamically generate FILES_${PN}, e.g. using do_split_packages(),
as in that case we can not easily modify what goes into
FILES_${PN}, because that list is based on filenames as seen
at the time do_split_packages() is executing.

Of couse one could explicitly specify the (renamed) file(s)
in the recipe, but that contradicts the intended usage of
do_split_packages().

Instead, if FILES_${PN} contains the file name as it was pre
renaming, we here modify this to reflect the new name.

This will allow usage of do_split_packages() to populate
FILES_${PN}.

[YOCTO #13058]

(From OE-Core rev: 5c23fe378732038643a450cbf916334d24764b70)

Signed-off-by: André Draszik <andre.draszik@jci.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-02 11:05:59 +00:00
André Draszik
0b782c39b9 update-alternatives: convert file renames to PACKAGE_PREPROCESS_FUNCS
At the moment, the update-alternatives file renaming is
happening right after copying into PKGD during packaging
time using an _append OVERRIDE to the copy function
perform_packagecopy().

This is not really readable and hard to maintain.

Additionally, this makes it impossible to e.g. populate
PACKAGES dynamically using do_split_packages() and still
do update-alternatives - update-alternatives file renaming
requires the PACKAGES variable to have been fully populated
to work correctly. On the other hand, do_split_packages()
can only execute after perform_packagecopy(), as it needs
PKGD populated; so it's impossible to insert
do_split_packages() early enough in a deterministic way in
this use-case.

As there doesn't seem to be a reason not to, convert
this to a proper function and use PACKAGE_PREPROCESS_FUNCS
instead - after all, that's what this is meant for.

No other classes or recipes in oe-core or meta-openembededd
seem to have a hard requirement on update-alterantives
executing before any other PACKAGE_PREPROCESS_FUNCS, so
this should be perfectly fine.

The only implication is that if compress_doc is inherited,
compressed man-page file names will end up being, e.g.
    eject.1.util-linux.gz
  or
    eject.1.gz.util-linux
based on the include of compress_doc.bbclass vs.
update-alternatives.bbclass order, but the symlink created
(alternative name) will always be correct.

This solves both problems:
* the code is easier to read / follow
* the above described use-case can be accomodated easily

[YOCTO #13058]

(From OE-Core rev: 8c3db2fa735e5933f842f80321bb3ed38753812d)

Signed-off-by: André Draszik <andre.draszik@jci.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-02 11:05:59 +00:00
Alexander Kanavin
99fc8d54d6 pixbufcache.bbclass: remove a dependency loop introduced by enabling gtk+3-native
(From OE-Core rev: f99921554fddef82eb80568125aa646a27859a4b)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-28 23:14:10 +00:00
Joshua Watt
3c34a844fc classes/icecc: Add ICECC_REMOTE_CPP variable
Newer versions of Icecream (1.2) add support for remotely pre-processing
files, controllable with the ICECC_REMOTE_CPP environment variable.

(From OE-Core rev: 8fe6728536bbee814c421afbbaa2022e0da0f65c)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-28 23:14:10 +00:00
Joshua Watt
ba3aa53112 Change default debug split to make separate source packages
Changes the default PACKAGE_DEBUG_SPLIT_STYLE to generate separate
source and debug packages. SDKIMAGE_FEATURES is updated to include the
source packages so that there is not change for the SDK contents.

[YOCTO #12931]

(From OE-Core rev: 8df14b678e44cc749b361224af05ccbcfa9ae9b5)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-28 23:14:09 +00:00
Richard Purdie
55ab45436d package: Add pkg_postinst_ontarget to PACKAGEVARS
Changes to pkg_postinst_ontarget were not triggering rebuilds, this fixes
that.

[YOCTO #13127]

(From OE-Core rev: 432d5a5481bd8efb848b95fbe6500a72fba9ac65)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-26 13:39:37 +00:00
Robert Yang
043d6ac1f6 native.bbclass/cross.bbclass: No strip sysroot when DEBUG_BUILD
This makes dbg work for native tools, and makes debug native tools problem
easier, otherwise, there is no symbol since trippped.

(From OE-Core rev: 1d903485da26fec991b4a940182e32934220e19b)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-26 13:39:37 +00:00