Existing compress framework for producing various compressed versions
of images is powerfull enough to be extended for other uses, e.g. to
convert types of images.
It is possible to use it also to produce image checksums at the time
of image generation. This commit adds support for all supported
at the moment coreutils hashing algorithms: md5, sha1, sha224,
sha256, sha384 and sha512
Usage:
IMAGE_FSTYPES_append = " hddimg.sha256sum"
(From OE-Core rev: 1fbd27122ac4403b8688bc41d128602564a05506)
Signed-off-by: Alexander D. Kanevskiy <kad@kad.name>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At present, if a recipe is built which creates users/groups via
useradd.bbclass, those users/groups are not removed from sysroot
when the recipe/package is cleaned using clean/cleansstate/cleanall
or when a recipe is rebuild and 'unstaged' from the the sysroot.
The "userdel_sysroot_sstate()" provides that functionality.
[YOCTO #9262]
(From OE-Core rev: b5304ce438666a7418746f4ddd32703ae3188089)
Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A path variable is assigned the value of PATH from the
environment, before the PATH is set on the environemnt
from the value of PATH in the bb datastore. This seems
to be an unnecessary leftover.
[YOCTO #8543]
(From OE-Core rev: a3c7c5148017c652661f2c5f79d90aa80dcb479b)
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a recipe passes for example extra_depends="${PN}-dev" to do_split_packages()
then it isn't expanded, so the check for the multilib prefix doesn't work.
Solve this centrally by expanding extra_depends inside do_split_packages().
[ YOCTO #9381 ]
(From OE-Core rev: d3875a8111f1d88dcc83219d4700784050988ca3)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The kernel tools assume git > 1.7.9.5, I'm unsure of the exact version but
the oldest in our infrastructure is 1.8.3.1. The git fetcher also currently
has nasty workarounds for git < 1.7.9.2.
Moving to 1.8.3.1 as our minimum version seems sane at this point as the oldest
we're testing/supporting.
[YOCTO #6162]
(From OE-Core rev: dbae075f62ecceadacc6d847e5697b9f3339b168)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An unprivileged "container" environment like this[1] doesn't have root
account (uid 0) which causes tons of "Invalid argument" warnings:
$ bitbake ...
...
WARNING: Could not copy license file [src] to [dest]: [Errno 22] Invalid argument: '[src]'
WARNING: Could not copy license file [src] to [dest]: [Errno 22] Invalid argument: '[src]'
WARNING: Could not copy license file [src] to [dest]: [Errno 22] Invalid argument: '[src]'
...
Fix it by handling EINVAL similar to existing handling of EPERM (which
was added for when not running under pseudo).
[1]: The real environemnt is buildFHSUserEnv from NixOS/nixpkgs, but a
demonstration of the issue can be done like this:
$ touch f
$ unshare --user --mount chown 0:0 f
chown: changing ownership of ‘f’: Invalid argument
(From OE-Core rev: d00b2250a6afebd7d1373c04b4006290f0cd4043)
Signed-off-by: Bjørn Forsman <bjorn.forsman@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use a unique tempfile as the temporary git index file when determining
the git hash of the source tree. A fixed filename was obviously causing
races (with the git index.lock file) under oe-selftest where multiple
bitbake instances were run against the same source tree at the same
time.
(From OE-Core rev: f81c641022c26a9b89fac769e0f2889eaec5d32f)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For images we produce a number of filtered dependency .dot files for
readability, the first of which is depends-nokernel.dot which filters
out just the kernel itself (not kernel modules). Unfortunately the
filter specifications hadn't been updated for the dash-to-underscore
removal or the 4.x kernel upgrade, thus the filtering wasn't actually
doing anything.
(From OE-Core rev: ec1077bed0c1538084bceeafb957abe725b13b76)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Invoking oe-pkgdata-util in turn for every package in the list was slow
with a large image. Modify oe-pkgdata-util's read-value command to take
an option to read the list of packages from a file, as well as prefix
the value with the package name; we can then use this to get all of the
package sizes at once. This reduces the time to gather this information
from minutes to just a second or two.
Fixes [YOCTO #7339].
(From OE-Core rev: 51c24904cc1bc823bccc3f179b8d7a192dace168)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise the task hash changes between server and worker context
leading to changing task checksums.
(From OE-Core rev: b5ab6111dbf21d2fc6a6e4aeaa9f105e3e0a23ac)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise the task hash changes between server and worker context
leading to changing task checksums. The dependency data here is
tracked by other pieces of the signature.
(From OE-Core rev: 1962571a7358fb9c9af2d6cfd1a01673b0d18193)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
During the simplication and cleanup of branches and kernel meta data
handling, the ability to force build a branch that didn't match the
meta data was dropped.
There are valid uses cases when a different branch should be built
(testing, development, etc), so we restore the capability with this
change.
If after the kernel meta data is processed the current branch does
not match the SRC_URI specified branch, a warning is generated
about the impending branch switch and that the user should double
check that they are building what they expect.
WARNING: After meta data application, the kernel tree branch is standard/base. The
WARNING: SRC_URI specified branch standard/gt. The branch will be forced to standard/gt,
WARNING: but this means the board meta data (.scc files) do not match the SRC_URI specification.
WARNING: The meta data and branch standard/gt should be inspected to ensure the proper
WARNING: kernel is being built.
Reported-by: Steve Sakoman" <steve.sakoman@intel.com>
(From OE-Core rev: d91a668bc0f6c2cfc52174b4039c7ea0d84e8d4d)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
package_qa_check_libdir() reports that the file libsoletta.so.0.0.1-gdb.py
in /usr/share/gdb/auto-load is in the wrong location. Before generating
a warning for files in non-standard locations, check that the file is an
actual elf file (and hence a real library file).
[YOCTO #9215]
(From OE-Core rev: a3ad36b9a435e7c3d97f114809561198b8abe6cf)
Signed-off-by: Bill Randle <william.c.randle@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use APPEND for grub's root device may cause confusion, for example, when
building efi + pcbios, there maybe be two root=/dev/ram0, one of them
would be carried to the installed target, and the target would fail to
boot. Use GRUB_ROOT to fix the problem, and remove SYSLINUX_ROOT from
APPEND will fix the problem.
[YOCTO #9354]
(From OE-Core rev: 1f46fe7d501644c83f81dc4cc3310073c804f797)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PACKAGECONFIG doesn't work in cross-canadian recipes at the moment as
DEPENDS are prepended with mlprefix.
A recipe is either nativesdk, native or target so adjust the if
statements accordingly, use inherits_class for more accurate recipe
classification and add cross-canadian support.
(From OE-Core rev: 4a172868a9423afb8a602413ef4935423ce51a5d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes the code a bit shorter and more readable.
(From OE-Core rev: f092f99a0d7116ba4347b22f3f81b4eac4808e62)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
base.bbclass sets 'check_license' to False (and therefore skips
license checking completely) for native, nativesdk, etc recipes
(ie anything which could potentially be classed as "host tools"),
so supporting a dedicated whitelist of GPLv3 host tools is not
necessary.
(From OE-Core rev: 8fc8b60005e7641861324c8541fb45058e7aab8e)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the common code to live_vm_common.bbclass and remove duplicated ones.
(From OE-Core rev: 4a70cc59a0350f06d4cc48c12c3053a39191ba07)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
They are doing the same things: create live images, merge them into one
bbclass makes it easy to understand.
(From OE-Core rev: bfd4d95210b3f841aa2e7c5a06ac89667523438d)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
They are doing the same things: create virtual machine images, merge
them into one bbclass makes it easy to understand.
(From OE-Core rev: c314fda9d739560b6a08e627e7aabf105d97c8c4)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously the list of packages that are considered unneeded for a
read-only rootfs was hardcoded. This made it impossible to, e.g., have
shadow installed on a system with a read-only rootfs, but where /etc
is mounted writable.
This also lists ${VIRTUAL-RUNTIME_update-alternatives} rather than
update-alternatives (as was previously the case) since this should
actually remove the intended package.
(From OE-Core rev: e3b881d4168e5b02ff00f5c470ba472ab8bbc747)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If SDK_EXT_TYPE is set to "minimal" then the SDK won't contain many
sstate artifacts, and you're required to set up an sstate mirror in this
case anyway so there's no point publishing the "stub" sstate-cache
directory from within the SDK since it won't be useful for update
purposes and may be confused with the real sstate-cache.
There is however a possibility that people might publish the real
sstate-cache directory under the same output directory provided to
oe-publish-sdk, thus deleting it after extracting (as we were doing with
other files we wanted to clean up at the end) would be problematic,
besides which extracting it and then deleting it is wasteful. Thus,
introduce a "-p" command line option to the SDK installer that we can
use to tell tar not to extract the items we don't want when publishing.
This has the added benefit of mostly keeping references to these in the
place they belong i.e. in populate_sdk_ext.bbclass.
(From OE-Core rev: 774b85d42db1d81936d4e4af4f6fb2c57cb51d2c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running bitbake you may pass in values of variables from the
external environment (making use of BB_ENV_EXTRAWHITE), and you may
choose to do this when building the extensible SDK, for example:
MACHINE=qemuarm bitbake -c populate_sdk_ext core-image-minimal
You would naturally expect those settings to be reflected in the
extensible SDK itself; however they were not, since we were only
considering local.conf and auto.conf. Check the variables mentioned in
BB_ENV_EXTRAWHITE to see if any are different than the values set in
local.conf/auto.conf and add lines setting them in the SDK's local.conf
if so.
Fixes [YOCTO #9339].
(From OE-Core rev: 2bfed75c48a6f6596ded9cb64cb96f00510f914e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The mkisofs requires syslinux/isolinux.bin which is provided by
target syslinux, for pcbios, the depends is already there, but for
EFI, it isn't, so add the depends.
(From OE-Core rev: e9adedd32a7f2015e58efe2f460a99e8d745b4c6)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the sanity code encounters a version change is doesn't understand, the current
output is unreadable and confusing for the user, particularly due to the presence
of the backtrace.
Use improved functionality in bitbake to improve this and correctly pass python
exceptions around.
[YOCTO #9291]
(From OE-Core rev: a0860e308645f17dbf4b9005b2fc0e9869d730bf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are various concerns about md5 so use sha256 instead.
(From OE-Core rev: a88603cb2ffd4f995e16349a389902eb884252e5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There was a do_deploy_archives_setscene, but no addtask.
(From OE-Core rev: 81db27c208468dc460e9d8f0063ad24b2b37dc97)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Found that ssize_t was being wrongly computed for mips64 musl
because it was using wrong cache files.
alias them to glibc versions, musl doesn't _yet_ have target
incompatibilities to note in siteinfo thankfully.
Add musl aliases for aarch64 and ppc64 targets
(From OE-Core rev: d3867743e7028046454dc135efeefadfa382f463)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildhistory_get_imageinfo() assumed that the buildhistory directory
for the image had already been created earlier. That assumption is not
true for special images (like the virtual swupd images from
meta-swupd) where the entire traditional do_rootfs/do_image is
skipped.
Creating files-in-image.txt still makes sense for such images, so
support them by creating the directory also in
buildhistory_get_imageinfo().
(From OE-Core rev: 723328319ee53235969ec9cce7ff5d7729dcf8d7)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tweak DEBUG_FLAGS to use "/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}"
as source target path in DWARF. While use gdb to debug binary, it could
work with sources in dbg package.
While -fdebug-prefix-map is used for compiling, we do not need invoking
debugedit to edit DWARF at do_package time, but list where sources files
are.
The copydebugsources uses the list to copy sources to dbg package. It
works whether -fdebug-prefix-map used or not.
[YOCTO #9305]
(From OE-Core rev: ecb56a6ae0c870af680da03db9d39703b525fc98)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extend the srctree_hash_files() function in externalsrc.bbclass to
handle non-Git source trees. If the source tree is not a git
repository, srctree_hash_files() now simply adds the whole source tree
as a dependency, causing bitbake to basically hash every file in it.
Hidden files and directories in the source tree root are ignored by the
glob currently used. This has the advantage of automatically ignoring
.git directory, for example.
During the first bitbake run preparing of the task runqueue may take
much longer if the source tree is not a git repository. The reason is
that all the files in the source tree are hashed. Subsequent builds are
not significantly slower because (most) file hashes are found from the
cache.
[YOCTO #8853]
(From OE-Core rev: a797abe08bfe4144e120c984d1718c2cc3d87c0a)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Be a bit more intelligent than mindlessly re-compiling every time.
Instead of always using 'nostamp' flag for do_compile run a python
function to get a list of files to add as 'file-checksums' flag. The
intention is to only re-run do_compile if something in the source tree
content changes.
This python function, srctree_hash_files(), works differently, depending
if the source tree is a git repository clone or not. If the source tree
is a git repository, the function runs 'git add .' and 'git write tree'
to get a hash of the working tree and writes this hash into a file under
the .git directory. This file containing the hash is then returned as
the file for the task to depend on. Hash is changed if any changes are
made in the source tree causing the task to be re-run. A trick is used
to parse the recipe every time so that the hash file gets updated.
If the source tree is not a git repository behaviour remains the same.
In this case srctree_hash_files() currently sets the 'nostamp' flag for
do_compile causing it to be re-run every time.
This method of tracking changes source tree changes to determine if
re-build is needed does not work perofectly, though. Many packages are
built under ${S} which effectively changes the source tree causing some
unwanted re-compilations. However, if do_compile of the recipe does not
produce new/different artefacts on every run (as commonly is and should
be the case) the re-compilation loop stops. Thus, you should usually see
only one re-compilation (if any) after which the source tree is
"stabilized" and no more re-compilations happen.
[YOCTO #8853]
(From OE-Core rev: a26becdf981b35d7ef8524f9e65c25a74b842f1d)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is pointless to archive several times the gcc source.
This change will archive gcc source once, for the gcc-source-{PV}
recipe.
[YOCTO #9327]
(From OE-Core rev: 2f53fb2ae561e2eb599682ba95ce83ad1b4b7ada)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new class uses gio-querymodules tool on postinst and postrm.
This regenerates the module cache which is useful to avoid loading
modules that are not needed at runtime: If a Gio module is not
listed in the cache file it will always get loaded.
* Add a postinst-intercept 'gio-module-cache': it runs
gio-querymodules using qemuwrapper. This is required because the tool
actually loads the modules to generate the cache.
* Add a gio-module-cache class that adds postinstall and postrm
scripts. In the sysroot population case use the new intercept.
* Inherit the new class in glib-2.0, glib-networking and gconf.
Fixes [YOCTO #9241].
(From OE-Core rev: 168cf5502a2dda78348cc9679e37bed0c69c0208)
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently do_wicenv task is run for all images. However, its
result is used only to produce wic image. It's better to
run this task only for wic images. If another rootfs is
required to produce wic image, dependency to its do_wicenv
must be added to the wic image recipy.
Stopped running do_wicenv for all images. Added explicit
dependency to this task in wic-image-minimal recipe.
[YOCTO #9095]
(From OE-Core rev: b81c176fb2f1ee818b6049c39ef353a7d7d5e078)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Return early in write_image_manifest () if the IMAGE_MANIFEST
variable is unset. This allows us to prevent writing of the
manifest where we prevent images being created for a recipe by
unsetting IMAGE_FSTYPES.
(From OE-Core rev: 54387be24d6a157574496f1b20b21c7fc2a51d3e)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new value is more general and better reflects what having the feature really means.
Introspection data, then, is built only if 'gobject-introspection-data' is in
DISTRO_FEATURES and 'qemu-usermode' is in MACHINE_FEATURES.
(From OE-Core rev: 9927a3d72e2272d8e3dc4785ba02e27802ee1c6c)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes kernels which package sb16 firmware e.g.
directories were installed but not shipped in any package:
/lib/firmware/sb16
/lib/firmware/sb16/ima_adpcm_playback.csp
/lib/firmware/sb16/ima_adpcm_init.csp
/lib/firmware/sb16/mulaw_main.csp
/lib/firmware/sb16/ima_adpcm_capture.csp
/lib/firmware/sb16/alaw_main.csp
(From OE-Core rev: 826bc56e77bd9a53f6bc19548549f6d92de09911)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make this class more clear and consistent, and avoid
confusion with the Intel C compiler (icc); the changes
affect function names, and calls to those functions
[YOCTO #8934]
(From OE-Core rev: daf59103c26ca8f1be7229f73a5b496d862e1f1e)
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Expanding PACKAGE_ARCH might be necessary as it is being
checked against to determine if this is allarch
[YOCTO #8934]
(From OE-Core rev: 45c59ffb742568e4f0d8cf53e4dfba5198bdc9f0)
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Additional check for inheritance of allarch.bbclass, for when
checking that PACKAGE_ARCH == "all" is not enough to be sure
a recipe is "allarch"; e.g. nativesdk-buildtools type recipes
[YOCTO #8934]
(From OE-Core rev: 0f9a3b445186b58d2c1f731da94c6b3a05494369)
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A python string literal isn't meant to be split over multiple lines
unless it is explicitly multi-line. Use three quotes to indicate that.
(From OE-Core rev: 8cdb505bb3d9748be1744f36cbbfb4f62fa20b67)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
patchelf will understandably error out if there isn't a .interp section to
relocate, so don't try to relocate static binaries.
(From OE-Core rev: 2a1803e64174825d9392094ae6e680a1ac96eb4a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When archiving the original source, the git repositories have the name as
they are in the $DL_DIR plus the source revision; i.e.
"git.yoctoproject.org.linux-yocto-4.4.git.89419d8b90_dadb436904.tar.gz".
This change set the tar name to $PF.tar.gz instead, to have consistency with
the others archives created by the class.
(From OE-Core rev: 3f903cb767150e316337929d72559cad6931039a)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
