Write a list of installed packages to a .manifest file next to the
image, so we can find out what went into the image after it has been
constructed without necessarily having to have buildhistory enabled
(although that will provide more detail.) We can make use of this for
example in the testimage class associated code that checks for installed
packages for determining whether or not to run specific tests.
Note: this replaces the previous ipk-specific manifest code with
something that works for ipk, rpm and deb, and instead of a pruned
status file, packages are listed one per line, in the following format:
<packagename> <packagearch> <version>
Tests for all three backends have shown that the performance impact of
this change is negligible (about 1.5s max).
Implements [YOCTO #5410]
(From OE-Core rev: 2978d1f2617a33e2e3a77e249d73e998d79b4ec9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
check_create_long_filename used a fixed filename for its test files. This
meant that os.remove(testfile) could fail with ENOENT if two instances were
running at the same time against the same sstate directory. Using a
randomly generated filename stops this from happening.
(Although it might seem unlikely, this race did appear to occur multiple
times with Jenkins - presumably because the matrix jobs were all kicked off
at the same time.)
(From OE-Core rev: bc28e3f26e7f85af82f403924c0ae29e1ad34a87)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Although ENAMETOOLONG is 36 on Linux x86 and x86_64 it does isn't on other
architectures so the value shouldn't be hard coded.
(From OE-Core rev: 11a9cf5ee0daf82097fb2f36b58016f20a5968f3)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids crashing on newer svn layouts where the entries files
don't contain three lines. If someone wants to fix this to
get the right version on newer subversion checkouts, patches
welcome but this at least stops things crashing.
[YOCTO #5363]
(From OE-Core rev: e850c53d4d8cb877a704a23f9ce02d6185ba3ffa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new do_bundle_initramfs task introduced in
609d5a9ab9e58bb1c2bcc2145399fbc8b701b85a defeats using the sstate
cache. The kernel is resurrected from the sstate cache but ends up being
built again since do_bundle_initramfs depends on do_compile.
The task is no longer nostamp to avoid causing unnecessary rebuilds. The
sstate checksum stamps should know when to rebuild.
The task now runs before do_deploy and part of the work has been moved to
do_deploy where it now writes to ${DEPLOYDIR} rather than
${DEPLOY_DIR_IMAGE} so that the files end up in sstate.
The task can also race against do_install since both call into the kernel
build system. This is fixed by making do_bundle_initramfs run after
do_install (which therefore also fixes the problem that
3baa63b4d588c3262254528b406ede265dd117bf was addressing.)
(From OE-Core rev: 55989cb509340bd265d0ce0d8bfe849681be4616)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 3baa63b4d588c3262254528b406ede265dd117bf. It broke
builds that aren't using kernel-yocto.
(From OE-Core rev: 81831db1c32afa3346f3ed9f4325ad280e5bb005)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When preping a read-only rootfs and finding some post-install
scripts that can not be run, list the names of said scripts to
avoid having to look around the rootfs to find a list.
(From OE-Core rev: 0188120691f433fdccf71b92618115195278c0af)
Signed-off-by: Jeffrey C Honig <jeffrey.honig@windriver.com>
Signed-off-by: Jeff Polk <jeff.polk@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On a multilib system when one of the multibs has a different OS then
other multilibs a failure can occur during the install process because
RPM assumes all systems have the same OS.
When an n32 platform is selected as an alternative multilib, it shows
up as mips64_n32-.*-linux-gnun32 in /etc/rpm/platform. This causes
problems when the smart tool tries to add a channel for the multilib.
RPM archScore call always returns zero for arch "mips64_n32" -
after appending default vendor and os, it finds "mips64_n32-wrs-linux"
doesn't match any predefined platforms. Fix this by removing the
restriction of -gnun32 suffix in platform file.
(From OE-Core rev: d9489c44ee4f195ae1b09f340b9545cddba58145)
Signed-off-by: Lei Liu <lei.liu2@windriver.com>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Jeff Polk <jeff.polk@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PREFERRED_PROVIDER_virtual_pkg has been incorrectly set with more
than one multilib prefixes. For example, if we have two alternative
multilibs lib64 and lib32, PREFERRED_PROVIDER_virtual_pkg will be
set to lib32-lib64-pkg or lib64-lib32-pkg, depending on which
multilib shows up first in the list.
(From OE-Core rev: 17a432dc059e24ba10d4baec988828c0025a5e46)
Signed-off-by: Lei Liu <lei.liu2@windriver.com>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The buildhistory code has fallback paths for older bitbakes for now. The
distrodata class is much less used and it can be assumed a recent bitbake
is used in that case rather than adding fallback code.
(From OE-Core rev: 570cc145029fd9d5528aef5c27cb65164265c799)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The license_create_manifest function contains bashism, this will lead
to unexpected results on ubuntu build host, as sh is linked to dash on
ubuntu. Even if COPY_LIC_MANIFEST and COPY_LIC_DIRS are enabled, the
license files will still be missing on target.
This patch fixes the above problem.
[YOCTO #5549]
(From OE-Core rev: 4df9daee5c732c0a20dabe8515577238a1508512)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
${KERNEL_OUTPUT} is being renamed/restored in bundle_initramfs task, so we
must ensure bundle_initramfs run after kernel_link_vmlinux where the link
of vmlinux is created as the bootable image.
(From OE-Core rev: 3baa63b4d588c3262254528b406ede265dd117bf)
Signed-off-by: Ming Liu <ming.liu@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If kernel_do_compile() encounters a .lzma image for the initramfs, it
attempts to decompress it using the little-known "lzmash" utility. This
may not be present, which will cause the build to fail.
Fortunately, it appears that the more mainstream "lzma" tool is
compatible so we can use that one instead.
(From OE-Core rev: f7a04cbdee67675ff6a8787709f3a312dc25bfec)
Signed-off-by: Phil Blundell <philb@gnu.org>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The icecc class often calls 'which' for determining paths. This leads
to many messages on stderr in case 'which' doesn't find the
executable. Using bb.utils.which is more appropriate here and doesn't
pollute stderr.
(From OE-Core rev: 7ed7aa38182ce8c7300a08e3aefcc65da2b524a8)
Signed-off-by: Tobias Henkel <tobias.henkel@bmw-carit.de>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The existing code in checkpkg from distrodata.bbclass had similar
functionality with fetch when searching for latest package version.
For packages that use svn protocol that part was rewrote in order
to use fetcher API.
It now calls latest_revision method from Svn class in fetch2 so
that it gets latest version.
[ YOCTO #1813 ]
(From OE-Core rev: 2686b92bbab88cc777fdc0e4dded5aeabca7ac77)
Signed-off-by: Irina Patru <irina.patru@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, it supported the situation that /boot area with separate boot
partition:
...
menuentry "Update bzImage-3.10.10-WR6.0.0.0_standard-3.10" {
set root=(hd0,1)
linux /bzImage-3.10.10-WR6.0.0.0_standard root=/dev/sdb1 rw ip=dhcp
}
...
But didn't consider the situation that /boot within root partition:
...
menuentry "Update bzImage-3.10.10-WR6.0.0.0_standard-3.10" {
set root=(hd0,1)
linux /boot/bzImage-3.10.10-WR6.0.0.0_standard root=/dev/sdb1 rw ip=dhcp
}
...
This fix supported them both.
[YOCTO #5514]
(From OE-Core rev: 53d342db9f9995564573715f28c3e4c9c8c68bf9)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The commit [kernel.bbclass: test for hardlinks before installing] doesn't
work on all build machines without a space between the ! and the test for
hardlink equivalance. The build continues, while the test fails and no
copy is made into the sysroot. Eventually tasks like build_hdimg will fail
with the missing kernel with the root cause being less than obvious.
(From OE-Core rev: 0704648e9dfd2897f6f54773a0e7d2762117970c)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
commit df564c4 [kernel: Use hardlinks for do_populate_sysroot for speed]
does indeed speed up the build, but it also means that the KERNEL_OUPUT
and KERNEL_IMAGETYPE may be hardlinks.
If they are hardlinks, install complains that they are actually the same
file and stops the build. We can easily test and avoid the copy if the
file is already in place.
[YOCTO #5527]
(From OE-Core rev: 6a3b3e501c66733216ac43d3f020cfdb512dd640)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The expected usage of COPY_LIC_MANIFEST and COPY_LIC_DIRS appears to be
to set them to "1" to enable; however the test here is just testing
whether they have a value at all, so setting them to "0" would also
enable them which is somewhat disingenuous. Actually check if they are
set to "1" instead in order to fix this.
(From OE-Core rev: 2d8e124adcf27af524eeeae61daf1b21a1c2f27c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is very old, and we now have pkgdata which is much more complete.
Nobody appears to be using this class, and even OE-Classic had no
current references to it.
(From OE-Core rev: e67dbb638044b804738bdd589d64d45963a3297a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This appears to be a very old way of archiving recipes and associated
files. We have better ways of doing this now, and nobody appears to be
using this class - even OE-Classic had no current references to it.
(From OE-Core rev: e3141844ef84d8a48efa81e0c9c85821ced16f7c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If SDKMACHINE is set then check that a configuration file matching it
actually exists, otherwise the user won't know that they've set it
incorrectly.
(From OE-Core rev: 8c984f92af821a4048c93f8e308c5f4a3fa39ca4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* If the md5 parameter is specified with no value, report that the
checksum is not specified instead of reporting that it has changed
* If the md5 checksum has changed, point directly to the license file in
a way that is easy to copy and paste and give the line numbers in an
easy to read form, as well as asking the user to verify that the new
contents matches the current LICENSE value.
(From OE-Core rev: ec8590aa81e201e28e500935d31cd7266114471f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't actually need this dummy class; "inherit" can be used with an
expression that evaluates to nothing with current BitBake.
(From OE-Core rev: f312eb2d2a2715e772ed9e8afc7aea326d8079ec)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This saves about 2GB on a core-image-sato build so is worth doing and is
consistent with our efforts to try and decrease our build footprint.
Build time in my local test seemed unaffected but on more IO bound
machines it should help.
(From OE-Core rev: 7136cbc64e5efb09f3fae3e2e35a181ca3d66dd4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bb.fetch.decodeurl() will throw if it doesn't like the look of the URL that
it's given. (Bitbake's idea of what constitutes a valid URL is somewhat
idiosyncratic so it is fairly easy to trip over this by mistake when writing
a recipe.)
If these exceptions are allowed to propagate all the way up to better_exec()
then we will get a large amount of python stack trace spew when they are
finally caught. Avoid that by catching them locally and throwing
bb.build.FuncFailed() with a suitable explanation instead.
(From OE-Core rev: ef35e164c62d89806367b822e3baeff482ec237f)
Signed-off-by: Phil Blundell <philb@gnu.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An empty distro value leads to OVERRIDES and FILESOVERRIDES containing
"::" entries which causes odd issues such as files being included when
they shouldn't be. We could put in anonymous python to guard against
empty entries but its messy and setting a default value for DISTRO to
something harmless is much easier.
This patch adds a weak default and ensures the sanity test doesn't
complain about it.
DISTRO_VERSION and SDK_VERSION are also updated to match.
(From OE-Core rev: b7279f99639774674da806d37d252f388f33055f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the current implementation there can be a race condition while
creating the toolchain archive causing the build to break.
This is fixed by locking the toolchain archiving step using flock.
(From OE-Core rev: 8a30be803e91e66688cfc27ca4c21f26fb22eed8)
Signed-off-by: Tobias Henkel <tobias.henkel@bmw-carit.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are some recipes which parse the PARALLEL_MAKE variable by their
own and set them to an empty string afterwards. This disables icecc
for this recipe.
Adding a whitelist for forcing icecc makes it possible to use icecc
also with these recipes.
(From OE-Core rev: d2735ac44887c7e01134d6870a4875a786501eba)
Signed-off-by: Tobias Henkel <tobias.henkel@bmw-carit.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the current implementation a KERNEL_CC variable containing shell
evaluation breaks the build process. Shell expansion is not happening
before general expansion in get_cross_kernel_cc which results in a
syntax error and an aborted parse process.
Before expanding the KERNEL_CC variable get_cross_kernel_cc now checks
for backticks or '$(' in the KERNEL_CC variable and performs a shell
evaluation using a call to echo if it finds one.
(From OE-Core rev: b28bae30fc5d8d1d7cc675ddb4159c39fb9bc3fd)
Signed-off-by: Tobias Henkel <tobias.henkel@bmw-carit.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some packages are calling the compiler in the install step. In this
case either the build breaks or icecc is not used for building. The
proper environment has to be set to enable icecc based building.
(From OE-Core rev: bb1366cebb60593fc21fd7a9a678a159da8ec81c)
Signed-off-by: Tobias Henkel <tobias.henkel@bmw-carit.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbakes prepend mechanism for the tasks disregards the type of the
function. Thus bitbaking recipes using python functions for configure,
compile or install steps fail due to the missing python version of
set_icecc_env.
Assuming that icecc doesn't need to be used in such situations adding
a dummy python version of set_icecc_env fixes this.
(From OE-Core rev: 365dec500166b8eb9d64c573dd7139d3a26ae445)
Signed-off-by: Tobias Henkel <tobias.henkel@bmw-carit.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changing any of the ICECC blacklist variables should not change the
sstate checksum as this doesn't influence the build result.
(From OE-Core rev: 0f7dc969e64bc7563e208a4fb7d3ebe7c4e69e79)
Signed-off-by: Tobias Henkel <tobias.henkel@bmw-carit.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the icecc class prints a note for every package which
disables parallel make at parse time. This is unneccessary as many
packages don't support parallel building. Changing the log level from
info to debug hides these messages in normal builds without removing
the information when needed.
(From OE-Core rev: 6d2735629c20560a9406964195726b1a6e2d7d99)
Signed-off-by: Tobias Henkel <tobias.henkel@bmw-carit.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current implementation doesn't give a hint about the cause in case
something went wrong in set_icecc_env. This makes it harder to find
out why a package is not being built using icecc. Therefore warnings
are inserted in the various error cases.
(From OE-Core rev: 0adea170c7019304471d442784fa0eb9bfcdf94b)
Signed-off-by: Tobias Henkel <tobias.henkel@bmw-carit.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
File command in some version could print extra space between
"LSB" and "executable" - it causes mklibs can't find any executables
using grep "LSB executable". Fix the grep pattern to catch
multiple spaces.
(From OE-Core rev: a52ef8c5dcd71f39bb48c71fb868cc0db662560e)
Signed-off-by: Lei Liu <lei.liu2@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changing APPEND wasn't causing syslinux to re-run, so add a manual dependency.
(From OE-Core rev: 3c2d7ae5d59cb76a838f227c4cb0c64cea0d9e03)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The kernel tree is large and doesn't need to be copied. Override
the default sysroot handling function to use a hardlink copying
function in python.
This commit also drops the copying of the /lib directory which
just contains the kernel modules. We never use those in the sysroot
so there is little point in carrying those around.
For linux-yocto this takes the do_populate_sysroot time 24s -> 14s.
(From OE-Core rev: 13259459e200a237ca486cbe1123a0b0a4d1eebf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whilst the comment says we can't do this, its incorrect, install operates
on WORKDIR and its hard for different parts of WORKDIR to be on different
filesystems. Hardlinking instead of copying is therefore a nice performance
gain.
Also, completely skip the Documentation directory (adding a dummy Makefile
to keep make happy) and tweak the other cp commands to link since if we
don't we'd get "this is the same file" type errors from cp for some kernel
versions.
For do_install on linux-yocto, this takes it from 227s -> 84s.
(From OE-Core rev: 0727e510f8533216518563b1533e804a1dc44402)
(From OE-Core rev: f6e98af6b9f6729bb61f17e3b4203c3c79829205)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There was still duplicated code in toolchain-scripts, this further cleans
up the functions to remove it. The now unused includedir parameter is also
dropped.
The final scripts do end up reordered slightly and the sysroot is parametrised
for the IDE scripts which is an improvement but should have no functional
difference.
(From OE-Core rev: 1dbd65b4d14319e784a66776c1e9943d0179d3ee)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you Ctrl+C ppackage_write_{deb/ipk} control files can get left lying around
and make it into another packaging format. This ensures we cleanup all known
control files before starting packaging. We can simplify some of the globbing
as a result.
(From OE-Core rev: 7f8c728c82a17621461be2114c1afab3713a808e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if multiple package tasks are running against a recipe, package_rpm
is restricted to the slowest speed of them due to the locking. This patch
explicitly ignores the opkg/debian artefacts and hence allows a speedup.
It also removes an issue were a Ctrl+C interrupting a deb.ipk packaging task
would end up with CONTROL/DEBIAN files in the spec file resulting in a build
failure.
(From OE-Core rev: 4c3b3a57aa9ced231f5c0340920195c15a26f2b1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code was clearly broken and isn't used, even by meta-darwin so we
might as well remove it.
(From OE-Core rev: b9d51bfe0d4821f5de75085ba8af732c46de3328)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having two scripts which do basically the same thing is a nightmare. This merges
them together. It also makes the sysroot location a variable in its own right
which may be more useful for end users wanting to change sysroot.
(From OE-Core rev: 213e0c67e938b802e6bb2246f00b7343a6f77b99)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
