If auto.conf exists in the user's configuration we need to also run it
through the same filter and write the result into the ext SDK, or we
risk missing configuration applied on an autobuilder.
Fixes [YOCTO #8904].
(From OE-Core rev: fb0a34a00f3eb10935a3fca4d0af74ae4de4f9a5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there is no newline at the end of local.conf, appending
INHERIT = "uninative" won't work, it will corrupt the line and the
installed eSDK will build things, making the "Preparing build system..."
step take an age.
Fixes [YOCTO #8897].
(From OE-Core rev: 1e641f09e8cf0ba716c3d2aade0845abecc27827)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
populate_sdk_ext requires uninative support, which is only available on
glibc based SDKMACHINES. For instance, when using mingw32 a dependency
error will occur:
NOTE: Runtime target 'nativesdk-glibc' is unbuildable, removing...
ERROR: Required build target 'core-image-minimal' has no buildable providers.
Missing or unbuildable dependency chain was: ['core-image-minimal', 'uninative-tarball', 'nativesdk-glibc']
This is dues to populate_sdk_ext.bbclass having:
do_populate_sdk_ext[depends] += "buildtools-tarball:do_populate_sdk uninative-tarball:do_populate_sdk"
addtask populate_sdk_ext
Since bitbake can't determine for dependency resolution if the task is going
to be run yet, it blows up and says it simply can't be resolved.
Workaround this problem by making the inherit conditional on the SDK_OS
containing 'linux'.
(From OE-Core rev: e471ce3464d5ae024315d4839cccd4c651f9ba83)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This tells APT that it can use such packages to resolve dependencies
from packages of any architecture in a multilib build.
(From OE-Core rev: 7158c79a70e3d820c9701dacfa7206d13f95845a)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Have DPKG_ARCH set by directly invoking a mapping function, rather
than using an anonymous Python function modify the variable under
the hood, so we can have proper handling of overrides.
Also bring in some additional mappings to Debian architecture names
that weren't being handled.
(From OE-Core rev: 8d042ea4e755cb0bb28b88333e10e04ec4e86a36)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
prune_lockedsigs expects excluded_targets to be a list, whereas
previously it was passed in as a string.
(From OE-Core rev: db3cd1e08b08b99342d269882f31ec7e1daba2c6)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Right now, the locked task hashes list for the extensible SDK locks
down only the sstate tasks.
Whilst asthetically pleasing, this gives two problems:
* Half the task are left floating meaning checksum mismatches
are a pain to debug
* The later code which copies relavent data files out the sstate
cache can't use any of this data.
This patch modifies things so all the checksums are listed in the locked
file. An exclusion of tasks probably makes more sense for the library
function rather than an allowed list.
The only sstate task being deliberaly excluded here was do_package
so add in a function to explictly exclude those sstate object files.
The net result of this that siginfo files for all tasks are included in
the SDK, which means commands like "bitbake -S printdiff" now function.
(From OE-Core rev: 6b70479e47b8a8743d8b410d6bc08da1607a318e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
xz gives *much* faster decompression times for the SDK which in itself
is a good reason to use it. It also gives better compression.
One downside is its slower but we care about the end user case
first, build performance secondary.
It also assumes the SDK user has a tar capable of understanding a xz
compressed file but that should be common enough now.
(From OE-Core rev: 193086137b899a0a4e774ffc337ed0da9947fd4f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After the separation of do_rootfs, some rootfs references need changing
to image_complete.
(From OE-Core rev: 59a5f596ca29b1eb8283706e3c60fbb39f9c2c23)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch splits the code in lib/oe/image into separate tasks, one per
image type. This removes the need for the simple task graph code and defers
to the bitbake task management code to handle this instead.
This is a good step forward in splitting up the monolithic code and starting
to make it more accessible to people.
It should also make it easier for people to hook in other tasks and processes
into the rootfs code.
Incidentally, the reason this code was all combined originally was due to
limitations of fakeroot where if you exited the session, you lost permissions
data. With pseudo this constraint was removed.
We did start to rework the rootfs/image code previously and got so far with
untangling it however we did prioritise some performance tweaks over splitting
into separate tasks and in hindsight, this was a mistake and should have been done
the other way around. That work was suspended due to changes in the people working
on the project but this split has always been intended, now is the time to finish
it IMO.
There were some side effects of doing this:
* The symlink for the manifest moves to the rootfs-postcommands class and into
the manifest function.
* There is no seperate "symlink removal" and "symlink creation", they are merged
* The date/time stamps of the manifest and the built images can now be different since
the tasks can be run separately and the datetime stamp will then be different
between do_rootfs and the do_image_* tasks.
(From OE-Core rev: c2dab181c1cdabac3be6197f4b9ea4235cbbc140)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As the next step in splitting up do_image, move the pre and post processing
commands to separate tasks. This also creates the do_image_complete task
which acts as the end marker task for image generation.
(From OE-Core rev: 800528eaa421d451b596545125cb218e08989151)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I've heard complaints from people trying to create more interesting image
types about how hard it is to understand the rootfs/image generation code
and that its a pain to develop/test/debug.
Having looked at it myself, the internal construction of shell functions which
then gets passed into a multiprocessing pool is rather convoluted and it places
rather odd constraints on when variables are expanded. Its therefore no wonder
people find it confusing/complex.
This patch starts the process of splitting this up by separating out image
generation from the do_rootfs task into a new do_image task.
(From OE-Core rev: 57578d0ca6c3aaf6edf0af2c4862d43c97415156)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
During extensible SDK installtion, if the build system preparation step
fails we try to put something at the end of the environment setup script
to show an error when it is sourced, in case the user doesn't realise
that the partially-installed SDK is broken. However, an apostrophe in
the message (actually a single quote) appears to terminate the string
and therefore breaks the command. Drop it to avoid that.
(From OE-Core rev: 21e591d182e24c399ae010a8eff9b89947061a46)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was supposed to be removed by a previous patch but was readded.
Really remove it.
(From OE-Core rev: 5661d8cb7849df62358368743134c0aaf523965e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The procces to do a wic image is to save a file with
variables required by wic and then call wic using this
file. Because this is external to bitbake if the vars
change, the image won't be rebuild; an example of such
is IMAGE_BOOT_FILES.
This patch adds these variables to vardeps of do_rootfs
when a wic image is build. This will rebuild the image
if a variable needed by wic changes.
[YOCTO #8693]
(From OE-Core rev: 12c54d50ed4c321dc272beb3c6cb770965c979f1)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that x11vnc won't be in Sato images we don't want to test it.
(From OE-Core rev: 90ff7219332d9651fcdfc895d61fb73a46ffde78)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I was using bytes here for the sizes and writing out KiB as the suffix.
Change it to actually write out size values in KiB.
(From OE-Core rev: e6b1840ee9aeb933f86fd2a92a90b94c1bc9db7c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sometimes it's hard to see what's wrong here
* error message:
Error, the PACKAGE_ARCHS variable does not contain TUNE_PKGARCH (cortexa7t2hfhf-vfp-vfpv4-neon).
doesn't help much to understand what exactly went wrong and where
(From OE-Core rev: 4bedd149964b3b594d7b1a233a9cc436925288e9)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Qt4 is no longer supportd upstream, but it is still needed for LSB compliancy.
Qt4 recipes have been moved to a separate meta-qt4 layer to be consistent with
meta-qt3 and meta-qt5.
(From OE-Core rev: cb89d2b25b4edb1241bc5426a69a6bc44df9be2c)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Various prefuncs and flags and addtask statements make sense to belong together
in one clearer function now, this patch cleans things up a bit.
(From OE-Core rev: eae0cf7875197f9520be54370bc670e27338aad3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reading image.bbclass is a little difficult as it has many post rootfs
helper functions and its hard to separate those from the core contents
of the rootfs/image code.
Moving it to a separate class would be one way of making it clearer
what these functions are. There are some comment layout improvements
but no code changes.
(From OE-Core rev: df4cb51c8e60fa46d4d15be8da3d84287ff08ae7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was added in aa3141e979 back in 2008
but I don't see why multiple images would need this now.
(From OE-Core rev: 2eaeac6d98c310cb2602ba194d934c5b7bed253d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Historical releases may disappear from the main mirrors, so
then we need to fall back to the archive (which, in turn, may
not carry the current release, and so can't be the primary mirror).
(From OE-Core rev: 2eb171a1c9f1d2ce37965aad7b3f470a2c44ed83)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Ensure extensible SDK and standard SDK go into their own directories
* Record extra extensible SDK variables
* Write sstate-package-sizes.txt and sstate-task-sizes.txt files so you
can analyse the size of the contents
* Add BUILDHISTORY_SDK_FILES (similar to BUILDHISTORY_IMAGE_FILES) and
default it to pick up config files installed in the extensible SDK
(From OE-Core rev: 5f87ff116d9ccd5e95e638dfa84583169a3f99f3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After OE-Core revision baa4e43a29e45df17eaa3456acc179b08d571db6 we lost
recording SDK the contents in buildhistory. This was due to the
SDK_POSTPROCESS_COMMAND variable being set with = in
populate_sdk_base.bbclass which overwrote any value set with += in
buildhistory.bbclass; to fix it, use _append in buildhistory.bbclass
instead.
Fixes [YOCTO #8839].
(From OE-Core rev: 11d1aa82ef4a00051e0a50a87a1efed1c50c73b5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The existing code doesn't tell regular (with .git) and bare cases and
just move the unpacked repo to the place of kernel source. But later
steps will fail on a bare-cloned repo because we can not checkout
directly in a bare cloned repo.
This change performs another clone to fix the issue.
Note: This change doesn't cover the case that S and WORKDIR are same
and the repo is bare cloned.
(From OE-Core rev: ccfa2ee5c4f509de4c18a7054b2a66fc874d5d69)
Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the installation of buildtools fails then we should fail the entire
installation instead of blindly continuing on.
(From OE-Core rev: 34bb63e6c72fb862e0ef0d2b26e1bfddaf7ddb99)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The configuration of the build system within the extensible SDK is
fixed, so there's not a lot of point in showing it; plus it just gets in
the way of the output that's interesting to the user in this context. So
let's hide it within the extensible SDK.
(From OE-Core rev: 6223b73c2dfc326382dbd3c0ae2d7a0cf2cc3b48)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we don't want a header printed at the start of task execution (which
we'd prefer not to within the extensible SDK) we can accomplish that by
clearing BUILDCFG_VARS and BUILDCFG_HEADER, but that was still printing
a load of blank lines. To keep things simple, check if BUILDCFG_HEADER
is set to something before printing a header.
(From OE-Core rev: d896d77c1d0fc56ff7019bfee3cf06fbc3ede8f3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We inherit uninative in the extensible SDK configuration, and uninative
sets NATIVELSBSTRING to a fixed value, so we don't need to force the
value ourselves.
Fixes [YOCTO #8662].
(From OE-Core rev: 918814c05d670bccb05d61fa848fd0d93da3a7b0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reading IO stats fails because the IO read/write bytes are
being converted to strings, then added to a numeric running
total.
Fix this by converting IO stats to integers.
(From OE-Core rev: 8e2475eecafc0161d25684f5b8239273739de759)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a check like what we does in package.bbclass
so that the already-stripped QA test can be skipped.
(From OE-Core rev: 2262fdb256954b22dadb2f7c6922e6046c269742)
Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The buildstats changes removed the no longer needed get_bn() function,
replace this with references to BUILDNAME.
(From OE-Core rev: e1a37899da56014693f08d1c39cb6ec0a4ed2bf4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This updates the toaster class to use the new data format from buildstats.
This does mean it will no longer read IO data from older builds, however
since that data is completely useless anyway, I don't consider that to be
an issue.
(From OE-Core rev: 5c3987226cd9669d5fc0993d0541048475af9fcb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add IO stats and getrusage() data to the task statistics. We
also drop the CPU percentage calculation since its pretty arbitrary
and not very accurate/useful.
In particular we can now see the user and sys times as well as the
wall clock times.
(From OE-Core rev: b849130f71d3ba32a6fa94c291ca6ce7c7c3b3d1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The combined build and task data code makes changing things hard, separate
out the functions so that changes can be made to the task data whilst the
build data remains unchanged.
(From OE-Core rev: c79cfce4f820f20346d0565df8df626832976e28)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than bb.data and e.data, cleanup to use 'd' and match the
standard coding style.
(From OE-Core rev: dbce2c6122d492ba86873db4d02322f0df7a6752)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current setting and getting of the "name" to use for buildstats is
convoluted and not particularly interesting. We only need this for the
e.getPkgs()[0] component of the path which is the first target listed
on the commandline. This is pretty arbitrary.
If we drop that piece, we can assume BUILDNAME is common for all events
and simply use this and query it. If BUILDNAME did change, that would
be a bug and it should be fixed elsewhere.
Also take the opportunity to share some common code since the function
now has the eventmask.
(From OE-Core rev: 918d83460639df273f38ae079ffeebd6a79b3373)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The existing diskstats data from buildstats simply isn't useful. It
gives stats on the total IO counts on some random disk within the
system. This means that the count includes data from all other tasks
running at the same time and from any other process running on the
system.
I've been unable to find any use for the data so as a start at
reworking the class, remove the related code.
(From OE-Core rev: f9fb02909f332365cad329352956a29cff6eba77)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously UNINATIVE_LOADER was always ld-linux-x86-64.so.2. That is
incorrect when the host is 32-bit.
This change also changes to using ?= so the user can override
UNINATIVE_LOADER if so desired.
[YOCTO #8124]
(From OE-Core rev: b78fa0bcadd54bb29b6f1bb3a9308d4c454bf4e2)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We shouldn't try and expand what is a python dict object
and we don't want it to influence the task hashes, task
dependencies are already accounted for.
(From OE-Core rev: 697072130a3298aede7166f125aa982be23793ca)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the autodebug package generation logic, specifically setting FILES_${PN}-dbg
isn't needed in most cases, we can remove them.
(From OE-Core rev: 3ab59d49dd7c18e194b58d1248b4b87709b5a738)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Creating FILES_${PN}-dbg is tedious and also pretty pointless. We might as well
assume ".debug" is a special directory name and split into -dbg automatically.
This change does so without changing the rest of the splitting logic too much.
It can be disabled for the cases where we really do want manual control of
the -dbg packages (e.g. qt4) with NOAUTOPACKAGEDEBUG = "1".
(From OE-Core rev: da5ec06814e105451cca11cce76b5c5231110524)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It turns out writing the same list of packaging tasks multiple
times in multiple places is error prone. Move this to a new class
'nopackages", migrate existing users and add glibc-initial and
libgcc-initial since we don't want packages for those recipes.
This means the sstate for those recipes won't be installed, saving
small amounts of build time and bandwidth.
A reference to the old package_write task is also dropped.
(From OE-Core rev: cece583d58f82a50c3a4eac876eb326ac3b8f5e5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was observed that do_fetch dependencies (e.g. subversion-native of tremor)
were being installed even when sstate was available and hence no fetch was
needed. This turned out to be due to the recursive nature of the rootfs
dependencies which include populate_lic.
We can explicitly whitelist these dependencies as being ones we don't
need to follow when installing sstate (the code defaults to being paranoid).
[YOCTO #8746]
(From OE-Core rev: 06c891f0fa6c7712eae233dd18612f1ab1889c45)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should always pass a parameter to getVar, add missing default value.
(From OE-Core rev: 31bc0a46a97d7dc98568a218c077c31d8b11dbd9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
archive.apache.org does not contain current releases, only historical ones,
so upstream checks aren't accurate. It's replaced with official mirrors
containing current versions.
(From OE-Core rev: e9c85489ae354f52ff5b78f4d7fb6fafd0050522)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
