For a bare-bone kernel recipe which specifies 32 bit x86 target,
a 64 bit .config will be generated from do_configure task when
building 32-bit qemux86, once all of these conditions are true:
* arch of host is x86_64
* kernel source tree used in build has commit ffee0de41 which
actually chooses i386 or x86_64 defconfig by asking host when
ARCH is "x86" (arch/x86/Makefile)
* bare-bone kernel recipe inherits directly from kernel without
other special treatments.
Build will fail because of the mismatched kernel architecture.
The patch sets ARCH i386 or x86_64 explicitly to configure
task to avoid this host contamination. Kernel artifact is also
changed so that it can map i386 and x64 back to arch/x86 when
needed.
(From OE-Core rev: 8d310b24927d0f348fb431895f0583733db2aad0)
Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
IMAGE_TYPES_MASKED support was accidentally removed. The original
idea behind it was to remove some of the hardcoding in the core
image code, so do that for image-live and ensure the dependency
and masked variables correctly reflect the needs of the class.
This means we can remove all the hardcoded special cases since
image-vm already has the needed markup.
(From OE-Core rev: 9a2d4a3b8d7bb1cf7f1fb7fe47d5c002d9941c89)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a base image type has an implicit dependency (from IMAGE_TYPEDEP)
this has to be taken into account. This is a regression introduced by
OE-Core:c2dab18 (image: Create separate tasks for rootfs construction).
The issue has been found when building meta-fsl-arm based images which
does not include the rootfs image type explicitly in IMAGE_FSTYPES but
instead is added, using IMAGE_TYPEDEP, for the 'sdcard.gz' image.
Reported-by: Fabio Berton <fabio.berton@ossystems.com.br>
(From OE-Core rev: 191c7be3a6cc52911f244323072433f6a1172bf1)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a separate task where we can just have the recursive dependencies
for the tasks corresponding to constructing the SDK content (i.e. from
the image contents). This avoids us recursing into dependencies from
buildtools and getting a bunch of nativesdk stuff, for example.
(This isn't an ideal way to have to implement it, but without
overcomplicating things on the BitBake side just for this use-case I
can't see a better way.)
(From OE-Core rev: eabeb26335b1a4eb1e68218160dbdbe8fdf36272)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using package_qa_handle_error(), we need to check QA_SANE and
give a fatal error if issues were raised. The main insane checks do
this, the anonymous python in insane does not, nor does the code in
package.bbclass. This was likely missed when the function was
introduced to package.bbclass.
The impact is that errors can be shown but don't stop the build
(do_package succeeds), the initial build fails due to the errors
shown but subsequent builds succeed. This is clearly broken.
The solution is to check QA_SANE in places we use the handle_error
function.
[YOCTO #8907]
(From OE-Core rev: 7e36d13212763fe409747aba011c020c9aeba6d7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After the change to use --setscene-only when running bitbake to prepare
the SDK at the end of installation, add a check that the SDK got
prepared correctly by doing a dry-run and looking at the output for any
real tasks that we don't expect. In order to make this easier, the
preparation shell script was rewritten in python.
(From OE-Core rev: 2306683634435b990e63020fc5cf91753bbaf7b6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These form part of the configuration for the extensible SDK, we should
really be recording what goes into them.
(From OE-Core rev: bd1557bfd726e44c23ab6220867119c57ac6b596)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If auto.conf exists in the user's configuration we need to also run it
through the same filter and write the result into the ext SDK, or we
risk missing configuration applied on an autobuilder.
Fixes [YOCTO #8904].
(From OE-Core rev: fb0a34a00f3eb10935a3fca4d0af74ae4de4f9a5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there is no newline at the end of local.conf, appending
INHERIT = "uninative" won't work, it will corrupt the line and the
installed eSDK will build things, making the "Preparing build system..."
step take an age.
Fixes [YOCTO #8897].
(From OE-Core rev: 1e641f09e8cf0ba716c3d2aade0845abecc27827)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
populate_sdk_ext requires uninative support, which is only available on
glibc based SDKMACHINES. For instance, when using mingw32 a dependency
error will occur:
NOTE: Runtime target 'nativesdk-glibc' is unbuildable, removing...
ERROR: Required build target 'core-image-minimal' has no buildable providers.
Missing or unbuildable dependency chain was: ['core-image-minimal', 'uninative-tarball', 'nativesdk-glibc']
This is dues to populate_sdk_ext.bbclass having:
do_populate_sdk_ext[depends] += "buildtools-tarball:do_populate_sdk uninative-tarball:do_populate_sdk"
addtask populate_sdk_ext
Since bitbake can't determine for dependency resolution if the task is going
to be run yet, it blows up and says it simply can't be resolved.
Workaround this problem by making the inherit conditional on the SDK_OS
containing 'linux'.
(From OE-Core rev: e471ce3464d5ae024315d4839cccd4c651f9ba83)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This tells APT that it can use such packages to resolve dependencies
from packages of any architecture in a multilib build.
(From OE-Core rev: 7158c79a70e3d820c9701dacfa7206d13f95845a)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Have DPKG_ARCH set by directly invoking a mapping function, rather
than using an anonymous Python function modify the variable under
the hood, so we can have proper handling of overrides.
Also bring in some additional mappings to Debian architecture names
that weren't being handled.
(From OE-Core rev: 8d042ea4e755cb0bb28b88333e10e04ec4e86a36)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
prune_lockedsigs expects excluded_targets to be a list, whereas
previously it was passed in as a string.
(From OE-Core rev: db3cd1e08b08b99342d269882f31ec7e1daba2c6)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Right now, the locked task hashes list for the extensible SDK locks
down only the sstate tasks.
Whilst asthetically pleasing, this gives two problems:
* Half the task are left floating meaning checksum mismatches
are a pain to debug
* The later code which copies relavent data files out the sstate
cache can't use any of this data.
This patch modifies things so all the checksums are listed in the locked
file. An exclusion of tasks probably makes more sense for the library
function rather than an allowed list.
The only sstate task being deliberaly excluded here was do_package
so add in a function to explictly exclude those sstate object files.
The net result of this that siginfo files for all tasks are included in
the SDK, which means commands like "bitbake -S printdiff" now function.
(From OE-Core rev: 6b70479e47b8a8743d8b410d6bc08da1607a318e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
xz gives *much* faster decompression times for the SDK which in itself
is a good reason to use it. It also gives better compression.
One downside is its slower but we care about the end user case
first, build performance secondary.
It also assumes the SDK user has a tar capable of understanding a xz
compressed file but that should be common enough now.
(From OE-Core rev: 193086137b899a0a4e774ffc337ed0da9947fd4f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After the separation of do_rootfs, some rootfs references need changing
to image_complete.
(From OE-Core rev: 59a5f596ca29b1eb8283706e3c60fbb39f9c2c23)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch splits the code in lib/oe/image into separate tasks, one per
image type. This removes the need for the simple task graph code and defers
to the bitbake task management code to handle this instead.
This is a good step forward in splitting up the monolithic code and starting
to make it more accessible to people.
It should also make it easier for people to hook in other tasks and processes
into the rootfs code.
Incidentally, the reason this code was all combined originally was due to
limitations of fakeroot where if you exited the session, you lost permissions
data. With pseudo this constraint was removed.
We did start to rework the rootfs/image code previously and got so far with
untangling it however we did prioritise some performance tweaks over splitting
into separate tasks and in hindsight, this was a mistake and should have been done
the other way around. That work was suspended due to changes in the people working
on the project but this split has always been intended, now is the time to finish
it IMO.
There were some side effects of doing this:
* The symlink for the manifest moves to the rootfs-postcommands class and into
the manifest function.
* There is no seperate "symlink removal" and "symlink creation", they are merged
* The date/time stamps of the manifest and the built images can now be different since
the tasks can be run separately and the datetime stamp will then be different
between do_rootfs and the do_image_* tasks.
(From OE-Core rev: c2dab181c1cdabac3be6197f4b9ea4235cbbc140)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As the next step in splitting up do_image, move the pre and post processing
commands to separate tasks. This also creates the do_image_complete task
which acts as the end marker task for image generation.
(From OE-Core rev: 800528eaa421d451b596545125cb218e08989151)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I've heard complaints from people trying to create more interesting image
types about how hard it is to understand the rootfs/image generation code
and that its a pain to develop/test/debug.
Having looked at it myself, the internal construction of shell functions which
then gets passed into a multiprocessing pool is rather convoluted and it places
rather odd constraints on when variables are expanded. Its therefore no wonder
people find it confusing/complex.
This patch starts the process of splitting this up by separating out image
generation from the do_rootfs task into a new do_image task.
(From OE-Core rev: 57578d0ca6c3aaf6edf0af2c4862d43c97415156)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
During extensible SDK installtion, if the build system preparation step
fails we try to put something at the end of the environment setup script
to show an error when it is sourced, in case the user doesn't realise
that the partially-installed SDK is broken. However, an apostrophe in
the message (actually a single quote) appears to terminate the string
and therefore breaks the command. Drop it to avoid that.
(From OE-Core rev: 21e591d182e24c399ae010a8eff9b89947061a46)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was supposed to be removed by a previous patch but was readded.
Really remove it.
(From OE-Core rev: 5661d8cb7849df62358368743134c0aaf523965e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The procces to do a wic image is to save a file with
variables required by wic and then call wic using this
file. Because this is external to bitbake if the vars
change, the image won't be rebuild; an example of such
is IMAGE_BOOT_FILES.
This patch adds these variables to vardeps of do_rootfs
when a wic image is build. This will rebuild the image
if a variable needed by wic changes.
[YOCTO #8693]
(From OE-Core rev: 12c54d50ed4c321dc272beb3c6cb770965c979f1)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that x11vnc won't be in Sato images we don't want to test it.
(From OE-Core rev: 90ff7219332d9651fcdfc895d61fb73a46ffde78)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I was using bytes here for the sizes and writing out KiB as the suffix.
Change it to actually write out size values in KiB.
(From OE-Core rev: e6b1840ee9aeb933f86fd2a92a90b94c1bc9db7c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sometimes it's hard to see what's wrong here
* error message:
Error, the PACKAGE_ARCHS variable does not contain TUNE_PKGARCH (cortexa7t2hfhf-vfp-vfpv4-neon).
doesn't help much to understand what exactly went wrong and where
(From OE-Core rev: 4bedd149964b3b594d7b1a233a9cc436925288e9)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Qt4 is no longer supportd upstream, but it is still needed for LSB compliancy.
Qt4 recipes have been moved to a separate meta-qt4 layer to be consistent with
meta-qt3 and meta-qt5.
(From OE-Core rev: cb89d2b25b4edb1241bc5426a69a6bc44df9be2c)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Various prefuncs and flags and addtask statements make sense to belong together
in one clearer function now, this patch cleans things up a bit.
(From OE-Core rev: eae0cf7875197f9520be54370bc670e27338aad3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reading image.bbclass is a little difficult as it has many post rootfs
helper functions and its hard to separate those from the core contents
of the rootfs/image code.
Moving it to a separate class would be one way of making it clearer
what these functions are. There are some comment layout improvements
but no code changes.
(From OE-Core rev: df4cb51c8e60fa46d4d15be8da3d84287ff08ae7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was added in aa3141e979 back in 2008
but I don't see why multiple images would need this now.
(From OE-Core rev: 2eaeac6d98c310cb2602ba194d934c5b7bed253d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Historical releases may disappear from the main mirrors, so
then we need to fall back to the archive (which, in turn, may
not carry the current release, and so can't be the primary mirror).
(From OE-Core rev: 2eb171a1c9f1d2ce37965aad7b3f470a2c44ed83)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Ensure extensible SDK and standard SDK go into their own directories
* Record extra extensible SDK variables
* Write sstate-package-sizes.txt and sstate-task-sizes.txt files so you
can analyse the size of the contents
* Add BUILDHISTORY_SDK_FILES (similar to BUILDHISTORY_IMAGE_FILES) and
default it to pick up config files installed in the extensible SDK
(From OE-Core rev: 5f87ff116d9ccd5e95e638dfa84583169a3f99f3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After OE-Core revision baa4e43a29e45df17eaa3456acc179b08d571db6 we lost
recording SDK the contents in buildhistory. This was due to the
SDK_POSTPROCESS_COMMAND variable being set with = in
populate_sdk_base.bbclass which overwrote any value set with += in
buildhistory.bbclass; to fix it, use _append in buildhistory.bbclass
instead.
Fixes [YOCTO #8839].
(From OE-Core rev: 11d1aa82ef4a00051e0a50a87a1efed1c50c73b5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The existing code doesn't tell regular (with .git) and bare cases and
just move the unpacked repo to the place of kernel source. But later
steps will fail on a bare-cloned repo because we can not checkout
directly in a bare cloned repo.
This change performs another clone to fix the issue.
Note: This change doesn't cover the case that S and WORKDIR are same
and the repo is bare cloned.
(From OE-Core rev: ccfa2ee5c4f509de4c18a7054b2a66fc874d5d69)
Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the installation of buildtools fails then we should fail the entire
installation instead of blindly continuing on.
(From OE-Core rev: 34bb63e6c72fb862e0ef0d2b26e1bfddaf7ddb99)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The configuration of the build system within the extensible SDK is
fixed, so there's not a lot of point in showing it; plus it just gets in
the way of the output that's interesting to the user in this context. So
let's hide it within the extensible SDK.
(From OE-Core rev: 6223b73c2dfc326382dbd3c0ae2d7a0cf2cc3b48)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we don't want a header printed at the start of task execution (which
we'd prefer not to within the extensible SDK) we can accomplish that by
clearing BUILDCFG_VARS and BUILDCFG_HEADER, but that was still printing
a load of blank lines. To keep things simple, check if BUILDCFG_HEADER
is set to something before printing a header.
(From OE-Core rev: d896d77c1d0fc56ff7019bfee3cf06fbc3ede8f3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We inherit uninative in the extensible SDK configuration, and uninative
sets NATIVELSBSTRING to a fixed value, so we don't need to force the
value ourselves.
Fixes [YOCTO #8662].
(From OE-Core rev: 918814c05d670bccb05d61fa848fd0d93da3a7b0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reading IO stats fails because the IO read/write bytes are
being converted to strings, then added to a numeric running
total.
Fix this by converting IO stats to integers.
(From OE-Core rev: 8e2475eecafc0161d25684f5b8239273739de759)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a check like what we does in package.bbclass
so that the already-stripped QA test can be skipped.
(From OE-Core rev: 2262fdb256954b22dadb2f7c6922e6046c269742)
Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The buildstats changes removed the no longer needed get_bn() function,
replace this with references to BUILDNAME.
(From OE-Core rev: e1a37899da56014693f08d1c39cb6ec0a4ed2bf4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This updates the toaster class to use the new data format from buildstats.
This does mean it will no longer read IO data from older builds, however
since that data is completely useless anyway, I don't consider that to be
an issue.
(From OE-Core rev: 5c3987226cd9669d5fc0993d0541048475af9fcb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add IO stats and getrusage() data to the task statistics. We
also drop the CPU percentage calculation since its pretty arbitrary
and not very accurate/useful.
In particular we can now see the user and sys times as well as the
wall clock times.
(From OE-Core rev: b849130f71d3ba32a6fa94c291ca6ce7c7c3b3d1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The combined build and task data code makes changing things hard, separate
out the functions so that changes can be made to the task data whilst the
build data remains unchanged.
(From OE-Core rev: c79cfce4f820f20346d0565df8df626832976e28)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than bb.data and e.data, cleanup to use 'd' and match the
standard coding style.
(From OE-Core rev: dbce2c6122d492ba86873db4d02322f0df7a6752)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current setting and getting of the "name" to use for buildstats is
convoluted and not particularly interesting. We only need this for the
e.getPkgs()[0] component of the path which is the first target listed
on the commandline. This is pretty arbitrary.
If we drop that piece, we can assume BUILDNAME is common for all events
and simply use this and query it. If BUILDNAME did change, that would
be a bug and it should be fixed elsewhere.
Also take the opportunity to share some common code since the function
now has the eventmask.
(From OE-Core rev: 918d83460639df273f38ae079ffeebd6a79b3373)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The existing diskstats data from buildstats simply isn't useful. It
gives stats on the total IO counts on some random disk within the
system. This means that the count includes data from all other tasks
running at the same time and from any other process running on the
system.
I've been unable to find any use for the data so as a start at
reworking the class, remove the related code.
(From OE-Core rev: f9fb02909f332365cad329352956a29cff6eba77)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously UNINATIVE_LOADER was always ld-linux-x86-64.so.2. That is
incorrect when the host is 32-bit.
This change also changes to using ?= so the user can override
UNINATIVE_LOADER if so desired.
[YOCTO #8124]
(From OE-Core rev: b78fa0bcadd54bb29b6f1bb3a9308d4c454bf4e2)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
