If multiple images are being built simultaneously against the same
sysroot then the call to os.makedirs in Image._write_wic_env can fail
with:
File: '.../meta/lib/oe/image.py', lineno: 341, function: _write_wic_env
0337: """
0338: stdir = self.d.getVar('STAGING_DIR_TARGET', True)
0339: outdir = os.path.join(stdir, 'imgdata')
0340: if not os.path.exists(outdir):
*** 0341: os.makedirs(outdir)
0342: basename = self.d.getVar('IMAGE_BASENAME', True)
0343: with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
0344: for var in self.d.getVar('WICVARS', True).split():
0345: value = self.d.getVar(var, True)
File: '/usr/lib/python2.7/os.py', lineno: 157, function: makedirs
0153: if e.errno != errno.EEXIST:
0154: raise
0155: if tail == curdir: # xxx/newdir/. exists if xxx/newdir exists
0156: return
*** 0157: mkdir(name, mode)
0158:
0159:def removedirs(name):
0160: """removedirs(path)
0161:
Exception: OSError: [Errno 17] File exists: '.../tmp-glibc/sysroots/cheetah/imgdata'
Using bb.utils.mkdirhier() protects against this.
There's also little point in checking to see if the directory already
exists - we might as well just try and create it regardless.
Once the directory has been created, there's no race on the actual file
since the filename contains IMAGE_BASENAME.
(From OE-Core rev: e3f216eef1b7a8ecfd00317e60704cff472238ba)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Creating a file for every image containing a few variables isn't
necessary if wic is not being used, so don't write the file if WICVARS
is empty.
(From OE-Core rev: d40779a7d3ce4f326c29ec7971731cab1f505a37)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This doesn't cause any issues right now but it make sense to standardise
on consistently using strings in the data store.
(From OE-Core rev: 99203fbe5ad470ef65cff93cec9d7f332883b5ee)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The dependency code needs to also include the dependency of base
types. For example:
- sdcard.gz image with ext4
The dependency chain needs to include:
- sdcard
- ext4
- gz
Until this change, the ext4 dependency were not being taken into
account when using the compressed one.
(From OE-Core rev: 10e5df3503632a6e1c54612055b19f7258c3ae2f)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The companion debug filesystem, enabled with IMAGE_GEN_DEBUGFS, was
creating the companion filesystem but was missing the code to actually
package it into a usable filesystem.
The code (and associated documentation) will allow the debugfs to generate a
companion tarball or other image.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a floating point multiplication done of a base image size
and an "overhead factor", which is currently rounded up to the next
integer. If the multiplication results in a whole number, the value
will still be a float. When this float is used to generate a shell
script, a buggy script is generated.
Fix this by always forcing to an integer.
(From OE-Core rev: bf74a002b8fa18d94cec93f0341cbe74cc010ca7)
Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When IMAGE_LINK_NAME is empty, OE will try to create a "blank" link instead
of just skipping it. The code checks for "link_name is not None" which will
never evaluate to true. Change the test to a simple "if link_name:" so it
no longer attempt to create links when the variable is an empty string.
(From OE-Core rev: 7f03dd50de76f0b5c3a10c514e920615ffaa846c)
Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Renamed this function as it's too generic name for it.
It writes variables, which are used by wic to .env file,
so _write_wic_env is better name for it.
Thanks Christopher Larson for poining out to this.
(From OE-Core rev: f8c7542164ebbe29613532c93ddc34c94238453c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Write set of bitbake variables used by wic into
build/tmp/sysroots/<machine>/imagedata/<image>.env
List of variables is defined in WICVARS variable in
meta/classes/image_types.bbclass.
This is needed for wic to be able to get bitbake variables without
running 'bitbake -e'.
(From OE-Core rev: 861ce6c5d4836df1a783be3b01d2de56117c9863)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Let's add output of image creation script to the bitbake log
as it can contain useful information.
One good example of such an information is wic report about
artifacts and .wks file used for image creation.
(From OE-Core rev: fd6eba587e39142134aeb9044393c08a3f79d28c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This variable is going to be used by wic to set partition
size. Setting it in image.py makes it possible for wic to
use it without calculating it again.
(From OE-Core rev: af37bb8ae71c4f932e2126bb620b3fb2b11cc466)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The following IMAGES_FSTYPES defaulted to ext3:
"vmdk", "vdi", "qcow2", "live", "iso", "hddimg"
This patch changes the default for those IMAGES_FSTYPES to
ext4 in order to bring the images more in line with other BSPs.
Besides improvements in performance and reliability ext4 provides
additional functionality as well (option to turn off the journaling,
dynamic resizing of VDI volumes etc.).
(From OE-Core rev: 2b56d671d2f0ef22786c97e29e1215eb80c94490)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added support for VirtualBox VDI format. The support was
implemented by merging with the already existing VMDK support
for VM player by creating a new class image-vm.bbclass.
This class replaces the previous VMDK only image-vmdk.class.
(From OE-Core rev: 0a3e8eb9f592c3f1edd2c7521855f7406541651a)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An invalid value in IMAGE_FSTYPES was triggering a traceback. Add a
check and a reasonable error message instead.
(From OE-Core rev: f38c35b925d6eab0d5a41b8592b0eb8b0a3b7b72)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When computing the dependency graph for the image generation, we need
to take into account the compression type and identify the base type
it relates to. This allow for a more robust graph generation even when
using composed image types.
(From OE-Core rev: 6e7d1de6cc99ed2def346dc40310573f5f0ce5ca)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The graph sorting algorithm for image dependencies does a look for an
occurrence of a searched string instead of comparing the chunk to the
searched string. This leads to the problem that ubifs is recognized as ubi aswell.
This fixes this by splitting up the string into chunks.
(From OE-Core rev: cec9725c540c2d54c27092e40d159694cea75b5f)
Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Check the rootfs size against IMAGE_ROOTFS_MAXSIZE (if set)
* Add comments for IMAGE_ROOTFS_SIZE to not confuse with IMAGE_ROOTFS_MAXSIZE
[YOCTO #2610]
(From OE-Core rev: 6acd4fc8d5e642b5c6c75fcc40dd8f37caf7ddcf)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When RM_OLD_IMAGE = "1", we delete old images but we didn't check they
actually exist...
[YOCTO #6029]
(From OE-Core rev: 8910d3cc94899ab4d509e681b438ae96218fa777)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a command executed with subprocess.check_output() fails, the
subprocess.CalledProcessError.output contains only STDOUT and the user
needs to check the log.do_rootfs to see any other details.
This commit forwards stderr to stdout so that, in case of failure, the
entire error output will be displayed in terminal.
[YOCTO #5902]
(From OE-Core rev: 4661f1441429952f30e19cebd93dc42ce31fd868)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit adds a dependency mechanism to image creation, so that we can
split the images creation execution in groups, that can be executed in
parallel, having the dependencies satisfied in the same time. The old
code didn't need this since everything was serialized.
Technically, it adds a dependency graph topological sort class that the
main Image class can use to sort out the dependencies.
Images that have dependencies have to declare them using the NEW
IMAGE_TYPEDEP variable, like in the example below:
For:
IMAGE_FSTYPES = "i1 i2 i3 i4 i5"
IMAGE_TYPEDEP_i4 = "i2"
IMAGE_TYPEDEP_i5 = "i6 i4"
IMAGE_TYPEDEP_i6 = "i7"
IMAGE_TYPEDEP_i7 = "i2"
We'll get the following image groups, sorted out by their dependencies:
[['i1', 'i3', 'i2'], ['i4', 'i7'], ['i6'], ['i5']]
The algorithm can probably be optimized but, given the small size of the
graphs, it'll do.
[YOCTO #5830]
(From OE-Core rev: db9dd4b4ef9120baccbccae77d9c31f54a6eb9a1)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 3f49597225.
This change appears to cause more problems than it fixes since the
compression commands usually work in the deploy dir but the archive ones
have always worked in the rootfs dir (which is clear from the tar command
we use).
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The working directory needs to be changed before the image creation
commands instead of afterwards.
(From OE-Core rev: 9d9bca8785911e8ae06d507bbfb99d6a811f072e)
Signed-off-by: Jonathan Liu <net147@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previous commit added support for sum 'compression' (rather
postprocessing) of jffs2 images. Drop support for sum.jffs2 image type.
(From OE-Core rev: 295c6c1452c8789a8b1d0090ff09a5cd7cf16507)
Signed-off-by: Dmitry Eremin-Solenikov <dmitry_eremin@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Print the entire command output in case of errors.
(From OE-Core rev: 2253c9ac2caa61dee0bd4fea04d4d77b79be7b36)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The IMAGE_ROOTFS_EXTRA_SPACE value was not correctly added to the base
size.
(From OE-Core rev: f3ac2a68ae0ae9bdf2a03c1b6d0f83c9918d58bd)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The value of IMAGE_ROOTFS_EXTRA_SPACE is "0 + 51200",
we should use eval rather than int in python.
(From OE-Core rev: dd3418e2db732ca1bc78fd93efa3f08da88a1183)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will replace the old bash image creation code. This needs the
rootfs to be already generated in order to work.
Usage:
Image(d).create()
or using the provided wrapper function:
create_image(d)
(From OE-Core rev: b75b78ce534fbf0d4de2f7f66af5b721d68b7471)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
