This will let folks extend the oe package with modules from other layers.
Given openembedded consists of more than just oe-core, I think this makes
sense, and adds some useful flexibility.
(From OE-Core rev: 8dfd28925fc47d8a4a1f10ec65df665c8326edc4)
Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding a common interface to add predefined package manager
channels to prebuilt rootfs:es.
Adding PACKAGE_FEED_URIS = "http://myre.po/repo/, will
assume repo directories named (rpm,ipk,deb) as subdirectories
and statically add them to the rootfs, using the same PKG_ARCHs
as the build which produced the images.
Tested with RPM, IPK and DEB.
deb feed functionality seem broken, is anyone using this ?
(From OE-Core rev: 9b8811045546ad67b4695d980f09636d5506e50c)
Signed-off-by: David Nyström <david.c.nystrom@gmail.com>
Signed-off-by: David Nyström <david.nystrom@enea.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* I've noticed errors like this in log.do_package:
DEBUG: Executing python function package_do_filedeps
sh: 1: Syntax error: "(" unexpected
sh: 1: Syntax error: "(" unexpected
DEBUG: Python function package_do_filedeps finished
which are actually caused by some filenames included in package
containing '()' characters
Maybe we should change meta/classes/package.bbclass to
fail when some filedeprunner call fails like this and fix
filedeprunner to escape '()' and other possibly dangerous chars
it's called like this:
processed = list(pool.imap(oe.package.filedeprunner, pkglist))
* don't use shell=True
* show the command when it fails and let do_package task to fail
(From OE-Core rev: 148c04c1bf39ca0d21288fdce61c51dc8e1c3226)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As we now require Python 2.7 and os.path.relpath() was added in 2.6 we can now
drop the reimplementation in oe.path.
oe.path.relative is simple now a wrapper that changes the order of the arguments
and it's use discouraged.
(From OE-Core rev: 1a03cd16401d2926bba902ffc5df30911b5c9394)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a command executed with subprocess.check_output() fails, the
subprocess.CalledProcessError.output contains only STDOUT and the user
needs to check the log.do_rootfs to see any other details.
This commit forwards stderr to stdout so that, in case of failure, the
entire error output will be displayed in terminal.
[YOCTO #5902]
(From OE-Core rev: 4661f1441429952f30e19cebd93dc42ce31fd868)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since tasks were renamed to packagegroups some time ago, this variable
name implies that its usage is necessarily related to them which is not
the case. Rename the variable to more closely represent what it does
(whilst still providing backwards-compatibility with a warning for
PACKAGE_GROUP).
(From OE-Core rev: d703e07ea5056624101e1bc48f10d25b602635e9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure the channel was not already added, before attempting to add.
[YOCTO #5890]
(From OE-Core rev: 0574f5390fb8958db5864f41a7bd8037ac046ac5)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Converting a list to a set does not keep the order of the items, as they
were added, which might result in the wrong packages being installed in
the final image...
This reverts commit 12f47c23df.
[YOCTO #5890]
(From OE-Core rev: 495cce9c9e63484c740aa669facb995b0ef74f85)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous implementation was checking IMAGE_PKGTYPE and created the
index files just for the backend used to create the image. Apparently,
'bitbake package-index' should attempt to create the index files for all
backends specified in PACKAGE_CLASSES.
[YOCTO #5827]
(From OE-Core rev: 0521d48a1612bfc735e2c86acc9b685c1dc389ef)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the output from opkg-query-helper.py is empty, output.split('\n')
would result in a list containing one element which is an empty string
while iterating over each line in the output. An exception is then
thrown by the line:
pkg, pkg_file, pkg_arch = line.split()
with the message:
Exception: ValueError: need more than 0 values to unpack
To avoid this, we add a condition to only split the output if it isn't
empty.
(From OE-Core rev: ee7b75c895e77ab20f728423c8efc2ced92265e8)
Signed-off-by: Jonathan Liu <net147@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We shouldn't be running this if it expands to None or "" or False
so update the code accordingly.
(From OE-Core rev: 255079780729f0dd9df6f35302169e0c8480eced)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The import oe.package_manager line is redundant, let's remove it.
(From OE-Core rev: ade1f3ce34489bba5a7ab23793c1d82559150583)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The _multilib_sanity_test installs multilib packages in a temporary
root fs, and compare with the current image to figure out duplicated
files that come from different packages.
While incremental image generation enabled and the previous image
was existed, there was an Multilib check error:
...
ERROR: Multilib check error: duplicate files tmp/work/qemux86_64-poky-
linux/core-image-minimal/1.0-r0/multilib/lib32/lib/libc.so.6 tmp/work/
qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/lib/libc.so.6
is not the same
...
The reason is the file in the existing image has been prelinked by
previous image generation and the file in a temporary root fs is not
prelinked, even though both of them came from the same package, the
Multilib check failed.
[YOCTO #1894]
(From OE-Core rev: 8d813f614cdfda31c85bbaf133f2822f90a4a78a)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
While incremental image generation enabled and the previous image is
existed, if BAD_RECOMMENDATIONS is changed, the operation on the
existing image is complicated, so remove the old image in this situation.
The same with PACKAGE_EXCLUDE and NO_RECOMMENDATIONS.
[YOCTO #1894]
(From OE-Core rev: 0566de3fa424af3bdfadcd0a08ce4c214abda083)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The incremental image generation is based on the previous existing
image, adds new packages, upgrades existing packages, and removes unused
packages.
[YOCTO #1894]
(From OE-Core rev: adf587e55c0f9bc74f0bef415273c937401baebb)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the following three functions to OpkgPM class:
- The 'dummy_install' is used to dummy install pkgs, and returns the log
of output;
- The 'backup_packaging_data' is used to back up the current opkg
database;
- The 'recover_packaging_data' is used to recover the opkg database
which backed up by the previous image creation;
Tweak 'remove' function in OpkgPM class, which the options for remove
with dependencies was incorrect.
Tweak 'handle_bad_recommendations' function in OpkgPM class:
- Fix none value check;
- Add the existance check of opkg status file;
[YOCTO #1894]
(From OE-Core rev: 5df18065e267a7e55a990ac3728414bb6e28a723)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The function create_full creates the manifest after the package in
initial manifest has been dummy installed. It lists all *to be
installed* packages. There is no real installation, just a test.
[YOCTO #1894]
(From OE-Core rev: 494adecd878496c2edc663ba09a456a9735d8252)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* change the text a bit, because I have trouble parsing two
verbs at the beginning of sentence.
(From OE-Core rev: 9837d194682f46e3a0363765bd6b317180a4cda0)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Small error in the package list creation routine. Buildhistory was
supposed to use this but was never called. Hence, it escaped tests...
(From OE-Core rev: eaa1994ad22730cec8e8c57736915da6b45a416e)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit adds a dependency mechanism to image creation, so that we can
split the images creation execution in groups, that can be executed in
parallel, having the dependencies satisfied in the same time. The old
code didn't need this since everything was serialized.
Technically, it adds a dependency graph topological sort class that the
main Image class can use to sort out the dependencies.
Images that have dependencies have to declare them using the NEW
IMAGE_TYPEDEP variable, like in the example below:
For:
IMAGE_FSTYPES = "i1 i2 i3 i4 i5"
IMAGE_TYPEDEP_i4 = "i2"
IMAGE_TYPEDEP_i5 = "i6 i4"
IMAGE_TYPEDEP_i6 = "i7"
IMAGE_TYPEDEP_i7 = "i2"
We'll get the following image groups, sorted out by their dependencies:
[['i1', 'i3', 'i2'], ['i4', 'i7'], ['i6'], ['i5']]
The algorithm can probably be optimized but, given the small size of the
graphs, it'll do.
[YOCTO #5830]
(From OE-Core rev: db9dd4b4ef9120baccbccae77d9c31f54a6eb9a1)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We override LDCONFIGDEPEND to be empty string for uclibc
however the current check is for it being None as a result
the function is still executed but ldconfig-native is not
built as dependency for rootfs when building with uclibc
Fixes errors like below
File:
'/home/kraj/work/angstrom-repo/sources/openembedded-core/meta/lib/oe/rootfs.py',
lineno: 191, function: _run_ldconfig
0187: def _run_ldconfig(self):
0188: if self.d.getVar('LDCONFIGDEPEND', True) is not None:
0189: bb.note("Executing: ldconfig -r" +
self.image_rootfs + "-c new -v")
0190: self._exec_shell_cmd(['ldconfig', '-r',
self.image_rootfs, '-c',
*** 0191: 'new', '-v'])
(From OE-Core rev: 315a71e00928fea6715087b1ce66f8a8a91d5fe0)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 3f49597225.
This change appears to cause more problems than it fixes since the
compression commands usually work in the deploy dir but the archive ones
have always worked in the rootfs dir (which is clear from the tar command
we use).
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The working directory needs to be changed before the image creation
commands instead of afterwards.
(From OE-Core rev: 9d9bca8785911e8ae06d507bbfb99d6a811f072e)
Signed-off-by: Jonathan Liu <net147@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its perfectly reasonable to call -c package-index with empty package
directories. This ensures the builds don't fail in such cases,
resolving autobuilder failures.
(From OE-Core rev: 11e2aaf2d751277e3e99ac2acbbeff2b7227be94)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the code from anonymous function in rootfs_rpm.bbclass has been
removed, MULTILIB_PREFIX_LIST variable was never set. Hence not all
directories got indexed.
This commit will move the multilib prefix list computation function from
RpmPM class to RpmIndexer, since the indexer needs it too. I was hoping
to avoid this but, unfortunately, I couldn't.
(From OE-Core rev: d3ba249aa1bf68aaeed226e934a4f4d5b7a19286)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previous commit added support for sum 'compression' (rather
postprocessing) of jffs2 images. Drop support for sum.jffs2 image type.
(From OE-Core rev: 295c6c1452c8789a8b1d0090ff09a5cd7cf16507)
Signed-off-by: Dmitry Eremin-Solenikov <dmitry_eremin@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Because the package-index.bb needs to create package indexes outside
do_rootfs environment, move the indexing capability out of
PackageManager class to a smaller Indexer class.
This commit:
* simply moves the indexing functions for ipk/deb with no changes;
* rewrites the RPM indexing function so that it can be easily moved out
of the PackageManager class;
* removes some RPM duplicate code, moves it into a method inside
RpmPM class and changes the RpmPM constructor so that the new method
is effective;
(From OE-Core rev: d339d6f48f81330e94162f333aad76f3c65d6bfd)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For some odd reason (at least I couldn't find an explanation to this,
yet), if a multilib version of a package is installed after the main one
(that is: in a different smart session), the main package binaries are
not overwritten.
This commit restores the functionality to the original one, before
migrating to python: feed all the packages to smart, apart from attempt
only ones which are installed separately.
(From OE-Core rev: 1fa94697163f16cdbb1499b57f1bc018546974ee)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit whitelists some common directories, so the multilib sanity
checks pass and also fixes an issue in lib/oe/rootfs.py when the
compared files do not exist.
(From OE-Core rev: 4615ac771175cd413f461c081f7d3191c49ccc4b)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Print the entire command output in case of errors.
(From OE-Core rev: 2253c9ac2caa61dee0bd4fea04d4d77b79be7b36)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use python sets instead of lists, to avoid duplicates. When doing a
multilib build, "smart channel --add" fails because it tries to add
'all' channel twice.
(From OE-Core rev: 730d675090eec5c03e444f0448d96a52035d0bef)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're just attempting to install packages and the package list is
empty, just return.
(From OE-Core rev: 8bf5d80e3502a71fdda688aace0799b47ae19b3b)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using opkg as the PM backend, one has the option to provide custom
feeds to create the rootfs from.
This commit:
* fixes this in the refactored code;
* moves the custom config creation code to python;
* clean up the package-ipk.bbclass;
(From OE-Core rev: 19c538f57c8fa7c566e88a6dbe13ea4826d4f26c)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The following commit:
rootfs_ipk: Ensure that BAD_RECOMMENDATIONS are honoured for all
architectures
changed the way BAD_RECOMMENDATIONS are handled. Make the change in the
new code too.
(From OE-Core rev: 7e518e399da51de3b159bd6804735b2f14c39357)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using stamps shortens rootfs/image creation because the package indexes
are not created again if no new package was installed in the deploy dir.
Unfortunately, there are some syncronization issues with
do_package_write_*() and the indexes are not properly created.
Will have to revisit this issue in the near future. For now, lose it.
(From OE-Core rev: 276f2a8a072ce243daa74210934ee63046e9c480)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit:
* adds missing mark_packages() function for OpkgPM. This is needed to
* manually change the packages installation status in the status file;
* fix OpkgPM remove_packaging_data() issue;
* fix OpkgPM handle_bad_recommendations() issue;
* improve OpkgPM/DpkgPM mark_packages() regex pattern;
* fix DpkgPM list_installed() problem;
* fix DpkgPM _create_configs() problem: status file should not be
truncated if it already exists/
(From OE-Core rev: d50a40100763dcbfabec2c147fdfcb46aa909329)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The IMAGE_ROOTFS_EXTRA_SPACE value was not correctly added to the base
size.
(From OE-Core rev: f3ac2a68ae0ae9bdf2a03c1b6d0f83c9918d58bd)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The idea of postinstall intercept hooks is to run CPU intensive
postinstalls (like the ones generating font/pixbuf/icon caches) on host,
at rootfs generation time. So, ideally, the user would like to know if
the intercepts fail on host.
With this patch, the user will see warnings on console if intercept
hooks fail to execute.
(From OE-Core rev: 5666ce9720132a76b5ff6e99712e240b8dc8e47a)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit:
* fixes a crash when handling interecept hook failures which happened when
MULTILIB_GLOBAL_VARIANTS was not set;
* convert dashes to underscores and use sets (so that we make sure the
items are unique) when creating RPM repos;
* uses a regex pattern to search for packages in the feeds list. The
old method could match also strings in the middle. For example: 'rpm'
matched 'kernel-module-lttng-probe-rpm" in qemux86_64 feeds;
* issue a bb.fatal if smart returns error while installing packages.
Otherwise we might end up with an incomplete image...
* fixes the /etc/rpm/platform file creation;
(From OE-Core rev: b98c7e4945f1c36a6e4f98144a3af4f3049450ae)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The value of IMAGE_ROOTFS_EXTRA_SPACE is "0 + 51200",
we should use eval rather than int in python.
(From OE-Core rev: dd3418e2db732ca1bc78fd93efa3f08da88a1183)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It failed to read Kernel version from kernel-abiversion
file, the reason was it didn't strip the readline.
...
Error: Kernel version 3.10.25-yocto-standard
does not match kernel-abiversion (3.10.25-yocto-standard)
...
(From OE-Core rev: 17a0a8f44fe344ab1ae49730d9f17705b43987e5)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
