A previous commit broke downloads
when proxies are involved, let's fix it.
(From OE-Core rev: 97e263b99cbe8184a74f80738fd471cfdef29e0c)
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Print the entire command output in case of errors.
(From OE-Core rev: 2253c9ac2caa61dee0bd4fea04d4d77b79be7b36)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use python sets instead of lists, to avoid duplicates. When doing a
multilib build, "smart channel --add" fails because it tries to add
'all' channel twice.
(From OE-Core rev: 730d675090eec5c03e444f0448d96a52035d0bef)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're just attempting to install packages and the package list is
empty, just return.
(From OE-Core rev: 8bf5d80e3502a71fdda688aace0799b47ae19b3b)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using opkg as the PM backend, one has the option to provide custom
feeds to create the rootfs from.
This commit:
* fixes this in the refactored code;
* moves the custom config creation code to python;
* clean up the package-ipk.bbclass;
(From OE-Core rev: 19c538f57c8fa7c566e88a6dbe13ea4826d4f26c)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The following commit:
rootfs_ipk: Ensure that BAD_RECOMMENDATIONS are honoured for all
architectures
changed the way BAD_RECOMMENDATIONS are handled. Make the change in the
new code too.
(From OE-Core rev: 7e518e399da51de3b159bd6804735b2f14c39357)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using stamps shortens rootfs/image creation because the package indexes
are not created again if no new package was installed in the deploy dir.
Unfortunately, there are some syncronization issues with
do_package_write_*() and the indexes are not properly created.
Will have to revisit this issue in the near future. For now, lose it.
(From OE-Core rev: 276f2a8a072ce243daa74210934ee63046e9c480)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit:
* adds missing mark_packages() function for OpkgPM. This is needed to
* manually change the packages installation status in the status file;
* fix OpkgPM remove_packaging_data() issue;
* fix OpkgPM handle_bad_recommendations() issue;
* improve OpkgPM/DpkgPM mark_packages() regex pattern;
* fix DpkgPM list_installed() problem;
* fix DpkgPM _create_configs() problem: status file should not be
truncated if it already exists/
(From OE-Core rev: d50a40100763dcbfabec2c147fdfcb46aa909329)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The IMAGE_ROOTFS_EXTRA_SPACE value was not correctly added to the base
size.
(From OE-Core rev: f3ac2a68ae0ae9bdf2a03c1b6d0f83c9918d58bd)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The idea of postinstall intercept hooks is to run CPU intensive
postinstalls (like the ones generating font/pixbuf/icon caches) on host,
at rootfs generation time. So, ideally, the user would like to know if
the intercepts fail on host.
With this patch, the user will see warnings on console if intercept
hooks fail to execute.
(From OE-Core rev: 5666ce9720132a76b5ff6e99712e240b8dc8e47a)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit:
* fixes a crash when handling interecept hook failures which happened when
MULTILIB_GLOBAL_VARIANTS was not set;
* convert dashes to underscores and use sets (so that we make sure the
items are unique) when creating RPM repos;
* uses a regex pattern to search for packages in the feeds list. The
old method could match also strings in the middle. For example: 'rpm'
matched 'kernel-module-lttng-probe-rpm" in qemux86_64 feeds;
* issue a bb.fatal if smart returns error while installing packages.
Otherwise we might end up with an incomplete image...
* fixes the /etc/rpm/platform file creation;
(From OE-Core rev: b98c7e4945f1c36a6e4f98144a3af4f3049450ae)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The value of IMAGE_ROOTFS_EXTRA_SPACE is "0 + 51200",
we should use eval rather than int in python.
(From OE-Core rev: dd3418e2db732ca1bc78fd93efa3f08da88a1183)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It failed to read Kernel version from kernel-abiversion
file, the reason was it didn't strip the readline.
...
Error: Kernel version 3.10.25-yocto-standard
does not match kernel-abiversion (3.10.25-yocto-standard)
...
(From OE-Core rev: 17a0a8f44fe344ab1ae49730d9f17705b43987e5)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The refactor of shell function rootfs_uninstall_unneeded is incorrect,
it should check and update the installed_pkgs.txt file for the existance
of the packages that were removed.
...
rootfs_uninstall_unneeded () {
if ${@base_contains("IMAGE_FEATURES", "package-management", "false", "true", d)}; then
if [ -z "$(delayed_postinsts)" ]; then
# All packages were successfully configured.
# update-rc.d, base-passwd, run-postinsts are no further
# use, remove them now
remove_run_postinsts=false
if [ -e ${IMAGE_ROOTFS}${sysconfdir}/init.d/run-postinsts ]; then
remove_run_postinsts=true
fi
# Remove package only if it's installed
pkgs_to_remove="update-rc.d base-passwd ${ROOTFS_BOOTSTRAP_INSTALL}"
for pkg in $pkgs_to_remove; do
# regexp for pkg, to be used in grep and sed
pkg_regexp="^`echo $pkg | sed 's/\./\\\./'` "
if grep -q "$pkg_regexp" ${WORKDIR}/installed_pkgs.txt; then
rootfs_uninstall_packages $pkg
sed -i "/$pkg_regexp/d" ${WORKDIR}/installed_pkgs.txt
fi
done
...
(From OE-Core rev: 9cdecb3935962653733705ad6313558bfd4fda29)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the bb.utils.remove(self.image_rootfs, True) from the base class
constructor, to Opkg/Ipkg constructors after super's constructor is
called.
(From OE-Core rev: 66d86c4bbb9423c9fe0fc89005530be54068d401)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This new file contains the python 'populate sdk' implementation of the
old bash populate_sdk_image() function for Opkg and Dpkg.
(From OE-Core rev: 6247efaba592db924e6466c39aef441f0e07c62a)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit contains the following fixes:
* pass the apt config directory to the DpkgPM constructor, so one can
instantiate this class multiple times and give it different config
files (like for creating SDK);
* change constructor argument name from 'dpkg_archs' to 'base_archs';
* export APT_CONFIG environment variable before calling apt-get, not in
constructor. If done in constructor, the last class instantiation,
sets the environment, which is note desireable;
(From OE-Core rev: dc626cbcfd37c940bb8739b14d3ab8097e1760ea)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the Manifest class has this property, use it. This contains the
default package installation order.
(From OE-Core rev: bb1fb4ceb544c161ed30cd102155657e3771859b)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit contains several changes:
* it is possible to create manifests for following types of images:
regular image, target SDK and host SDK. To distinguish between these
types of manifests, one has to pass the manifest_type argument to the
contructor or create_manifest() wrapper. The manifest type can have
the following values: image, sdk_host, sdk_target;
* move image_rootfs variable to _create_dummy_initial() since it's used
only here. This function will probably be removed in the future;
* fix a bug in the Dpkg class;
* add INSTALL_ORDER property to Manifest class which contains the
default install order for the packages and will be used Rootfs/Sdk
classes;
(From OE-Core rev: a8c1b7504bf9cd5625fdecfdc3c3adce53aa164c)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove the directory, manually, in the Rootfs.create() function.
(From OE-Core rev: 37a9dc05272e7e49c8a475ccb3d10880dcee4763)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Additionaly, the commit contains a couple of minor changes
(comments, error printing, etc).
(From OE-Core rev: ef3faaef6b1a25c943a8d5594ce55b0e558b62f3)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit will revert on using the bitbake APT_ARGS variable, so users
can alter the way apt is called without needing to change it in code.
(From OE-Core rev: ae337edc8ac441c947d1543c2800f37104a0be70)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is needed in order to serialize the index file creation when
multiple do_rootfs tasks are running in the same time.
(From OE-Core rev: cb03d15482569c2e56232c921526938dcecfdb68)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Manifest class clients don't really need to know how package types are
encoded.
(From OE-Core rev: bac2e279005b601daff4d53549612ceb76a6a857)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In this commit:
* add ability to create initial manifest for opkg;
* make var_map available to all backends;
(From OE-Core rev: 8d14fbedcee02b723288004ae29a5c29524eec5a)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This library will be used to generate the rootfs.
Recommended usage:
create_rootfs(d, manifest_file)
this will determine the PM backend used, save the shell environment and
will call the appropriate backend implementation (DpkgRootfs(d,
manifest_file).create()).
NOTE: this commit adds Dpkg support.
(From OE-Core rev: 5ad8c21bc7f2213d1287cecf07d00a61d1456ff7)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This new library is intended to be used by the new python rootfs
creation code.
It implements the rpm/dpkg/opkg package management backends: RpmPM,
DpkgPM and OpkgPM.
The base API is this:
update()
install()
install_complementary()
remove()
write_index()
remove_packaging_data()
list_installed()
All implementations have to provide these functions. Some backends may
need to implement additional functions though.
(From OE-Core rev: 224e5053044b4c7966fea3b8d199a3dfd4fe438e)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This new library allows for the creation of 2 types of manifests:
* initial manifest - used by the new rootfs creation routines to
generate the rootfs;
* final_manifest - this will contain all the packages in the image,
after all installations finished;
Usage:
Manifest(d, manifest_dir).create_initial()
Manifest(d, manifest_dir).create_final()
or using the provided wrapper function:
create_manifest(d, False, manifest_dir) -> creates initial manifest
create_manifest(d, True, manifest_dir) -> creates final manifest
If manifest_dir argument is ommited, it defaults to ${WORKDIR}.
NOTE: this commit creates fixed manifests for minimal/sato/sato-sdk
images, for Rpm & Opkg backends, in order to help speed up
development of rootfs refactoring. Dpkg initial manifest creation is
implemented.
(From OE-Core rev: a9d8e5e5878d14b4804317a7f7ea6394fca5e010)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will replace the old bash image creation code. This needs the
rootfs to be already generated in order to work.
Usage:
Image(d).create()
or using the provided wrapper function:
create_image(d)
(From OE-Core rev: b75b78ce534fbf0d4de2f7f66af5b721d68b7471)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This helper function will be used to execute pre/post process commands.
(From OE-Core rev: 23d409558cb12cbf0300156e67f768a13442910a)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running tests outside of the build system we can't use
bb.fetch anymore. It was nice but tests and their modules
need to rely on the data storage only as that gets exported.
This module is used by the oeqa/runtime/build* tests.
(From OE-Core rev: 3caf8e244ea94f62a93f3b40e73e15ea78fc2880)
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This script will run the exported tests outside of the build system.
Simplest way to test this is with a qemu image that you manually start.
For an already build image use this in local.conf:
TEST_EXPORT_ONLY = "1"
TEST_TARGET = "simpleremote"
TEST_TARGET_IP = "192.168.7.2"
TEST_SERVER_IP = "192.168.7.1"
Export the tests: bitbake core-image-sato -c testimage
Then: runqemu core-image-sato
And: cd build/tmp/testimage/core-image-sato
./runexported.py testdata.json
The contents of build/tmp/testimage/core-image-sato can be moved on another machine
as long as some paths are updated in the json.
The exported data contains paths to the build dir. We only care about DEPLOY_DIR/rpm (
if the rpm and smart tests are enabled), so running the tests on other machine
means that the user has to move the contents and call runexported with --deploy-dir PATH:
./runexported.py --deploy-dir /path/on/another/machine testdata.json
runexported.py accepts other arguments as well, see --help.
[YOCTO #5613]
(From OE-Core rev: 087ee840ad642bada6fe0b02311f05a595ea2e65)
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the ability to export the tests so that they can run independently of
the build system, as is required if you want to be able to hand the test
execution off to a scheduler.
Booting/deployment of the target is still handled by the build system,
as before, only the execution of the tests happens outside of the build system.
Tests exported are the ones defined in TEST_SUITES.
No tests have been changed as interesting parts of the data store have been
exported and tests can continue to query them as before. Small adjustments were made
for a couple of oeqa modules though.
[YOCTO #5613]
(From OE-Core rev: 155dd52e0f707e06f50756584a50f744ba6b7844)
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The settle() function had a race where services could still be activating at two
minutes but then when the final log is output, they've activated.
Remove this race and generally clean up the code.
(From OE-Core rev: 8d107e0a828868702cfe035104c1f0b51da4291e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use a python module "folder" rather than a single module within
layers to ensure multiple layers can define a TEST_TARGET class.
Current implementation using controllers.py module will only allow
a single layer to define test targets.
Add a controllers folder as well as a TestTargetLoader class whose
job is to load the given TEST_TARGET class from any number of
python modules within the oeqa/controllers/ directory of any
layer.
The only condition will be that layers will need to ensure
the TEST_TARGET class name they provide is unique otherwise there
is no guarantee which class is instantiated. a bb.warn is used
to alude to this if it happens.
(From OE-Core rev: 3f25705f4a986e06cbd397aaea52b841c1a1e054)
Signed-off-by: Sipke Vriend <sipke.vriend@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Split the class in two, one to handle the process and the
timeout based on output and one for the actual ssh/scp commands.
The ssh/scp methods now use the same run method.
It does the same thing as before but:
- it looks cleaner.
- adds support for using a different user than root
- optionally, raises an exception when exit code != 0
(that's useful for code outside of tests, where you wouldn't want
to check the return code every time as the tests do)
(From OE-Core rev: bb14a7598d3c0636dc249f719bde0d9d65b2694d)
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this it won't add core2-64 and similar channels, as the
directory name in deploy_dir/rpm uses _ not - as the package arch.
(From OE-Core rev: 14e3a8f58e309121760fec70619633281dd9d88c)
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
