Commit Graph

508 Commits

Author SHA1 Message Date
Peter Kjellerstedt
aa13b97286 image.bbclass: Make unneeded packages for a read-only rootfs configurable
Previously the list of packages that are considered unneeded for a
read-only rootfs was hardcoded. This made it impossible to, e.g., have
shadow installed on a system with a read-only rootfs, but where /etc
is mounted writable.

This also lists ${VIRTUAL-RUNTIME_update-alternatives} rather than
update-alternatives (as was previously the case) since this should
actually remove the intended package.

(From OE-Core rev: e3b881d4168e5b02ff00f5c470ba472ab8bbc747)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-31 23:01:37 +01:00
Richard Purdie
b3e4a31cb4 sstatesig: Ensure we keep native depends for allarch recipes
Without this, do_package_write_rpm doesn't depend on rpm-native which
it really should since that is needed to build rpms.

[YOCTO #8047]

(From OE-Core rev: 3fab4f9920d004fe13fb01434d4c7f3b8bbd7895)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-29 23:20:12 +01:00
Ross Burton
c3c0d0ac55 lib/oe/qa: add method to check if static or dynamic linked
It's useful to know if a binary is statically or dynamically linked, so add a
method to determine this.

(From OE-Core rev: 96813445e6618fd8442600d81e53c448310b6e8b)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-26 07:34:58 +00:00
Robert Yang
3e5c5fe330 gpg_sign.py: get rid of pexpect
The python-expect is not installed on the distro such as Ubuntu by
default, and we can get rid of it.

Use RPM_GPG_PASSPHRASE to replace of RPM_GPG_PASSPHRASE_FILE which is
more straightforward.

(From OE-Core rev: 4a8a74c62836a20610daf029d4cec0b3087758b2)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-25 10:29:14 +00:00
Joshua Lock
d915965117 lib/package_manager: remove RPM4 support code
Simplify the RPM code by removing support for RPM 4 now that we've
dropped the RPM 4 recipe.

(From OE-Core rev: 7db6f0a402948ce489bafadf2e389802f764f122)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-20 23:12:30 +00:00
Mariano Lopez
36bf66654a package_manager.py: Fix race condition in OpkgIndexer.write_index()
When writing the index using ipk packages there could be a race condition
when populate the index. This happens because the architectures
are repeated (specially all) and the commands generated to write the index
run in parallel.

This change avoid the duplication of commands using a set instead of a list.

[YOCTO #8924]

(From OE-Core rev: 74adb14b0002e20099cc2c34e01862e8ddb8e013)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-12 22:11:47 +00:00
Ioan-Adrian Ratiu
81d58d6b24 sign_package_feed: add feed signature type
Signing package feeds will default to ascii armored signatures (ASC) the
other option being binary (BIN). This is for both rpm and ipk backends.

(From OE-Core rev: 862a3892feb2628282e1d6f2e4498a7a3bd60cbf)

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-11 16:50:45 +00:00
Ioan-Adrian Ratiu
42f612c42a package_manager: sign IPK package feeds
Create gpg signed ipk package feeds using the gpg backend if configured

(From OE-Core rev: a2ee831cfb688bc64c071f75a1dff8a963abe287)

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-11 16:50:45 +00:00
Ioan-Adrian Ratiu
14e809e03a gpg_sign: export_pubkey: add signature type support
Add support for multiple types of signatures (binary or ascii)
in export_pubkey(). There is no change in behaviour for the function,
the previous implicit default is the new parameter "armor" default.

(From OE-Core rev: 95ba4a982b887444908207e3180fe4bc46281d3b)

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-11 16:50:45 +00:00
Ioan-Adrian Ratiu
0b088e0829 gpg_sign: detach_sign: fix gpg > 2.1 STDIN file descriptor
Starting from v2.1 passing passwords directly to gpg does not work
anymore [1], instead a loopback interface must be used otherwise
gpg >2.1 will error out with:
"gpg: signing failed: Inappropriate ioctl for device"

gpg <2.1 does not work with the new --pinentry-mode arg and gives an
invalid option error, so we detect what is the running version of gpg
and pass it accordingly.

[1] https://wiki.archlinux.org/index.php/GnuPG#Unattended_passphrase

(From OE-Core rev: 0413bd8e294ca8ac972ac68662b43a981952f5ae)

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-11 16:50:45 +00:00
Ioan-Adrian Ratiu
2fccd8aa1c gpg_sign: add local ipk package signing functionality
Implement ipk signing inside the sign_ipk bbclass using the gpg_sign
module and configure signing similar to how rpm does it. sign_ipk uses
gpg_sign's detach_sign because its functionality is identical to package
feed signing.

IPK signing process is a bit different from rpm:
    - Signatures are stored outside ipk files; opkg connects to a feed
server and downloads them to verify a package.
    - Signatures are of two types (both supported by opkg): binary or
ascii armoured. By default we sign using ascii armoured.
    - Public keys are stored on targets to verify ipks using the
opkg-keyrings recipe.

(From OE-Core rev: a40f27aa7802e8a0bd87a5417e35adbface62d05)

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-11 16:50:45 +00:00
Paul Eggleton
91455005b6 recipetool: create: split npm module dependencies into packages
Rather than rolling all of an npm module's dependencies into the same
package, split them into one module per package, setting the SUMMARY and
PKGV values from the package.json file for each package. Additionally,
mark each package with the appropriate license using the license
scanning we already do, falling back to the license stated in the
package.json file for the module if unknown. All of this is mostly in
aid of ensuring all modules and their licenses now show up in the
manifests for the image.

Additionally we set the main LICENSE value more concretely once we've
calculated the per-package licenses, since we have more information at
that point.

(From OE-Core rev: 8226805f83d21e7c1d2ba21969f3e8ee4b137496)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 17:00:29 +00:00
Joshua Lock
9dca5c8426 lib/sstatesig: skip shared_workdir when checking locked sigs
do_shared_workdir is not a proper sstate task, it always reruns if
needed, so special-case it in warnings when checking locked sigs.

(From OE-Core rev: 4b08f982a2b15bff9092f60f7957301bb2d2108b)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 17:00:28 +00:00
Mark Hatle
007c284cb8 rpm: Uprev to rpm-5.4.16 (pre) and rpm-5.4+cvs to current CVS head
meta/lib/oe/package_manager.py was also updated.  This ensures that any
diagnostic messages are ignored from the output of rpmresolve.

The patches have been split into bug fixes (things that belong upstream)
and local changes that are OE specific.

The following patches are obsolete and have been removed:

rpm-remove-sykcparse-decl.patch
fstack-protector-configure-check.patch
rpm-disable-Wno-override-init.patch
rpm-lua-fix-print.patch
rpm-rpmpgp-fix.patch
verify-fix-broken-logic-for-ghost-avoidance-Mark-Hat.patch

(From OE-Core rev: ee97e53fcceabc6ef4ddc68f38c5fa0e05c5d9a8)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 17:00:27 +00:00
Richard Purdie
0b84897ea8 insane/prelink: Handle nonstandard library paths
Prelink contains some hardcoded assumptions about the path layout of
the target system. Unfortunately if the system doesn't match, prelink
doesn't work. This breaks:

a) prelink of those images
b) the unsafe-references-in-binaries QA test (which uses prelink-rtld)

One way to work around this is to construct an ld.so.conf file which
lists the library paths in question. We do this in sanity QA check and
in the rootfs prelink code, being careful not to trample any existing
target ld.so.conf.

There is an additional problem that $LIB references in RPATHs won't be
handled correctly, I've not see any system use these in reality though
so this change at least improves things.

(From OE-Core rev: 7fd1d7e639c2ed7e0699937a5cb245c187b7c811)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 00:11:39 +00:00
Paul Eggleton
25d9c4e02a devtool: add build-sdk subcommand
Add a build-sdk command which is only available within the extensible
SDK that builds a derivative extensible SDK. The idea is recipes in the
workspace become a part of the new SDK - for example, this allows taking
a vendor provided SDK, adding a few libs and then producing a new SDK
with those included.

When normally building the extensible SDK, the workspace is excluded;
here we need to copy into the new SDK (renaming it in the process); the
recipes' task signatures become locked and thus the sources are no
longer needed, so they are removed along with the workspace bbappends
which would interfere with the locked signatures. Additionally we need
to just copy the configuration files (i.e. local.conf and auto.conf)
rather than filtering and appending to them since that work has already
been done when constructing the original SDK. The extra sstate artifacts
from workspace recipes are also determined and copied into the new SDK
in minimal mode (on the assumption that you won't set up a new sstate
mirror).

This reuses some code from build-image, so that needed to be
generalised to allow that.

Implements [YOCTO #8892].

(From OE-Core rev: 59e207ff6dd4b50a8905e14bc9292cf2794f4e7a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 00:11:38 +00:00
Ross Burton
4495e8bae0 lib/oe/qa: add explicit exception for 'file isn't an ELF'
(From OE-Core rev: 4c1fe0cbcb98b0a69ad5b3a04432055d773ee4ba)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:32:59 +00:00
Randy Witt
7bb9e8ddbf signing-keys: Make signing keys the only publisher of keys
Previously the keys were put into the os-release package. The package
indexing code was also deploying the keys rather than only using the keys.

This change makes signing-keys.bb the only publisher of the keys and also
uses standard tasks that already have sstate.

(From OE-Core rev: 1e38068ac38dfd067655dfd41464e28439179306)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-26 17:16:25 +00:00
Markus Lehtonen
db7c7c2ece oe/gpg_sign: add 'passphrase' argument to detach_sign method
This allows directly giving the passphrase, instead of reading from a
file.

[YOCTO #9006]

(From OE-Core rev: fd55c6e86b38b33f62006324e73678a13a534220)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-19 01:01:25 +00:00
Markus Lehtonen
e845b75f8f sign_rpm.bbclass: do not store key details in signer instance
Refactor the LocalSigner class. Do not store keyid or passphrase file in
the signer object as they are only needed for some of the methods. For
example, the newly added verify() method does not need any key
parameters and export_pubkey only uses keyid.

(From OE-Core rev: e2412294b6b1d3a80ee97a0706613349edc51d33)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-19 01:01:25 +00:00
Markus Lehtonen
d5be8666a1 oe/gpg_sign: add 'armor' argument to detach_sign()
[YOCTO #9006]

(From OE-Core rev: ce653694a87fd77d79ec3d28ed3365a2c8e57ad6)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-19 01:01:25 +00:00
Markus Lehtonen
03554b77cd oe/gpg_sign: add verify() method
A new method for verifying detached signatures.

[YOCTO #9006]

(From OE-Core rev: 4445c645c71151bd56ff7e133343a6f1e30cc3b3)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-19 01:01:25 +00:00
Ross Burton
7a700f59d9 lib/qa.py: raise ValueError if file isn't an ELF
Instead of raising a generic Exception that can't be handled specifically, raise
a ValueError.  Also update the callers so any unexpected exceptions are not
ignored.

Also, rename isBigEngian() to isBigEndian().

(From OE-Core rev: c136652f9c0b35aafa393e63567daf029ae03929)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-16 11:17:10 +00:00
Ross Burton
334e1b5e0e lib/oe/qa: ELFFile: check that a path is a file before opening it
When opening an ELF file check that a filename points to a normal file before
attempting to open it, as if the file turns out to be something more exotic like
a FIFO it could hang forever.

(From OE-Core rev: 4b3576bc30d8f8cdcde25189def8b059fc92b27c)

(From OE-Core rev: d3af2058e2753516b9aaf7f6d71162363eea11d4)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-16 11:17:10 +00:00
Pascal Bach
36b43b233c lib/oe/terminal: set workdir for konsole terminal
It seems that if the --workdir option is not set konsole does open in the users
home directory. By setting --workdir . konsole opens in the recipes work
directory. This is the same behavior as observed for other consoles.
(Tested with Konsole 2.14.2 on Debian Jessie).

(From OE-Core rev: bd06944249c3de3f629c013e14f446464441c4da)

Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-14 11:42:19 +00:00
Paul Eggleton
ddfe74447b devtool: commit for extra tasks that modify source when extracting
When extracting source for a recipe, if there are additional custom
tasks run that make changes to the source, create a commit in the
generated git branch so they are contained. This is particularly
useful for tasks that come before do_patch since otherwise the changes
might get incorporated in the first applied patch, but otherwise it
helps avoid the tree being dirty at any point.

Fixes [YOCTO #7626].

(From OE-Core rev: 997a77d9b20af1778b804778e5d8c8a7424f7582)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:02 +00:00
Markus Lehtonen
8cb1e83a57 oe/gpg_sign: fix incorrect variable name
Prevents crash in signing if GPG_PATH is defined.

(From OE-Core rev: 0ecd748258abfe5ed9e9e2505aeb42b232389968)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:27:46 +00:00
Ioan-Adrian Ratiu
bbdbe00c2a package_manager.py: fix python indentation bug (opkg)
This if branch is causing the following error during do_rootfs:
"Exception: UnboundLocalError: local variable 'cfg_file' referenced
before assignment". This happends because the cfg_file variable is
defined at a deeper nesting level (just above the if branch).

(From OE-Core rev: 555981c10e6b7f5c2be7f2c3fefb89b1add72cab)

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-06 23:10:45 +00:00
Randy Witt
f1f8447750 copy_buildsystem.py: Pass the nativelsb argument to gen-lockedsig-cache
If the nativelsb argument is not used, then create_locked_sstate_cache()
can get collisions when moving the files from the input_sstate_cache
to the output_sstate_cache.

The specific case where this was encountered was when a "universal"
nativelsb directory already existed in the input_sstate_cache.

(From OE-Core rev: 760f7178e0267f930c8af9cb59039e317149f944)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-06 23:10:44 +00:00
Chen Qi
090fb5146f copy_buildsystem: add ability to exclude layers
In some cases, we may have some kind of download layers in BBLAYERS, so
that we can set BB_NO_NETWORK to "1". This results in extremely large
extensible SDK. And we actually don't need these download layers in the
SDK.

Add a new variable, SDK_LAYERS_EXCLUDE, to enable users to explicitly
exclude some layers when generating the extensible SDK.

[YOCTO #8878]

(From OE-Core rev: acf1148bf3f4e489e9e2b0b8745753e1311ee812)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:39:01 +00:00
Richard Purdie
7fa6eeba1c classes/lib: Add expand parameter to getVarFlag
This sets the scene for removing the default False for expansion
from getVarFlag. This would later allow True to become the default.

On the most part this is an automatic translation with:

sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, True):g'  -i `grep -ril getVar *`

In this case, the default was False, but True was used since in most
cases here expansion would be expected.

(From OE-Core rev: 42a10788e89b07b14a150ced07113566cf99fcdd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:39:00 +00:00
Stefan Agner
db81fc967a lib/oe/package_manager: remove package feed lists
Remove opkg package feed lists after generating the rootfs. The
lists have been generated by the local feed but are no longer
necessary after image generation. This still leaves the package
management fully usable (and hence is different from omitting
package-management from IMAGE_FEATURES).

(From OE-Core rev: 25477874ef737777cedc623a8e1c5aedf2f4bae1)

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-31 13:29:48 +00:00
Peter Kjellerstedt
eb7e554cbd lib/oe/patch: Make GitApplyTree._applypatch() support read-only .git/hooks
Rather than modifying files in .git/hooks, which can be read-only
(e.g., if it is a link to a directory in /usr/share), move away the
entire .git/hooks directory temporarily.

(From OE-Core rev: a88d603b51a9ebb39210d54b667519acfbe465c3)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:37:01 +00:00
Markus Lehtonen
bb971577ab meta/lib: new module for handling GPG signing
Add a new Python module (oe.gpg_sign) for handling GPG signing
operations, i.e. currently package and package feed signing. The purpose
is to be able to more easily support various signing backends and to be
able to centralise signing functionality into one place (e.g.  package
signing and sstate signing). Currently, only local signing with gpg is
implemented.

[YOCTO #8755]

(From OE-Core rev: 9b3dc1bd4b8336423a3f8f7db0ab5fa6fa0e7257)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 22:31:58 +00:00
Richard Purdie
a1c35f302d lib/oe/sdk: Partially revert "sdk.py: fix conflicts of packages"
OE-Core rev: f2b64f725803ad8be7c2876c531e057a4fe5ca7c (poky
1362986886) unintentionally broke opkg/dpkg
multilib support within the SDK by making things not honour
self.install_order. This reinstates that code for opkg/dpkg but
not rpm where the original problem was.

(From OE-Core rev: 98b585120137a3db07ed742a8f18223883ad6dc5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:34 +00:00
Joshua Lock
0aeb33f6e0 lib/oe/package_manager: prevent testing an undefined variable
This prevents an error in do_populate_sdk when building
buildtools-tarball with ipk as the package manager:

Exception: UnboundLocalError: local variable 'pkg' referenced
before assignment

(From OE-Core rev: f971ae290cbbc51ec3669b57c55b2b7c0bd9bc87)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton
477fa84390 gen-lockedsig-cache: copy correct native sstate into ext SDK
When constructing the sstate-cache directory for the extensible SDK,
we were copying in any matching native sstate packages, and as the
signature doesn't actually change when the distro changes (since
NATIVELSBSTRING is just a path separator for the artifacts and is not
part of the signature) we ended up copying duplicated packages when the
distro changed e.g. upon host distro upgrade. Only search in the
NATIVELSBSTRING-named subdirectory for native packages and the issue
goes away.

Fixes [YOCTO #8885].

(From OE-Core rev: 6c6baf6aa1823b8b20123f505e45c2768a193ad5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton
d3a4f72896 classes/populate_sdk_ext: add option to bring in pkgdata for world
Add a variable SDK_INCLUDE_PKGDATA which you can set to "1" to include
pkgdata for all recipes in the world target. There are a couple of uses
for this:

1) If you use "devtool add" to add a recipe that builds something which
   depends on anything in world, the dependency can then be correctly
   mapped to the recipe providing it and that recipe can be added to
   DEPENDS, since we have the pkg-config and shared library dependency
   data within pkgdata.
2) You'll be able to search for these recipes and any files they
   package for the target with "devtool search" since that also uses
   pkgdata

This of course assumes you've tailored world through EXCLUDE_FROM_WORLD
to only include recipes you'd want built in your distro, but I think
that's a reasonable assumption; failing that there is a
WORLD_PKGDATA_EXCLUDE variable that you can set to exclude any recipes
you don't want.

Note that this patch relies on functionality implemented in a recent
BitBake patch and will not work without it.

Implements [YOCTO #8600].

(From OE-Core rev: 67149ea097d6fab7496b43e85a40853f40bd527e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:32 +00:00
Mariano Lopez
571289df7b lib/oe/package_manager.py: Remove list() from PkgsList class
Now that the method list() is not used anymore, remove it.

[YOCTO #7427]

(From OE-Core rev: 01e7dfbce972cfb926668e5ee194c83838e1e1b6)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:15 +00:00
Mariano Lopez
6ebda8e659 lib/oe/rootfs: Use list_pkgs() instead of list()
This patch changes the use list_pkgs() instead of list()
from class RpmPkgsList. The change is in two functions,
image_list_installed_packages from rootfs.py and
sdk_list_installed_packages from sdk.py.

With this change the functions calling the functions
listed above, must format the output as they required.
The formatting can be done using format_pkg_list() from
oe.utils.

The classes calling the afected functions are changed too
with this patch, to keep the same functionality using the
new data structure.

[YOCTO #7427]

(From OE-Core rev: 983ea373362514e5888bd1d7d9c4f136c94b00f2)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:15 +00:00
Mariano Lopez
03075f671c lib/oe/utils: Add function format_pkg_list()
The class PkgsList returns a dictionary with all the installed
packages, because the data structure is a dictionary there is
needed to format the data in order to write to a file.

The function format_pkg_list returns a formated sting with all
packages installed. The output will depend on the requested format
when calling the function.

[YOCTO #7427]

(From OE-Core rev: 25725e6e5fff8017aaf3a6fcd9b1b893c22630b5)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:15 +00:00
Mariano Lopez
c708411f20 lib/oe/package_manager: Add list_pkgs() to PkgsList class
Currently the class PkgList returns a formated string of the
installed packages. It would be more clean to pass a standard
data structure to the callers instead to format the output
inside PkgsList class.

This patch adds list_pkgs() method to PkgsList class to get the
all the information for installed packages and return a dictionary
with the info.

[YOCTO #7427]

(From OE-Core rev: 6cbb144a86a8188fad102bb281fd9e8d0a4b9142)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:15 +00:00
Aníbal Limón
b8ebac9794 DpkgRootfs: Fix logcheck_error false-positive when use multilib
Rootfs with dpkg was failing due to false-positive in logcheck_error
because current logic of DpkgPM handles missing dependencies failure
using apt-get -f install [1][2].

This support was broken due to addition of logcheck and don't take into
account dpkgpm cases, in order to fix add an attr for specify expected
errors regex'es by package manager.

[1] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/rootfs.py#n659
[2] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/package_manager.py#n2038

(From OE-Core rev: fa7a5ebef87883755491b847c2f4e1a7b021d585)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:31 +00:00
Matt Madison
4aeb69d35f package_manager.py: fixes for multilib deb packaging builds
* tmp/deploy/deb subdirectories do not get hyphens replaced
  with underscores, so don't do that translation when building
  the sources list.

* Fix MULTILIB_VARIANTS handling to be more general and
  work for all architectures

* Also include a fix for a warning generated by apt
  due to missing apt/preferences.d directory.

(From OE-Core rev: c3ee9b12ae704eadf02ff288d8035b5885d6218e)

Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:31 +00:00
Richard Purdie
e306d5495a populate_sdk_ext: Change to include siginfo and non sstate task sigs
Right now, the locked task hashes list for the extensible SDK locks
down only the sstate tasks.

Whilst asthetically pleasing, this gives two problems:

* Half the task are left floating meaning checksum mismatches
  are a pain to debug
* The later code which copies relavent data files out the sstate
  cache can't use any of this data.

This patch modifies things so all the checksums are listed in the locked
file. An exclusion of tasks probably makes more sense for the library
function rather than an allowed list.

The only sstate task being deliberaly excluded here was do_package
so add in a function to explictly exclude those sstate object files.

The net result of this that siginfo files for all tasks are included in
the SDK, which means commands like "bitbake -S printdiff" now function.

(From OE-Core rev: 6b70479e47b8a8743d8b410d6bc08da1607a318e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Richard Purdie
0a4e1f968a image: Create separate tasks for rootfs construction
This patch splits the code in lib/oe/image into separate tasks, one per
image type. This removes the need for the simple task graph code and defers
to the bitbake task management code to handle this instead.

This is a good step forward in splitting up the monolithic code and starting
to make it more accessible to people.

It should also make it easier for people to hook in other tasks and processes
into the rootfs code.

Incidentally, the reason this code was all combined originally was due to
limitations of fakeroot where if you exited the session, you lost permissions
data. With pseudo this constraint was removed.

We did start to rework the rootfs/image code previously and got so far with
untangling it however we did prioritise some performance tweaks over splitting
into separate tasks and in hindsight, this was a mistake and should have been done
the other way around. That work was suspended due to changes in the people working
on the project but this split has always been intended, now is the time to finish
it IMO.

There were some side effects of doing this:

* The symlink for the manifest moves to the rootfs-postcommands class and into
  the manifest function.
* There is no seperate "symlink removal" and "symlink creation", they are merged
* The date/time stamps of the manifest and the built images can now be different since
  the tasks can be run separately and the datetime stamp will then be different
  between do_rootfs and the do_image_* tasks.

(From OE-Core rev: c2dab181c1cdabac3be6197f4b9ea4235cbbc140)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Richard Purdie
fdced52387 image: Move pre/post process commands to bbclass
As the next step in splitting up do_image, move the pre and post processing
commands to separate tasks. This also creates the do_image_complete task
which acts as the end marker task for image generation.

(From OE-Core rev: 800528eaa421d451b596545125cb218e08989151)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:29 +00:00
Richard Purdie
1ee08426d7 sstatesig: Handle special case of gcc-source shared-workdir for printdiff
Often, bitbake -S printdiff would show that there was a checksum not found
which would turn out to be from gcc-source. This is due to it being a
shared-workdir recipe.

For now, hardcode the special case into the sstatesig code to stop people
(including me) puzzling over this.

If/as/when we add any other shared workdir recipes, we'll need to rethink
this.

(From OE-Core rev: f11342f0c838b520828927c9d69f7c17309c1b48)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:47 +00:00
Patrick Ohly
c91078954e package_manager.py: add debugging support for rpm scriptlet execution
By default, smart is invoked with --log-level=warning, which hides all
output from pre/post install scriptlets. That makes it hard to debug
scriptlet failure or why they get postponed to first-boot via
/etc/rpm-postinst.

The new ROOTFS_RPM_DEBUG variabled is expected to be set to an integer in
local.conf an incrementally adds more output:
  0 = default, only warnings
  1 = --log-level=info (includes information about executing scriptlets and their output)
  2 = --log-level=debug
  3 = --log-level=debug plus dumps of scriplet content and command invocation

The default behavior is not changed yet, but it seems that level 1 would
be a better default.

(From OE-Core rev: 5cb597a19dbfe825e5b26d828e18644c9ee58f86)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:19 +00:00
Mariano Lopez
9c0186fd57 rootfs.py: Change logic to unistall packages
In the current state some of the base utils (update-rc.d,
base-passwd, shadow, and update-alternatives) are unistalled
when there is no package manager in the image. Checking for
previous commits, the unistall of these utils were to be
done in a read-only filesystem.

It is a valid option to have these utils without a package
manager, and also make sense to remove them when building a
read-only filesystem.

This changes the check logic from having a package mananger
to if is a read-only filesystem to remove the utils.

Another change implemented with this patch is that delayed
post installs now doesn't depend if there is a package manager.
Also it is a valid option to have post install scripts without
package manger.

[YOCTO #8235]

(From OE-Core rev: 5aae19959a443c6ac4b0feef10715c8acf3c6376)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-12 23:42:51 +00:00