The original idea here was that changes to certain fields might be able
to be explained if there was a change to another field, for example if
RDEPENDS changed it might be because DEPENDS changed. Thus we were
printing this kind of thing out with each change. Unfortunately in
practice this turned out to be noisy and not particularly useful, so we
might as well remove it.
Fixes [YOCTO #7336].
(From OE-Core rev: 8658b3677b9f7cb70806061c41570c709086ef05)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is helpful when exclude a lot of layers. It uses python re, and
supports multiple patterns (separated by space).
(From OE-Core rev: b5170882feb0f3bc2dddc213b6d115dfa87b7cc1)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is better to return False for None. E.g. checking an undefined
variable returned d.getVar().
(From OE-Core rev: 3048e9fa0df6b1edf79bd1723e0fc022c3332af1)
Signed-off-by: Binghua Guan <freebendy@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a function much like shutil.which or bb.utils.which, retaining
shutil.which-like function semantics, bb.utils.which's support for
returning available candidates for signatures, and most importantly,
supports wildcards, returning only the first occurrance of each found
pathname in the search path.
(From OE-Core rev: ca276fe139129eec383d77768ba91b808c462b04)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're in a multilib context already and want a non-multilib context
this function returned incorrect values.
Try and retain optimisations for the common case not needing to request
a datastore but allow the different multilib/non-multilib combinations
to work too.
This fixes bugs where rootfs generation of a multilib image would
write into incorrect locations, or be unable to find sstate manifest
files due to incorrect data stores being used to expand data.
(From OE-Core rev: af7cd7823a8cc95c9f849498b2416cdb0d4d103e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently this function assumes that no multilib is applied and that
we're applying a multilib. This means if we're in multilib context
and want the non-multilib context we can't obtain it (and no other
function exists for this either).
Improve the function to allow this to be requested.
(From OE-Core rev: 295c5a3d19834a2fac255346d0a373449cfdd776)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As of commit 2c5c6e3ff we create multilib variants of intercept
hooks but we did not account for delay_to_first_boot variants.
This was covered up until commit a335e7867, but will now cause
an error.
(From OE-Core rev: 77f7c75481dceec36b7373f277c3bac811de9ef2)
Signed-off-by: Joe Slater <joe.slater@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Exception: TypeError: boolean accepts a string, not '<class 'bool'>
is a bit annoying if you pass in True/False. Tweak the function
to make it forgive that situation.
(From OE-Core rev: 147f5a665fe5073027d92e4acac631f15f08f79f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously a warning was printed regardless of context and nature
of the failure, and because it was only a warning, it was mostly ignored.
Now, the following is considered when a failure happens:
1) whether we are installing packages into a target image, or populating
a SDK with host or target packages.
2) whether the failure was due to qemu not supporting the target machine.
Accordingly, warnings, notes, and failures are printed, and postponing
to first boot happens if possible.
(From OE-Core rev: a335e78672b1e1ae3ea6427f6a805218e513bb52)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If package management is present opkg/dpkg will bring the original
copy of the postinsts scripts with the metadata and will be able to
handle postinsts just fine. In fact, it is preferred to let package
management handle the postinsts scripts in this case since it will
keep the package managers database up-to-date too. The run-postinsts
scripts will make sure the package manager gets invoked instead of
the scripts directly.
Note: Before commit 5aae19959a44 ("rootfs.py: Change logic to
unistall packages") rootfs.py did not install /etc/$pm-postinsts
too. It is not clear whether that change was intentionally or just
a bug. This commit fixes/reverts that aspect of the commit.
(From OE-Core rev: 4b571c59e3e6bd8b22a63f547a95757aeba5b638)
Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When generating the extended SDK, there is a copy step where this class
goes through the layers and other stuff that have been copied to
generate the SDK. The corebase; ie. the folder that contains the core
layer 'meta' is treated in a special way. Unfortunately in our tree, we
have:
sources/meta/meta
| `- core layer
`------- corebase
In populate_sdk_ext's copy_buildsystem, the heuristic to determine which
element of the list returned by copy_bitbake_and_layers is corebase is
fooled by such layout.
In copy_bitbake_and_layers, corebase is already handled specifically and
reliably, so we should let that function tell us which folder is
corebase instead of trying to determine it.
To do so, change the return type of copy_bitbake_and_layers to a tuple
that contains (corebase, copied_layers). It also simplifies the code on
the caller side.
(From OE-Core rev: 5368bc5d0d3606198b93e877bcafcd77bb5f4fd1)
Signed-off-by: Damien Riegel <damien.riegel@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was undocumented, and it's better to just always enable
full debug output, as this allows immediate generation of logs
with full diagnostics when things go not as expected.
Also, change the output of dnf from note to debug level; this
does not affect what is written to log file, but does reduce the
verbosity of bitbake -v.
(From OE-Core rev: 9128fd1396729a71b4832a597cf070c2be922d63)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If package management has been disabled then we've already removed all the
state, and running 'dnf clean all' again will simply recreate a lot of the
files.
(From OE-Core rev: 4524068ad2248b37fb08a24828d018e2f7e6a761)
(From OE-Core rev: f23332dc6a0c5b49246b4d17ec4e6fcad0cb7d47)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If package management is disabled we remove the state and configuration for the
package manager after the rootfs has been populated. This list wasn't complete
and the DNF/RPM configuration files were left behind.
As we've added files to the list (and not just directories), expand the
backup/restore package management state code to handle this.
(From OE-Core rev: c8a6422457b4d29e6afd1e193b8b921287472137)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The isELF function works by running:
result = file <pathname>
if 'ELF' in result
By default 'file' will prepend the result with the path name of the file
that is being checked. This usually works fine, such as:
$ file /home/foo/openembedded-core/meta/classes/package.bbclass
/home/foo/openembedded-core/meta/classes/package.bbclass: Python script, ASCII text executable, with very long lines
However, if the path includes 'ELF', ELF will end up in the result, and then
the check will return positive.
$ file /home/ELF/openembedded-core/meta/classes/package.bbclass
/home/ELF/openembedded-core/meta/classes/package.bbclass: Python script, ASCII text executable, with very long lines
This will then result in the isELF coming back true, and possibly causing the
checks that use isELF, such as the 'is it already stripped' check, to do the
incorrect thing.
Adding the '-b' option to file will result in the path being omitted in the
result:
$ file /home/ELF/openembedded-core/meta/classes/package.bbclass
Python script, ASCII text executable, with very long lines
(From OE-Core rev: 5a324e9b2cf6378f8eaa4e394f9cb36d4e2680ac)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously this wasn't done, and so any packages installed from
populate_sdk would not have the postinsts fully executed
(particularly generation of various caches via running nativesdk or target
binaries with qemu wasn't working).
[YOCTO #12630]
(From OE-Core rev: a484ff072eea3f47de2c3348048201249cefa46b)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows running the intercepts when creating SDKs, which previously
wasn't possible, as SDK code does not use the rootfs class, and calls
into PackageManager methods directly.
(From OE-Core rev: f830388c5e9125f385a42acd7365d1235967b57c)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will allow handling postinst_intercepts when populating SDKs (which
use PackageManager class directly, and do not utilize RootFS class).
(From OE-Core rev: 9454fd328040fd58c981d028a74fcf181bde8e89)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will be useful when also moving postinst_intercept handling to
package manager class from rootfs class.
(From OE-Core rev: a4cd69bdd5b9dfa1125887f4d9038d41996e39c7)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default, RPM_SIGN_PACKAGES is not defined. Add gpgcheck=0 to
oe-remote-repo.repo file, otherwise dnf will complain during
install operation on target
Note, RPM_SIGN_PACKAGES is set only when you inherit sign_rpm explicitly
(From OE-Core rev: 002a71eaa7606828c399972d8fd35e19e7b71929)
Signed-off-by: Manjukumar Matha <manjukumar.harthikote-matha@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of assuming order has changed if no package has been added or
removed, loop through packages to check if order has changed. This will
prevent the script from falsely reporting "changed order" if a version
has increased.
Fixes [YOCTO #12334]
(From OE-Core rev: 77d701c5fb5961bd818810a4d4cb3a9bd2432fae)
Signed-off-by: Amanda Brindle <amanda.r.brindle@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* --no-signature saves unnecessary .patch modifications when executed on
host with different git version
* --no-numbered saves unnecessary .patch modifications when number of the
applied patches is changed (the number is still in the filename so the
order how they should be applied is still preserved)
* both options exist for very long time, I've tested them with git 1.9.1
from Ubuntu 14.04 and I'm quite sure they were available even in much
older releases, so there shouldn't be any issue on relatively new sanity
tested distros
(From OE-Core rev: ad76fa92c3a5be38962aff09df070ffd9756f777)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ignoring patch context increases the chances of patches being
applied incorrectly. Depending on what code is being patched, this can go
completely unnoticed and create subtle bugs, sometimes with security implications.
Please see here for a specific example:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=10450
On the other hand, we cannot simply force all patch context to match exactly:
doing this would break a lot of recipes suddenly, across all layers.
So let's try a softer approach: issue a warning, and gently update
patches over a longer span of time. When most of the warnings are eliminated,
we can start enforcing a strict patch application policy.
I do understand that this patch creates a lot of warnings all of a sudden, however
I believe the problem does need to be addressed. All of oe-core recipes have their
context already fixed.
Sample warning:
WARNING: vulkan-1.0.61.1-r0 do_patch:
Some of the context lines in patches were ignored. This can lead to incorrectly applied patches.
The context lines in the patches can be updated with devtool:
devtool modify <recipe>
devtool finish --force-patch-refresh <recipe> <layer_path>
Then the updated patches and the source tree (in devtool's workspace)
should be reviewed to make sure the patches apply in the correct place
and don't introduce duplicate lines (which can, and does happen
when some of the context is ignored).
Details:
Applying patch demos-Don-t-build-tri-or-cube.patch
patching file demos/CMakeLists.txt
Hunk #1 succeeded at 63 (offset 2 lines).
Hunk #2 succeeded at 76 with fuzz 1 (offset 2 lines).
[YOCTO #10450]
(From OE-Core rev: 5133fd46bccf14e21680f8d94e952914edccb113)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously this was done only for rpm packages; now also ipk/deb scriptlet
failures are reported.
In the future this will become a hard error, but it can't yet happen
due to the legacy 'exit 1' way of deferring scriptlet execution to first boot which
needs a deprecation period.
(From OE-Core rev: a36671faf6e0b7623185b0e22814a786d5444592)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Particularly, this was causing 'devtool modify' to erroneously add those
.orig files into commits. This was getting in the way, if the goal
was to amend/update those existing patches.
(From OE-Core rev: f4f3406c3bd9599d7a19275475d52bda4c42f2ab)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running bitbake -c populate_sdk <image_name>, it is expected that
packages matching SDKIMAGE_INSTALL_COMPLEMENTARY name mask (unless
declared in PACKAGE_EXCLUDE_COMPLEMENTARY) are installed to resulting
SDK. Underlying mechanism issues a package manager install call for set
of complementary packages. However the mechanism doesn't seem to inform
the user all too obviously in case the package manager command behind
install_complementary() method fails -- and since it is combined with
attempt_only=True option, user might end up wondering why several *-dev,
*-dbg packages are missing from resulting SDK.
Improve associated install() method behaviour in affected OpkgPM and
DpkgPM classes so that a problematic state of affairs becomes directly
obvious for bitbake user, resulting in shell output like:
WARNING: someimage-1.0-r0 do_populate_sdk: Unable to install packages.
Command '...' returned 1:
Collected errors:
* Solver encountered 1 problem(s):
* Problem 1/1:
* - package somepkg-dev-1.0-r0.x86 requires somepkg = 1.0-r0, but
none of the providers can be installed
*
* Solution 1:
* - allow deinstallation of someotherpkg-1.1-r1.x86
* - do not ask to install a package providing somepkg-dev
* Solution 2:
* - do not ask to install a package providing somepkg-dev
(From OE-Core rev: 2502bd591c37bf532d02dc6b37fc1e8b5224fb0a)
Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In cases when a package like qemu might have files with same names
in multiple directories, the rename logic might go wrong and create
multiple rename pair for a single directory.
Make sure that we process each rename pair once. Also, don't print
FILELIST as part of PKGSIZE to ensure that it gets printed only once
when reporting package changes.
Fixes [YOCTO #12559]
(From OE-Core rev: cff000c43d6e9a183911338951026dfbef88f838)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you have a package that does not generate a manifest due to using a
noexec rule, the package name should be printed so the problem can be
tracked down. With out the patch you get an error that makes it look
more like the package_manager is broken as shown below.
oe-core/meta/lib/oe/package_manager.py', lineno: 534, function: create_packages_dir
0530:
0531: for dep in rpmdeps:
0532: c = taskdepdata[dep][0]
0533: manifest, d2 = oe.sstatesig.find_sstate_manifest(c, taskdepdata[dep][2], taskname, d, multilibs)
*** 0534: if not os.path.exists(manifest):
0535: continue
0536: with open(manifest, "r") as f:
0537: for l in f:
0538: l = l.strip()
File: '/usr/lib/python3.5/genericpath.py', lineno: 19, function: exists
0015:# This is false for dangling symbolic links on systems that support them.
0016:def exists(path):
0017: """Test whether a path exists. Returns False for broken symbolic links"""
0018: try:
*** 0019: os.stat(path)
0020: except OSError:
0021: return False
0022: return True
0023:
Exception: TypeError: stat: can't specify None for path argument
(From OE-Core rev: 21924fdba286e5962b1680601664dc0491527e25)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using glibc-locale to install locales only makes sense if we're using glibc.
(From OE-Core rev: 8fc80734053645fa893694dfe33ddaee99aa9a1a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- No need to use bb.utils.which() as subprocess will search $PATH
- Clarity flow by moving the install inside the try/except
(From OE-Core rev: f4d22b7195dd8f08fe26dd353c7e860208e87d6a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prompted by bitbake commit 2ba8a6b25ccc12e7b543e8450121e5311c7a701d,
add .txz to the list of archives used within get_recipe_local_files()
here as well.
(From OE-Core rev: 28466586f4777f11f0a29ed5ebe9734a4e15920e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In a few places we have checks to see path B is the parent of path A, by
adding / to the end of the path B and then seeing if path A starts with
the suffixed path B. Unfortunately there are two potential flaws:
(1) path A needs to be suffixed with / as well or the directory itself
won't match (semantics perhaps, but in a lot of scenarios returning True
is correct); (2) you need to run os.path.abspath() on both paths first
or you will wrongly return False for some relative paths where you
should return True. Let's solve this once and for all by writing a
function that takes care of these and put it in oe.path.
(From OE-Core rev: dd3d4b0367272a5826a9a51afa26f426dd003e5d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS supports the wildcard syntax,
these exclusions no longer need to be hardcoded
(From OE-Core rev: 891ca3c891f2ed5476442442bd9784ba10b9a94d)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The special string "*" on the left hand side of the dependency
specification matches all recipes except the one on the right hand side.
(From OE-Core rev: 526855f6573317ced913f39e878beac1d5d294de)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changes to the icecc-create-env recipe should not cause all recipes to
rebuild just because the have inherited icecc.
(From OE-Core rev: b088a9c34c34c4bda54d93907e04972d3f8ba5a1)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a common function for locating task manifest files rather than
several implementations with missing pieces.
(From OE-Core rev: 68150bac7444f089f19c789e9f6602d59f605d7a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently do_rootfs gets to see all rpms in the deploy directory. This filters
that view to only rpms which the image recipe has actual depends upon which
potentially removes some sources of confusion in the image construction.
This makes builds more reproducibile and also fixes contamination issues
where dnf picks up packages it shouldn't be able to 'see'.
[YOCTO #12039]
(From OE-Core rev: 85e72e129362db896b0d368077033e4a2e373cf9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
parallel_make_argument() was incorrectly taking the maximum of the limit
and the calculated value instead of the minimum.
(From OE-Core rev: 45205be547967c84dff96403c3a6825a62e3ca6a)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code to extract the integer number of parallel build threads and
construct a new argument from them has started to be copied in multiple
locations, so create two new helper utilities to aid recipes.
The first helper (parallel_make()) extracts the integer number of
parallel build threads from PARALLEL_MAKE. The second
(parallel_make_argument()) does the same and then puts the result back
into a format string, optionally clamping it to some maximum value.
Additionally, rework the oe-core recipes that were manually doing this
to use the new helper utilities.
(From OE-Core rev: ccd1142d22b31ed85d8823b1bc9e11ccfd72b61f)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that the first boot deferral needs to be requested explicitly,
it's not really something to be concerned about.
(From OE-Core rev: 19f3f4c60060ff5ed3c1c9871e18fcd4d128c5a3)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'exit 1' is not optimal for two reasons:
1) Code is hard to read; it is not obvious that it means 'defer what follows to first boot'.
2) Worse, this hides actual errors in the scriptlets; there is no difference between scriptlet
failing because it's intended to be run on target and scriptlet failing because there's a bug or
a regression somewhere.
The new, supported way is to place the code that has to run on target into pkg_postinst_ontarget(),
or, if a more fine-tuned control is required, call 'postinst-intercepts defer_to_first_boot' from
pkg_postinst() to explicitly request deferral to first boot.
(From OE-Core rev: d12cf56e9ff2a4f13dfbef9290ea5647b52b3f6d)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
That hook is empty, and doesn't need to be executed; it merely indicates that packages
that have used it are requesting to defer their postinst scripts to first boot
unconditionally.
(From OE-Core rev: 939f7f1a06cd2db05aeb5e75a66322314e10aa6d)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
