Commit Graph

32 Commits

Author SHA1 Message Date
Andres Beltran
f3518bcdab buildhistory: Fix package output files for SDKs
Currently, installed packages are listed for images in image-info.txt, but
not for SDKs in sdk-info.txt. Add TOOLCHAIN_HOST_TASK and
TOOLCHAIN_TARGET_TASK to the output variables in sdk-info.txt.

Moreover, package output files for the SDK host are empty because
PKGDATA_DIR defaults to the target directory. Fix this bug and create a new
variable called PKGDATA_DIR_SDK which stores the correct path for the SDK
host package data.

(From OE-Core rev: 82e6172c1df378dff4e503aa878501c08937b5bb)

Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14 11:22:18 +01:00
Richard Purdie
bb6ddc3691 Convert to new override syntax
This is the result of automated script conversion:

scripts/contrib/convert-overrides.py <oe-core directory>

converting the metadata to use ":" as the override character instead of "_".

(From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02 15:44:10 +01:00
Fabio Berton
1f3094e775 lib/oe/package_manager: Don't ignore installation failures in install_complementary
When attempt_only flag is set to True all installation failures are
ignored. Disable attempt_only, that is the default in install function,
to raise error in install_complementary and to prevent do_populate_sdk
task to treat errors as warnings.

[RP: Historically some of our addtional packages were in bad shape and
wouldn't all install. This is no longer the case so we shouldn't need
a hack like this any longer.]

(From OE-Core rev: 8042ad9ca91ddaaf741366e1068c63c284ff98f5)

Signed-off-by: Fabio Berton <fabio.berton@criticaltechworks.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-29 22:24:29 +01:00
Armin Kuster
75f691d728 package_manager/rpm: decode systemctl failures
Parse systemctl error to provide more info

[Yocto #14395]

(From OE-Core rev: 9d33a3ad68aeadd359e978b35097ece507a92b14)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-01 14:03:49 +01:00
Richard Purdie
eff192abe2 lib/package_manager: Use shutil.copy instead of bb.utils.copyfile for intercepts
If the scripts/postinst-intercepts is owned by root/root then the copyfile() calls
will fail due to chown issues. We don't care about ownership of these files so
use shutil.copy() instead which won't perform any chown.

(From OE-Core rev: 1a03c70c282b3445b93a4c70ea6d40a1778750c5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06 11:16:24 +01:00
Devendra Tewari
b71375304f classes/lib/scripts: Use bb.utils.rename() instead of os.rename()
Incremental build in Docker fails with:

OSError: [Errno 18] Invalid cross-device link

when source and destination are on different overlay filesystems.

Rather than adding fallback code to every call site, use a new wrapper
in bitbake which detects this case and falls back to shutil.move
which is slower but will handtle the overlay docker filesystems correctly.

[YOCTO #14301]

(From OE-Core rev: 656a65b2b84e7d529b89cf5de7eb838f902d84a2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06 11:16:23 +01:00
hongxu
9ca4b13bd5 deb: apply postinstall on sdk
If not postinstall applied, some nativesdk command could not be found
in sdk due to update-alternatives in postinst not be executed, such as chroot:

$ which chroot
/sbin/chroot
$ which chroot.coreutils
path-to-sdk/sysroots/x86_64-wrlinuxsdk-linux/usr/bin/chroot.coreutils

After applying the fix
$ which chroot
path-to-sdk/sysroots/x86_64-wrlinuxsdk-linux/usr/bin/chroot
$ which chroot.coreutils
path-to-sdk/sysroots/x86_64-wrlinuxsdk-linux/usr/bin/chroot.coreutils

(From OE-Core rev: 2a9bf19502766baa4087456649d5471483d04f6a)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-18 11:37:25 +01:00
Richard Purdie
06e6505abf package_manager/deb: Fix image generation with package removal
When building SDKs with the deb backend you could see errors like:

Setting up nativesdk-python3-ndg-httpsclient (0.5.1-r0) ...
mkdir: cannot create directory ‘/usr/lib/opkg’: Permission denied
dpkg: error processing package nativesdk-python3-ndg-httpsclient (--configure):

which is due to environment misconfiguration when removing packages.

Fix this by setting the same environment variables as used for installation.

(From OE-Core rev: f23c7e319a192ada14bb9a82822ef2967309aaea)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-16 11:28:17 +00:00
Tomasz Dziendzielski
f22225a413 lib/oe/package_manager: Do not pass stderr to package manager as an argument
The cmd redirected stderr to stdout that was then assigned to variable
with pkgs to install. Then this variable was passed to package manager
that then tried to install it and generated confusing warnings.

For example this variable could contain:
| ['(en_US.UTF-8)', 'LC_ALL:', 'bash:', 'cannot', 'change', 'locale', 'setlocale:', 'warning:']

and the warning was:

| WARNING: addon-bci-1.0-r0 do_populate_sdk: Unable to install packages.
| Command 'PATH/usr/bin/opkg ... install (en_US.UTF-8) LC_ALL: bash:
| cannot change locale setlocale: warning:' returned 255:
| Collected errors:
|  * opkg_prepare_url_for_install: Couldn't find anything to satisfy '(en_US.UTF-8)'.

In this change I remove stderr redirection to stdout and pass it to
bb.note instead.

(From OE-Core rev: 70d8ced3d7f53f988c02ff03d8dfa448f088fdf1)

Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-27 10:17:45 +00:00
Ross Burton
bca05547e2 lib/oe/package_manager: ensure repodata is wiped
When building package indexes for RPM feeds, delete repodata first to
be absolutely sure that the indexes are up to date.

We've seen some mysterious failures where the repodata doesn't match
the repository, so hopefully this will stop that happening.

[ YOCTO #14190 ]

(From OE-Core rev: 88cd40c08109bf732934fa4c8d602c701c1eb2a0)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-23 17:08:54 +00:00
Hongxu Jia
57706c0fb3 deb: do not insert feed uris if apt not installed
- The dir /etc/apt was created in package apt, if package apt was not
installed, there is no need to insert package feed. Otherwise, it
will fail with no such dir

- Output the result of apt install

- Explicitly trust the deb package repository from build
This could avoid apt install warning:
...
WARNING: The following packages cannot be authenticated!
...

- Also trust the inserted deb package repository from PACKAGE_FEED_URIS

(From OE-Core rev: 9ec65b77c9a4a0ba240117edee0e84208c58328e)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-16 22:39:36 +00:00
Michael Ho
2fa0b3278c rootfs_ipk: allow do_populate_sdk in parallel to do_rootfs
Switch do_populate_sdk for the ipk package manager to use a separate target
opkg config file and separate the lockfiles restricting do_rootfs and
do_populate_sdk from running in parallel.

This way if an image recipe includes a dependency to do_populate_sdk by
default then it will run in parallel to do_rootfs saving time compared to the
sequential execution.

(From OE-Core rev: 8c210407d07483075a70c8b97ad52b5eae062c9c)

Signed-off-by: Michael Ho <Michael.Ho@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-13 23:13:45 +00:00
Ross Burton
b603dc96b3 package_manager/ipk: improve remove_packaging_data
/var/cache/opkg wasn't being deleted, and /var/lib/opkg doesn't need
to exist as there are no lockfiles that write into it after this step.

(From OE-Core rev: 2209cef2cbe5fbdd5562f13f84ae2a3935f4fb61)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-06 23:17:28 +00:00
Ross Burton
13bc112474 package_manager/ipk: neaten OPKGLIBDIR logic
oe.path.join handles path components starting with / for us.

(From OE-Core rev: ff003e076511fdbe9a6e775e987726dae43e2003)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-03 22:53:09 +00:00
Fredrik Gustafsson
e51345b507 package management: Allow dynamic loading of PM
Dynamic loading of package managers will allow other layers to simply
add their package manager code in package_manager/ and have bitbake find
it according to the package manager configuration. This is useful for
adding new (faster) package managers to Open Embedded while not increasing the
test scope or require Open Embedded to support more package managers.

How this is tested:
* Build core-image-minimal with all three package managers
* Build the sdk with all three package managers. dpkg fails, but
  it fails on master as well.
* Run the complete test suite, all tests passed except 16
* Run those 16 tests on master and verify that they fail there as well
* Fix errors making tests works on master but not with this patch.

(From OE-Core rev: 02670501dea192879ddf9f8048eea57a94719fc1)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-24 15:53:07 +00:00
Richard Leitner
7a9019a8ec deb: export INTERCEPT_DIR for remove actions
During the do_populate_sdk task apt-get purge is called by deb's remove
function. This fails with error messages similiar to the following one
if any of the included packages uses intercepts as the INTERCEPT_DIR
isn't exported:

.../*.postinst: line 4: /postinst_intercept: No such file or directory

Therefore fix it by exporting the INTERCEPT_DIR variable within the
remove function.

(From OE-Core rev: f18adf53dd4bf5dd3adef82b2dcc34a6cdfd0c89)

Signed-off-by: Richard Leitner <richard.leitner@skidata.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-31 08:54:12 +00:00
Richard Leitner
6fd014b7c4 deb: replace deprecated apt force-yes argument
apt-get deprecated --force-yes in favor of various options starting with
--allow [1]. Replace it to avoid the following warning:

W: --force-yes is deprecated, use one of the options starting with --allow instead.

[1] https://salsa.debian.org/apt-team/apt/-/blob/master/debian/changelog

(From OE-Core rev: 4af737e2643f498d1ff4c387207bd8c4f3d405b8)

Signed-off-by: Richard Leitner <richard.leitner@skidata.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-30 13:04:39 +00:00
Michael Thalmeier
acc22a2753 IMAGE_LOCALES_ARCHIVE: add option to prevent locale archive creation
Under some circumstances it is not desirable to create a combined locale
archive (/usr/lib/locale/locale-archive).
The new variable IMAGE_LOCALES_ARCHIVE defaults to '1', so the default
behaviour is not changed.

(From OE-Core rev: 8d78b819c2ec33fce3a34254fa90864ee5fa7617)

Signed-off-by: Michael Thalmeier <michael.thalmeier@hale.at>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-13 09:42:08 +01:00
Richard Purdie
b70c39767c lib/package_manager: Fix missing imports
The package_manager code rearranging had some issues with module imports that
were now missing. Fix all the ones I could spot from quick inspection.

(From OE-Core rev: 287eccd7af7d97604ca68d456c23655fd1b6c40b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-01 19:57:49 +01:00
Fredrik Gustafsson
f5f6ae2986 deb: Move package manager to its own dir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

RP: Fixes to parse/build
(From OE-Core rev: 510d5c48c0496f23a3d7aede76ea8735da2d371d)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
5bc67f5502 ipk: Move package manager to its own dir
This is part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

RP: Fixes to parse/build
(From OE-Core rev: 3ef5a3c885e1010cddfe7eba1cd3728f15270d78)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
c5bea36b72 rpm: Move package manager to its own dir
This is part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

RP: Fixes to parse/build
(From OE-Core rev: 8b776ed9ed291dd8e112621561762449c7eb5ee2)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
635305fe27 deb: Move sdk to its own dir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: 2b305d5f48be84b6362be850d6e0b2bd7f4691ce)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
bfa8f4c098 ipk: Move sdk to its own dir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: 3f9cec50065eec5a02ffcc8ccc2986f2027b44b5)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
e9e2e32999 rpm: Move sdk to its own dir
This is part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: 75066e78d92a23516fd9c6d538c4f991d1504839)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
6fd8b17748 deb: Move rootfs to its own dir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: abadf053371ce863bf21b4a9474eb61761545de1)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
873aee855f ipk: Move rootfs to its own dir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

RP: Fixes to parse/build
(From OE-Core rev: 67fa086589bae484a9beca50b627b007766dcb93)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
b8d3d22ccd rpm: Move rootfs to its own dir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: e4f07444276a88f336d5fe5e60688fcfb2f22df7)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
9129dfe523 deb: Move deb manifest to its own subdir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: f8ee8bc737a982001b9fd0ad441495a52f12e9b4)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
59397b3a01 ipk: Move ipk manifest to its own subdir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: 405cd8560fed2e05fc82919d728c42516793cc0f)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
a0416ca06e rpm: Move manifest to its own subdir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: 87a1c8ee406f73e53888df3b682e8a5f0f610c2f)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
34dc8571d2 package_manager: Move to package_manager/__init__.py
This is part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: 66ccc7a228bf73df0a4dd846bf2c8e99eaa79580)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:09 +01:00