Commit Graph

1000 Commits

Author SHA1 Message Date
Tomasz Dziendzielski
f22225a413 lib/oe/package_manager: Do not pass stderr to package manager as an argument
The cmd redirected stderr to stdout that was then assigned to variable
with pkgs to install. Then this variable was passed to package manager
that then tried to install it and generated confusing warnings.

For example this variable could contain:
| ['(en_US.UTF-8)', 'LC_ALL:', 'bash:', 'cannot', 'change', 'locale', 'setlocale:', 'warning:']

and the warning was:

| WARNING: addon-bci-1.0-r0 do_populate_sdk: Unable to install packages.
| Command 'PATH/usr/bin/opkg ... install (en_US.UTF-8) LC_ALL: bash:
| cannot change locale setlocale: warning:' returned 255:
| Collected errors:
|  * opkg_prepare_url_for_install: Couldn't find anything to satisfy '(en_US.UTF-8)'.

In this change I remove stderr redirection to stdout and pass it to
bb.note instead.

(From OE-Core rev: 70d8ced3d7f53f988c02ff03d8dfa448f088fdf1)

Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-27 10:17:45 +00:00
Ross Burton
bca05547e2 lib/oe/package_manager: ensure repodata is wiped
When building package indexes for RPM feeds, delete repodata first to
be absolutely sure that the indexes are up to date.

We've seen some mysterious failures where the repodata doesn't match
the repository, so hopefully this will stop that happening.

[ YOCTO #14190 ]

(From OE-Core rev: 88cd40c08109bf732934fa4c8d602c701c1eb2a0)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-23 17:08:54 +00:00
Lee Chee Yang
3807c6d9a7 cve-check: replace Looseversion with custom version class
The way distutils.version.LooseVersion compare version are tricky, it treat
all these ( "1.0-beta2", "1.0-rc1", "1.0A", "1.0p2" and "1.0pre1") as greater
version than "1.0". This might be right for "1.0A" and "1.0p1" but not for
the rest, also these version could be confusing, the "p" in "1.0p1" can be
"pre" or "patched" version or even other meaning.

Replace Looseversion with custom class, it uses regex to capture common
version format like "1.1.1" or tag format using date like "2020-12-12" as
release section, check for following known string/tags ( beta, rc, pre, dev,
alpha, preview) as pre-release section, any other trailing characters
are difficult to understand/define so ignore them. Compare release
section and pre-release section saperately.

included selftest for the version class.

[YOCTO#14127]

(From OE-Core rev: 6ced85e9ddd3569240f1e8b82130d1ac0fffbc40)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-23 17:08:54 +00:00
Hongxu Jia
57706c0fb3 deb: do not insert feed uris if apt not installed
- The dir /etc/apt was created in package apt, if package apt was not
installed, there is no need to insert package feed. Otherwise, it
will fail with no such dir

- Output the result of apt install

- Explicitly trust the deb package repository from build
This could avoid apt install warning:
...
WARNING: The following packages cannot be authenticated!
...

- Also trust the inserted deb package repository from PACKAGE_FEED_URIS

(From OE-Core rev: 9ec65b77c9a4a0ba240117edee0e84208c58328e)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-16 22:39:36 +00:00
Anton Kachalov
49c4fdcb88 rootfs: add option to allow delayed postinsts on read-only rootfs
Example use case in OpenBMC: rootfs is squashfs and the system has either
overlayfs for whole rootfs or for some parts (e.g. /etc).

This option will allow to create migration one-shot postinsts using
"pkg_postinst_ontarget_${PN}" routines defined in recipes to fix
files under upper workdir in overlayfs.

(From OE-Core rev: 0977204e16279b117811b5d5cdac5918287e95ac)

Signed-off-by: Anton D. Kachalov <rnouse@google.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-16 22:39:35 +00:00
Tomasz Dziendzielski
ca608639f8 lib/oe/utils: Return empty string in parallel_make
In cmake.bbclass we set CMAKE_BUILD_PARALLEL_LEVEL using parallel_make
function and if PARALLEL_MAKE is set to empty string then this variable
is exported as "None" causing cmake to fail with:
"'CMAKE_BUILD_PARALLEL_LEVEL' environment variable
invalid number 'None' given."

(From OE-Core rev: 2f790ded554a52ac18d1c28002142f9c62abec8b)

Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-16 22:39:35 +00:00
Michael Ho
2fa0b3278c rootfs_ipk: allow do_populate_sdk in parallel to do_rootfs
Switch do_populate_sdk for the ipk package manager to use a separate target
opkg config file and separate the lockfiles restricting do_rootfs and
do_populate_sdk from running in parallel.

This way if an image recipe includes a dependency to do_populate_sdk by
default then it will run in parallel to do_rootfs saving time compared to the
sequential execution.

(From OE-Core rev: 8c210407d07483075a70c8b97ad52b5eae062c9c)

Signed-off-by: Michael Ho <Michael.Ho@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-13 23:13:45 +00:00
Ross Burton
949646a9e9 lib/oe/qa: handle the 'no specific instruction set' ELF e_machine value
[RP: Update OEQA selftest to match change]
(From OE-Core rev: b7cfc0f51cc0b4866f913f6eae4fcc6f72d2578c)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-20 00:03:05 +00:00
Peter Kjellerstedt
c37aed34af lib/oe/path: Add canonicalize()
oe.path.canonicalize() is used to canonicalize paths (i.e., remove
symbolic links and "..", and make them absolute). It takes a string
with paths separated by commas, and returns the canonicalized path in
the same format.

(From OE-Core rev: 282b19c0e27488ec119f00fb2542ffdc1af54e2a)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-20 00:03:04 +00:00
Alexander Kanavin
9b534f334d meta/lib/oe/reproducible.py: gitsm:// works just as fine as git:// for timestamps
This in particular addresses vulkan-samples reproducibility which made me scratch my
head for a while.

(From OE-Core rev: 4a2936126f12eeacecced051fa339c32c1f16576)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-09 12:20:33 +00:00
Ross Burton
b603dc96b3 package_manager/ipk: improve remove_packaging_data
/var/cache/opkg wasn't being deleted, and /var/lib/opkg doesn't need
to exist as there are no lockfiles that write into it after this step.

(From OE-Core rev: 2209cef2cbe5fbdd5562f13f84ae2a3935f4fb61)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-06 23:17:28 +00:00
Ross Burton
13bc112474 package_manager/ipk: neaten OPKGLIBDIR logic
oe.path.join handles path components starting with / for us.

(From OE-Core rev: ff003e076511fdbe9a6e775e987726dae43e2003)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-03 22:53:09 +00:00
Richard Purdie
3ecf5d9692 uninative: Don't use single sstate for pseudo-native
pseudo-native is a bit special. It conditionally compiles in support for
xattr, statx and statvfs amongst other options. If a pseudo-native binary is
used on a system where these functions are present but it wasn't compiled in
we see hard to debug permissions problems.

An example is the devtool.DevtoolExtractTests.test_devtool_deploy_target
oe-selftest which shows a cryptic error:

  File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/devtool.py", line 1388, in test_devtool_deploy_target
    self.assertEqual(filelist1, filelist2)
  File "/usr/lib64/python3.9/unittest/case.py", line 831, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib64/python3.9/unittest/case.py", line 1037, in assertListEqual
    self.assertSequenceEqual(list1, list2, msg, seq_type=list)
  File "/usr/lib64/python3.9/unittest/case.py", line 1019, in assertSequenceEqual
    self.fail(msg)
  File "/usr/lib64/python3.9/unittest/case.py", line 670, in fail
    raise self.failureException(msg)

AssertionError: Lists differ: ['-rwxr-xr-x 6000 6000 /etc/init.d/mdmonitor', '-rw-r-[10124 chars]n.8'] != ['-rwxr-xr-x root root /etc/init.d/mdmonitor', '-rw-r-[10124 chars]n.8']

First differing element 0:
'-rwxr-xr-x 6000 6000 /etc/init.d/mdmonitor'
'-rwxr-xr-x root root /etc/init.d/mdmonitor'

This is due to a version of pseudo without statx being used on a system where
ls uses statx, hence the files are displayed as 6000.6000 instead of root.root.

Avoid this by always building pseudo-native for the specific distro in question rather
than using a universal sstate feed.

This hopefully fixes one of the mysterious AB-INT issues.

(From OE-Core rev: 6e3785a3f1f3cf68f5fe101cd6bebe91db165973)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-24 15:53:07 +00:00
Fredrik Gustafsson
e51345b507 package management: Allow dynamic loading of PM
Dynamic loading of package managers will allow other layers to simply
add their package manager code in package_manager/ and have bitbake find
it according to the package manager configuration. This is useful for
adding new (faster) package managers to Open Embedded while not increasing the
test scope or require Open Embedded to support more package managers.

How this is tested:
* Build core-image-minimal with all three package managers
* Build the sdk with all three package managers. dpkg fails, but
  it fails on master as well.
* Run the complete test suite, all tests passed except 16
* Run those 16 tests on master and verify that they fail there as well
* Fix errors making tests works on master but not with this patch.

(From OE-Core rev: 02670501dea192879ddf9f8048eea57a94719fc1)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-24 15:53:07 +00:00
Alexander Kanavin
acfdab8daa meta: drop _PYTHON_SYSCONFIGDATA_NAME hacks
(From OE-Core rev: d3a81dd0e72a3495bfc7cc969c2bb806b666023d)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-16 14:27:46 +00:00
Richard Purdie
235101a662 sstatesig: Remove workaround for bitbake taskhash bug
When trying to lock an individual signature, we see the checksum calculations
of dependent tasks failing. The fix is to remove a bad optimisation within
bitbake but with the removed, we need to remove some bogus code with
OE-Core's sstatesig code too.

(From OE-Core rev: 9923392539b1ce6d70f713527373d6bbc03f3021)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-03 08:21:10 +00:00
Richard Leitner
7a9019a8ec deb: export INTERCEPT_DIR for remove actions
During the do_populate_sdk task apt-get purge is called by deb's remove
function. This fails with error messages similiar to the following one
if any of the included packages uses intercepts as the INTERCEPT_DIR
isn't exported:

.../*.postinst: line 4: /postinst_intercept: No such file or directory

Therefore fix it by exporting the INTERCEPT_DIR variable within the
remove function.

(From OE-Core rev: f18adf53dd4bf5dd3adef82b2dcc34a6cdfd0c89)

Signed-off-by: Richard Leitner <richard.leitner@skidata.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-31 08:54:12 +00:00
Richard Purdie
98df63020d scripts/buildhistory_analysis: Avoid tracebacks from file comparision code
We're seeing tracebacks from buildhistory analysing the python 3.8 -> 3.9
upgrade due to the significant file renames. Avoid these by checking before
removal as they can happen multiple times.

(From OE-Core rev: b1eb390bbcb995c0da70478e17f9170721c75341)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-30 13:04:39 +00:00
Richard Leitner
6fd014b7c4 deb: replace deprecated apt force-yes argument
apt-get deprecated --force-yes in favor of various options starting with
--allow [1]. Replace it to avoid the following warning:

W: --force-yes is deprecated, use one of the options starting with --allow instead.

[1] https://salsa.debian.org/apt-team/apt/-/blob/master/debian/changelog

(From OE-Core rev: 4af737e2643f498d1ff4c387207bd8c4f3d405b8)

Signed-off-by: Richard Leitner <richard.leitner@skidata.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-30 13:04:39 +00:00
Richard Purdie
cc840dfc9d sstatesig: Log timestamps for hashequiv in reprodubile builds for do_package
Currently if a task generates the same output with different timestamps,
hasequiv won't detect it but reproducibile builds will fail tests due
to the different timestamps.

Add do_package timestamps to the hash when reproducibile builds are enabled
to avoid this.

(From OE-Core rev: 11e8200ccec765ff6a4263e06512e5751eca261a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-30 13:04:39 +00:00
Konrad Weihmann
7048d3d1f0 lib/oe/rootfs: introduce IMAGE_LOG_CHECK_EXCLUDES
When using rpm as package manager and trying to install a file called
'/usr/share/doc/What to when an Error occurs.txt'
log_check falsely errors out on the build, because used regex match on
'DEBUG:  Removing manifest: /path/usr/share/doc/What
to when an Error occurs.txt'.
To handle such cases introduce IMAGE_LOG_CHECK_EXCLUDES, to allow
user to add custom exclude regex to log_check exclude list

(From OE-Core rev: 7c7555a79b8bdef0a8d7fbd57e9ddf059066de76)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-20 11:11:46 +01:00
Michael Thalmeier
acc22a2753 IMAGE_LOCALES_ARCHIVE: add option to prevent locale archive creation
Under some circumstances it is not desirable to create a combined locale
archive (/usr/lib/locale/locale-archive).
The new variable IMAGE_LOCALES_ARCHIVE defaults to '1', so the default
behaviour is not changed.

(From OE-Core rev: 8d78b819c2ec33fce3a34254fa90864ee5fa7617)

Signed-off-by: Michael Thalmeier <michael.thalmeier@hale.at>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-13 09:42:08 +01:00
Richard Purdie
a2db4fa127 base/bitbake.conf: Enable pseudo path filtering
This is a pretty big change to the way pseudo operates when used in OpenEmbedded.
Normally, pseudo monitors and logs (adds to its database) any file created or
modified whilst in a fakeroot environment. There are large numbers of files
we simply don't care about the permissions of whilst in that fakeroot context,
for example ${S}, ${B}, ${T}, ${SSTATE_DIR}, the central sstate control
directories,

This change uses new functionality in pseudo to ignore these directory trees,
resulting in a cleaner database with less chance of "stray" mismatches if files
are modified outside pseudo context. It also should reduce some overhead from
pseudo as the interprocess round trip to the server is avoided.

There is a possible complication where some existing recipe may break, for
example, we found a recipe which was writing to "${B}/install" for
"make install" in do_install and since we listed ${B} as not to be tracked,
there were errors trying to chown root for files in this location.

This patch fixes a few corner cases in OE-Core when used with this new
ignore list:

* The archiver directory matched a "${WORKDIR}/deploy*" pattern so was renamed
  to something else since that directory does need its root permissions
* The ${S} and ${B} ignoring is conditional on them being different to ${WORKDIR}
* package_write_* task output (the debs/rpms/ipks) are now owned by the build
  user so we don't want the file ownership information in the hashequiv outhash
  calculation even if they are built under pseudo.
* The fontcache postinstall intercept is run under qemu outside of pseudo context
  so delete files it may delete up front where pseudo can see this.
* SSTATE_DIR is in PSEUDO_PATHS_IGNORE, which is in FAKEROOTENV which is cached
  by bitbake. We therefore need to trigger reparsing if this changes, which means
  SSTATE_DIR can be in BB_HASHBASE_WHITELIST but not BB_HASHCONFIG_WHITELIST.
  Rework the variables to handle this. This otherwise breaks some of our sstate
  tests in oe-selftest.
* Ignore the temp directory wic uses for rebuilding rootfs.

(From OE-Core rev: ad8f5532ffaead9a5ad13e1034fe9e5e1b7979f4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-07 11:18:38 +01:00
Ross Burton
c2a11e18c8 utils: respect scheduler affinity in cpu_count()
cpu_count() returns multiprocessing.cpu_count() but that is simply returns
os.cpu_count() so we could use that directly.

However this returns the number of CPUs on the host, not the number of
usable CPUs on the host. If the user is using scheduler affinity then
the number of usable CPUs may be less, so when determining how many cores
we can use check the affinity instead.

(From OE-Core rev: e2e8ccbe410b2f38bcd9525982b2261cf71aab60)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30 15:01:51 +01:00
Naveen Saini
3204982fd7 lib/oe/recipeutils.py: add support for BBFILES_DYNAMIC
Instead of relying on value of BBFILES from bitbake, devtool
parses the layer.conf because the layer might not be in bblayers.conf.
And it currently does not consider the value of BBFILES_DYNAMIC because
of which recipes, in paths defined by BBFILES_DYNAMIC, upgraded
using devtool end up in wrong location.

Include the code from bitbake to append values to BBFILES based on
what is in BBFILES_DYNAMIC too.

(From OE-Core rev: c7bbb98ea8ccd3568dd8bded6e404e2f781e6841)

Signed-off-by: Naveen Saini <naveen.kumar.saini@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-27 08:28:03 +01:00
Richard Purdie
b70c39767c lib/package_manager: Fix missing imports
The package_manager code rearranging had some issues with module imports that
were now missing. Fix all the ones I could spot from quick inspection.

(From OE-Core rev: 287eccd7af7d97604ca68d456c23655fd1b6c40b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-01 19:57:49 +01:00
Joshua Watt
20e9df5721 lib/oe/reproducible.py: Fix git HEAD check
The check for a git HEAD still wasn't quite correct because it was using
the .git directory as the current working directory. Instead, it should
be passed as the --git-dir argument when running git. Running `git
rev-parse HEAD` in a .git directory with no HEAD reports 'HEAD' and
exits with success but then 'git log' will fail, which is not what we
want.

(From OE-Core rev: cdbd47dd7e1657b91b65a0940b7cbf119764240f)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-29 11:37:31 +01:00
Richard Purdie
4e3240220b Revert "lib/oe/log_colorizer.py: add LogColorizerProxyProgressHandler"
This reverts commit 312fb3c86a3d84e60867b132666c01859f73ceb2 as this
wasn't meant to merge as yet.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-29 11:35:51 +01:00
Chris Laplante
1739fee633 lib/oe/log_colorizer.py: add LogColorizerProxyProgressHandler
This progress handler intercepts log output, stripping any ANSII color
escape codes. Then the stripped output is fed to the underlying progress
handler which will render the progress bar as usual.

(From OE-Core rev: 312fb3c86a3d84e60867b132666c01859f73ceb2)

Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-29 10:12:31 +01:00
Fredrik Gustafsson
f5f6ae2986 deb: Move package manager to its own dir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

RP: Fixes to parse/build
(From OE-Core rev: 510d5c48c0496f23a3d7aede76ea8735da2d371d)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
5bc67f5502 ipk: Move package manager to its own dir
This is part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

RP: Fixes to parse/build
(From OE-Core rev: 3ef5a3c885e1010cddfe7eba1cd3728f15270d78)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
c5bea36b72 rpm: Move package manager to its own dir
This is part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

RP: Fixes to parse/build
(From OE-Core rev: 8b776ed9ed291dd8e112621561762449c7eb5ee2)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
635305fe27 deb: Move sdk to its own dir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: 2b305d5f48be84b6362be850d6e0b2bd7f4691ce)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
bfa8f4c098 ipk: Move sdk to its own dir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: 3f9cec50065eec5a02ffcc8ccc2986f2027b44b5)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
e9e2e32999 rpm: Move sdk to its own dir
This is part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: 75066e78d92a23516fd9c6d538c4f991d1504839)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
6fd8b17748 deb: Move rootfs to its own dir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: abadf053371ce863bf21b4a9474eb61761545de1)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
873aee855f ipk: Move rootfs to its own dir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

RP: Fixes to parse/build
(From OE-Core rev: 67fa086589bae484a9beca50b627b007766dcb93)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
b8d3d22ccd rpm: Move rootfs to its own dir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: e4f07444276a88f336d5fe5e60688fcfb2f22df7)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
9129dfe523 deb: Move deb manifest to its own subdir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: f8ee8bc737a982001b9fd0ad441495a52f12e9b4)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
59397b3a01 ipk: Move ipk manifest to its own subdir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: 405cd8560fed2e05fc82919d728c42516793cc0f)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
a0416ca06e rpm: Move manifest to its own subdir
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: 87a1c8ee406f73e53888df3b682e8a5f0f610c2f)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:10 +01:00
Fredrik Gustafsson
34dc8571d2 package_manager: Move to package_manager/__init__.py
This is part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.

(From OE-Core rev: 66ccc7a228bf73df0a4dd846bf2c8e99eaa79580)

Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:09 +01:00
Naoto Yamaguchi
49e92f8d6d patch.py: Change to more strictly fuzz detection
When applying patch subject is including a " fuzz ", do_patch_qa detect
Fuzz.

After the patch is applied, the following log message appears.

  Applying: meson: treat all fuzz cases as unit tests

The current Fuzz detection checks for the presence of "fuzz" in this
log message.  The log in this example will be treated as Fuzz,
despite its success.

This patch change to more strictly fuzz detection.
if log message is including " fuzz " and "Hunk " in log message,
it will be treated as Fuzz.

(From OE-Core rev: a8605c66ef5afe7c3583366781dfd90fe3526398)

Signed-off-by: Naoto Yamaguchi <wata2ki@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27 19:58:09 +01:00
Joshua Watt
5843460575 lib/oe/reproducible: Fix error when no git HEAD
Fixes an error that occurs when attempting to get the timestamp of the
latest commit when there is no HEAD in the git repository. The easiest
way to trigger this condition is to use the 'subdir=' option when
specifying a 'git://' SRC_URI.

(From OE-Core rev: a64caca5b5dbe4a76acd0b5709b2c3e75b245863)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-22 12:45:56 +01:00
Joshua Watt
ebb40b027c classes/reproducible: Move to library code
Moves most of the python code used for dealing with the source date
epoch to library code.

(From OE-Core rev: a7ede90955bc0c8bec1cbb3cab498ef2583b2f4e)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-22 12:45:56 +01:00
Tim Orling
c9ba6e7bfe lib/oe/recipeutils.py: add AUTHOR; BBCLASSEXTEND
If you try to create a plugin for recipetool that
adds the AUTHOR field, it is impossible to put it
in the recommended position [1] without adding to the
recipe_progression variable.

While we are at it, also add BBCLASSEXTEND at the
end, as also recommended by [1].

[1] http://www.openembedded.org/wiki/Styleguide

(From OE-Core rev: d687c5b7b10b3decdd80d5c2fd61072a87f061f2)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-12 22:39:41 +01:00
Damian Wrobel
061eb6f501 rootfs: do not let ldconfig to create symlinks
Adds -X option to ldconfig to not create symlinks
as part of the ldconfig cache creation process.

It is much better to keep all needed symlinks as part of
of the package then let ldconfig do the job.

In the case symlink was created by ldconfig and later on the
package got removed the dangling link will remain on the
filesystem. Not to mention that such a symlink is orphaned
(e.g. rpm -qf <file-path> will not show the package the symlink
is part of).

It also align the behaviour with libc-musl where the ldconfig
is not invoked by default.

(From OE-Core rev: ad4ff7c18cc6a5c15a0525149dbe2dbd5ae39129)

Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-03 11:38:24 +01:00
Joshua Watt
48a883bb6e sstatesig: Account for all dataCaches being passed
Bitbake now passes all the dataCaches to the taskhash API, so use this
to correctly filter mcdepends.

[YOCTO #13724]

(From OE-Core rev: 749731a420fb905b6af97ce2909f06b2bcd14fe2)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-15 14:53:46 +01:00
Richard Purdie
eb226b897f buildhistory: Add simplistic file move detection
We'd like to use buildhistory more during patch review however its
proving hard, particularly where whole subtrees of files move,
such as a kernel version upgrade, or where a software module moves
include directory.

This adds file rename matching which covers our common case of library
moves, kernel upgrades and more.

A new test case is also added so that someone in the future can change
the code and test the logic is still doing the expected things.

(From OE-Core rev: 791ce304f5e066759874beac0feef5ee62a1c255)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-15 14:53:45 +01:00
Rasmus Villemoes
6710572f90 glibc: move ld.so.conf back to main package
There are cases where one doesn't want ldconfig on target (e.g. for
read-only root filesystems, it's rather pointless), yet one still
needs ld.so.conf to be present at image build time:

When some recipe installs libraries to a non-standard location, and
dutifully drops in a file in /etc/ld.so.conf.d/foo.conf, we need the
ld.so.conf containing the

  include /etc/ld.so.conf.d/*.conf

stanza to get those other locations picked up.

So change the packaging logic so that there's always an ld.so.conf
present when the build-time ldconfig runs.

The ld.so.conf and ld.so.conf.d/*.conf files don't take up much
room (at least not compared to the 700K binary ldconfig), and they
might be needed in case ldconfig is installable, so leave them
alone.

In case of a read-only rootfs, one could add some logic to remove them
if one really wants to shave those few dozens of bytes off.

While here, fix typos in the bb.note (add spaces) so one can just
copy-paste the line from the log-file and redo the command.

(From OE-Core rev: a4cdda012f613d8d80203b9f5fc737d8511d16ce)

Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-09 13:13:59 +01:00