mirrors/poky
1
0
Fork 0
mirror of https://git.yoctoproject.org/poky synced 2026-05-02 18:32:15 +02:00
Code Issues Packages Projects Releases Wiki Activity

package: Build pkgdata specific to the current recipe

Browse Source 
This switches the code to build pkgdata specific to the current recipe
which means that its filtered to the recipes dependencies and can perform
better as we can drop the lockfile.

It uses a similar method to the staging code to do this, using BB_TASKDEPDATA
to construct a list of packagedata task output which this recipe should "see".

The original pkgdata store is left unaltered so existing code works.

The lock file was there to prevent files disappearing as they were read or as
directories were listed. Since we have a copy of the data and only access output
from completed tasks (as per their manifests), we can remove the lock.

The lock was causing starvation issues on systems with parallelism.

There was also a potential determinism problem as the current code could "see"
data from recipes which it doesn't depend upon.

[YOCTO #13412]

(From OE-Core rev: 1951132576bfb95675b4879287f8b3b7c47524fa)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit is contained in:
Richard Purdie
2019-06-28 16:45:53 +01:00
parent b9bf7d2682
commit 99206cb7b4
3 changed files with 173 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
meta/classes/package.bbclass
View File

@@ -40,6 +40,7 @@
inherit packagedata
inherit chrpath
inherit package_pkgdata
# Need the package_qa_handle_error() in insane.bbclass
inherit insane
@@ -1571,7 +1572,7 @@ python package_do_filedeps() {
d.setVar("FILERPROVIDESFLIST_" + pkg, " ".join(provides_files[pkg]))
}
SHLIBSDIRS = "${PKGDATA_DIR}/${MLPREFIX}shlibs2"
SHLIBSDIRS = "${WORKDIR_PKGDATA}/${MLPREFIX}shlibs2"
SHLIBSWORKDIR = "${PKGDESTWORK}/${MLPREFIX}shlibs2"
python package_do_shlibs() {
@@ -1729,10 +1730,7 @@ python package_do_shlibs() {
needed = {}
# Take shared lock since we're only reading, not writing
lf = bb.utils.lockfile(d.expand("${PACKAGELOCK}"), True)
shlib_provider = oe.package.read_shlib_providers(d)
bb.utils.unlockfile(lf)
for pkg in shlib_pkgs:
private_libs = d.getVar('PRIVATE_LIBS_' + pkg) or d.getVar('PRIVATE_LIBS') or ""
@@ -1918,9 +1916,6 @@ python package_do_pkgconfig () {
f.write('%s\n' % p)
f.close()
# Take shared lock since we're only reading, not writing
lf = bb.utils.lockfile(d.expand("${PACKAGELOCK}"), True)
# Go from least to most specific since the last one found wins
for dir in reversed(shlibs_dirs):
if not os.path.exists(dir):
@@ -1936,8 +1931,6 @@ python package_do_pkgconfig () {
for l in lines:
pkgconfig_provided[pkg].append(l.rstrip())
bb.utils.unlockfile(lf)
for pkg in packages.split():
deps = []
for n in pkgconfig_needed[pkg]:
@@ -2134,6 +2127,7 @@ def gen_packagevar(d):
PACKAGE_PREPROCESS_FUNCS ?= ""
# Functions for setting up PKGD
PACKAGEBUILDPKGD ?= " \
package_prepare_pkgdata \
perform_packagecopy \
${PACKAGE_PREPROCESS_FUNCS} \
split_and_strip_files \
@@ -2261,12 +2255,8 @@ do_packagedata () {
addtask packagedata before do_build after do_package
SSTATETASKS += "do_packagedata"
# PACKAGELOCK protects readers of PKGDATA_DIR against writes
# whilst code is reading in do_package
PACKAGELOCK = "${STAGING_DIR}/package-output.lock"
do_packagedata[sstate-inputdirs] = "${PKGDESTWORK}"
do_packagedata[sstate-outputdirs] = "${PKGDATA_DIR}"
do_packagedata[sstate-lockfile] = "${PACKAGELOCK}"
do_packagedata[stamp-extra-info] = "${MACHINE_ARCH}"
python do_packagedata_setscene () {