Commit Graph

1322 Commits

Author SHA1 Message Date
Khem Raj
bf2b73e1f3 classes, conf, lib: Add support for powerpc64le
LE is default for modern powerpc64, power8+

PowerPC64 Little Endian Linux ABI specifies Power8 as the minimum ISA.
The basic ABI can run on earlier versions of the 64 bit PowerPC ISA,
but it was helpful to define a new, minimum instruction set for Linux
distribution releases during the switch to Little Endian.

(From OE-Core rev: b9c73fb6d1afb3367d871a3d6bf7d0d6a53968a9)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-19 23:49:39 +00:00
Richard Purdie
ecbf203e34 sstatesig: Improve debug output if getpwuid() fails
If getpwduid fails, we don't see which file it failed on which is key information
to aid debugging. Print this information when exceptions are raised.

(From OE-Core rev: 9d7a5219713af8117eda145052c6d9abdf577d8f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-16 22:33:09 +00:00
Alexander Kanavin
25fd2bf815 lib/oe/package_manager.py: put the sdk_provides_dummy_target upfront
Otherwise, the package utilizing this architecure does not get
properly prioritized by rpm/dnf. This isn't seen single-lib builds
(as the architecture ends up upfront anyway), but is triggered in
multulib where the architecture may end up in the middle of the list
due to multilib expansion/trimming duplicates.

(From OE-Core rev: a807d54655d7377ef8ac49abd5ab428303831663)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:21 +00:00
Richard Purdie
76af397213 sstatesig: Avoid resetting taskhash within siggen for locked sigs
Since get_unihash uses taskhash as a key internally, changing it means
different bebahour when locked sigs are active verses not active. Under
corner cases this leads to a signature mismatch.

Avoid this by by adding a wrapper for the place its externally exposed
and then not changing the internals.

(From OE-Core rev: 23b7484d326cf7fac013384598d7745a042eaa75)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-03 22:35:48 +00:00
Richard Purdie
737c5b27bb sstatesig: Fix locked signature handling with unihashes
get_taskhash will call get_unihash internally in the parent class. We
need to disable our filter of it whilst this runs else incorrect hashes
can be calculated.

This is believed to be causing the locked signatures test to fail under
some circumstances (depending on whether earlier hashes are being
remapped).

[YOCTO #13605]

(From OE-Core rev: 523c093a882f6831ba75b5c4513837554d7e2414)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-03 22:35:48 +00:00
Richard Purdie
7538be0a50 sstatesig: Test cross/native hashserv method extension
We can have one taskhash which represents multiple native/cross sstate objects
since they're stored by BUILD_ARCH or possibly host distro (or host gcc version).
We need to put these into separate namespaces on hashserv since their outhashes
will never match and we need deterministic lookups for the different namespaces.

Use this extramethod option to handle this. This fixes several problematic
failures on the autobuilder.

(From OE-Core rev: 2a76082363d189880613765ad339718e3614049d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-02 16:43:01 +00:00
Ross Burton
3780744968 chrpath: Cleanup and fix previous patch
Ensure self.data isn't accessed without assignment. Also clean up old style
popen use and replace with modern/simpler subprocess.

(From OE-Core rev: 39825cba4761a6b4b2473825705975f9f421ec8b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-29 09:27:16 +00:00
Alexander Kanavin
40f28d15c5 recipeutils: set SRCPV for devupstream variant
Otherwise bitbake is unable to expand it if the main
variant is not using a SCM fetcher.

(From OE-Core rev: 24e59094de42b4b0bed3584ca8571c4c19080428)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-28 23:25:42 +00:00
Ross Burton
1ac2b6c82f rootfs: don't use oe.cachedpath
Unless cachedpath is used correctly then it's just a glorified clone of
os.walk, but without any of the recent optimisations in os.walk.  In
this codepath there is no point to using cachedpath.

(From OE-Core rev: 765e0c3410d6bf3b4a50c4a036f555eae760acd2)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-28 23:25:41 +00:00
Richard Purdie
ba4dbacedd sstatesig: Further optimise
Optimise the call into the parent function to be only when needed.

(From OE-Core rev: 198bacaa1068fd68075d9182318fe64e9809dd36)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-16 23:25:49 +00:00
Richard Purdie
58726c4a56 sstatesig: Optimise get_taskhash for hashequiv
With hashequiv the get_taskhash function is called much more regularly
and contains expensive operations. This these don't change based upon
hash in a given build, improve the caching within the function to
reduce overhead.

(From OE-Core rev: de98cfe3cde4b8d5f4b163b5fba3f129651ef06a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-15 09:10:46 +00:00
Haris Okanovic
dfb8d75ae6 meta/lib/oe/package_manager.py: Enable sha256 checksums in opkg indexer
Pass `--checksum md5` and `--checksum sha256` to opkg-make-index.

Sha256 checksum enables more reliable install-time validation of IPKs.
This is particularly useful when installing from signed feeds --
I.e. feeds using signed Packages index files that deliver otherwise
unsigned IPKs. Such feeds rely on hash validation of enclosed IPKs to
thwart tampering. After download, opkg verifies IPK's checksum against
the (signed) Packages index file. Weak hashes like md5 are prone to
collision and therefore tampering.

The md5 checksum is purely for backward compatibility. Sha256 validation
was recently added to opkg. Newer builds of opkg will use it. Older
builds still look for an md5 checksum. Md5 is deprecated and should be
removed once old build are phased out.

Testing: I ran `bitbake package-index` after building a few IPKs and
verified MD5Sum and SHA256sum attributes are present in Packages.
Using opkg-utils 0.4.0.

Performance Impact: It takes about 40 seconds to cleanly re-index 8000
IPKs on an Intel Xeon E5-1620 machine. This was previously about
20 seconds.

NOTE: It's recommended to delete all Packages* files after applying this
patch. Otherwise, some IPKs won't have sha256.

(From OE-Core rev: e462f47489f35902b6972f9837d9adfa542fc796)

Signed-off-by: Haris Okanovic <haris.okanovic@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-10 14:06:10 +00:00
Andreas Müller
29b733d7b8 IMAGE_LINGUAS_COMPLEMENTARY: auto-add language packages other than locales
This change allows layers adding packages other than locales automatically to
images based on languages selected in IMAGE_LINGUAS.

E.g if a layer has recipes creating packages as:
| ${PN}-foo-en / ${PN}-foo-de / ${PN}-foo-it / ..

it would set
| IMAGE_LINGUAS_COMPLEMENTARY_append = " *-foo-%s"

in its layer.conf to enable auto-adding.

(From OE-Core rev: 3a9e9585aeb67234901b2f842113dbe20c3801b3)

Signed-off-by: Andreas Müller <schnitzeltony@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-31 16:09:34 +00:00
Ross Burton
37388b6899 buildhistory-analysis: filter out -src changes by default
Like the -dbg package, this package is automatically generated and contains
source filenames.  We expect this to change on every upgrade, so don't show the
differences unless the user wants to see all changes.

(From OE-Core rev: 30acbf6f89ba76d6fab8987ed20f72d1fa3d70fa)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-28 05:31:42 +00:00
Ola x Nilsson
f3f1597249 lib/oe/package_manager: Use with to control file handle lifetime
(From OE-Core rev: e3e86d6683e305865c9fd171c6081aec037d9870)

Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-23 16:30:36 +01:00
Ola x Nilsson
3af1c66939 lib/oe/packagedata: Use with to control file handle lifetime
(From OE-Core rev: 28b3635b7f40b627a9d6294a78589c78f0c682c8)

Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-23 16:30:36 +01:00
Ola x Nilsson
c4f2a115f7 oe.types.path: Use with to control file handle lifetime
(From OE-Core rev: b6e7fee68d018e5f148a7a02cf110bc210022a72)

Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-23 16:30:36 +01:00
Trevor Gamblin
621a5836dd lib/oe/terminal.py: fix gnome-terminal start behavior
[Bugzilla Bug 13201] -- https://bugzilla.yoctoproject.org/show_bug.cgi?id=13201

Newer versions of gnome-terminal (3.32.0 and up) are not starting
as expected for commands e.g. "bitbake -c devshell zlib". This
manifests as the instance appearing as a new tab rather than a
new window. Fix this (and maintain new window preferred behavior)
by changing the "-x" option to "--" as per the warning message,
avoiding deprecated options:

    # Option “--command” is deprecated and might be removed in a later version of gnome-terminal.
    # Use “-- ” to terminate the options and put the command line to execute after it.

(From OE-Core rev: d5fd205239c8e3a1d68649562a7e91c8fbbc805e)

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-08 11:22:24 +01:00
Peter Kjellerstedt
9a07399607 lib/oe/lsb: Make sure the distro ID is always lowercased
In commit 8689e561 (lib/oe/lsb: attempt to ensure consistent distro id
regardless of source), the distro ID returned by
oe.lsb.distro_identifier() was lowercased, but only if a release
version is also present.

This changes the code to always lowercase the distro ID, including the
default distro ID "unknown", which is used if no other ID can be
identified.

(From OE-Core rev: 4ba7ef79d23a4cf688d7a794064893fe5f2f473b)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-30 16:55:21 +01:00
Richard Purdie
d80dd1c3f9 lib/sstatesig: Fix class inheritance problems
The locked sigs class needs to be inherited after the hashequiv mixin so
that get_unihash can correctly wrap the underlying hashequiv function.

To do this turn the locked sigs class into a second mixin, then the order
can be correctly handled. Tweak the get/set_taskdata to match.

(From OE-Core rev: 73d3414ecf594e426e6024b71931038a201eca52)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-30 16:55:21 +01:00
Richard Purdie
c92aca546e sstatesig: Fix hash equivlanency locked signature issues
Using locked signatures with the hash equivalency server ran into
problems. We need to:

a) Ensure the lockedhashes data object is passed from the core to
   any individual tasks using the get/set_taskdata methods

b) Return a locked singature instead of a unihash

c) Write the unihash being used to locked signature lists rather than
   the calculated taskhash

d) Skip warnings of hash mismatch if the hash is a unihash

These changes fix esdk builds (which use locked sigs) when a hash equivalence
server is in use.

(From OE-Core rev: 25dc3d78de01dffa77a3a2452d6a97d741b446d9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-30 16:55:21 +01:00
Khem Raj
990fedd595 sdk: Install nativesdk locales for all TCLIBC variants
install_locales() here is actually operating on nativesdk and only glibc
is the default library for nativesdk, since thats what most of
desktop/server distros use, therefore bailing out based on TCLIBC is not
needed here, since nativesdk-glibc would be required for all non-glibc
targetting SDKs as well.

Fixes SDK install time error

ERROR:  OE-core's config sanity checker detected a potential misconfiguration.
Either fix the cause of this error or at your own risk disable the checker (see sanity.conf).
Following is the list of potential problems / advisories:
Your system needs to support the en_US.UTF-8 locale.
ERROR: SDK preparation failed

(From OE-Core rev: ba2df1fa43b79c4959911c1b0bcad996f255b7a8)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27 13:02:16 +01:00
Jaewon Lee
88db9e41c1 populate_sdk_ext: Introduce mechanism to keep nativesdk* sstate in esdk
When doing a devtool build-sdk from within an esdk all nativesdk
components would be rebuilt. This patch introduces SDK_INCLUDE_NATIVESDK
flag to toggle the inclusion of nativesdk packages when creating the
esdk sstate

Currently locked-sigs.inc is generated during do_sdk_depends which
doesn't pull in nativesdk packages. Generating another locked-sigs.inc
in do_populate_sdk_ext and pruning it to only nativesdk* packages by
using a modified version of the already existing function
prune_locked_sigs and merging it with the current locked-sigs.inc
Also adding SDK_INCLUDE_NATIVESDK tasklistfn to the logic surrounding
setting tasklist file to not prune esdk sstate during creation

[YOCTO #13261]

(From OE-Core rev: d046afd12e1c209b29dca6ba402b9aa14680c5ce)

Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-19 15:27:02 +01:00
Joshua Watt
34923e4f77 Remove SSTATE_HASHEQUIV_SERVER
Removes all references to the SSTATE_HASHEQUIV_SERVER variable. This
variable is redundant now that BB_HASHSERVE is present.

(From OE-Core rev: 54b3adb6bc90a8e4b9e92952688772ee074d36e5)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-18 17:52:01 +01:00
Joshua Watt
f12e41c1bb sstatesig: Update server URI
The server no longer uses a "http://" URI, since it has been updated to
use a different protocol.

(From OE-Core rev: 519561172c48bc7f7a61a3d02edd418fc0895b7b)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-18 17:52:01 +01:00
Changqing Li
da61ee5541 classextend.py: don't extend file for file dependency
Fix error like:
lib32-e2fsprogs-1.45.3-r0 do_package_qa: QA Issue: /usr/sbin/e2scrub_all
contained in package lib32-e2fsprogs-e2scrub requires /bin/bash,
but no providers found in RDEPENDS_lib32-e2fsprogs-e2scrub

For some lib32 packages(eg: lib32-bash, lib32-sed) which probvides files,
extend is not needed

Eg: RPROVIDES of lib32-bash expects to have /bin/bash, with original
extend, it will become lib32-/bin/bash, then will cause above error

Fix by don't extend file dependency, and skip multilib check for file dependency
in do_package_qa to avoid error like:
WARNING: lib32-bash-5.0-r0 do_package: QA Issue: lib32-bash package lib32-bash
 - suspicious values '/bin/bash /bin/sh' in RPROVIDES [multilib]

(From OE-Core rev: a9163120ed52534e7dbf4db50dc2b03bbf69f06b)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-16 23:02:45 +01:00
Vijai Kumar K
550d3541ae lib/oe/gpg_sign.py: Remove unwanted import
Remove unwanted import of oe.utils.

(From OE-Core rev: 42e9de68384fb24d5c9a5f161810be90c45942d2)

Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-06 14:58:09 +01:00
Richard Purdie
359d61067a sstatesig: Fix leftover splitting issue from siggen change
(From OE-Core rev: 8b76048da99ec3be7d763b7ccc81f767c7015bc7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-12 16:23:57 +01:00
Richard Purdie
a0542ed3ff sstatesig: Updates to match bitbake siggen changes
Update the metadata to correspond to the bitbake siggen task specification
format change. This standardises on "<fn>:<task>" everywhere rather than
the "." delimiter that was being used in some places.

This is an API breaking change but means we now have a consistent format
being used throughout the codebase without compatibility APIs.

(From OE-Core rev: 23db236a054ee7a989cdbbcb42ad5c6eefd4a6ae)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-06 11:24:26 +01:00
Richard Purdie
bcc9f1b5f1 sstate/sstatesig: Update to new form of BB_HASHCHECK_FUNCTION
Bitbake has updated to a cleaned up form of BB_HASHCHECK_FUNCTION,
adapt to this cleanup. This is an API breaking change.

(From OE-Core rev: a4d413d8d809132b0e0a5dd673a36e2bd0e0be4c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-06 11:24:26 +01:00
Richard Purdie
8d7f663136 sstatesig: Update to handle BB_HASHSERVE
Update the code to handle BB_HASHSERV, an autostarted bitbake internal hash
equivalency server suited to local developer usage.

(From OE-Core rev: 9b3d7b0b0ff27281391c2b8f2511d312d6995ed6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-06 11:24:26 +01:00
Richard Purdie
56b178ca2f sstatesig: Adpat to recent bitbake hash equiv runqueue changes
Upstream bitbake now hanes preserving the unihash data itself, drop
this usage of persist_data which was extremely problematic due to
concurrent task access issues, particulary on loaded systems.

(From OE-Core rev: 034d91c2c94b201797a7830b0af6141132f9bad1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-06 11:24:26 +01:00
Richard Purdie
00f40114b4 sstatesig: Add debug for incorrect hash server settings
If the hash server settings are incorrect, show the user useful
error messages instead of tracebacks.

(From OE-Core rev: 2df5d95b9b63e30ddaa7c72a4173b9a05b3f15f9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-06 11:24:26 +01:00
Richard Purdie
62c8b8c4d9 sstatesig: Move unihash siggen code to bitbake
This code is closely tied with the hash server in bitbake and also means
we can't relibably test the hashserv runqueue functionality without OE
metadata. Moving this to bitbake as a MixIn class makes most sense
and encourages code collaboration and reuse as well as enabling easier
and more accurate testing of the APIs.

(From OE-Core rev: a2a9c6092d4dde706ed071b08a972d1d87184295)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-06 11:24:26 +01:00
Richard Purdie
546c32753b classes/lib: Remove bb.build.FuncFailed
Whilst seemingly a good idea, this exception doesn't really serve any purpose
that bb.fatal() doesn't cover. Wrapping exceptions within exceptions isn't
pythonic.

Its not used in many places, lets clean up those and remove usage of it
entirely. It may ultimately be dropped form bitbake entirely.

(From OE-Core rev: efe87ce4b2154c6f1c591ed9d8f770c229b044ad)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-06 11:24:26 +01:00
Richard Purdie
41a5dbd16b package: Improve determinism
Its possible in cases with multiple shlib providers we were not being
deterministic. Add in a couple of sorted() calls to fix the shlibs and
pkgconfig cases with this potential issue.

(From OE-Core rev: 5b9a4214ee17e1a39dd5a1742f2ac5ed25a11310)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-26 08:41:39 +01:00
Alistair Francis
991cd48af5 package_manager: Ensure the base-feed directory exists
Ensure that the /etc/opkg directory exists before we try to create a
file there.

(From OE-Core rev: 30ff50223cd0b79fd3b8aa393ea1e621282773ac)

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-18 12:16:19 +01:00
Andrej Valek
c41acbcf8a oe/copy_buildsystem: move layer into layers directory
Layers could be located outside from poky but inside the build directory.
This case should be covered in eSDK.
meta-abc
meta-def/meta-ghi
meta-def/poky
meta-def/meta-oe/meta-oe
...

It should take all enabled layers and put them into 'layers' dir during
build-time with respecting new relative path to poky.
layers/meta-abc
layers/meta-ghi
layers/poky
layers/meta-oe/meta-oe
...

(From OE-Core rev: 55ecf6988d3e3c0935cb6324a6ad2c75f1191a1d)

Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-18 12:16:19 +01:00
Ross Burton
1b7876b060 buildhistory_analysis: ignore ownership for sysroot diffs
The sysroot isn't populated under pseudo, so ownership differences should be ignored.

(From OE-Core rev: 01b816be4adff8f3992c1369810bdcf11a26fd6c)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-05 12:00:20 +01:00
Ross Burton
adaba4ee4f buildhistory: report sysroot changes
Now that the sysroot is written into the build history, write it out.

(From OE-Core rev: e9df98ba63e6d2baefee550170dbdd11ed2ad03a)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-28 13:28:37 +01:00
Ricardo Ribalda Delgado
931ed3e4f8 rootfs: Fix dependency for every dpkg run
Avoid getting a warning on do_rootfs.

Fixes:

NOTE: Installing the following packages: libgdk-pixbuf-2.0-locale-en-gb glibc-locale-en-gb libatspi-locale-en-gb gstreamer1.0-locale-en-gb gtk+3-locale-en libatk-1.0-locale-en-gb gtk+3-locale-en-gb gstreamer1.0-plugins-good-locale-en-gb gstreamer1.0-plugins-base-locale-en-gb gstreamer1.0-plugins-bad-locale-en-gb libglib-2.0-locale-en-gb avahi-locale-en-gb vte-locale-en-gb xkeyboard-config-locale-en-gb
WARNING: Unable to install packages. Command '/workdir/build/tmp/work/qt5122-poky-linux/bottlecam-image/1.0-r0/recipe-sysroot-native/usr/bin/apt-get  install --force-yes --allow-unauthenticated libgdk-pixbuf-2.0-locale-en-gb glibc-locale-en-gb libatspi-locale-en-gb gstreamer1.0-locale-en-gb gtk+3-locale-en libatk-1.0-locale-en-gb gtk+3-locale-en-gb gstreamer1.0-plugins-good-locale-en-gb gstreamer1.0-plugins-base-locale-en-gb gstreamer1.0-plugins-bad-locale-en-gb libglib-2.0-locale-en-gb avahi-locale-en-gb vte-locale-en-gb xkeyboard-config-locale-en-gb' returned 100:
Reading package lists...
Building dependency tree...
Reading state information...

(From OE-Core rev: 4e315dd23c52b75fd64dca89e7bffe25a0b0e6f9)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-19 22:13:42 +01:00
Richard Purdie
32df42cff5 multiconfig: Adapt to bitbake switch 'multiconfig' -> 'mc'
(From OE-Core rev: 8a6f7c1e455156966f467008645fef14db679ccf)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-10 14:46:38 +01:00
Richard Purdie
1334b064d3 gpg_sign/selftest: Fix secmem parameter handling
We keep seeing "cannot allocate memory" errors from rpm when signing packages
on the autobuilder. The following were tried:

* checking locked memory use (isn't hitting limits)
* Restricting RPM_GPG_SIGN_CHUNK to 1
* Limiting to 10 parallel do_package_write_rpm tasks
* Allowing unlimied memory overcommit
* Disabling rpm parallel compression

and the test still failed. Further invetigation showed that the --auto-expand-secmem
wasn't being passed to gpg-agent which meant the secmem couldn't be expanded hence the
errors when there was pressure on the agent.

The reason this happens is that some of the early gpg commands can start the agent
without the option and it sticks around in memory so a version with the correct
option may or may not get started.

We therefore add the option to all the key gpg calls.

(From OE-Core rev: c7e131a76e522503df55e211dd261829feacfa28)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-07 09:11:49 +01:00
Hongxu Jia
c113a83dd5 lib/oe/reciputils.py: support character `+' in git pv
While pv contains char `+' (such as ncurses 6.1+20181013),
it was incorrectly converted to `6.1'

In this commit:
- Convert [^\+]* to .*?
  Support pv to contain char `+' and not greedy match
  (Such as ncurses 6.1+20181013)

- Add [^\+] to sfx match
  Support sfx contains extra chars between `+' and `git'
  (such as asciidoc "8.6.9+py3-gitAUTOINC+618f6e6f6b")

- Make sfx and rev greedy match

Run `devtool check-upgrade-status --all' in poky and compare results,
only one difference on ncurses version:
Without the commit:
INFO: ncurses                   6.1             6.1+20181013    Hongxu Jia <hongxu.jia@windriver.com> 7a97a7f937762ba342d5b2fd7cd090885a809835

With the commit:
INFO: ncurses                   6.1+20181013    MATCH           Hongxu Jia <hongxu.jia@windriver.com> 7a97a7f937762ba342d5b2fd7cd090885a809835

(From OE-Core rev: 8049bd34b89e710f7bb20883813ba3f929d9e997)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-04 09:09:42 +01:00
Ross Burton
92290d3b70 patch: add minver and maxver parameters
Add minver/maxver parameters to limit patch application by comparing PV.

(From OE-Core rev: 3fac9f884ac6bcc0280e1bb5d0f0e397bb53678f)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09 16:31:56 +01:00
Richard Purdie
ffae400179 meta/lib+scripts: Convert to SPDX license headers
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.

The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).

More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.

The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.

(From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09 16:31:55 +01:00
Paul Barker
10a14af4ca oe.path: Add copyhardlink() helper function
This function creates hard links if possible, falling back to copying
the file if the destination is on a different volume to the source.

The docstring for copyhardlinktree() is also updated to make the
difference between the two functions a little clearer.

(From OE-Core rev: 5437efa16f9bec914e417c6c939a39c247084f52)

Signed-off-by: Paul Barker <paul@betafive.co.uk>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-07 10:12:53 +01:00
Richard Purdie
244cbcce0e utils/multiprocess_launch: Improve failing subprocess output
Output before this patch:

ERROR: bash-4.4.18-r0 do_package_write_ipk: Fatal errors occurred in subprocesses:
Command 'PATH="X" opkg-build -Z xz -a "--memlimit=50% --threads=88" Foobar /media/build1/poky/build/nodistro-glibc/work/core2-64-oe-linux/bash/4.4.18-r0/deploy-ipks/core2-64' returned non-zero exit status 1.: Traceback (most recent call last):
  File "/media/build1/poky/meta/lib/oe/utils.py", line 272, in run
    ret = self._target(*self._args, **self._kwargs)
  File "/media/build1/poky/meta/classes/package_ipk.bbclass", line 230, in ipk_write_pkg
    shell=True)
  File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'PATH="X" opkg-build -Z xz -a "--memlimit=50% --threads=88" Foobar /media/build1/poky/build/nodistro-glibc/work/core2-64-oe-linux/bash/4.4.18-r0/deploy-ipks/core2-64' returned non-zero exit status 1.

Note how stdout/stderr from the failing command isn't shown.

After this patch:

ERROR: bash-4.4.18-r0 do_package_write_ipk: Fatal errors occurred in subprocesses:
Command 'PATH="X" opkg-build -Z xz -a "--memlimit=50% --threads=88" Foobar /media/build1/poky/build/nodistro-glibc/work/core2-64-oe-linux/bash/4.4.18-r0/deploy-ipks/core2-64' returned non-zero exit status 1.
Subprocess output:Foobar
*** Error: Package name Foobar contains illegal characters, (other than [a-z0-9.+-])

opkg-build: Please fix the above errors and try again.

We suddenly get a much more usable error message. The traceback is supressed
as its distracting from the real problem in this case.

Ideally python itself would handle this but it doesn't so we have to
wrap the exception. We already do this in bitbake itself for the same reason.

(From OE-Core rev: 09276dc76a8bda237b0b0b6d117a1980ae9dbfcc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-26 10:09:08 +01:00
Alexander Kanavin
ac5b7cef59 buildhistory: call a dependency parser only on actual dependency lists
Previously it was also called on filelists and possibly other items which
broke the parser.

(From OE-Core rev: 90bbe1bbc1667bf836d93df1e1ecca0c43315d06)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-25 00:05:20 +01:00
Andreas Müller
b54bff841b patch/insane: Rework patch fuzz handling
Currently there are three issues which can be enhanced:

1. Fuzz warnings cannot be configured as errors for hardening. It happened
   often to me that these warnings were overseen and detected after commits
   were already out.
2. The output is too verbose - particularly when more than one file is
   affected. Meanwhile all users should know why patch fuzz check is performed.
   So move links with background information to insane.bbclass.
3. Reduce copy & paste effort slightly by printing PN (nit: <recipe> was not
   a correct suggestion e.g for native extended recipe - see example below)

To achieve patch.py drops patch-fuzz info encapsulated by a header- and footer-
string into log.do_patch. With this insane.bbclass can drop warnings/errors
depending on 'patch-fuzz' in ERROR_QA or WARN_QA. Default remains unchanged:
Spit out warnings only.

A message for two fuzzed patches and 'pact-fuzz' in ERROR_QA now looks like:

| ERROR: autoconf-native-2.69-r11 do_patch: Fuzz detected:
|
| Applying patch autoreconf-exclude.patch
| patching file bin/autoreconf.in
| Hunk #1 succeeded at 73 with fuzz 1 (offset -3 lines).
| Hunk #2 succeeded at 143 (offset 6 lines).
| Hunk #3 succeeded at 167 (offset 6 lines).
| Hunk #4 succeeded at 177 (offset 6 lines).
| Hunk #5 succeeded at 281 (offset 15 lines).
| Hunk #6 succeeded at 399 (offset 15 lines).
| Hunk #7 succeeded at 571 (offset 20 lines).
| Hunk #8 succeeded at 612 (offset 20 lines).
| Hunk #9 succeeded at 636 (offset 20 lines).
| Hunk #10 succeeded at 656 (offset 20 lines).
| Hunk #11 succeeded at 683 (offset 20 lines).
|
| Applying patch autoreconf-gnuconfigize.patch
| patching file bin/autoreconf.in
| Hunk #1 succeeded at 55 with fuzz 1 (offset -3 lines).
| Hunk #3 succeeded at 663 (offset 18 lines).
|
| The context lines in the patches can be updated with devtool:
|
|     devtool modify autoconf-native
|     devtool finish --force-patch-refresh autoconf-native <layer_path>
|
| Don't forget to review changes done by devtool!
|
| ERROR: autoconf-native-2.69-r11 do_patch: QA Issue: Patch log indicates that patches do not apply cleanly. [patch-fuzz]

(From OE-Core rev: c762c0be43a3854a43cb4b9db559b03126d50706)

Signed-off-by: Andreas Müller <schnitzeltony@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-09 13:44:39 +01:00