The mechanism used to get the hash for a stamp file is split out so that
it can be overridden by derived classes
[YOCTO #13030]
(Bitbake rev: ce241534d19b2f1c51dbdb3b92419676d234e464)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Pass the task hash as a parameter to the 'runtask' message instead of
passing the entire dictionary of hashes when the worker is setup. This
is possible less efficient, but prevents the worker taskhashes from
being out of sync with the runqueue in the event that the taskhashes in
the runqueue change.
[YOCTO #13030]
(Bitbake rev: 1e86d8c1bec7ea5d016a5ad2097f999362e29033)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Turns off the shared cache. It isn't a significant factor in performance
(now that WAL is enabled), and is a really bad idea to have enabled in
processes that fork() (as bitbake it prone to do).
[YOCTO #13030]
(Bitbake rev: 71b427bf01374973a971c10cb64024c8ef2a11eb)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enabling the write ahead log improves database reliability, speeds up
writes (since they mostly happen sequentially), and speeds up readers
(since they are no longer blocked by most write operations). The
persistent database is very read heavy, so the auto-checkpoint size is
reduced from the default (usually 1000) to 100 so that reads remain
fast.
[YOCTO #13030]
(Bitbake rev: 79100fa67539f9654af9bf6d3e6842eb5c12e989)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Constructs the "key" column in the persistent database as a non-NULL
primary key. This significantly speeds up lookup operations in large
databases.
[YOCTO #13030]
(Bitbake rev: f5ba7775cfcb90401522d977cc66fe0f5aeb7a66)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The original implementation of persistent data executed all SQL
statements via sqlite3.Connection.execute(). Behind the scenes, this
function created a sqlite3 Cursor object, executed the statement, then
returned the cursor. However, the implementation did not account for
this and failed to close the cursor object when it was done. The cursor
would eventually be closed when the garbage collector got around to
destroying it. However, sqlite has a limit on the number of cursors that
can exist at any given time, and once this limit is reached it will
block a query to wait for a cursor to be destroyed. Under heavy database
queries, this can result in Python deadlocking with itself, since the
SQL query will block waiting for a free cursor, but Python can no longer
run garbage collection (as it is blocked) to free one.
This restructures the SQLTable class to use two decorators to aid in
performing actions correctly. The first decorator (@retry) wraps a
member function in the retry logic that automatically restarts the
function in the event that the database is locked.
The second decorator (@transaction) wraps the function so that it occurs
in a database transaction, which will automatically COMMIT the changes
on success and ROLLBACK on failure. This function additionally creates
an explicit cursor, passes it to the wrapped function, and cleans it up
when the function is finished.
Note that it is still possible to leak cursors when iterating. This is
much less frequent, but can still be mitigated by wrapping the iteration
in a `with` statement:
with db.iteritems() as it:
for (k, v) in it:
...
As a side effect, since most statements are wrapped in a transaction,
setting the isolation_level when the connection is created is no longer
necessary.
[YOCTO #13030]
(Bitbake rev: e8b9d3f534ef404780be23b601d5a4bb9cec928a)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06 23:19:24,564 - oe-selftest - INFO - Traceback (most recent call last):
File "/media/build1/poky-sumo/meta/lib/oeqa/core/case.py", line 32, in _oeSetUpClass
clss.setUpClassMethod()
File "/media/build1/poky-sumo/meta/lib/oeqa/selftest/cases/eSDK.py", line 78, in setUpClass
cls.tmpdir_eSDKQA = cls.tempdirobj.name
AttributeError: type object 'oeSDKExtSelfTest' has no attribute 'tempdirobj'
(From OE-Core rev: c6de27c2f6f598849dcb8036ee849f449ba7f327)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The following patches are rebased.
fix_parallel_build_error.patch
pam.conf.patch
(From OE-Core rev: 30fa97ec4c9d0553f1566bc26a6949e93cb6da90)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The following patch is rebased.
0001-don-t-try-to-run-generated-binaries.patch
(From OE-Core rev: ee57d79aec06e9b160cf2713636cda650ba68d5a)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
License-Update: include more files to check, but license remains the same.
(From OE-Core rev: ad0f26263eb51cda4cf96cd2fe7f6e7f950b9e8e)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PACKAGE_NO_GCONV is set in libc-package.bbclass if not all of
'libc-charsets libc-locale-code libc-locales' included in
DISTRO_FEATURES. And then no packages glibc-gconv-* glibc-charmap-* and
glibc-localedata-* is created. Update recipes and conf file which depend
on these packages to check required distro features.
(From OE-Core rev: 58446992de0f16a345f1f55b66d0d34d31dc341b)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It installs locale-base-* packages according to var IMAGE_LINGUAS.
Packages locale-base-* are split in libc-package.bbclass if variable
PACKAGE_NO_GCONV is not set. When none of ditro features libc-charsets
libc-locales and libc-locale-code is set, PACKAGE_NO_GCONV is set. Then
no locale-base-* is created and fails to create image.
Clear IMAGE_LINGUAS in such situation.
(From OE-Core rev: 85240094175a8ea726bfba19c00d4556a62862fc)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Variable PACKAGE_NO_GCONV is set in libc-package.bbclass when none of
libc-charsets, libc-locales and libc-locale-code set in DISTRO_FEATURES.
Then it causes installed-vs-shipped qa issue of glibc-locale:
| ERROR: glibc-locale-2.28-r0 do_package: QA Issue: glibc-locale:
| Files/directories were installed but not shipped in any package:
| /usr/share/i18n
| /usr/share/i18n/charmaps
| /usr/share/i18n/locales
| /usr/share/i18n/charmaps/CP737.gz
| ...
| /usr/share/i18n/locales/ru_RU
| ...
| /usr/lib64/gconv/gconv-modules
| Please set FILES such that these items are packaged. Alternatively if they
| are unneeded, avoid installing them or delete them within do_install.
| glibc-locale: 843 installed and not shipped files. [installed-vs-shipped]
So check PACKAGE_NO_GCONV during do_install and not copy those files if
PACKAGE_NO_GCONV has been set.
Simplify call of bb.utils.contains() in libc-package.bbclass as well.
(From OE-Core rev: 39840ed10af559bbcc306b378baa4723921668f5)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
autoconf-doc package contains autoconf.info.
This file contains date when this file was created, i.e:
"This manual (31 January 2018) .."
Therefore, two builds done on two different days will show different dates for
otherwise identical files, hence breaking reproducibility.
The date is obtained from mtime of "autoconf.texi", unfortunately we patch this
file and change the mtime as a consequence.
We restore reproducibility by removing the patch modifying "autoconf.texi".
As a consequence certain supplemental information will be not added to the documentation.
In particular: more details on usage, and on workarounds for non-updated projects.
[YOCTO #12524]
(From OE-Core rev: 1eaa918ae3663e3f863c025744a9ced0300b6b78)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
With applying this fix:
"This manual (24 April 2012) .."
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Waf requires that the current working directory be ${S} (the location of
the wscript) when building. Most of the time, this was true only because
B defaults to S. However, anything that changed that behavior (notably,
using externalsrc) would break the recipe. Remedy this by explicitly
changing cwd to ${S} when running waf commands. As a happy side effect,
B can be set up for "out of tree" builds to keep the source directory
clean.
(From OE-Core rev: 62dffb71ce22222c635bd90eaa47dd01f70f9c0f)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Small tweak to ensure these items are printed into the log since there
is other logging code which looks for the header and this makes it clearer
the server did start but is slow somewhere in startup.
(Bitbake rev: 7c0b84cac892744225fa0462f918ea9a79b356cb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its possible the read may return a smaller number of characters. Remove
the possibility by using a single character to signal the server is ready.
(Bitbake rev: 767c9596d129d21ddf5d2e00c55f0a0525c641dc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This simplifies the code and makes it easier to read but has the
same functionality.
(Bitbake rev: a0b9cfaf2d03fd047a79d32e668001718d02c4bf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous check was in data.py which only can check code like "python
funcname()" in the dependency chain, but there are 3 kinds of python functions:
- python()
- def py_funcname()
- python funcname()
Add the checking to BBHandler to check and warn for all of them.
The warning looks like:
WARNING: /path/to/recipes-core/busybox/busybox_1.29.2.bb: python should use 4 spaces indentation, but found tabs in busybox.inc, line 75
(Bitbake rev: 0cdc5b81fc1f5e5281a525a657e420ebc3bb9e90)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've seen issues with rootfs size calculations and we've seen systems
like opensuse which have btrfs mounted on /tmp causing selftest failures.
(From OE-Core rev: 61be3cd748d1b7321a1fc4cfe84efa9b26a6aee0)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we need to create a temporary directory in targetbuild or buildproject use
tempfile.TemporaryDirectory so that when the test case is finished, the
directory is deleted.
Also synchronise the logic and don't possibly store the temporary directory in
self.tmpdir as nothing uses that.
(From OE-Core rev: db0e658097130d146752785d0d45f46a3e0bad71)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reorder the shutdown/teardown to avoid:
File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/utils/qemurunner.py", line 224, in launch
op = self.getOutput(output)
File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/utils/qemurunner.py", line 90, in getOutput
fl = fcntl.fcntl(o, fcntl.F_GETFL)
ValueError: I/O operation on closed file
(From OE-Core rev: 8e7d756862d2a8d62f3c87497d6d65ddb3c1b962)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
assertTrue doesn't give good debug information when things fail. Update
several to use assertIn which gives information upon failure, for the
others print the log information upon failure.
(From OE-Core rev: c29cb75d5ce6b0873a934f4709b0c8824f7164d3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've seen a cryptic:
"ERROR: Fatal errors occurred in subprocesses, tracebacks printed above"
message from oe-selftest with no other traceback information. Improve the
traceback logging to try and give a better indication of any errors that is
ocurring.
(From OE-Core rev: 521dd3d00979a27b6932e58d5497de68abac26e1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch altered the clean target's behaviour to skip the ipkg-install
directory. However this directory isn't created by opkg, opkg-utils, or the
package_ipk class; and we don't invoke the clean target as we perform
out-of-tree builds.
(From OE-Core rev: 9f8bd475701e5d797d3ffc1ba97647101ba0b9b0)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We stop distutils for *native* Python from rewriting hashbangs when installing
(so installed scripts don't have a hashbang that refers to sysroot paths), but
this isn't needed nor desirable for the *target* Python.
(From OE-Core rev: 52e128619803907c804d42815ea979b1848529c4)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rpm use nss as digest crypto library and which will cause an error as follows:
error: test-manual-1.2.3-20181012.noarch.rpm: Header SHA1 digest: BAD (Expected
f1deb7dc4a10742d88ccd1e967dbc62ae45095a5 !=4ad9d7dad6d70d6086eefec62612ad5d77f2fe81) => this value is wrong
error: test-manual-1.2.3-20181012.noarch.rpm: not an rpm package (or package manifest)
The error is caused by SHA_HTONL in nss, for there is no need to reverse the host value for arm 32be, so fix it.
(From OE-Core rev: 50a554a8ffff0bb34f844fb6cb9dd69362caafa5)
Signed-off-by: Zheng Ruoqin <zhengrq.fnst@cn.fujitsu.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a special library for memcheck tool, where it needs to have the
symbols intact for the stack traces to work on target, current option is
to install valgrind-dbg ( 151 MB uncompressed ) is quite big for some
systems which may not have space to install it all. Leaving it
unstripped adds about 200KB to image which is much better, this alone
gets memcheck working, as an aside we might need same solution for other
tools e.g. helgrind etc. when needed, they also have leading libraries
installed
(From OE-Core rev: 23da8f50b1e0a74777035c9f7b65b81456908f9f)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ccache disable flag was misspelled, preventing it from being
disabled.
(From OE-Core rev: 33fba601a7365aced9f4b206c1fadda997076d4c)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For systems that don't otherwise depend on the full
util-linux package, blkdiscard is a mere 18k (on
cortexa7t2hf-neon).
(From OE-Core rev: 96bfdea8e0ec891b44fc4b0c69120669954426ad)
Signed-off-by: André Draszik <andre.draszik@jci.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add various fixes and improvements including the ability to export
patch statsitics as json data.
(From OE-Core rev: aa4a4b3ca799948047337e006ee9bf482be7b409)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have a fatal error if ELF objects don't have GNU_HASH segments but it
doesn't explain what the problem is. At least give a hint to users by
suggesting that LDFLAGS wasn't passed to the compiler.
(From OE-Core rev: 5d4da6713b40e10e853eb746f700096307ffe158)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the linux-firmware-wl12xx contains all wl12xx and
wl18xx firmwares, except for the wl18xx compatibility symlinks
for old kernels and the linux-firmware-wl18xx contains just
those compatibility symlinks and nothing else. This doesn't
make sense...
Be more specific about what to package into each package, in
particular because the existing wl12xx package is specific
about symlinks already.
At the same time, we split the common bits into a -wlcommon
package, so that the wl18xx package doesn't need to depend
on all the wl12xx firmwares, saving several MiB in the file
system.
(From OE-Core rev: e4423eef71c463efe7a1cdf1a426cc9619c78b8a)
Signed-off-by: André Draszik <andre.draszik@jci.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case of SDK generation, /usr/bin/ path are not correct
and must be replaced by ${bindir}.
(From OE-Core rev: 0fa7d99444763192914e798d8bc9dba1d9cdae42)
Signed-off-by: Christophe Priouzeau <christophe.priouzeau@st.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This recipe doesn't ship a *-config binary, so don't inherit binconfig.
(From OE-Core rev: acfcebec2d0849cc52abed31663da888e2a230f9)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop backported and obsolete patches.
Respect systemd DISTRO_FEATURE when enabling/disabling journal support.
(From OE-Core rev: 33d2cee29758d2a827baa2dda98b10c993101be2)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise .gir files installed in the sysroot will not be found when multilib is in use
(multilib configurations use libdir rather than datadir).
(From OE-Core rev: cfae171964ba85298d883e70635324a33b8539ac)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This should also fix Rygel builds in meta-oe
(From OE-Core rev: 8332fe551f018339b93841609625121145e97684)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
setUp() is used to populate a directory of temporary files, and deleted in
__del__. However setUp() is called once *per test* so __del__ would only be
able to remove the last directory created.
Fix the code by using the natural counterpart to setUp, tearDown(), to clean up.
(From OE-Core rev: 68b4723e6fb11d171869185bccf28f32f6284c18)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The segfault on append was fixed upstream with a different patch, so apply that
instead.
(From OE-Core rev: 24000d1fdba2684202e15371f80bb385722c9d91)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
