This doesn't save much time but does make the profile counts for the
function more accurate which is in itself useful.
(Bitbake rev: d446fa89d206fbc6d098215163c968ea5a8cf4a9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were seeing this running thousands of times with hashequiv, do
the filtering where it makes more sense and make it persist.
(Bitbake rev: 2cfeb9998a8ad5b1dcda0bb4e192c5e4306dab17)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the calls to difference_update out a code level which improves efficiency
significantly.
Also further combine the outer loop for efficiency too.
These two changes remove a bottleneck from the performance charts.
(Bitbake rev: e28ec69356f1797de3e4e3fca0fef710bc4564de)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Creating a new sorted list of sstate tasks each iteration through runqueue is
extremely ineffecient and was compounded by the recent change from a list to set.
Create one sorted list instead of recreating it each time.
(Bitbake rev: de18824996841c3f35f54ff5ad12f94f6dc20d88)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are two operations happening in get_taskhash, the building of the
underlying data and the calculation of the hash.
Split these into two funtions since the preparation part doesn't need
to rerun when unihash changes, only the calculation does.
This split allows sigificant performance improvements for hashequiv
in builds where many hashes are equivalent and many hashes are changing.
(Bitbake rev: 6a32af2808d748819f4af55c443578c8a63062b3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "no dependencies" task case was not being correctly considered in this
code and seemed to be the cause of occasionaly task hash mismatch errors
that were being seen as the dependencies were never accounted for properly.
(Bitbake rev: 608b9f821539de813bfbd9e65950dbc56a274bc2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The looping over multiple changed hashes causes many calls to get_taskhash
and get_unihash which are potentially slow and then overwritten.
Instead, batch up all the tasks which have changed unihashes and then
do one big loop over the changed tasks rather than each in turn.
This makes worlds of difference to the performance graphs and should speed
up build where many tasks are being rehashed.
(Bitbake rev: c9c68d898985cf0bec6fc95f54c151cc50255cac)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Discovered with a recipe under devtool. The ${S}/singletask.lock file (added by
externalsrc.bbclass) was leaked, giving a warning like:
WARNING: <PN>+git999-r0 do_populate_lic: /home/laplante/yocto/sources/poky/bitbake/lib/bb/build.py:582: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/laplante/yocto/build/workspace/sources/<PN>/singletask.lock' mode='a+' encoding='UTF-8'>
exec_func(task, localdata)
(Bitbake rev: 6beddf6214e22b4002626761031a9e9d34fb04db)
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The local hash equivalence server used by bitbake requires python 3.5,
so bump up the minimum required version.
[YOCTO #13678]
(Bitbake rev: 1412dcc077b1bea10b72fc8b525d6258dca46d97)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of reporting ${LICENSE} when a package cannot be installed
into an image because it is using an incompatible license, report the
license(s) that are actually incompatible.
(From OE-Core rev: b1863e570d4b169cd2f0ea7b4fe7c2348943cb2c)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When excluding a package from being packaged due to incompatible
licenses, report the license(s) that are actually incompatible.
(From OE-Core rev: 7d5c07e4a32a0968942ae538023c2891b59d8ab5)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of reporting ${LICENSE} when a package is identified as using
an incompatible license, report the license(s) that are actually
incompatible.
(From OE-Core rev: fb3405fac7b933a3f9d23e5faf6a3cf2d1990982)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After a number of rewrites, the code checking if a package has been
whitelisted for an incompatible license was calculating the
whitelisted packages twice (as 'whitelist' and as 'incompatwl').
(From OE-Core rev: 01dcc8c94b882d865fc72a067a90d9721ac5fea7)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes it possible to report the incompatible licenses.
(From OE-Core rev: 02a3189b4902e616f5b4277ee3d0fca10b73b4dc)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The SRC_DISTRIBUTE_LICENSES variable and its static list of licenses
has been replaced by AVAILABLE_LICENSES, which automatically contains
all available licenses.
(From OE-Core rev: 64daaf29e2c12c8b587bafdebf9409433187ddf7)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, there was SRC_DISTRIBUTE_LICENSES, an undocumented
variable that contained a static list of licenses. It was used by
expand_wildcard_licenses() to expand any wildcards used in, e.g.,
INCOMPATIBLE_LICENSE. However, since this static list of licenses has
not been kept up-to-date, many licenses were missing, with the result
that if one tried to use any of those licenses with a wildcard, no
licenses would be found, effectively ignoring that they should be
marked as incompatible.
This introduces a new (documented) variable, AVAILABLE_LICENSES, that
is automatically updated to contain all licenses found in any
directories specified by ${COMMON_LICENSE_DIR} and ${LICENSE_PATH},
and uses it instead of SRC_DISTRIBUTE_LICENSES when expanding
wildcards.
(From OE-Core rev: 8c9ef587fe499c612a878a1ab42092eb79b334ef)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upstream mesa can either be built as a debug release (the default) or a
production release. This patch allows the user to choose which one they
want by setting MESA_BUILD_TYPE to either 'production' or 'debug' as
they see fit. Under OpenEmbedded a production build will be performed by
default.
(From OE-Core rev: 6a6b775d69aa59102e1f6a68c88276be6d54a8e6)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some upstream projects are employing the buildtype parameter so users
can create, say, "production" versus "debug" builds. Therefore create a
configurable parameter so recipes/users can tweak it.
(From OE-Core rev: 5b12211ef7856bedafd356f6b7a03add46eca6c7)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Patches have been refreshed and all of meta-oe and oe-core was sucessfully built.
(From OE-Core rev: 469bcfbfe25afd9c480c26f117b81d65cc305fbe)
Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user specified an http port to use for serving
files, use that instead of a random one. At the same
time, have the http server bind to all interfaces.
Binding to the server_ip might not always be possible,
e.g. in the case of running bitbake / oeqa from within
a docker container. In this case, the ip address is valid
outside the container, but not inside, and hence can't
be bound to. So switch to simply binding to all interfaces.
(From OE-Core rev: da529e6522959c2024023e90de52b5e42f160596)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow tests to access the listening port as just introduced.
Note that when using qemu this infrastructure shouldn't be
needed, but we still need to set the port to 0 so that
a listening port is determined automatically (e.g. by the
python http server).
(From OE-Core rev: ca58559fc30842c123907fb5cc68b3356c530862)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Similar to the existing possibility to specify a port in
TEST_TARGET_IP, allow TEST_SERVER_IP to also contain a
port.
The intention is for this port to be passed into e.g.
the http server from the apt / dnf / opkg tests, or
any other (custom) tests that might need the target to
connect to a service spawned by bitbake / oeqa, where
bitbake is actually running inside a docker container.
(From OE-Core rev: 9f401cd8b010ca613a151d0323b251f2243f399f)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Being able to specify the listening port is useful when
running OEQA from within a docker container, e.g.
crops or any other solution.
In that case, a port on the outside must be mapped to a
specific port inside the container. If no port is specified
for the http server module in this case, the http server
would choose a random port, which is unlikely to be mapped
and thus won't be reachable from the outside.
(From OE-Core rev: e54391ca9068661693275fbfcfded9bc0b2606f3)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
nf_tables_inet.ko and nf_tables_ipv4.ko and nf_tables_ipv6.o
(and nf_tables_arp.ko and nf_tables_netdev.ko) were merged into
the core nf_tables.ko before v4.17 in kernel commit
02c7b25e5f54 netfilter: nf_tables: build-in filter chain type
nf_meta.ko was merged to be a builtin of nf_tables.ko before
v4.18 in kernel commit
8a22543c8e70 netfilter: nf_tables: make meta expression builtin
(From OE-Core rev: 98715c9dc8dd925cffce9bd5d57172206d1e2a7b)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a mere 24KB (on 32bit arm), avoiding to pull in
all of the remainder +300KB in small initramfs images
when not needed.
Add this new package to RDEPENDS of e2fsprogs so as to
not break existing users.
Also, remove a stale reference to the non-existent
e2fsprogs-fsck package which was removed in 2012 from here
in commit 98b1b9f047ef ("e2fsprogs: Drop fsck binary"),
commit 93d6471727 in poky, and sort some of the lists
alphabetically.
(From OE-Core rev: 19dd3453da39066564d899d561e4814822dcbd18)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The script scans for services, parses them and makes sure those targets are still
there. The exists file check fails if the target is an alias, such as default.target
so add an additional test.
[YOCTO #13685]
(From OE-Core rev: 10bdbf033e51c97f6408c9114d480372135a2c2e)
Signed-off-by: Werner Grift <sky.captin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
0001-Unset-need_charset_alias-when-building-for-musl.patch is no longer
necessary as upstream has removed the code.
dm_check.patch is replaced with 0002-tests-use-skip_-rather-than-skip_test_-which-is-unde.patch
Rework ptests, in particular remove the generated Makefile
from the layer, and use the upstream Makefile.
Ptest pass rate is 100%.
(From OE-Core rev: 5dc6529d0b5b8b1bef12b90008f0a25612bd453e)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove 0001-Use-labs-instead-of-abs.patch as problem is fixed upstream.
(From OE-Core rev: 6ecc17f83d57d08bbeb6366cb586993a8844c8a6)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Optimise the call into the parent function to be only when needed.
(From OE-Core rev: 198bacaa1068fd68075d9182318fe64e9809dd36)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On test machines we have python3 available at alternative locations. Use
these rather from the evnrionment rather than a hardcoded path.
(From OE-Core rev: 81ee68f39a155c79443d45321ce4b81d9bf6675e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With hashequiv the get_taskhash function is called much more regularly
and contains expensive operations. This these don't change based upon
hash in a given build, improve the caching within the function to
reduce overhead.
(From OE-Core rev: de98cfe3cde4b8d5f4b163b5fba3f129651ef06a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use wic instead of the live/hddimg filesystem type for x86 machines, as it
produces better filesystems and doesn't have a hard limit of 4GB.
(From OE-Core rev: 04e4e93efa4d8e2bdde950fe95c2fd95f89c13e7)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python 3.8 has started to throw this as a warning but usage is wrong see
[1], it seems to be working by accident and not design.
Fixes
SyntaxWarning: "is" with a literal. Did you mean "=="?
[1]: https://bugs.python.org/issue34850
(From OE-Core rev: c3cb6bc21f3f84e2ed103b6438516ab85222faa9)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Configures the options passed to nasm to remove build paths in the
generated object files.
(From OE-Core rev: 9119a882adbf19df9f47f66cebaa7f730d3e9462)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds an option to nasm to change the prefix for file paths encoded in
the object files. This allows builds to be reproducible regardless of
the build directory.
(From OE-Core rev: ad5c914933c7b38296dcb8bba3c36aed45aacc32)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
