Allow tests to access the listening port as just introduced.
Note that when using qemu this infrastructure shouldn't be
needed, but we still need to set the port to 0 so that
a listening port is determined automatically (e.g. by the
python http server).
(From OE-Core rev: ca58559fc30842c123907fb5cc68b3356c530862)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Similar to the existing possibility to specify a port in
TEST_TARGET_IP, allow TEST_SERVER_IP to also contain a
port.
The intention is for this port to be passed into e.g.
the http server from the apt / dnf / opkg tests, or
any other (custom) tests that might need the target to
connect to a service spawned by bitbake / oeqa, where
bitbake is actually running inside a docker container.
(From OE-Core rev: 9f401cd8b010ca613a151d0323b251f2243f399f)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Being able to specify the listening port is useful when
running OEQA from within a docker container, e.g.
crops or any other solution.
In that case, a port on the outside must be mapped to a
specific port inside the container. If no port is specified
for the http server module in this case, the http server
would choose a random port, which is unlikely to be mapped
and thus won't be reachable from the outside.
(From OE-Core rev: e54391ca9068661693275fbfcfded9bc0b2606f3)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
nf_tables_inet.ko and nf_tables_ipv4.ko and nf_tables_ipv6.o
(and nf_tables_arp.ko and nf_tables_netdev.ko) were merged into
the core nf_tables.ko before v4.17 in kernel commit
02c7b25e5f54 netfilter: nf_tables: build-in filter chain type
nf_meta.ko was merged to be a builtin of nf_tables.ko before
v4.18 in kernel commit
8a22543c8e70 netfilter: nf_tables: make meta expression builtin
(From OE-Core rev: 98715c9dc8dd925cffce9bd5d57172206d1e2a7b)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a mere 24KB (on 32bit arm), avoiding to pull in
all of the remainder +300KB in small initramfs images
when not needed.
Add this new package to RDEPENDS of e2fsprogs so as to
not break existing users.
Also, remove a stale reference to the non-existent
e2fsprogs-fsck package which was removed in 2012 from here
in commit 98b1b9f047ef ("e2fsprogs: Drop fsck binary"),
commit 93d6471727 in poky, and sort some of the lists
alphabetically.
(From OE-Core rev: 19dd3453da39066564d899d561e4814822dcbd18)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The script scans for services, parses them and makes sure those targets are still
there. The exists file check fails if the target is an alias, such as default.target
so add an additional test.
[YOCTO #13685]
(From OE-Core rev: 10bdbf033e51c97f6408c9114d480372135a2c2e)
Signed-off-by: Werner Grift <sky.captin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
0001-Unset-need_charset_alias-when-building-for-musl.patch is no longer
necessary as upstream has removed the code.
dm_check.patch is replaced with 0002-tests-use-skip_-rather-than-skip_test_-which-is-unde.patch
Rework ptests, in particular remove the generated Makefile
from the layer, and use the upstream Makefile.
Ptest pass rate is 100%.
(From OE-Core rev: 5dc6529d0b5b8b1bef12b90008f0a25612bd453e)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove 0001-Use-labs-instead-of-abs.patch as problem is fixed upstream.
(From OE-Core rev: 6ecc17f83d57d08bbeb6366cb586993a8844c8a6)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Optimise the call into the parent function to be only when needed.
(From OE-Core rev: 198bacaa1068fd68075d9182318fe64e9809dd36)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On test machines we have python3 available at alternative locations. Use
these rather from the evnrionment rather than a hardcoded path.
(From OE-Core rev: 81ee68f39a155c79443d45321ce4b81d9bf6675e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With hashequiv the get_taskhash function is called much more regularly
and contains expensive operations. This these don't change based upon
hash in a given build, improve the caching within the function to
reduce overhead.
(From OE-Core rev: de98cfe3cde4b8d5f4b163b5fba3f129651ef06a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use wic instead of the live/hddimg filesystem type for x86 machines, as it
produces better filesystems and doesn't have a hard limit of 4GB.
(From OE-Core rev: 04e4e93efa4d8e2bdde950fe95c2fd95f89c13e7)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python 3.8 has started to throw this as a warning but usage is wrong see
[1], it seems to be working by accident and not design.
Fixes
SyntaxWarning: "is" with a literal. Did you mean "=="?
[1]: https://bugs.python.org/issue34850
(From OE-Core rev: c3cb6bc21f3f84e2ed103b6438516ab85222faa9)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Configures the options passed to nasm to remove build paths in the
generated object files.
(From OE-Core rev: 9119a882adbf19df9f47f66cebaa7f730d3e9462)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds an option to nasm to change the prefix for file paths encoded in
the object files. This allows builds to be reproducible regardless of
the build directory.
(From OE-Core rev: ad5c914933c7b38296dcb8bba3c36aed45aacc32)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
goarch is used in all classes of recipes ranging from native to target,
therefore its best to contain the variables and not spill over into
recipe classes where they can adversely affect shared state reuse e.g.
go-native currently gets TUNE_FEATURES into dependency chain of
referenced variables which means go-native gets rebuilt when we change
from qemuarm to qemux86 machine types since TUNE_FEATURES is defined
with DEFAULTTUNE which would change as machines are switched
These variables are specific to arm/mips/x86 so marking them with
appropriate overrides for native recipes will be right thing here
Chose 'hardfloat' for mips which is default too, 7 for arm and sse2 for x86
somehow go-native bootstrap compiler (1.4) still needs them so feed
commonly used values or defaults.
Fixes
ERROR: go-native different signature for task do_configure.sigdata between qemux86copy and qemuarm
...
List of dependencies for variable TUNE_FEATURES changed from '{'DEFAULTTUNE', 'TUNE_FEATURES_tune-core2-32'}' to '{'DEFAULTTUNE', 'TUNE_FEATURES_tune-armv7vethf-neon'}'
changed items: {'TUNE_FEATURES_tune-core2-32', 'TUNE_FEATURES_tune-armv7vethf-neon'}
(From OE-Core rev: 83eebf5577dd0a23be937375c1a8a15e3da4fa64)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes the log very large and breaks the ptest result
parser.
(From OE-Core rev: 78221af4c0297e0278c042eed390ad0e53f2548a)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop a big pile of backports.
Disable zsh completions, as they're unlikely to be useful.
(From OE-Core rev: dad8cd50bcc4203a65d153dc2445502c1e728975)
(From OE-Core rev: 1a31a31a3bd68f027ba0a13f8e66ce92104a4d4c)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
busybox as well as vlock utility from meta-oe provides vlock utility
which can conflict when with kbd if pam is a enabled distro_feature
Fixes image build errors
update-alternatives: Error: not linking <rootfs>/usr/bin/vlock to /bin/busybox.suid since <rootfs>/usr/bin/vlock exists and is not a link
ERROR: yoe-qt5-wayland-image-1.0-r0 do_rootfs: Postinstall scriptlets of ['busybox'] have failed. If the intention is to defer them to first boot,
then please place them into pkg_postinst_ontarget_${PN} ().
Deferring to first boot via 'exit 1' is no longer supported.
(From OE-Core rev: 08636708f6ca677a6ee6c88fa2999c7b70b7d474)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
The vlock doesn't exist if PAM isn't enabled.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When doing reproducible builds inherit podfix to remove the possibility
of differing Perl versions causing manpages to change.
(From OE-Core rev: 5dc43edab4787906f2c9f51e51bf78d1dc966286)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Manpages generated by Pod::Man contain the version number, which isn't
reproducible if we're using the host Perl to generate manpage.
One option is to always depend on perl-native when generating manpages
but this is a heavy dependency, so instead strip out the versions in
do_install().
(From OE-Core rev: 18d8e5ac689d6eb6098f68ac785f43e9d5f5938a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we fail to run the command to generate the tap devices then we should
show a reasonable message and then exit, without showing a traceback.
"return 1" at this point in the code does nothing because the caller
doesn't check the return, so just use sys.exit().
(From OE-Core rev: 47e40fdd7dd58bde4e017e2375c16450fcb14eca)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is not a good idea to mix logging and calls to print() - if the
output is being captured the result can be that the two types of output
are not recorded contiguously; this could be observed if an error
occurred running runqemu from inside testimage:
---------- snip ----------
ERROR: core-image-minimal-1.0-r0 do_testimage: Output from runqemu:
runqemu - INFO - Continuing with the following parameters:
runqemu - INFO - Setting up tap interface under sudo
sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
runqemu - ERROR - Setting up tap device failed:
Command '('sudo', '/home/paul/poky/poky/scripts/runqemu-ifup', '1000', '1000', '/home/paul/poky/poky/build/tmp/work/x86_64-linux/qemu-helper-native/1.0-r1/recipe-sysroot-native/usr/bin')' returned non-zero exit status 1.
Run runqemu-gen-tapdevs to manually create one.
runqemu - INFO - Cleaning up
KERNEL: [/home/paul/poky/poky/build/tmp/deploy/images/qemux86-64/bzImage--5.2.20+git0+bd0762cd13_dd25a04fc5-r0-qemux86-64-20191205213021.bin]
MACHINE: [qemux86-64]
FSTYPE: [ext4]
ROOTFS: [/home/paul/poky/poky/build/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.ext4]
CONFFILE: [/home/paul/poky/poky/build/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.qemuboot.conf]
---------- snip ----------
What we should see here is the KERNEL, MACHINE, etc. lines appearing
immediately after the "Continuing with the following parameters:" line
as they do when you run runqemu directly. If we put all of the lines
through the logger instead then it works properly.
(From OE-Core rev: ca64a3d490fbe1bf87c9f1dd6d87a1ecdeba8325)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we get here in the code it's because runqemu has failed, this is not
a debug situation - we need to see the output, so print it as an error.
Fixes [YOCTO #13681].
(From OE-Core rev: c1def38c680d64c992839166bbf9bec51eb0788a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If runqemu fails, there is some logic to run a set of commands to dump
various bits of information that might help debug the issue
(particularly in a busy situation such as on the autobuilder). One of
those commands is "top -bn1", however top restricts the output to the
width of the calling terminal, and for whatever reason this is a little
restrictive when called from inside testimage, so set COLUMNS in the
environment to a high value to fix it. (Another way is to use the -w
option, but that is not supported by our default busybox configuration
so it will then fail when this same list of commands is used on the
target).
(From OE-Core rev: 07da998f5aa6c3f5ffeee5150fc579e8a1051ba4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If runqemu fails, there is some logic to run a set of commands to dump
various bits of information that might help debug the issue
(particularly in a busy situation such as on the autobuilder). However,
when we try to run these we are inside the normal build environment
which restricts commands to be run on the host to those specified in
HOSTTOOLS. Since this isn't a place where host contamination is going to
be a problem, override PATH to a reasonable default so that we run the
actual host tools directly to avoid the issue. (Logically we would want
to use the original PATH value here, but it is not easily accessible.)
(From OE-Core rev: 3f6dcf67bccb4f0fc1a9a64e02bd0e2da620ad0e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If mtools isn't present on the underlying system this test would fail.
Ensure wic-tools is available in a similar way to other wic selftests
in OE-Core.
(From meta-yocto rev: 5d48d41a3f93ca0e7085ce3c0f646b1d0c8d52d1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
replace .wks with .wks.in and refer loader to EFI_PROVIDER
to enable different bootloader.
soft assign default EFI_PROVIDER "grub-efi" for genericx86.
(From meta-yocto rev: 11ecd2febed95e923c8cc68e2c438cd846a88d02)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a significant performance issue in get_unihash(). The issue turns out
to be the lookups of setscene tasks. We can fix this by using a set() instead of
the current list.
(Bitbake rev: 1e561672d039ebfb8cd0e0654a44dcf48513317c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit c4b8440f730c33eaf9f818b856ae81b2f1017fec.
The logic in this change is flawed and needs to be re-thought.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're seeing huge slowdowns on large builds on the autobuilder. A qemux86 world
build was running really slowly, a primary feature was lots of rehashing going on
due to an sstate change which caused a rebuild when all output should be identical.
This was traced to the hundreds of thousands of calls to get_unihash() from
get_taskash(). If we simplify the unitaskhashes data structure, we can bypass
the function call and access the data directly. In local profile charts,
this significanly sped up process_possible_migrations(), both on profiles
and visually.
Whilst this change doesn't aid readability, it does solve an otherwise
huge performance issue.
(Bitbake rev: c4b8440f730c33eaf9f818b856ae81b2f1017fec)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Batch all updates to scenequeue data together in a single invocation
instead of checking each task serially. This allows the checks for
sstate object to happen in parallel, and also makes sure the log
statement only happens once (per set of rehashes).
(Bitbake rev: db033a8f8a276d864bdb2e1eef159ab5794a0658)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids a directory not present error.
Fix a comment typo whilst here.
(From OE-Core rev: 1360d8d7d99b70a80c8cdbc1fc6d9e6752483139)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous patches meant the mkdir might no longer match the final target
directory. Fix this.
(From OE-Core rev: 0af4dae84099e8632a9ea6a4afdbea2f232bb170)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
