Sporadic errors have been observed in autobuilder when trying to store new
tests results:
error: failed to push some refs to 'push.yoctoproject.org:yocto-testresults'
hint: Updates were rejected because the tag already exists in the remote.
The new tag name is generated by gitarchive based on known tags from the
repository (learnt with git tag). In autobuilder case, this repository is a
shallow clone, so git tag only returns most recent tags, which mean we
could miss some older tags which exist in remote but not locally. In this
case, gitarchive will likely create a tag which already exists in remote,
and so will fail to push
Fix this tag duplication by using git ls-remote to learn about existing
tags instead of git tag. To do so, create a helper ("get_tags") which
manages both nominal case (target directory is a git repository with a
proper remote) and fallback case (target directory is not from a clone, no
remote has been configured)
Fixes [YOCTO #15140]
(From OE-Core rev: 9cbbe9689866158825a7ae774b7965b41ff5c461)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 5a0a7da85a3acfd4a20a07478eabefdab60f313a.
This caused failres on the build performance tests on the autobuilder.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sporadic errors have been observed in autobuilder when trying to store new
tests results:
error: failed to push some refs to 'push.yoctoproject.org:yocto-testresults'
hint: Updates were rejected because the tag already exists in the remote.
The new tag name is generated by gitarchive based on known tags from the
repository (learnt with git tag). In autobuilder case, this repository is a
shallow clone, so git tag only returns most recent tags, which mean we
could miss some older tags which exist in remote but not locally. In this
case, gitarchive will likely create a tag which already exists in remote,
and so will fail to push
Fix this tag duplication by using git ls-remote to learn about existing
tags instead of git tag. Two places which wrongly read only local tags has
been identified in gitarchive: expand_tag_strings and get_test_runs
Fixes [YOCTO #15140]
(From OE-Core rev: 5a0a7da85a3acfd4a20a07478eabefdab60f313a)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The LTP test reporting appears to be a little fragile so I tried to make
it more reliable.
Primarily this is done by not passing -p to runltp, which results in
machine-readable logfiles instead of human-readable. These are easier
to parse and have more context in, so we can also report correctly
skipped tests.
(From OE-Core rev: d585c6062fcf452e7288f6f8fb540fd92cbf5ea2)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The intent behind these functions was to dump the system state when issues occured
but it has never really worked as we'd planned. Regular monitoring as the build
runs has largely replaced this as that allows a trend to be seen rather than a spot
value which was never really useful. The code is bitrotting and not functioning
correctly so drop it.
[YOCTO #13872]
RP: Reword commit message
(From OE-Core rev: dea37ba49a236029da73d5cfbfc069bffc38b508)
Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Occasionally we see:
File "/media/build/poky/meta/lib/oeqa/runtime/cases/ptest.py", line 27, in test_ptestrunner_expectfail
self.do_ptestrunner()
File "/media/build/poky/meta/lib/oeqa/runtime/cases/ptest.py", line 77, in do_ptestrunner
results, sections = parser.parse(ptest_runner_log)
File "/media/build/poky/meta/lib/oeqa/utils/logparser.py", line 80, in parse
self.results[current_section['name']][result.group(1).strip()] = t
KeyError: 'No-section'
which occurs when there are "results" outside the main log section. The strace
tests do then upon failure as they dump logs there.
Add code to avoid the tracebacks and just make them warnings.
(From OE-Core rev: d9bf95d8cfb123f9d992fd2a95099bdcece97be8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The error message is a little misleading as the qmp module is a
directory with __init__.py file, not qmp.py file. Also, put the
path where we try to import it from in the error message to make
the message more indicative.
(From OE-Core rev: 08bacbf797f6a50ae8abe8fc3455b3a15a0a94b3)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Complex boot sequences take a lot of time to reach serial console login.
In our case boot involves multiple bootloaders like u-boot and grub,
then hypervisors like xen, then initramfs images which do things like
validating secure boot and encrypting the main rootfs. All these happen
before main rootfs serial console login prompt is reached and thus many
things can go wrong, including race conditions, hangs and deadlocks, and
thus logging the details is important for development.
This change adds the qemu serial console logs to do_testimage task logs while
they are being read out. The logs are not added to bitbake output. This
enables debugging and tracing the do_testimage via task log file while the boot
and tests are running.
(From OE-Core rev: cb62fae6ea3e411643395b12e56c363ce4d73e5d)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runqemu script works with qemu machines which provide login
and serial console to the qemu process stdout. Add the same support
to qemurunner so that testing with testimage.bbclass is possible.
Default qemu machines provide serial console boot logs and login via
socket to qemu process but I don't see a reason why qemu process stdout
should not be supported too since they work with runqemu as well.
(From OE-Core rev: a3d4b80c5b4cce933c759d023c75b8671c56fe12)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that even if a ptests results section is empty, the log parser adds that
empty section. Then ensure that empty sections trigger warnings.
This means if a ptest suddently stops returning any results, we notice and see
warnings about it. This has gone unnoticed on the autobuilder far too many times
so is very much worth highlighting as a regression. We shouldn't have empty ptests.
(From OE-Core rev: 5ad0cf57b41ec7f44647a03bc568d0b24906cc8d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* There is no need to be that precise. It just irritates in the logs.
* There is also no point in printing plain time.time() value a single
time while only using formatted printout everywhere else, thus remove
it.
* Use %d for printing integer times
(From OE-Core rev: f0988cb8cf8d03708490cca4eba345492ef78d52)
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since login user is configurable with boot_patterns['send_login_user'],
unconditionally using 'root' in the debug message can be confusing.
Also fix the debug message to say 'Logged in' instead of 'Logged'.
(From OE-Core rev: 900e3d42b918b5a33d8d952b3a8078fbe72ba98f)
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Read from serial console with a small delay to bundle data to e.g.
full lines. Reading one character at a time is not needed and causes
busy looping.
(From OE-Core rev: 0049f6757f6f956fb4cc77b3df6a672c20b53cf4)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a qemu machine hangs, the QMP calls can hang for ever
too, and when this happens any failing test commands from ssh
runner may be followed by dump_monitor() calls which
then also hang. Hangs followed by hangs.
Use runqemutime at setup and run_monitor() specific timeout
for later calls.
(From OE-Core rev: 3a07bdf77dc6ecbf4c620b051dd032abaaf1e4ff)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use a short sleep to bundle serial console reads so that
we are not reading one character at a time which reduces busy
looping.
(From OE-Core rev: cafe65d8cf7544edbd387f7f5f6d77c64c6b18fa)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If test target qemu machine hangs completely, dump_target() calls
over serial console are taking a long time to time out, possibly
for every failing ssh command execution and a lot of test cases,
and same with dump_monitor().
Instead of trying for ever, count errors and after 5 stop trying
to dump_target() and dump_monitor() completely.
These help to end testing earlier when a test target is completely
deadlocked and all ssh, serial and QMP communication with it are
failing.
(From OE-Core rev: d9ad0a055abba983c6cee1dca4d2f0a8a3c48782)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Call logger.info() in the log_message handler so that we get request
logging, and hopefully even error messages.
Create a child logger to be neat and compartmentalise the logging.
Add a __main__ entrypoint so this class can be exercised outside of oeqa.
Remove unused traceback import.
(From OE-Core rev: 8fc939d942cec1662ca386b87c63f167bb10fd05)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This does not actually guarantee that the child runqemu process has completely exited:
poll() may return prematurely while the SIGTERM handler in runqemu is still running.
This thwarts the rest of the processing, and may terminate the handler before
it completes.
Use Popen.communicate() instead: this is what python documentation recommends as well:
https://docs.python.org/3/library/subprocess.html#subprocess.Popen.communicate
(From OE-Core rev: cd3e55606c427287f37585c5d7cde936471e52f4)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It seems some layers want to subvert the intent of LAYERSERIES_COMPAT
so bitbake is going to have to become stricter about the values there.
To work with this, use LAYERSERIES_CORENAMES to generate the entries in
LAYERSERIES_COMPAT instead of the current magic LAYERSERIES_COMPAT_core
value which may not continue to work.
The downside to this is when migating between releases, people would
need to update devtool workspace layer.conf files. I guess you could
argue this is a feature!
(From OE-Core rev: 96ff9baa8ead57504f40f362ed3a4aaa776d1b58)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the first release in 13 years;
I have reviewed the accumulated patches, and dropped some of them
where purpose or issue being fixed is not clear. Specifically:
0001-Add-listen-action-for-a-tcp-socket.patch
0001-daemon.c-Libtirpc-porting-fixes.patch
fixed upstream in
84ab475f93
0001-attr-fix-utime-for-symlink.patch
addresses an open issue in
https://github.com/unfs3/unfs3/issues/4
please rebase and re-submit as a PR if the problem is still present.
alternate_rpc_ports.patch
unnecessary as of
https://git.yoctoproject.org/poky/commit/?id=6bb9860ef7ba9c84fe9bd3a81aa6555f67ebd38e
Command line options introduced by the patch no longer used anywhere.
fix_compile_warning.patch
merged upstream.
fix_pid_race_parent_writes_child_pid.patch
rebased and re-submitted upstream.
no-yywrap.patch
dropped as backport.
relative_max_socket_path_len.patch
needs to be re-submitted by the original author, purpose and reproducer scenario unclear.
rename_fh_cache.patch
merged upstream.
tcp_no_delay.patch
purpose and use case for oe unclear.
unfs3_parallel_build.patch
fixed upstream in
987d32ca12a39a78995c
Drop -N option from oeqa nfs helper and runqemu helper;
the option was provided by tcp_no_delay.patch
and is not needed for the tests or qemu.
Drop ad hoc libtirpc support; upstream supports it directly now.
Drop the check for portmap/rpcbind, it is unnecessary as of
https://git.yoctoproject.org/poky/commit/?id=6bb9860ef7ba9c84fe9bd3a81aa6555f67ebd38e
(From OE-Core rev: fa2f7cf545137b071db97015bca5b70d77566cd8)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Qemurunner should not pass rootfs to runqemu in case
rootfs is not a filesystem itself. Some images could
be built into some disk format like vmdk and this
commit makes qemurunner handle such images properly.
(From OE-Core rev: 5aea74046cf4c1aa7fa9e2402788d662268ccf53)
Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default host side IP address is not set and qemu listens
on all IP addresses on the host machine which is not a good
idea when images have root login enabled without password.
It make sense to listen only on localhost IP address 127.0.0.1 using
config change like:
QB_SLIRP_OPT = "-netdev user,id=net0,hostfwd=tcp:127.0.0.1:2222-:22"
This config works for qemu itself, but breaks runqemu which tries to
parse the host side port number from qemu process command line arguments.
So change the runqemu side hostfwd parsing for port number to ignore
the host IP address field.
Reviewed-by: Quentin Schulz <foss+yocto@0leil.net>
(From OE-Core rev: bdbd52082eb26f418000eb4e424baae9babc272c)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now there are no users of getResults, drop it. The code doesn't work with
python 3.8 onwards and is pretty horrible with internal python knowledge.
We don't need this.
(From OE-Core rev: 058d00386f13c48e84081adc0504fc71a06c6529)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These decorators aren't used anywhere in core and broke from python 3.8
onwards. The code implementing them (in getResults) is pretty horrible
and I'm happy to see them and it removed.
(From OE-Core rev: 1d7ff45e2bd48c613a0757491daaa3bedc97cb5e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using a binary string as the login banner search expression is fraught with
risks. We've seen cases on the autobuilder where "login:" is clearly shown
but the code hasn't triggered. The most likely cause is hidden control characters
in the output causing the search to fail.
Take the opportunity to remove the horrible binary string search, at the expense of
decoding the bootlog multiple times.
Tweak the logging so we can know which log was printed (self.msg or bootlog)
just in case this isn't the issue and we need more information in future.
(From OE-Core rev: 91b9e30e08695e715ef14c3df7471e8c99f9deb5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updates the log message printed when login banner is seen in QEMU to
report the UNIX Epoch time in addition to the human readable time. This
makes it much easier and accurate to correlate logs with the guest, in
particular with the guest journalctl which prints log timestamps in
human readable format and the oeqa SSH debug logging which prints the
UNIX Epoch.
(From OE-Core rev: 275b6f3c8d0eeafa3902c48a49655491a89c47bc)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upstream made changes to the qmp module. We need to use the legacy one
for now since that matches the interface we use, ultimately we likely
need to update our code.
Also fix the generic exception handler to show the actual exception
which helps debugging when something does break.
(From OE-Core rev: 6e90b0d909d3c8b1be5cb19f2411cd0e89735c84)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If native_sysroot is passed, also support the caller passing in the
target_sys and add that to the path if so. This allows runCmd() to be
used to invoke the cross tools.
(From OE-Core rev: afa3d3ba00b40fd29e9852eeaa2c2c9b68f18659)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a comment explaining the non-obvious return codes.
(From OE-Core rev: 6572baffa02ba6b8a686490d55af17cacb528920)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Where there isn't a copyright statement, add one to make it explicit.
Also add license identifiers as MIT if there isn't one.
(From OE-Core rev: bb731d1f3d2a1d50ec0aed864dbca54cf795b040)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
| Failed to dump QMP CMD: query-status with
| Exception: [Errno 2] No such file or directory: '.../tmp/log/runtime-hostdump/qmp_00_query-status'
| Failed to dump QMP CMD: query-block with
| Exception: [Errno 2] No such file or directory: '.../tmp/log/runtime-hostdump/qmp_00_query-block'
| Failed to dump QMP CMD: dump-guest-memory with
| Exception: [Errno 2] No such file or directory: '.../tmp/log/runtime-hostdump/qmp_00_dump-guest-memory'
The qmp dump commands could fail, because of missing root directory.
So create it before any log writing.
(From OE-Core rev: c4dc5d674afe65fedb5195f187b68f23720646ba)
Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ltp compliancy parser is rewritten to actually
match the logs: they seem to be unstructured, test case names
are not printed and the only indication of failure is appearance of
FAIL[ED] somewhere.
(From OE-Core rev: 52766561dbfee625c89393905a85e10d85f69c6c)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
sys.exit will cause finally statements and other code to run at exit. Since
we're using os.fork() here, os._exit() is apprioriate in this codepath.
(From OE-Core rev: ec08498ff29de9ccd23be88b9d7af3dab6bbb81e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is done when starting up qemu has failed, but is not done
when qemu started ok, but fails later in QMP communication.
Output from runqemu does contain valuable information to find out
why, so rather than fix all the QMP fails to include it, let's just
print it in stop().
(From OE-Core rev: 6e2bf68e4401db747484c2c8ba0f77500b1d2d49)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Files in /proc/xxx/map_files/ may no longer exist, just ignore this rather than
raising an exception.
(From OE-Core rev: fb1027896a263cd91e2378a4e97dbdf0807b306b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
