Commit Graph

11 Commits

Author SHA1 Message Date
Mikko Rapeli
bd1fc66e4c testimage.bbclass: remove QEMU_USE_SLIRP variable
QEMU_USE_SLIRP is replaced by TEST_RUNQEMUPARAMS with "slirp" and
possibly other arguments to runqemu script.

(From OE-Core rev: 99fd24f0d9ff79fed389ae5a01c3031d7e7167d0)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-14 21:57:16 +01:00
Ross Burton
a96f735ff9 testimage: respect target/server IPs when using slirp
We can't hardcode these IPs when using slirp.

The target IP will need a port to be specified as this controls what port
the SSH connection uses, and when slirp is used it can't bind to port
22. The qemu runner (OEQemuTarget) assumes that the first port forward
is the SSH forward, but this may be wrong or a different target may be
used.

The server IP depends on how the virtual networking is configured.
runqemu defaults to 10.0.2.x for the guests so that is a wise default,
but that may be configured differently.

(From OE-Core rev: 81b304e2558730de285f2773371340fc636a8ed1)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-14 09:19:38 +01:00
Mikko Rapeli
8af6f66bb4 testimage.bbclass: detect slirp from TEST_RUNQEMUPARAMS
Also set target and server IP addresses correctly to 127.0.0.1
so that TEST_TARGET_IP and TEST_SERVER_IP don't
need to be set manually with slirp.

Users run qemu images with slirp networking like this:

$ bitbake core-image-minimal
$ ../poky/scripts/runqemu slirp

slirp networking means that only one TCP port is forwarded from host
system to the qemu machine so that ssh into the machine works. ping
and other low level networking from host to target machine do not work,
but networking from target via host does work. This is much more easy to
setup than the bridge networking used by default, which I've never
managed to get running on various host machines.

To use slirp networking wita qemu and core-image-minimal to run testimage.bbclass tests, add
these to local.conf:

IMAGE_CLASSES += "testimage"
IMAGE_FEATURES += "ssh-server-dropbear"
TEST_RUNQEMUPARAMS += "slirp"

Then image can be compiled and tested with:

$ bitbake core-image-minimal
$ bitbake -c testimage core-image-minimal

Note that only few of the current oeqa runtime tests work against
core-image-minimal, but test results look like:

RESULTS:
RESULTS - date.DateTest.test_date: PASSED (3.64s)
RESULTS - df.DfTest.test_df: PASSED (0.55s)
RESULTS - oe_syslog.SyslogTest.test_syslog_running: PASSED (0.56s)
RESULTS - oe_syslog.SyslogTestConfig.test_syslog_logger: PASSED (1.88s)
RESULTS - oe_syslog.SyslogTestConfig.test_syslog_restart: PASSED (0.93s)
RESULTS - pam.PamBasicTest.test_pam: PASSED (2.20s)
RESULTS - parselogs.ParseLogsTest.test_parselogs: PASSED (4.98s)
RESULTS - ping.PingTest.test_ping: PASSED (0.05s)
RESULTS - ssh.SSHTest.test_ssh: PASSED (1.28s)
RESULTS - systemd.SystemdBasicTests.test_systemd_basic: PASSED (0.56s)
RESULTS - systemd.SystemdBasicTests.test_systemd_failed: PASSED (1.10s)
RESULTS - systemd.SystemdBasicTests.test_systemd_list: PASSED (0.92s)
RESULTS - systemd.SystemdJournalTests.test_systemd_boot_time: PASSED (0.56s)
RESULTS - systemd.SystemdJournalTests.test_systemd_journal: PASSED (0.54s)
RESULTS - apt.AptRepoTest.test_apt_install_from_repo: SKIPPED (0.00s)
RESULTS - buildcpio.BuildCpioTest.test_cpio: SKIPPED (0.00s)
RESULTS - buildgalculator.GalculatorTest.test_galculator: SKIPPED (0.00s)
RESULTS - buildlzip.BuildLzipTest.test_lzip: SKIPPED (0.00s)
RESULTS - connman.ConnmanTest.test_connmand_help: SKIPPED (0.00s)
RESULTS - connman.ConnmanTest.test_connmand_running: SKIPPED (0.00s)
RESULTS - dnf.DnfBasicTest.test_dnf_help: SKIPPED (0.00s)
RESULTS - dnf.DnfBasicTest.test_dnf_history: SKIPPED (0.00s)
RESULTS - dnf.DnfBasicTest.test_dnf_info: SKIPPED (0.00s)
RESULTS - dnf.DnfBasicTest.test_dnf_search: SKIPPED (0.00s)
RESULTS - dnf.DnfBasicTest.test_dnf_version: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_exclude: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_install: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_install_dependency: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_install_from_disk: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_install_from_http: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_installroot: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_installroot_usrmerge: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_makecache: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_reinstall: SKIPPED (0.00s)
RESULTS - dnf.DnfRepoTest.test_dnf_repoinfo: SKIPPED (0.00s)
RESULTS - gcc.GccCompileTest.test_gcc_compile: SKIPPED (0.00s)
RESULTS - gcc.GccCompileTest.test_gpp2_compile: SKIPPED (0.00s)
RESULTS - gcc.GccCompileTest.test_gpp_compile: SKIPPED (0.00s)
RESULTS - gcc.GccCompileTest.test_make: SKIPPED (0.00s)
RESULTS - gi.GObjectIntrospectionTest.test_python: SKIPPED (0.00s)
RESULTS - go.GoHelloworldTest.test_gohelloworld: SKIPPED (0.00s)
RESULTS - kernelmodule.KernelModuleTest.test_kernel_module: SKIPPED (0.00s)
RESULTS - ldd.LddTest.test_ldd: SKIPPED (0.00s)
RESULTS - logrotate.LogrotateTest.test_logrotate_newlog: SKIPPED (0.00s)
RESULTS - logrotate.LogrotateTest.test_logrotate_wtmp: SKIPPED (0.00s)
RESULTS - oe_syslog.SyslogTestConfig.test_syslog_startup_config: SKIPPED (0.00s)
RESULTS - opkg.OpkgRepoTest.test_opkg_install_from_repo: SKIPPED (0.00s)
RESULTS - perl.PerlTest.test_perl_works: SKIPPED (0.00s)
RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectfail: SKIPPED (0.00s)
RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectsuccess: SKIPPED (0.00s)
RESULTS - python.PythonTest.test_python3: SKIPPED (0.00s)
RESULTS - rpm.RpmBasicTest.test_rpm_help: SKIPPED (0.00s)
RESULTS - rpm.RpmBasicTest.test_rpm_query: SKIPPED (0.00s)
RESULTS - rpm.RpmBasicTest.test_rpm_query_nonroot: SKIPPED (0.00s)
RESULTS - rpm.RpmInstallRemoveTest.test_check_rpm_install_removal_log_file_size: SKIPPED (0.00s)
RESULTS - rpm.RpmInstallRemoveTest.test_rpm_install: SKIPPED (0.00s)
RESULTS - rpm.RpmInstallRemoveTest.test_rpm_remove: SKIPPED (0.00s)
RESULTS - rust.RustCompileTest.test_cargo_compile: SKIPPED (0.00s)
RESULTS - rust.RustCompileTest.test_rust_compile: SKIPPED (0.00s)
RESULTS - scp.ScpTest.test_scp_file: SKIPPED (0.00s)
RESULTS - stap.StapTest.test_stap: SKIPPED (0.00s)
RESULTS - systemd.SystemdServiceTests.test_systemd_disable_enable: SKIPPED (0.00s)
RESULTS - systemd.SystemdServiceTests.test_systemd_disable_enable_ro: SKIPPED (0.00s)
RESULTS - systemd.SystemdServiceTests.test_systemd_status: SKIPPED (0.00s)
RESULTS - systemd.SystemdServiceTests.test_systemd_stop_start: SKIPPED (0.00s)
RESULTS - weston.WestonTest.test_wayland_info: SKIPPED (0.00s)
RESULTS - weston.WestonTest.test_weston_can_initialize_new_wayland_compositor: SKIPPED (0.00s)
RESULTS - weston.WestonTest.test_weston_running: SKIPPED (0.00s)
RESULTS - weston.WestonTest.test_weston_supports_xwayland: SKIPPED (0.00s)
RESULTS - xorg.XorgTest.test_xorg_running: SKIPPED (0.00s)
SUMMARY:
core-image-minimal () - Ran 70 tests in 22.173s
core-image-minimal - OK - All required tests passed (successes=14, skipped=56, failures=0, errors=0)
NOTE: recipe core-image-minimal-1.0-r0: task do_testimage: Succeeded
NOTE: Tasks Summary: Attempted 1305 tasks of which 1304 didn't need to be rerun and all succeeded.

(From OE-Core rev: f4e865062cec06586d8c38c05d86a5b0d727625c)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-12 12:23:09 +01:00
Thomas Roos
f82c3bd37d testimage/oeqa: Drop testimage_dump_host functionality
The intent behind these functions was to dump the system state when issues occured
but it has never really worked as we'd planned. Regular monitoring as the build
runs has largely replaced this as that allows a trend to be seen rather than a spot
value which was never really useful. The code is bitrotting and not functioning
correctly so drop it.

[YOCTO #13872]

RP: Reword commit message
(From OE-Core rev: dea37ba49a236029da73d5cfbfc069bffc38b508)

Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-29 10:57:27 +01:00
Richard Purdie
5967a90e8a testimage: Only note missing target directories, don't warn
We don't need to see warnings for missing target debug directories. Just
show a note in the logs instead.

(From OE-Core rev: 52db25c58069c4f440da33daf0474255c9fa870b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-17 11:50:57 +01:00
Alexis Lothoré
58206ea8d9 testimage: implement test artifacts retriever for failing tests
Add a basic artifacts retrievers in testimage class which:
- triggers when at least one runtime test fails but tests execution
  encountered no major issue
- reads a list of paths to retrieve from TESTIMAGE_FAILED_QA_ARTIFACTS
- checks for artifacts presence on target
- retrieve those files over scp thanks to existing ssh class
- store those files in an "artifacts" directory in "tmp/log/oeqa/<image>"

This implementation assumes that the SSH or Qemu target has run and
finished gracefully. If tests do not finish because of an exception,
artifacts will not be retrieved

Bring partial solution to [YOCTO #14901]

(From OE-Core rev: 36ef582b8c1c99e6af1ce79ea79f5b059d2a1aad)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-17 11:50:57 +01:00
Martin Jansa
3f0acc2a61 runqemu: respect IMAGE_LINK_NAME
* when searching for qemuboot.conf
* don't assume that IMAGE_LINK_NAME is always
  <rootfs>-<machine> (with <rootfs>-<machine>.qemuboot.conf)

* runqemu: use IMAGE_LINK_NAME set by testimage.bbclass or query with bitbake -e

* testimage.bbclass was setting DEPLOY_DIR which I don't see used
  anywhere else, so I assume it was supposed to be DEPLOY_DIR_IMAGE as mentioned
  in corresponding runqemu code, do the same with IMAGE_LINK_NAME variable

* add virtual/kernel as bitbake -e target in run_bitbake_env to make
  sure IMAGE_LINK_NAME is defined (kernel-artifact-names.bbclass inherits
  image-artifact-names.bbclass as well)

* improve .qemuboot.conf search
  1st search for file matching the rootfs and only when not found
  try again with .rootfs suffix removed

[YOCTO #12937]

(From OE-Core rev: 716eb55bb963db7b02d985849cb025898aabc855)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-04 13:11:49 +01:00
Alejandro Hernandez Samaniego
61c35514cf testimage: Fix error message to reflect new syntax
(From OE-Core rev: ec73d19d78e8f30ff9b817490c23bcdf8ea47c86)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-02 10:27:30 +00:00
Ross Burton
868eb84303 oeqa/runtime/dnf: use dnf-test packages
Instead of installing run-postinsts with it's postinst scripts causing
systemd restarts, use the new dnf-test-* packages instead.

Remove from the installroot tests entirely as they're exercised enough
using just busybox.

Rewrite the exclude test to be simplier now these packages are not going
to be part of an existing dependency chain.

[ YOCTO #14787 ]

(From OE-Core rev: fb1de2abc53bd742bc55cfecd384b78852c10d80)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-26 22:04:35 +01:00
Richard Purdie
2d96d3f5ac selftest/qemurunner: Work around possible control character contamination
Using a binary string as the login banner search expression is fraught with
risks. We've seen cases on the autobuilder where "login:" is clearly shown
but the code hasn't triggered. The most likely cause is hidden control characters
in the output causing the search to fail.

Take the opportunity to remove the horrible binary string search, at the expense of
decoding the bootlog multiple times.

Tweak the logging so we can know which log was printed (self.msg or bootlog)
just in case this isn't the issue and we need more information in future.

(From OE-Core rev: 91b9e30e08695e715ef14c3df7471e8c99f9deb5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-13 23:03:36 +01:00
Richard Purdie
fd1517e2b5 classes: Update classes to match new bitbake class scope functionality
Move classes to classes-global or classes-recipe as appropriate to take
advantage of new bitbake functionality to check class scope/usage.

(From OE-Core rev: f5c128008365e141082c129417eb72d2751e8045)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-12 15:27:17 +01:00