Commit Graph

4068 Commits

Author SHA1 Message Date
Martin Jansa
7eef95d723 oeqa: loader.py: show warning when skipping selected module and abort if all are skipped
* skipped modules were triggering an ERROR before:

  poky $ oe-selftest -v -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers imagefeatures.ImageFeatures.test_image_gen_debugfs -K -B /OE/build/poky/build-eSDK
  2023-03-13 15:07:53,430 - oe-selftest - ERROR - Not found eSDK.oeSDKExtSelfTest.test_install_libraries_headers in loaded test cases

* but didn't show the reason why it wasn't loaded and more importantly -r
  was ignored when all selected modules were silently skipped

* add a warning when skipping some module and abort if some modules were
  selected, but all ended being skipped:

  poky $ oe-selftest -v -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers -K -B /OE/build/poky/build-eSDK
  2023-03-13 15:11:51,028 - oe-selftest - WARNING - module 'eSDK.oeSDKExtSelfTest.test_install_libraries_headers' was skipped from selected modules, because it doesn't match with module name assumptions: package and module names do not contain upper case characters, whereas class names do
  2023-03-13 15:11:51,028 - oe-selftest - ERROR - All selected modules were skipped, this would trigger selftest with all tests and -r ignored.

* I was hit by this in oe-selftest -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers
  which is skipped due to upper case characters in module name and selftest started to run
  all tests (archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name as first from 529)

  poky $ oe-selftest -v -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers -K -B /OE/build/poky/build-eSDK
  2023-03-13 14:00:52,955 - oe-selftest - DEBUG - Selected tests with -r: ['eSDK.oeSDKExtSelfTest.test_install_libraries_headers']
  2023-03-13 14:00:55,531 - oe-selftest - INFO - Changing cwd to /OE/build/poky/build
  ..
  2023-03-13 14:00:58,128 - oe-selftest - INFO - test_archiver_allows_to_filter_on_recipe_name (archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name)

  I'll rename eSDK to esdk in next commit to avoid this.

* also fix small typo in context I've noticed when debugging this

(From OE-Core rev: 15229c9abaf4cc398c31d97d171b9fac57141873)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
248f6f01cf selftest: imagefeatures.py: respect IMAGE_LINK_NAME for debugfs and manifest as well
* these cases were correctly respecting IMAGE_LINK_NAME in most tests

* the only exception was relatively wide glob for manifest:
  "test-empty-image-*.manifest"
* and even wider glob for -dbg:
  "*-dbg.rootfs.tar.bz2"

* replace them with the exact filename we expect for given image

* be aware that gzip won't accept the symlink in IMAGE_LINK_NAME causing:
  2023-03-13 08:58:23,845 - oe-selftest - INFO -  ... FAIL
  2023-03-13 08:58:23,845 - oe-selftest - INFO - Traceback (most recent call last):
    File "/OE/build/poky/meta/lib/oeqa/selftest/cases/imagefeatures.py", line 124, in test_bmap
      self.assertTrue(runCmd('gzip -t %s' % gzip_path))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/OE/build/poky/meta/lib/oeqa/utils/commands.py", line 214, in runCmd
      raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
  AssertionError: Command 'gzip -t /OE/build/poky/build/build-st-2023-03-12-todo-patch2/imagefeatures.ImageFeatures.test_bmap/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.ext4.bmap.gz' returned non-zero exit status 1:
  gzip: skipping: /OE/build/poky/build/build-st-2023-03-12-todo-patch2/imagefeatures.ImageFeatures.test_bmap/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.ext4.bmap.gz is a symbolic link

* and "*-dbg.rootfs.tar.bz2" doesn't work if IMAGE_NAME_SUFFIX
  is changed to anything else than ".rootfs" or moved into
  IMAGE_LINK_NAME (like I plan in future changes where this will
  need to be updated again).

  Luckily we can use the symlink which currently doesn't have
  IMAGE_NAME_SUFFIX nor the DATETIME, so we don't need to search
  for it with glob, e.g. currently:

  core-image-minimal-qemux86-64-dbg-20230313112546-dbg.rootfs.tar.bz2
  core-image-minimal-qemux86-64-dbg.tar.bz2 -> core-image-minimal-qemux86-64-dbg-20230313112546-dbg.rootfs.tar.bz2

[YOCTO #12937]

(From OE-Core rev: 39285e981343930e41afe4eb8f2db675a85d54c2)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
752d8ab3ed selftest: minidebuginfo.py respect IMAGE_LINK_NAME
* use IMAGE_LINK_NAME instead of hardcoding
  core-image-minimal-${MACHINE} assumption

[YOCTO #12937]

(From OE-Core rev: 9f8ffe22d9bb7cd93b9bc9ece917a48b27ab22d3)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
2e7d36b542 selftest: gdbserver.py: respect IMAGE_LINK_NAME
* use IMAGE_LINK_NAME instead of hardcoding
  core-image-minimal-${MACHINE} assumption

[YOCTO #12937]

(From OE-Core rev: 3ee65ecdf841d74e2767d620e7f4a62e3a8863dc)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
923a9e11b1 selftest: fitimage.py: respect INITRAMFS_IMAGE_NAME and KERNEL_FIT_LINK_NAME
* use these variables instead of assuming:
  "fitImage-its-%s-%s-%s" % (image_type, machine, machine)
  notice that these files have no extension and the -machine suffix
  is duplicated (once from core-image-minimal and once from
  kernel-fitimage.bbclass:kernel_do_deploy:append
  through KERNEL_FIT_LINK_NAME which is MACHINE by default:

  KERNEL_FIT_NAME ?= "${KERNEL_ARTIFACT_NAME}"
  KERNEL_FIT_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}"

  KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
  KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}"

[YOCTO #12937]

(From OE-Core rev: d6e52428ca25d9b6ec2a52406f2d1859d05867d3)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
a2b8fec289 selftest: runqemu: respect IMAGE_LINK_NAME
* don't assume that every built image is named:

  <recipe>-<machine>.<suffix>

[YOCTO #12937]

(From OE-Core rev: d3c1c983d062f0731610f4a6984126ec6dab8caf)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
e1334b120e selftest: wic: respect IMAGE_LINK_NAME also in test_rawcopy_plugin_qemu
* this one is more tricky, because the test_rawcopy_plugin.wks.in file
  is used while building core-image-minimal-mtdutils, but the image filename
  inside wks.in is from core-image-minimal, so we cannot just let bitbake
  expand IMAGE_LINK_NAME, use separate variable set in the same config fragment
  IMAGE_LINK_NAME_CORE_IMAGE_MINIMAL

[YOCTO #12937]

(From OE-Core rev: 8864ee825fdc52fe7a8cf93876c81c639fba66f4)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
af9f67606c selftest: wic: respect IMAGE_LINK_NAME
* use IMAGE_LINK_NAME instead of hardcoding
  core-image-minimal-${MACHINE} assumption

[YOCTO #12937]

(From OE-Core rev: d6151aeb1e8f6162113b1123fa0f162536e48a44)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
9dd62390e2 selftest: devtool: set BB_HASHSERVE_UPSTREAM when setting SSTATE_MIRROR
* with my build/conf/local.conf:
  SSTATE_DIR = "/OE/build/poky/build/sstate-cache"
  these devtool tests will first set own SSTATE_DIR and the original one set as SSTATE_MIRROR:

2023-03-11 11:51:46,837 - oe-selftest - INFO - test_devtool_update_recipe_append (devtool.DevtoolUpdateTests.test_devtool_update_recipe_append)
2023-03-11 11:51:46,846 - oe-selftest - DEBUG - Appending to: /OE/build/poky/build/build-st-2023-03-11-patch2/devtool.DevtoolUpdateTests.test_devtool_update_recipe_append/build-st/conf/selftest.inc
SSTATE_DIR = "/OE/build/poky/build/build-st-2023-03-11-patch2/devtool.DevtoolUpdateTests.test_devtool_update_recipe_append/build-st/sstate_devtool"
SSTATE_MIRRORS += "file://.* file:////OE/build/poky/build/sstate-cache/PATH"

* but that unfortunately leads to a warning from sanity.bbclass
  about SSTATE_MIRRORS without matching BB_HASHSERVE, because
  BB_HASHSERVE is set to "auto" by default

  these tests failing with:

2023-03-11 11:55:39,610 - oe-selftest - INFO - ======================================================================
2023-03-11 11:55:39,610 - oe-selftest - INFO - FAIL: test_devtool_update_recipe_append_git (devtool.DevtoolUpdateTests.test_devtool_update_recipe_append_git)
2023-03-11 11:55:39,610 - oe-selftest - INFO - ----------------------------------------------------------------------
2023-03-11 11:55:39,611 - oe-selftest - INFO - Traceback (most recent call last):
  File "/OE/build/poky/meta/lib/oeqa/selftest/cases/devtool.py", line 1118, in test_devtool_update_recipe_append_git
    self.assertNotIn('WARNING:', result.output)
AssertionError: 'WARNING:' unexpectedly found in 'NOTE: Starting bitbake server...\nWARNING: You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror.\nLoading cache...done.\nLoaded 0 entries from dependency cache.\nParsing recipes...done.\nParsing of 947 .bb files complete (0 cached, 947 parsed). 1764 targets, 52 skipped, 0 masked, 0 errors.\n\nSummary: There was 1 WARNING message.\nINFO: Updating SRCREV in recipe mtd-utils-selftest_git.bb\nNOTE: Writing append file /tmp/devtoolqa1m2lh02v/layer/recipes-devtools/mtd/mtd-utils-selftest_git.bbappend'
----------------------------------------------------------------------

* just setting BB_HASHSERVE to empty doesn't work, because then we
  would need to disable OEEquivHash as well as it fails with:

  ERROR: OEEquivHash requires BB_HASHSERVE to be set

(From OE-Core rev: 96d4392ee9c5c3674e5c4c4512f527a2ca6765e4)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Mikko Rapeli
ac59487364 oeqa rtc.py: skip if read-only-rootfs
hwclock command fails on read-only-rootfs:

AssertionError: 1 != 0 : Failed to reset RTC time, output: hwclock: cannot open /etc/adjtime: Read-only file system

(From OE-Core rev: 017bf8c160f6ab67d9f8e8d9e30b15bf84f73807)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-14 17:13:11 +00:00
Richard Purdie
2ba8593b95 oeqa/selftest/sstate: Split classes to allow more parallelism
By splitting the single class into multiple classes, it allows more parallelism
in the execution. These tests don't have interdependencies that benefit
from sstate reuse or anything like that so this makes sense to improve overall
test execution time if/where possible.

(From OE-Core rev: 9dbc2f4ebbde47b1c997948a9690ffb8eb29b552)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-13 17:35:22 +00:00
Richard Purdie
2420625414 oeqa/selftest/sstate: Move common code to base class
Move the other common shares test functions to the base class to
improve the code structure.

(From OE-Core rev: de3e6f85c5537a3571ffbe2326b73f2c2526bce2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-13 17:35:22 +00:00
Richard Purdie
a4c484b8cb oeqa/selftest/sstate: Merge sstate test class with tests themselves
Having this base class as a separate file is just confusing. Merge with
the rest of the test code.

(From OE-Core rev: 977522a3b063225e22e2fd04b8265a4595606db2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-13 17:35:22 +00:00
Richard Purdie
4eb546cf1c selftest/recipetool: Stop test corrupting tinfoil class
Running recipetool.RecipetoolTests.test_recipetool_handle_license_vars
followed by wic.Wic2.test_biosplusefi_plugin_qemu would show a failure of:

  File "/media/build/poky/meta/lib/oeqa/utils/commands.py", line 351, in runqemu
    qemu = oeqa.targetcontrol.QemuTarget(recipedata, targetlogger, image_fstype)
  File "/media/build/poky/meta/lib/oeqa/targetcontrol.py", line 116, in __init__
    use_kvm = oe.types.qemu_use_kvm(d.getVar('QEMU_USE_KVM'), d.getVar('TARGET_ARCH'))
  File "/media/build/poky/meta/lib/oe/types.py", line 182, in qemu_use_kvm
    if kvm and boolean(kvm):
  File "/media/build/poky/meta/lib/oe/types.py", line 128, in boolean
    raise ValueError("Invalid boolean value '%s'" % value)
ValueError: Invalid boolean value '/media/build/poky/meta/files/common-licenses'

which made no sense until you realise the recipetool test is corrupting
the tinfoil class. Work on a copy instead to avoid this.

(From OE-Core rev: 7fe76fe17f67c1bbd108d02836692fed20d24771)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-13 17:35:22 +00:00
Alexis Lothoré
c31c140746 oeqa/selftest/resulttool: add test for error propagation in test name filtering
(From OE-Core rev: 17d825c88499c6f7e3ffe22fae099871d0850abc)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-12 23:39:13 +00:00
Alexis Lothoré
754d270ab8 oeqa/selftest/resulttool: fix fake data used for testing
resulttool searches for "status" field, not "STATUS", in results.
This fix is more to avoid confusion than fixing anything, since the updated
tests are about regression.can_be_compared, which does not check for "status"

(From OE-Core rev: df342c81d02dea3f677966602089d4a972bb09bd)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-12 23:39:13 +00:00
Alexis Lothoré
e3b005d8eb oeqa/selftest/resulttool: fix ptest filtering tests
There are multiple issues with the tests for the ptest fixup mechanism
introduced in c45d58f003:
- the feature does not impact regression.can_be_compared but directly the
  comparison step in regression.regression_common
- the "status" field was wrong ("STATUS"), which prevents tests comparison
- there is a typo babeltrace2 ptest result name

(From OE-Core rev: 69205e2e2117fc469fcf3ae4b2174482db1d6297)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-12 23:39:12 +00:00
Alexander Kanavin
03e900a748 selftest/overlayfs: enable systemd via INIT_MANAGER
Latest systemd version needs the full set of settings enabled
by that (if also r/o rootfs is enabled), and refuses to boot
otherwise.

(From OE-Core rev: 16f7661000a50e58d512a5d47794c5f8be2c49e6)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-11 00:08:40 +00:00
Michael Halstead
45d0ab66fd selftest/runtime_test/virgl: Disable for all Rocky Linux
RHEL compatible kernels do not support vgem so we disable virgl headless
testing.

(From OE-Core rev: 78bc1ba2361249ec845d1b9e28d429a37dd83910)

Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-09 23:51:29 +00:00
Ross Burton
f3c5db2465 oeqa/selftest/imagefeatures: add test for man-db
Add a test to verify that manpages are built with api-documentation
enabled, and apropos works on the target (so mandb has been ran).

(From OE-Core rev: c085c335319bb98e09177c09feb1241a8560b606)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-08 16:41:55 +00:00
Tom Hochstein
011cc9a104 oeqa/sdk: Improve Meson test
The meson wrapper setup command detection is broken in the case of an
implicit setup command with an option with a space-separated argument,
but the test was not detecting it since the case was not covered.

Add the option `--warnlevel 1` to the meson command line to cover this
case.

(From OE-Core rev: 54e9ee8a0c6c9fc89cbb743f0e4fc18607d503cf)

Signed-off-by: Tom Hochstein <tom.hochstein@nxp.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-06 09:52:16 +00:00
Alexis Lothoré
9b8665bb96 oeqa/selftest/resulttool: add test for metadata filtering on regression
Multiple filters have been added to resulttool to ensure that test
results are compared only to relevant older tests results

Add some unit tests about added filters:
- tests should only be compared when machine matches
- tests run as oeselftest should only be compared when
  OESELFTEST_METADATA matches
- many ptests should match despite non-static names
- runtime tests should match with matching tests regarding ltp tests
  content

(From OE-Core rev: b84302ef56f2516742a496aef43b89d4c3decd37)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-28 07:53:54 +00:00
Alexander Kanavin
5a5ab7cdec scripts/runqemu: move render nodes check to runqemu from selftest
This will produce a more useful hint for those setting
up or testing virgl headless:

runqemu - ERROR - No render nodes found in /dev/dri: ['by-path', 'card0']. If /dev/dri/renderD* is absent due to lack of suitable GPU, 'modprobe vgem' will create one suitable for mesa llvmpipe software renderer.

as qemu itself isn't helpful:

alex@Zen2:/srv/storage/alex/yocto/build-64-alt$ qemu-system-x86_64 -display egl-headless
qemu-system-x86_64: egl: no drm render node available
qemu-system-x86_64: egl: render node init failed

(From OE-Core rev: cbbada6a6c9b0a2e97f7395117dad986555f2db9)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-27 13:27:37 +00:00
Alexander Kanavin
a87811f255 selftest: do not check for dri.pc in the headless test
runqemu (which the test also runs) already performs this check,
and does it better, as it additionally unsets a number of environment
variables that can confuse and misdirect host's pkg-config.

(From OE-Core rev: ad179728a6978d929e2a876abe7e1cbd6b76a502)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-27 13:27:37 +00:00
Alexis Lothoré
ffaee6bd48 oeqa/selftest: add test for yocto_testresults_query.py
Add some tests for new yocto_testresults_query.py helper. First test is taken
from yocto-autobuilder-helper feature which has moved in yocto_testresults_query

(From OE-Core rev: 9ce28f685e4282d81f179877cbafd0a52fa887bd)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Alexis Lothoré
e557cd92d6 scripts/oe-selftest: append metadata to tests results
Many stored results TEST_TYPE are set to "oeselftest", however all those
tests are not run with the same sets of parameters, so those tests results may
not be comparable.

Attach relevant parameters as tests metadata to allow identifying tests
configuration so we can compare tests only when they are run with the same
parameters.

(From OE-Core rev: 406ec0a87e80d1ee0dd1d246adfe9507c10450ff)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Pavel Zhukov
2e18bbb307 oeqa/selftest/wic: Add test for uefi-kernel loader
This is regression test for [1].
[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15033

(From OE-Core rev: d8ef15bb9ef652bf85002fe523ff5060b47bca6a)

Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:49:00 +00:00
Richard Purdie
0ab3325ffb oeqa/selftest/prservice: Improve debug output for failure
We keep seeing this failure on the autobuilder but the output amounts
to "False is not True". Improve the debug message on the chance it may
make the issue clearer.

(From OE-Core rev: d03f4cf19c2cc96e9d942252a451521dfec42ebc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-24 15:38:06 +00:00
Pedro Baptista
803e781e42 create.py: add command arg to add layer to bblayers.conf
Add command arg `--add-layer` which enables the create
and add layer in a single step.

(From OE-Core rev: f3be788a55a2dde1f627aa85b08dc3ffa55d751f)

Signed-off-by: Pedro Baptista <pedro.miguel.baptista@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-24 13:31:45 +00:00
Mikko Rapeli
c7b05d5657 oeqa ssh.py: fix hangs in run()
When qemu machine hangs, the ssh commands done by tests
are not timing out. do_testimage() task has last logs like this:

DEBUG: time: 1673531086.3155053, endtime: 1673531686.315502

The test process is stuck for hours, or for ever if the
executing command or test case did not set a timeout correctly.
The default 300 second timeout is not working when target hangs.
Note that timeout is really a "inactive timeout" since data returned
by the process will reset the timeout.

Make the process stdout non-blocking so read() will always return
right away using os.set_blocking() available in python 3.5 and later.

Then change from python codec reader to plain read() and make
the ssh subprocess stdout non-blocking. Even with select()
making sure the file had input to be read, the codec reader was
trying to find more stuff and blocking for ever when process hangs.

While at it, add a small timeout to read data in larger chunks if
possible. This avoids reading data one or few characters at a time
and makes the debug logs more readable.

close() the stdout file in all cases after read loop is complete.

Then make sure to wait or kill the ssh subprocess in all cases.
Just reading the output stream and receiving EOF there does not mean
that the process exited, and wait() needs a timeout if the process
is hanging. In the end kill the process and return the return value
and captured output utf-8 encoded, just like before these changes.

This fixes ssh run() related deadlocks when a qemu target hangs
completely.

(From OE-Core rev: 9c63970fce3a3d6029745252a6ec2bf9b9da862d)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-23 12:11:29 +00:00
Martin Jansa
149ea9db96 insane.bbclass: move Upstream-Status logic to oe.qa
* to be used by standalone script scripts/contrib/patchreview.py
  as well

(From OE-Core rev: c326efeec8f576200728a44c694becdeab4fe2db)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-19 07:47:53 +00:00
Joshua Watt
ceb95cf9c2 classes/create-spdx-2.2: Report downloads as separate packages
Moves the downloaded items from SRC_URI into separate packages in the
recipe document. This is much better than the previous implementation
because:
 1) It can report multiple download locations in SRC_URI, instead of
    just the first one reported.
 2) It prevents the assumption that the source files listed in the
    recipe are the exact file from the source URL; in particular, files
    that come from file:// SRC_URI entries, and source files that have
    been patched were problematic, since these aren't from the upstream
    source.
 3) It allows the checksums to be specified

(From OE-Core rev: 1dd4369b3638637a2cbba2a3c37c6b6f4df335cd)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-19 07:47:53 +00:00
Alexis Lothoré
abf2776641 oeqa/selftest/resulttooltests: fix minor typo
(From OE-Core rev: 80cfa56d133bd3abbb1f37272607d8e15ce70861)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-17 18:01:01 +00:00
Mateusz Marciniec
c9273b8f2f sstatesig: Improve output hash calculation
Symbolic links to the files are included during the output hash
calculation but symlinks to the directories are missed.
So if the new symlink to a directory was the only change made,
then the output hash won't change,
and the Hash Equivalence server may change unihash.
In the next run bitbake may use an older package from sstate-cache.

To fix this followlinks=True flag could be set for os.walk
but it can lead to infinite recursion if link points
to a parent directory of itself.
Also, all files from a directory to which symlink points
would be included in depsig file.
Therefore another solution was applied, I added code that will loop
through directories and process those that are symlinks.

(From OE-Core rev: ee729163f31f26b1462a47e1e53f7a0f9de9b464)

Signed-off-by: Mateusz Marciniec <mateuszmar2@gmail.com>
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-15 10:21:34 +00:00
Alexander Kanavin
314944eaab bblayers/makesetup: skip git repos that are submodules
(From OE-Core rev: d8bc9cd4ca8ae268a61024f8ac5083a2bbdc432f)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-15 10:21:34 +00:00
Adrian Freihofer
ac60e31af3 bblayers/setupwriters/oe-setup-layers: create dir if not exists
Without this patch:

$ bitbake-layers create-layers-setup /home/adrian/temp/poky-clone
NOTE: Starting bitbake server...
Traceback (most recent call last):
  File "/home/adrian/projects/poky/bitbake/lib/bb/process.py", line 169, in run
    pipe = Popen(cmd, **options)
  File "/home/adrian/projects/poky/bitbake/lib/bb/process.py", line 73, in __init__
    subprocess.Popen.__init__(self, *args, **options)
  File "/usr/lib64/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib64/python3.10/subprocess.py", line 1847, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/adrian/temp/poky-clone'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/adrian/projects/poky/bitbake/bin/bitbake-layers", line 95, in <module>
    ret = main()
  File "/home/adrian/projects/poky/bitbake/bin/bitbake-layers", line 88, in main
    return args.func(args)
  File "/home/adrian/projects/poky/meta/lib/bblayers/makesetup.py", line 90, in do_make_setup
    p.do_write(self, args)
  File "/home/adrian/projects/poky/meta/lib/bblayers/setupwriters/oe-setup-layers.py", line 36, in do_write
    repos = parent.make_repo_config(args.destdir, args.include_layer_repo)
  File "/home/adrian/projects/poky/meta/lib/bblayers/makesetup.py", line 55, in make_repo_config
    destdir_repo = self._get_repo_path(destdir)
  File "/home/adrian/projects/poky/meta/lib/bblayers/makesetup.py", line 30, in _get_repo_path
    repo_path, _ = bb.process.run('git rev-parse --show-toplevel', cwd=layer_path)
  File "/home/adrian/projects/poky/bitbake/lib/bb/process.py", line 172, in run
    raise NotFoundError(cmd)
bb.process.NotFoundError: Execution of 'git rev-parse --show-toplevel' failed: command not found

with this patch:

$ bitbake-layers create-layers-setup /home/adrian/temp/poky-clone
NOTE: Starting bitbake server...
NOTE: Created /home/adrian/temp/poky-clone/setup-layers.json
NOTE: Created /home/adrian/temp/poky-clone/setup-layers

(From OE-Core rev: 2da12ccada46443d58dd8fab463156fa763b84cc)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-15 10:21:34 +00:00
Mikko Rapeli
9437c73905 oeqa qemurunner.py: try to avoid reading one character at a time
Read from serial console with a small delay to bundle data to e.g.
full lines. Reading one character at a time is not needed and causes
busy looping.

(From OE-Core rev: 0049f6757f6f956fb4cc77b3df6a672c20b53cf4)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-10 09:25:44 +00:00
Mikko Rapeli
18bcccc8f8 oeqa qemurunner.py: add timeout to QMP calls
When a qemu machine hangs, the QMP calls can hang for ever
too, and when this happens any failing test commands from ssh
runner may be followed by dump_monitor() calls which
then also hang. Hangs followed by hangs.

Use runqemutime at setup and run_monitor() specific timeout
for later calls.

(From OE-Core rev: 3a07bdf77dc6ecbf4c620b051dd032abaaf1e4ff)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-10 09:25:44 +00:00
Mikko Rapeli
d61df4b220 oeqa qemurunner: read more data at a time from serial
Use a short sleep to bundle serial console reads so that
we are not reading one character at a time which reduces busy
looping.

(From OE-Core rev: cafe65d8cf7544edbd387f7f5f6d77c64c6b18fa)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-10 09:25:44 +00:00
Mikko Rapeli
49da864246 oeqa dump.py: add error counter and stop after 5 failures
If test target qemu machine hangs completely, dump_target() calls
over serial console are taking a long time to time out, possibly
for every failing ssh command execution and a lot of test cases,
and same with dump_monitor().

Instead of trying for ever, count errors and after 5 stop trying
to dump_target() and dump_monitor() completely.

These help to end testing earlier when a test target is completely
deadlocked and all ssh, serial and QMP communication with it are
failing.

(From OE-Core rev: d9ad0a055abba983c6cee1dca4d2f0a8a3c48782)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-10 09:25:44 +00:00
Mikko Rapeli
a0ccfba547 oeqa ssh.py: add connection keep alive options to ssh client
Configure ssh client to test that connection with server is up.
If the server does not respond within a minute then the connection,
target machine or sshd daemon are stuck and it's better to exit
the command execution with errors.

Some tests can execute a long time without returning stdout/stderror
data and it's difficult to adjust timers for those cases if
connection to target machine or the target machine itself hangs
and output is not expected in minutes or even hours.

(From OE-Core rev: ba68ff04c5786eca7cd8dd44056705867dea8ac4)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-10 09:25:44 +00:00
Mikko Rapeli
487cb16119 oeqa ssh.py: move output prints to new line
The output from is garbled otherwise and it's not
easy to remove debug output form real command output on target.

(From OE-Core rev: 917a70cbc43ac1c70c477b220c4115735457ef04)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-10 09:25:44 +00:00
Mikko Rapeli
75b70b6257 oeqa context.py: fix --target-ip comment to include ssh port number
Providing ssh port number is supported too with
"--target-ip 192.168.0.10:22".

(From OE-Core rev: 637919b9df0abc06da5b2f9b389cf25376bd6b7c)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-08 10:29:01 +00:00
Richard Purdie
8e035db1e5 oeqa/selftest/locales: Add test for disabled binary locale generation
Similarly to the recently added test for binary generated locales, add
a version to test on target locale generation. This was broken but should
be fixed now so we can add the test sharing code from the previous test.

(From OE-Core rev: 50a29c167eb9fe9fa96aa53a379ae7597cefd1cc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-04 17:02:10 +00:00
Xiaotian Wu
324b03e15f base: add support for loongarch64
(From OE-Core rev: 759baaceb4dd623d5da12ba0d01540fa080154ba)

Signed-off-by: Xiaotian Wu <wuxiaotian@loongson.cn>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-04 17:02:10 +00:00
Louis Rannou
912977a373 oeqa/selftest/locales: Add selftest for locale generation/presence
[YOCTO #9070]

Add a new selftest to validate locale generation. This selftest builds a
complete target with GLIBC_GENERATE_LOCALES, IMAGE_LINGUAS,
ENABLE_BINARY_LOCALE_GENERATION set.

(From OE-Core rev: 53258fd810bea6475af9f908f7b712a13a02b628)

Signed-off-by: Louis Rannou <lrannou@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-02 09:52:52 +00:00
Martin Jansa
94149c4f37 meta: remove True option to getVar and getVarFlag calls (again)
* True is default since 2016 and most layers were already updated
  not to pass this parameter where not necessary, e.g. oe-core was
  updated couple times, first in:
  https://git.openembedded.org/openembedded-core/commit/?id=7c552996597faaee2fbee185b250c0ee30ea3b5f

  Updated with the same regexp as later oe-core update:
  https://git.openembedded.org/openembedded-core/commit/?id=9f551d588693328e4d99d33be94f26684eafcaba

  with small modification to replace not only d.getVar, but also data.getVar as in e.g.:
  e.data.getVar('ERR_REPORT_USERNAME', True)

  and for getVarFlag:
  sed -e 's|\(d\.getVarFlag \?\)( \?\([^,()]*, \?[^,()]*\), \?True)|\1(\2)|g' \
      -i $(git grep -E 'getVarFlag ?\( ?([^,()]*), ?([^,()]*), ?True\)' \
          | cut -d':' -f1 \
          | sort -u)

(From OE-Core rev: 26c74fd10614582e177437608908eb43688ab510)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-02 09:50:02 +00:00
Ross Burton
c24a83ed4a httpserver: add error handler that write to the logger
(From OE-Core rev: a4bcca3123685f410fc99e5cc23f2b8f39fd0a63)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-01 14:05:15 +00:00
Ross Burton
f6a1c8c309 oeqa/utils/httpserver: connect up the request logging
Call logger.info() in the log_message handler so that we get request
logging, and hopefully even error messages.

Create a child logger to be neat and compartmentalise the logging.

Add a __main__ entrypoint so this class can be exercised outside of oeqa.

Remove unused traceback import.

(From OE-Core rev: 8fc939d942cec1662ca386b87c63f167bb10fd05)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-01 14:05:15 +00:00
Ross Burton
d8ec2bbc77 sdkext/cases/devtool: pass a logger to HTTPService
Pass our logger to the HTTPService instance so we can see the requests
and any errors.

(From OE-Core rev: 3485df2ff61143aac03d92300b7bac4e5d6b2427)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-01 14:05:15 +00:00