Commit Graph

36703 Commits

Author SHA1 Message Date
Mingli Yu
cc4ce7db19 mdadm: improve the run-ptest
* There are 120+ cases under ${libdir}/mdadm/ptest/tests,
  but the test will break if one test fails as
  below logic in run-ptest.
  ./test &>./test.log

  That's to say, the tests after the failed test
  have no chance to run with the current logic.

  To guarantee all the tests can run even one
  of the tests fails, the option --keep-going
  should be added.

* Refactor the test report to make the report
  more detailed and more common

(From OE-Core rev: 80d17497b719efb2ca9f36b8a730815547e93aa7)

(From OE-Core rev: 1b84064b15022a0834d7f443a8d6bde7f4cfeb6d)

Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Peter Kjellerstedt
f48d96e4df meson: Correct use of the _append operator
The value to SRC_URI_append_class-native was not prefixed with a space.
This was not noticed as the SRC_URI before applying the _append contains
trailing spaces. However, if one, e.g., has a .bbappend and adds to the
SRC_URI using SRC_URI += "file://foo.patch", then there no longer is any
trailing space and the _append concatenates the two URIs together,
leading to a build failue.

(From OE-Core rev: c07ee11e99dfe28405a7225903a541b33aeb1de6)

(From OE-Core rev: 53594c9fd01f7bd02e24aa82a5bd3a14d3dd465e)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Richard Purdie
b55b254f4e meson: Disable rpath stripping at install time
As discussed in https://github.com/mesonbuild/meson/issues/2567 there
needs to be a way to allow our rpath options passed to the linker to be
preserved, else we run into weird build failures.

(e.g. libmodulemd-native used by libdnf can't find libyaml)

Disable this for now until upstream come up with a better way of handling
this.

(From OE-Core rev: b4e36281631e0b59d1058f5cf391eb8b15e605cf)

(From OE-Core rev: 98a76c3171f9080d6246aac0188ef05c40852adc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Alexander Kanavin
1cfa86d80f meson: do not manipulate the environment when looking for python via pkg-config
meson does it in a way that breaks oe builds (they export a bunch of PKG_CONFIG_ variables)

(From OE-Core rev: f071c5eb0a46b8ac5424c5baeb471a8080d4a078)

(From OE-Core rev: d5f1211b0820d5039ecec932fda92cbe1ba9132e)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Changhyeok Bae
b4e664db71 ethtool: 4.17 -> 4.19
(From OE-Core rev: 565bbbf43da14de466fccdfaa259bdb9b50b686e)

(From OE-Core rev: 4bcacd05428498158ae2fedc11bf4d6f11824967)

Signed-off-by: Changhyeok Bae <changhyeok.bae@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Changhyeok Bae
34a2953a95 iproute2: 4.18.0 -> 4.19.0
(From OE-Core rev: b5acefc041b2316c75eefae745d894412ac7bd78)

(From OE-Core rev: 875ebdcaf479b38f7564d68f1530de08e50f8ba8)

Signed-off-by: Changhyeok Bae <changhyeok.bae@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Robert Yang
90e0658f2e sstate.bbclass: Only remove sstate file when task is existed
This can improve the performance a lot for "bitbake <recipe-native/cross/crosssdk>
-ccleansstate" when there are a lot of sstate files.

For example:
* Before
  $ bitbake quilt-native -ccleansstate
  - Check log.do_cleansstate:
  Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*
  Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package_qa.tgz*
  Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package_write_rpm.tgz*
  Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_packagedata.tgz*
  Removing /sstate-cache/*/sstate:quilt-native::0.65:r0::3:*_populate_lic.tgz*
  Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*

  There are no package tasks for quilt-native, so the first 4 lines doesn't
  make any sense, but the glob pattern "sstate-cache/*/*" is very time
  consuming when there are no disk caches. E.g., I have more than 600,000
  sstate files:
  - Without disk caches
  # echo 3 >/proc/sys/vm/drop_caches
  $ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*")'
    real    4m32.583s
    user    0m5.768s
    sys     0m12.892s

  - With disk caches (e.g., run it in the second time)
  $ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*")'
    real    0m5.128s
    user    0m2.772s
    sys     0m2.308s

  So the 4 removing *package* commands cost more than 20s or 272s in theory.

* After
  $ bitbake quilt-native -ccleansstate
  - Check log.do_cleansstate:
  Removing /sstate-cache/*/sstate:quilt-native::0.65:r0::3:*_populate_lic.tgz*
  Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*

  We can see that it saved 20s or 272s in theory.

(From OE-Core rev: bb2d6349ea87f090c58001f0d4348b24c2982cde)

(From OE-Core rev: aa35fbeb995b62523d44c8fad17f67d9852c594f)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Robert Yang
037a69934c bugzilla.bbclass: Remove it since obsoleted
It is a still python2 bbclass, so it has been broken since bitbake changed to
python3 which was 2 years ago. No one reported/fixed it for python3 in recent 2
years. So we can assume that no one uses it anymore.

(From OE-Core rev: 7f6da5fb54cbcf8e358e988382f45839a8b80019)

(From OE-Core rev: 7264c248946c09b139110b512dd12ce6d8d72ddc)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Hongxu Jia
26750a75dc libgcrypt: upgrade 1.8.3 -> 1.8.4
(From OE-Core rev: 1100e7f1519be91c90b139c337799c7ea635a8b3)

(From OE-Core rev: 6187f5eff556d5f308fa6812dc5335a2769cb249)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Hongxu Jia
ce93ac6395 gnupg: upgrade 2.2.10 -> 2.2.11
(From OE-Core rev: 4a373ce7d718ee3299bcf7f9fa62e7337d41e40a)

(From OE-Core rev: 65134404a6572f126e159503a079d5b8d3e1c1d5)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Hongxu Jia
98d689400d gnupg: upgrade 2.2.9 -> 2.2.10
(From OE-Core rev: 5efe9eb79ac325f55fc52f67b522afaf7ebb847a)

(From OE-Core rev: 5c8a8465d24b33f53a7e3bc4dc1d0dfbb14685d6)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Armin Kuster
6b673012d9 gnutls: update to 3.6.4
Notable change:

libgnutls: Added the final (RFC8446) version numbering of the TLS1.3 protocol.
see: https://lists.gnupg.org/pipermail/gnutls-help/2018-September/004457.html

(From OE-Core rev: 0697141e7be0b755db600aa0d5a975eac62cc7b8)

(From OE-Core rev: 7c062c9d2c48cd758b3ca9a4c7a5b26d74b9c1e3)

Signed-off-by: Armin Kuster <akuster808@gmail.com>

--
[v2]
Fix typo in version in subject

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Hongxu Jia
d50041ea78 elfutils: 0.174 -> 0.175
- Drop backport CVE patches
  0001-libdwfl-Sanity-check-partial-core-file-data-reads.patch
  0001-size-Handle-recursive-ELF-ar-files.patch
  0001-arlib-Check-that-sh_entsize-isn-t-zero.patch

- Drop patches that upstream has fixed
  0005-fix-a-stack-usage-warning.patch [9a74c19 backends: ppc use define
  instead of const for size of dwarf_regs array.]

- Update debian patches to 0.175

- Rebase local patch to 0.175
  0008-build-Provide-alternatives-for-glibc-assumptions-hel.patch

(From OE-Core rev: 8748de4df5a4ece303f07f8bbb248920a199478a)

(From OE-Core rev: 81ae67e603087166ec5583cc9686a60f769be799)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Robert Yang
38887e6c1a recipes: Remove tab indentations in python code
Use 4 spaces to replace a tab.

(From OE-Core rev: cbb6743d46752481782789fa1a0dfade11057114)

(From OE-Core rev: 42ebdc7253c3a319d671a7f924603d85a22bbb4e)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Robert Yang
bc246384a1 classes: Remove tab indentations in python code
Use 4 spaces to replace a tab.

(From OE-Core rev: 55eaf8779170b9396e94dc4a44667824c4f36363)

(From OE-Core rev: dadd66706515ada3fab50779bfb75be8e776c975)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Douglas Royds
1dc65214c8 ptest: Reproducibility: Take control of umask
The build host umask was leaking into the thing-ptest packages
at do_install_ptest() time.

(From OE-Core rev: 891343e8ba6490ca3e1876c892269b611ddc7877)

(From OE-Core rev: 6cbc54790d9a0784fb0df4772aa38392ddea682b)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Douglas Royds
c90a56ed6d reproducible: Don't look for youngest file when no source tarball
Some packages (eg. init-ifupdown) take their source files entirely from
openembedded-core, that is, they download no source tarball.
These recipes either don't use S at all (ie. it is empty at unpack time),
or they set S = WORKDIR (as in init-ifupdown).
Looking at the file timestamps in the WORKDIR causes a non-reproducible
SOURCE_DATE_EPOCH, as files taken from file:// URIs do not have
reproducible timestamps.

If S == WORKDIR, we are better to assume that there is no source tarball,
and to fall back to a fixed timestamp for the SOURCE_DATE_EPOCH.
This makes the init-ifupdown build reproducible.

(From OE-Core rev: d395bad0179037eb5d0fa4d921985c87ae13f3a4)

(From OE-Core rev: cd56795a1588d780ca6a0cb974bf4024ab636be7)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Douglas Royds
e2a2f8cd89 reproducible: Refactor: Break out fixed_source_date_epoch() function
(From OE-Core rev: 4eb6def4fe82959c2a348142b9eada27d3354aef)

(From OE-Core rev: 98b7d22ed563efdf58beba1ba65270b731673103)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Douglas Royds
d513b1d6bb boost-context: Reproducibility: Set .file section for all *_elf_gas.S files
Add a .file directive explicitly for all *_elf_gas.S files to prevent the linker
adding a host build-system path as a FILE symbol to the object file.

This replaces the existing patch that added the .file directive to a small
subset of these files.

Upstream-Status: Submitted [https://github.com/boostorg/context/issues/91]
(From OE-Core rev: 5ff5f89f2db079a6baf0275ebf1333b4b9642504)

(From OE-Core rev: 763398b87c257ab672430bedd01004b9317d02b2)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Niko Mauno
da35b05c7f opkg-utils: Fix update-alternatives link relocation
Recently Debian-style support for link relocation was added to
'update-alternatives' script, but it fails under circumstances where
host rootfs root directory differs from target rootfs root directory
and two alternative packages provide a symbolic link with source
located in different directories.

An example of the case is busybox provided /bin/rev (symlinking to
/bin/busybox.nosuid) and util-linux provided /usr/bin/rev (symlinking
to /usr/bin/rev.util-linux) in which case following failure occurs
during image recipe's do_rootfs() task:

  ERROR: core-image-minimal-1.0-r0 do_rootfs: Postinstall scriptlets of ['util-linux'] have failed. If the intention is to defer them to first boot,
  then please place them into pkg_postinst_ontarget_${PN} ().
  Deferring to first boot via 'exit 1' is no longer supported.
  Details of the failure are in .../tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_rootfs.
  ERROR: core-image-minimal-1.0-r0 do_rootfs: Function failed: do_rootfs

Looking in log.do_rootfs file, following relevant lines can be observed:

  update-alternatives: renaming rev link from /bin/rev to /usr/bin/rev
  mv: cannot stat '/bin/rev': No such file or directory

Mitigate issue by applying patch which adds target root filesystem root
directory path prefix to failing 'mv' calls relevant variable references

(From OE-Core rev: f0912e23629758fe4303284e7db8f4089bb7b4cb)

(From OE-Core rev: 58f062843008c42d28f14c42fb5f991aef73728e)

Signed-off-by: Niko Mauno <niko.mauno@iki.fi>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Joshua Watt
7e27a9c351 meta/icecc.bbclass: Update system blacklists
Updates the system blacklists to include packages that are known to have
problems compiling under icecream

(From OE-Core rev: fc5418e7bbdecfb27bafe595084e0fd0f991a388)

(From OE-Core rev: be54e1e0e769a9833b9b595e7a820ea9e098b91d)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:42 +00:00
Joshua Watt
f85b494630 meta/icecc.bbclass: Move system blacklist to variables
The system blacklists are moved to variables which are ignore when
hashing. This prevents changes to the blacklists from causing all
taskhashes to change (and thus rebuild).

(From OE-Core rev: f5be9f6e9180ace3362bba52c7ced3b039441d7d)

(From OE-Core rev: 6f001b7b2ec3a6fb77184ca32664f89fee7ff5c3)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:41 +00:00
Paul Eggleton
c56383ce65 socat: fix LICENSE
According to both the README and source headers, the LICENSE value for
socat is explicitly GPLv2, not v2 or later, so adjust LICENSE
accordingly (leaving aside whether "GPL-2.0+-with-OpenSSL-exception"
should actually be considered a valid LICENSE string or not).

(From OE-Core rev: 466044a341a8b42159bd9388950c9079e0d7a2c3)

(From OE-Core rev: 8d5565274545628cb0a7125b660e94f763cc7f49)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 20:14:41 +00:00
Richard Purdie
074cd4f22c oeqa/selftest/esdk: Fix typo causing test failure
2018-12-06 23:19:24,564 - oe-selftest - INFO - Traceback (most recent call last):
  File "/media/build1/poky-sumo/meta/lib/oeqa/core/case.py", line 32, in _oeSetUpClass
    clss.setUpClassMethod()
  File "/media/build1/poky-sumo/meta/lib/oeqa/selftest/cases/eSDK.py", line 78, in setUpClass
    cls.tmpdir_eSDKQA = cls.tempdirobj.name
AttributeError: type object 'oeSDKExtSelfTest' has no attribute 'tempdirobj'

(From OE-Core rev: 75cd4edaa8a42f76c0594ce26df05c7a51d620df)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
2cafee8620 oeqa/selftest/esdk: Ensure parent directory exists
INFO - ======================================================================
INFO - ERROR: setUpClass (eSDK.oeSDKExtSelfTest)
INFO - ----------------------------------------------------------------------
INFO - Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/core/case.py", line 32, in _oeSetUpClass
    clss.setUpClassMethod()
  File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/selftest/cases/eSDK.py", line 76, in setUpClass
    cls.tmpdirobj = tempfile.TemporaryDirectory(prefix="selftest-esdk-", dir=bb_vars["WORKDIR"])
  File "/usr/lib/python3.5/tempfile.py", line 929, in __init__
    self.name = mkdtemp(suffix, prefix, dir)
  File "/usr/lib/python3.5/tempfile.py", line 507, in mkdtemp
    _os.mkdir(file, 0o700)
FileNotFoundError: [Errno 2] No such file or directory: '/home/pokybuild/yocto-worker/oe-selftest-debian/build/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/selftest-esdk-q7ln84gc'

(From OE-Core rev: eca3c0a6aa4bce48f295ba25f613da8dcaefac20)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
237ff3e462 testimage/testsdk/selftest: Avoid platform.distro_identifier deprecation warnings
Use our own lsb function instead as used elsewhere by the codebase.

(From OE-Core rev: acac45a6fd604d28ef7c23d67482af3d7e8bcfe3)

(From OE-Core rev: 570256a64af5a3fa994a20a5cc4c74d59ffc361f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
ea1ffae258 oeqa/concurrencytest: Avoid unclosed file warnings
Avoid an unclosed file per thread warning when running selftests concurrently
by closing the result stream.

(From OE-Core rev: 33a4a076e8aa72a872807332501e7f5ae1cee0e2)

(From OE-Core rev: a7dceca55b169bcdb8d1528238cbdedfd131f37f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
19287294c1 oeqa/utils/qemurunner.py: Fix python regex warnings
Fix the warnings:

meta/lib/oeqa/utils/qemurunner.py:250: DeprecationWarning: invalid escape sequence \.
  ips = re.findall("((?:[0-9]{1,3}\.){3}[0-9]{1,3})", cmdline.split("ip=")[1])
meta/lib/oeqa/utils/qemurunner.py:343: DeprecationWarning: invalid escape sequence \-
  if re.search("root@[a-zA-Z0-9\-]+:~#", output):
poky/meta/lib/oeqa/utils/qemurunner.py:350: DeprecationWarning: invalid escape sequence \-
  if re.search("root@[a-zA-Z0-9\-]+:~#", output):
meta/lib/oeqa/utils/qemurunner.py:448: DeprecationWarning: invalid escape sequence \-
  if re.search("[a-zA-Z0-9]+@[a-zA-Z0-9\-]+:~#", data):

by correctly marking the regexs.

(From OE-Core rev: 8e6987735002560fca714f77ea8ece9d4b28f7fa)

(From OE-Core rev: a980cb8a0940d4db4bb5d338650cf848cd292f5b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
0a5b7850e5 oeqa/selftest/context: Replace deprecated imp module usage
Avoid the warning:

meta/lib/oeqa/selftest/context.py:8: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp

In this case importlib is a direct replacement.

(From OE-Core rev: db7a60c36a2d3eefc61ae6e1ede01680dc932035)

(From OE-Core rev: 1e7bbdaf78fd2a75d03c6a0c48afed13fffd4397)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
01d00cd776 oeqa/utils/commands: Avoid unclosed file warnings
Avoid warnings such as:

meta/lib/oeqa/utils/commands.py:213: ResourceWarning: unclosed file <_io.BufferedReader name=4>
  return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options)

(From OE-Core rev: 6a68c42de08cffbadb59ebda63fa5e19f6e5acef)

(From OE-Core rev: f65a5fbd4fd13a52b54c808a6f5d2afab426e050)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
f349ec0424 oeqa/core/runner: Correctly markup regexs
Avoid the warning "DeprecationWarning: invalid escape sequence \(" by marking
the regexs correctly.

(From OE-Core rev: cb49980fa4a158d5529902df731dec61a8c9b3d4)

(From OE-Core rev: bb2cddb08d7c4c6c56dfe6b2f0d26dce1cdf20e4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
e09a79e997 oeqa/loader: Fix deprecation warning
Clean up the warning:
meta/lib/oeqa/core/loader.py:27: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
  _failed_test_args = inspect.getargspec(unittest.loader._make_failed_test).args

(From OE-Core rev: d2deb66830be2d44532fea3d5db763b57778252a)

(From OE-Core rev: e65e9492acad4861b22e6f29d5f470a82778f2b4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
86dbdfa00d oeqa/runner: Sort the test result output by result class
We want to see failures/errors listed last since this is the most easily
visible part of the log on consoles or autobuilder output and makes
human processing easier rather than having to scroll up and scan for
a single failure.

(From OE-Core rev: 7954b19020c28a4120bc1671aa81b9e1e2b05fa2)

(From OE-Core rev: 260738158b09aea0beeca85b778aa4ab08ba1c4c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
359ce89adf oeqa/runner: Always show a summary of success/fail/error/skip counts
Its useful to have the counts of success/failure/error/skipped at the end of the
results to allow for easier human reading of what happened.

(From OE-Core rev: 080d8900d470a8e7f929b0c5c2765ad461744fbb)

(From OE-Core rev: 5bc862d8f16df5611537cb51da95812e519c61f6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Ross Burton
510e6e2143 oeqa/selftest/esdk: run selftest inside workdir not /tmp
We've seen issues with rootfs size calculations and we've seen systems
like opensuse which have btrfs mounted on /tmp causing selftest failures.

(From OE-Core rev: 61be3cd748d1b7321a1fc4cfe84efa9b26a6aee0)

(From OE-Core rev: 63eefbe21612e58e88f1eb3fde3d314da53927fa)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Ross Burton
29361d7aeb oeqa: don't litter /tmp with temporary directories
If we need to create a temporary directory in targetbuild or buildproject use
tempfile.TemporaryDirectory so that when the test case is finished, the
directory is deleted.

Also synchronise the logic and don't possibly store the temporary directory in
self.tmpdir as nothing uses that.

(From OE-Core rev: db0e658097130d146752785d0d45f46a3e0bad71)

(From OE-Core rev: 6e2c6668791a80ee0ffe44c756cc0caceebea0e2)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
c7ff99ab44 oeqa/utils/qemurunner: Avoid tracebacks on closed files
Reorder the shutdown/teardown to avoid:

  File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/utils/qemurunner.py", line 224, in launch
    op = self.getOutput(output)
  File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/utils/qemurunner.py", line 90, in getOutput
    fl = fcntl.fcntl(o, fcntl.F_GETFL)
ValueError: I/O operation on closed file

(From OE-Core rev: 8e7d756862d2a8d62f3c87497d6d65ddb3c1b962)

(From OE-Core rev: 11d4bf460030eb6f072bd0e15550e26e055e632b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
d69080b9da oeqa/selftest/runqemu: Improve testcase failure handling
assertTrue doesn't give good debug information when things fail. Update
several to use assertIn which gives information upon failure, for the
others print the log information upon failure.

(From OE-Core rev: c29cb75d5ce6b0873a934f4709b0c8824f7164d3)

(From OE-Core rev: e9c6e974b49b8821cce8f2c7e3ba0dc16a5b46a0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
d6ba484506 lib/oe/utils: Improve multiprocess_lauch exception handling
We've seen a cryptic:

"ERROR: Fatal errors occurred in subprocesses, tracebacks printed above"

message from oe-selftest with no other traceback information. Improve the
traceback logging to try and give a better indication of any errors that is
ocurring.

(From OE-Core rev: 521dd3d00979a27b6932e58d5497de68abac26e1)

(From OE-Core rev: 2696e69af0b32e03692d8644cc01b28dcf221aa1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Ross Burton
72d7b5b5d4 oeqa/oelib/path: don't leak temporary directories
setUp() is used to populate a directory of temporary files, and deleted in
__del__.  However setUp() is called once *per test* so __del__ would only be
able to remove the last directory created.

Fix the code by using the natural counterpart to setUp, tearDown(), to clean up.

(From OE-Core rev: 68b4723e6fb11d171869185bccf28f32f6284c18)

(From OE-Core rev: eee30d799ff892443d21de4128e7d643215b0ac1)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
3c27f3b3a7 oeqa/selftest/context: Improve log file handling
The existing logfile is simply placed in the current directory. Since the test
changes cwd to BUILDDIR, the symlink to the log can be placed in an invalid
directory. We also see trackbacks if the symlink is invalid.

Improve things by:

* Placing logs in LOG_DIR (or BUILDDIR if unset).
* Using a full path to the log meaning the log and link are placed in the same directory.
* Using lexists instead of exists so invalid symlinks are handled correctly.

(From OE-Core rev: 750ece11bed0e62a11e0003d1d16a81f7c219761)

(From OE-Core rev: 1ec53b8d82491aeb9f49e7a78f531e98b5608f0f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
c414b06ea6 oeqa/selftest/buildoptions: Ensure diskmon tests run consistently
Heartbeat events default to once a second and we need to ensure we have
enough time in the task to see them.

Add a nostamp delay task 5s long so we can have a consistently timed
task which doesn't need cleanup or have unneeded dependencies. This
ensures we should deterministically see the disk moinitor events
regardless of the state of the build. This is done in a way which
doesn't corrupt build state or need cleanup and is efficient.

(From OE-Core rev: ecc49ee8986929e2429d948000a0ca588fe63959)

(From OE-Core rev: a883aa053ddeb4591109c7c1374525e63a59bd80)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:33 +00:00
Richard Purdie
d7594fd2ae oeqa/selftest/buildoptions: Improve ccache test
This test occisionally fails as m4 doesn't recompile, meaning the logfile test
then doesn't find mention of ccache.

To ensure m4 does recompile, clean m4 before force compiling it.

(Reading the test is confusing due to the test cleanup also involving a clean)

(From OE-Core rev: 6e0b9214a0d57ed45a5df0ba5c9887a9045b89b1)

(From OE-Core rev: bddb09a411ce69f49a37260e2188bbd9b02f5902)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:32 +00:00
Richard Purdie
5e12af7587 oeqa/qemurunner: Remove resource python warnings
If runqemu fails it would leak an unclosed socket and file. Ensure we
close these in all cases to remove the resource warning.

(From OE-Core rev: ed80e46ccbc8fe8e9148d80723152066fa00ba28)

(From OE-Core rev: 81ce67fb1274faafb9e7386c5982aeaf4c74294d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:32 +00:00
Richard Purdie
1cd8b3b789 oeqa/utils/commands: Avoid log message duplication
Each time a runqemu() fails, the log handler would be left behind meaning
messages from any subsequent run would be duplicated (or worse/more).

This ensures we remove the handler regardless and means we no longer
have the duplication.

(From OE-Core rev: 532984708436bdfa3a8cac2c684a425eb249bad0)

(From OE-Core rev: 018549bf94d1e5d33344691fde0fb27ca50f675f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:32 +00:00
Richard Purdie
ce291652c5 oeqa/utils/qemurunner: Fix python ResourceWarning for unclosed file
Fixes:

Stderr:
/media/build1/poky/meta/lib/oeqa/utils/qemurunner.py:381: ResourceWarning: unclosed file <_io.BufferedWriter name=16>
  self.runqemu = None

(From OE-Core rev: b9e0bf919e6fc1a58e02145a363ebe7066e5bf4f)

(From OE-Core rev: 43b7ee731d2cfc3fd4a4c26381c8d035b5fbcdeb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:32 +00:00
Richard Purdie
594679bb48 oeqa/utils/commands: Add extra qemu failure logging
Rather than just referring the user to the logs containing the failure, print
them on the console. This aids debugging with oe-selftest with parallelisation
as the logs may otherwise be lost.

(From OE-Core rev: 36a018e245a232f520ff946f152cc875927a6fb4)

(From OE-Core rev: 85b373f571cf2076d93e96db2aca295c53d3c16a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:32 +00:00
Richard Purdie
9cf4132158 oeqa/selftest/buildoptions: Improve ccache test failure output
The current failure mode doesn't show us what the logs actually looked like
and later cleans can lose them. Show the whole log in case of failure
to aid debugging intermittent problems on the autobuilder.

(From OE-Core rev: 7c3a0dc5978cea898b1ca51decf4d6e7cf9d519f)

(From OE-Core rev: 8a9fa1597245d13db89361c40db7867786f137ff)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:32 +00:00
Richard Purdie
835cf20001 oeqa/selftest/case: Use bb.utils.remove() instead of shutil.remove()
This avoids problems where shutil.remove will error with:

  File "/usr/lib/python3.5/shutil.py", line 436, in _rmtree_safe_fd
    os.unlink(name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'S.gpg-agent.extra'

when there are races over file deletion (gpg agent may be slow to exit).

We already worked around speed and race issues in bb.utils.

(From OE-Core rev: 00a8fd5b93a5c19ce0b7498e2bc653ce8ad58aaf)

(From OE-Core rev: ee8255128e11d69d82e6093b58b761dfe773e71d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:32 +00:00
Richard Purdie
588480ff19 oeqa/selftest/signing: Use do_populate_lic target instead of do_package
This should speed the test up signficiantly without any loss of functionality
for the purposes of the test.

(From OE-Core rev: 3dde0b749643575878bfbca2f8d2d9ec30bad166)

(From OE-Core rev: 9fcfc43e1a51520a86761cf5534cff9e70167b77)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-16 14:31:32 +00:00