Commit Graph

1365 Commits

Author SHA1 Message Date
Armin Kuster
e782da1fbd lib/oe/terminal.py: use an absolute path to execute oe-gnome-terminal-phonehome
A flaw was found on my Ubuntu 14.04.5 LTS, on which that gnome-terminal is
the default terminal, when I run any of the tasks:
bitbake busybox -c menuconfig/devshell/devpyshell
bitbake virtual/kernel -c menuconfig/devshell/devpyshell

I got a error as follows:
"Failed to execute child process "oe-gnome-terminal-phonehome" (No such file or directory)"

Seems the environment of the process calling Popen is not passed to the
child process, this behaviour is a known issue in Python bug tracker:
http://bugs.python.org/issue8557

It could be fixed by using an absolute path instead per test.

(From OE-Core rev: 5055bcabca542074cad00c92edf79f3a1551ac32)

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2117c148ef07d84bc605768e3b3671b0126b9337)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 15:13:48 +01:00
Ross Burton
fc0a773246 sdk: only install locales if we're using glibc
Using glibc-locale to install locales only makes sense if we're using glibc.

(From OE-Core rev: 8fc80734053645fa893694dfe33ddaee99aa9a1a)

(From OE-Core rev: 1ca33a798f5edf4bb1e695a79a46088dd23b6858)

(From OE-Core rev: 75918061572c0943aa4ea24df07b4ec1ae1aa09f)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[Fixup for the getVar True bit]
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 15:13:47 +01:00
Ross Burton
f1de1761b3 sdk: install specified locales into SDK
(From OE-Core rev: 9b1c3dbe79f67d3b46e0f90a73bce6c61f094a50)

(From OE-Core rev: f223f90d07965eef8ddbb047abd1f75da99b1c08)

(From OE-Core rev: 691925df192b4b1671ba9d19cbc3602654826451)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[Fixup for getVar True bit]
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 15:13:47 +01:00
Ross Burton
eff7238cf3 package-manager: add install_glob()
(From OE-Core rev: 8d1b530c82de386d4183f5673c060b9d416a3835)

(From OE-Core rev: b9a7821086b5165fda9f1c8a7c79a7997803f2a6)

(From OE-Core rev: c01a2c01a0c78a72ed715ac31a4578013ff44231)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[Fixup for getVar True bit]
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 15:13:47 +01:00
Ross Burton
9ac4f5b88e package_manager: improve install_complementary
- No need to use bb.utils.which() as subprocess will search $PATH
- Clarity flow by moving the install inside the try/except

(From OE-Core rev: f4d22b7195dd8f08fe26dd353c7e860208e87d6a)

(From OE-Core rev: 1267df89164d7dcbbf77668ac49452bd0df87b54)

(From OE-Core rev: fdc95793dff3dbfa49faa19c51c7e8c9cd67dc6c)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[Fixup do to merge conflict]
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 15:13:47 +01:00
Martin Jansa
c9159f18ed package_manager: flush installed_pkgs file before oe-pkgdata-util uses it
* since this commit:
  commit f5a1013ffa9815f22e13989e2bcb83f966e7ce2c
  Author: Ross Burton <ross.burton@intel.com>
  Date:   Tue Apr 18 16:19:12 2017 +0100

    package_manager: don't race on a file when installing complementary
    packages

  the file isn't closed before oe-pkgdata-util uses it and this
  temporary file might look empty to oe-pkgdata-util, because it
  wasn't flushed yet. Which resulted in almost empty debugfs tarballs
  and no locale packages in regular rootfs.
* without this change:
  124K May 30 07:41 core-image-full-cmdline-raspberrypi3-64-20170530054003-dbg.rootfs.tar.gz
* with this change:
  173M May 30 07:29 core-image-full-cmdline-raspberrypi3-64-20170530052715-dbg.rootfs.tar.gz

(From OE-Core rev: 9b34200048b3d2b477a19b7ddc8d447f873adbb2)

(From OE-Core rev: 667c43354e38c2fec1829089de0b7de076498e85)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 877d38db08aa7060d16405443cf70539c559fe82)
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 15:13:47 +01:00
Ross Burton
0652fb68b2 package_manager: don't race on a file when installing complementary packages
PackageManager.install_complementary() uses WORKDIR/installed_pkgs.txt as a
temporary file but if two tasks are executing for the same recipe which uses
this file (e.g. bitbake my-image my-image:do_populate_sdk) then it's possible
for the file to be overwritten or deleted.

Instead of using a static filename, use tempfile to generate a unique name and
ensure it is cleaned up when finished.

Also move the glob generation/expansion earlier in the function as if there are
no globs to install, we don't need to generate a package list.

(From OE-Core rev: f5a1013ffa9815f22e13989e2bcb83f966e7ce2c)

(From OE-Core rev: b02b54192ce71606aac30c21f3ff2199fa70a529)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[Fixup do to merge conflicts]
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 15:13:47 +01:00
Richard Purdie
0200ab2a22 lib/oe/package_manager/sdk: Ensure do_populate_sdk_ext and do_populate_sdk repos don't conflict
(From OE-Core rev: c9c0927bb6e71253cbdd5b6b780dca829526e1d8)

(From OE-Core rev: a18942b0b3b848ccfa4bc50c56a565c279da9d74)

(From OE-Core rev: 94e21cf21f8d8adad352ef2f94d4f75e36cd843a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[Fixup do to merge conflicts]
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 15:13:47 +01:00
Ross Burton
55049700e4 lib/oe/package_manager: bail if createrepo can't be found
If createrepo isn't found then the errors later are mysterious, so explicitly
check and error out early if it isn't there.

(From OE-Core rev: e09636bbb3ea8ec58984197fd9c691bb908efe00)

(From OE-Core rev: c87361fc886432a9db584712bf3e41ecd0541960)

(From OE-Core rev: 0adc876ed8be21190510a91220f9755c8e9df021)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[Fixup do to merge conflicts]
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 15:13:47 +01:00
Stephano Cetola
8cdf2a2e36 package_manager: remove strings and migrate to direct arrays
When using subprocess call and check_output, it is better to use arrays
rather than strings when possible to avoid whitespace and quoting
problems.

[ YOCTO #9342 ]

(From OE-Core rev: b12cec9a5ef14ecb02be7feec65508cf5d65c795)

(From OE-Core rev: 60ba1d424636bdd5700ec3ee0acec5c19550b884)

Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 15:13:47 +01:00
Linus Wallgren
2d5e8043be lib/oe/package_manager: .deb pre/postinst args
The debian policy manual and MaintainerScripts wiki page states that the
postinst script is supposed to be called with the `configure` argument
at first install, likewise the preinst script is supposed to be called
with the `install` argument on first install.

https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html
https://wiki.debian.org/MaintainerScripts

(From OE-Core rev: 3d9c3aae54589794ce3484fa1b21d1af2bd32661)

(From OE-Core rev: ba59b6416f24ad53f1caccf9185b46cb60da213a)

Signed-off-by: Linus Wallgren <linus.wallgren@scypho.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 15:13:47 +01:00
Leonardo Sandoval
913b20f799 selftest/cases/signing: ignore removal errors when cleaning temporary gpg directory
The high-level method tempfile.TemporaryDirectory give us no way to
ignore erros on removal thus use tempfile.mkdtemp instead. Ignoring possible issues
on removal is neccesary because it contains gpg sockets that are automatically removed
by the system once the process terminates, otherwise the following log is observed:

  ..
  ..
  File "/usr/lib/python3.5/shutil.py", line 436, in _rmtree_safe_fd
    os.unlink(name, dir_fd=topfd)
  FileNotFoundError: [Errno 2] No such file or directory: 'S.gpg-agent.browser'

[YOCTO #11821]

(From OE-Core rev: e7f139c5d97a3871215c88c5bfc07ecf4e8fd7f3)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-10 17:27:39 +01:00
libertad
e2fcc721c2 devtool.py: update testcase Ids in morty
eSDK testcases Ids were modified on testopia, hence there needs to be a change in the devtool script.

[YOCTO #11603]

(From OE-Core rev: b835e9bf9da6ebc8fd5728b4587470e9a57d5966)

Signed-off-by: libertad <libertad.cruz@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-25 22:21:45 +01:00
Richard Purdie
a17d574f09 oeqa/selftest: Drop http sstate sharing
Using httpServer from python for sharing sstate is known to be buggy, it can't
cope with the number/type of requests coming from bitbake and quietly fails
to share files.

This causes intermittent build failures which are hard to debug. We can
use a file:// url for the sstate mirror instead, removing the need for
the http server.

The sdk-update test is simply dropped since the SDK is never published
to this location and hence it would never have any update. Its equiavalent
to pointing at an empty web server. There is a better eSDK update test in
testsdk so rather than improve this one, lets drop it and concentrate on
the one there.

(From OE-Core rev: 7606f05e48ad2e31650e3a56bfcd04b4fbfad1e3)

(From OE-Core rev: 840a317e8a5518dbd1a025381441e13e906519ff)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11 17:15:55 +01:00
Mariano Lopez
ce4016c070 selftest/eSDK.py: Cleanup when there is an error in setUpClass
Lately autobuilders are experiencing hangs with selftest,
it seems it is cause if an error happens in setUpClass
method of oeSDKExtSelfTest class because HTTP server
keeps running in background.

This patch will ensure tearDownClass will be run if there
is an error in setUpClass.

(From OE-Core rev: eb1383949f76c6eb36f86c051057f761a71016a3)

(From OE-Core rev: 5dc68a378d9f4ec2c313ac395e91225a02e5b2c7)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11 17:15:55 +01:00
Francisco Pedraza
4ff1c8ddba oeqa/selftest: Adds test case for sdk-update eSDK
1. Test case adds perl recipe before build eSDK.
2. After this added recipe, the script verifies the update:

    ${SDK_DEPLOY}/${TOOLCHAINEXT_OUTPUTNAME}.sh
    oe-publish-sdk [esdk] [path-to-http]
    CORE_IMAGE_EXTRA_INSTALL = "perl"
    bitbake -c populate-sdk-ext [some-image]
    devtool sdk-update

This should fix [YOCTO #9369]

(From OE-Core rev: 21bd406bf89e9ceafe1a807877406be817cacca6)

(From OE-Core rev: 7824ca273e7e4b25d4efe5461c0402833f535d06)

Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11 17:15:55 +01:00
Chen Qi
ffaf0ea565 selftest/eSDK.py: fix sstate dir not found error
Fix the error below when SSTATE_DIR is not "${BUILDDIR}/sstate-cache".

  FileNotFoundError: [Errno 2] No such file or directory: '/xxx/../sstate-cache'

(From OE-Core rev: 785f0343d04c1684363b5289a3012cf7e1caa95f)

(From OE-Core rev: 9d04fe3a0ce7eb418cd48438e1699825904e2dfb)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11 17:15:55 +01:00
Mike Crowe
2a70e84643 kernel, license, sstate, rootfs.py: Remove deploy directory README
It isn't clear that the README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt
file in the deploy directory warrants the complexity it brings elsewhere.
Let's just remove it entirely.

In particular, if two do_image_complete tasks run in parallel they risk
both trying to put their image into ${DEPLOY_DIR_IMAGE} at the same time.
Both will contain a README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt
file. In theory this should be safe because "cp -alf" will just cause one
to overwrite the other. Unfortunately, coreutils cp also has a race[1]
which means that if one copy creates the file at just the wrong point the
other will fail with:

 cp: cannot create hard link ‘..../tmp-glibc/deploy/images/pantera/README_-_DO_NOT_DELETE_FILES_IN_THIS_D.txt’ to
+‘..../tmp-glibc/work/rage_against-oe-linux-gnueabi/my-own-image/1.0-r0/deploy-my-own-image-complete/README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt’: File exists

[1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25680

(From OE-Core rev: 71e9e88847d7000781642ea6187ebd8f40dfdcfe)

(From OE-Core rev: 20c39fdbb25c1b1867709c5bfb3ae2baef249be9)

Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-29 23:43:50 +01:00
Robert Yang
d4b41223d3 oe/path.py: fix for "Argument list too long"
Issue: LIN9-1648

Fixed when len(TMPDIR) = 410:
$ bitbake core-image-sato-sdk
[snip]
Subprocess output:
/bin/sh: /bin/cp: Argument list too long

ERROR: core-image-sato-sdk-1.0-r0 do_rootfs: Function failed: do_rootfs
[snip]

This is because "copyhardlinktree(src, dst)" does "cp -afl src/* dst",
while src/* is expanded to "src/file1 src/file2, src/file3..." which
causes the "Argument list too long", use ./* as src and change cwd in
subprocess.check_output() to fix the problem.

(From OE-Core rev: a3dc93eb25fba32109edd1db6e8766074fb52e4b)

(From OE-Core rev: befda6ce3fd916ab04c035d1d82ed173759f7f09)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-27 11:28:04 +01:00
Ross Burton
8a96509017 oeqa/selftest: lock down Meson git revision for reliability
The test_recipetool_create_github test fetches HEAD of the repository so
upstream changes can (and do) break the test.  Avoid these problems by passing
the rev= argument in the URL to lock the checkout to the same version that is
fetched in the github_tarball test.

Also pass the commands to runCmd() as a list instead of a string, the semicolon
in the URL needs more quotes if the shell is involved and passing a list
bypasses the shell entirely.

(From OE-Core rev: cbb5961d3db27f81967345a73267920f2623ccac)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-05 18:00:02 +01:00
Choong YinThong
b6a1004c73 selftest/recipetool: replace fedorahosted.org SRC_URI with github.com source
fedorahosted.org was retired on March 1st, 2017. This is to
update the SRC_URI to point to github.com.

[YOCTO #11226]

(From OE-Core rev: 9b3c9193d4a04061e64e43d55782a4d73ee166ad)

(From OE-Core rev: c9c0528d7f298dae4f543ab4006057022abe71e5)

Signed-off-by: Choong YinThong <yin.thong.choong@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-05 18:00:01 +01:00
Mariano Lopez
48a8c893ad selftest: Avoid sstate corruption by calling cleansstate
Currently selftest doesn't use sstates because some tests
clean sstate cache; using sstates would give a performance
boost instead of building everything from scratch.

With this sstates are not corrupted using different methods
depending on tests:

devtool: These tests needed to delete the cache so SSTATE_DIR
as SSTATE_MIRRORS and set a temporal SSTATE_DIR.

sstatetests: This module already used a temporal SSTATE_DIR, so
just set up the SSTATE_MIRRORS.

Rest: Removed cleansstate, some of them required to force a
certain task, others were just removed or changed for another
task.

[YOCTO #10929]

(From OE-Core rev: 62c61087a10cc3b26fbff32c9e2efd1704a39724)

(From OE-Core rev: 44ca8b1c23e4b2cf470e9be42705e3cca23b7f85)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-27 00:21:04 +01:00
Richard Purdie
078915fbf5 selftest: Disable SSTATE_MIRRORS for sstate signing test
Building ed into an sstate mirror, then leaving it enabled for
oe-selftest -r signing.Signing.test_signing_sstate_archive results in:

NOTE: recipe ed-1.14.1-r0: task do_populate_lic_setscene: Started
WARNING: ed-1.14.1-r0 do_populate_lic_setscene: Failed to fetch URL file://29/sstate:ed::1.14.1:r0::3:290bbc39f3b25cc27028dafce4b5aa48_populate_lic.tgz.sig;downloadfilename=29/sstate:ed::1.14.1:r0::3:290bbc39f3b25cc27028dafce4b5aa48_populate_lic.tgz.sig, attempting MIRRORS if available
ERROR: ed-1.14.1-r0 do_populate_lic_setscene: Fetcher failure: Unable to find file file://29/sstate:ed::1.14.1:r0::3:290bbc39f3b25cc27028dafce4b5aa48_populate_lic.tgz.sig;downloadfilename=29/sstate:ed::1.14.1:r0::3:290bbc39f3b25cc27028dafce4b5aa48_populate_lic.tgz.sig anywhere. The paths that were searched were:
    /media/build1/poky/build/test-sstate
    /media/build1/poky/build/test-sstate
WARNING: ed-1.14.1-r0 do_populate_lic_setscene: Cannot verify signature on sstate package /media/build1/poky/build/test-sstate/29/sstate:ed::1.14.1:r0::3:290bbc39f3b25cc27028dafce4b5aa48_populate_lic.tgz
NOTE: recipe ed-1.14.1-r0: task do_populate_lic_setscene: Succeeded

so we need to disable SSTATE_MIRRORS for this test.

(From OE-Core rev: 2ce9962851fe58c099599679340fd87e90f426ec)

(From OE-Core rev: 9bb708cf31850f5a3e53660ee9ea5a9ba7615e43)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-27 00:15:04 +01:00
Ross Burton
c2ade72828 oeqa/selftest: remove test_sanity_unsafe_binary_references
This test was attempting to exercise a broken test, for some reason this broke
with patches under review but investigation revealed that the test itself is
broken.  The test has been removed, so there's no need to test it.

(From OE-Core rev: 4f5f552cd6601ba244ef4efc782616bc477e1340)

(From OE-Core rev: 0b9bb6d9ec4e030ab64bac73680df4657cb7d066)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-25 17:38:36 +01:00
Alexander Kanavin
e7f948f28a selftest: do not perform a full build in test_continue
This was fetching and building the toolchain and everything else
against empty download dir and sstate cache, and so was enormously slow.
The test does not need that, it only checks that one fetch task fails and
another succeeds when using bitbake's -k option.

(From OE-Core rev: 6c69b5edc5711d2e5304a931e43eba562a095191)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-25 17:38:36 +01:00
Sullivan, California L
be54ac0ebd parselogs: Whitelist NUC6 firmware load error message on genericx86-64
This was already whitelisted, but the 4.8 kernel changed the error
message, causing it to get caught by parselogs again.

Fixes [YOCTO #10494].

(From OE-Core rev: a163060c25615fd29cdc477319f83997e65d0846)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
(cherry picked from commit e1bad14231115f3b1a2bf844ef5b2022c648b55d)
Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-18 13:07:33 +01:00
Martin Jansa
a8500d9c75 rootfs.py: Respect OPKGLIBDIR variable
* when OPKGLIBDIR doesn't have the default /var/lib value it will
  silently fail to copy package database from normal rootfs to debugfs
  rootfs and then when trying to install *-dbg complimentary packages
  it won't install anything, because installed_pkgs.txt file generated
  from debugfs is empty

(From OE-Core rev: 0554537271f56d95cb43e09bb831f57f82e8f3ac)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-18 13:07:33 +01:00
Martin Jansa
46b3cc9d8d package_manager.py: respect OPKGLIBDIR
* respect it for incremental rootfs generation
* add lists_dir option to opkg.conf
* also fix setting info_dir and status_file when they use default value, the
  problem is that self.opkg_dir is already prefixed with rootfs directory,
  comparing it with /var/lib/opkg always returned false and the options were
  appended to config file unnecessary
* with opkg 0.3.4 we can use VARDIR prefix added in:
  commit d2a8e23dc669adc398f4bb8bcfcabfcf925708f7
  Author: Florin Gherendi <floring2502@gmail.com>
  Date:   Mon Dec 19 12:25:38 2016 +0200
  libopkg: make the /var and /etc directories configurable at compile time.

(From OE-Core rev: f1d21d1d408d89cc3bd59cd30ab10c902863c7c0)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-18 13:07:33 +01:00
Markus Lehtonen
6c08cf277a lib/oe/gpg_sign: fix rpm signing with gpg > 2.1
We need to check the gpg version and alter its command line options
accordingly.

[YOCTO #11054]

(From OE-Core rev: 863459e6b565eb6b1a74b78bc01e884ffeac2085)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
(cherry picked from commit 44a44b7e582a5a654baf21829d168568481c13d9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-14 15:21:07 +00:00
Markus Lehtonen
35c95af7a8 lib/oe/gpg_sign: make gpg version a property of the signer
(From OE-Core rev: 951d22cb62891c4bc078925c4af8445d7be940e0)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
(cherry picked from commit a00a362e3dc18ba04230cbbd6f91264e5d76f40d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-14 15:21:07 +00:00
Markus Lehtonen
fd78c35086 lib/oe/gpg_sign: sign rpm packages in chunks of 100
Split the file list into chunks in order to avoid
"OSError: [Errno 7] Argument list too long"

This would happend when a package has huge amount of subpackages, e.g.
glibc-locale.

[YOCTO #11069]

(From OE-Core rev: 786eafd7b1080eccfe1c7d417eede20d75d80cb0)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
(cherry picked from commit 874f5016fd4dc76bc867b68470297fe59e78a9e6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-14 15:21:07 +00:00
Ed Bartosh
124df4ee35 selftest: wic: fix test_qemu
Setting WKS_FILE variable in qemux86-64 made wic test to
use wrong wks file to produce an image and resulted in
test_qemu failure.

Used conditional assignment in qemux86-64 and explicitly
set WKS_FILE in wic testing suite to make the suite to use
wic-image-minimal.wsk. This should fix test_qemu failure.

(From OE-Core rev: 3bca4d18c2712e3b154bacfb917f0a749ebaddeb)

(From OE-Core rev: bda4e3cceda2205a0a5d916ef5b674df560d43f9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-11 17:21:46 +00:00
Ross Burton
6c5a52ca8f utils: Always use datastore's PATH for host_gcc_version
BUILD_CC may reference something like ccache and expect this to come from
ccache-native, we at least have some selftests which assume this. Modify the
code to use PATH when runnig BUILD_CC to ensure the tests continue to work
as expected.

(From OE-Core rev: f3e753372baac43d0921186340cf260df056de20)

(From OE-Core rev: e7ec3228d9a2f40165b60f273205c17438b2c9bb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

Hand applied and used d.getVar(True)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-11 17:21:46 +00:00
Ed Bartosh
8108c47740 uninative: rebuild uninative for gcc 4.8 and 4.9
Some c++ libraries fail to build if uninative is built
with gcc 5.x and host gcc version is either 4.8 or 4.9.

The issue should be solved by making separate uninative sstate
directory structure sstate-cache/universal-<gcc version> for host gcc
versions 4.8 and 4.9. This causes rebuilds of uninative if host gcc
is either 4.8 or 4.9 and it doesn't match gcc version used to build
uninative.

[YOCTO #10441]

(From OE-Core rev: d36f41e5658bbbb6080ee833027879c119edf3e0)

(From OE-Core rev: 3d39ca5c91dbb62fb43199f916bd390cd6212e3d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-11 17:21:46 +00:00
Paul Eggleton
41e8df8228 lib/oe/rootfs: fix log_check warnings being printed twice with RPM packaging
We were calling _log_check() in the RPM-specific rootfs class as well as
in the base class; this is unnecessary and resulted in any errors/warnings
generated during the actual package installation time triggering two warnings
instead of one. Drop the call from RpmRootfs._create() to fix this.

(From OE-Core rev: 541c56d755ba0354297673e857628026ad9e4df2)

(From OE-Core rev: 9654a6a238a44ed1025a45e4ad1cb779a8684344)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-11 17:21:45 +00:00
Juro Bystricky
1933b492ca targetloader.py: drop test for ClassType
ClassType was removed from python3.
The code testing for ClassType kept throwing AttributeError exceptions:

    module 'types' has no attribute 'ClassType'

The exceptions prevented loading of any dynamically resolved target
controllers.

(From OE-Core rev: d62f18c39bc0ed3b0f5ac8465b393c15f2143ecf)

(From OE-Core rev: 6258471b7077839519898d537b99dce0a0fc8aa4)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-11 17:21:45 +00:00
Jair Gonzalez
bbc7130507 parselogs: Whitelist GPT warnings as the device is fully functional
The warning occurs when the GPT image is not the same size than the
media into which it's being flashed, causing the backup GPT table
not being at the end of the disk. However, this is expected as the
image is created before having the information about the destination
media. The error is harmless, so it will be whitelisted.

Fixes [YOCTO 10481].

(From OE-Core rev: 5cc5cdc788308a79f8f0706e6d794c602ef427ed)

(From OE-Core rev: 941194d29d8e4c334d3f85ce7709c894cf34b8d1)

Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-11 17:21:43 +00:00
California Sullivan
745517685b parselogs.py: Whitelist iwlwifi firmware load error messages
The iwlwifi module of any given kernel has a minimum and maximum
supported firmware version. The kernel begins by attempting to load the
maximum version, and decrements until it is successful. The 4.8 kernel's
maximum supported firmware version is 24, but thus far only 22 has been
released, meaning we get errors for 24 and 23.

Filter out iwlwifi firmware load error messages, as they are not
necessarily indicative of real problems.

(From OE-Core rev: 7df570c2310efac8f9898da15deaac2b7df16655)

(From OE-Core rev: 23aa95a8a15cd0b7bdc43dba9375e139f4b20607)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-11 17:21:43 +00:00
Martin Vuille
926f3dfe2d terminal.py: Pass string instead of bytes to ExecutionError to avoid exception
Based on run() in bitbake/lib/bb/process.py, ExecutionError() expects strings
not bytes. Passing bytes results in a "TypeError: Can't convert 'bytes' object
to str implicitly" exception.

Fixes Bug 10729

(From OE-Core rev: 063b63d4d324c23322ac1b6b7c7928e725d7b968)

(From OE-Core rev: df4d5246d47acc54aa322b777364c9b86f9a4c54)

Signed-off-by: Martin Vuille <jpmv27@yahoo.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-11 17:21:43 +00:00
Ross Burton
101173d6cf lib/oe/qa: handle binaries with segments outside the first 4kb
The ELF parser was assuming that the segment tables are in the first 4kb of the
binary.  Whilst this generally appears to be the case, there have been instances
where the segment table is elsewhere (offset 2MB, in this sample I have).  Solve
this problem by mmap()ing the file instead.

Also clean up the code a little whilst chasing the problem.

(From OE-Core rev: b88c6bd61e7c3388b3902de1adb5e48a88f2e235)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-11 17:21:40 +00:00
Samuli Piippo
ee6ff5ce83 package_manager.py: correctly remove all dependent packages
Do not use --force-depends when trying to remove all dependent packages,
as it removes only the selected package and not the dependent packages.

(From OE-Core rev: a82e8725902086dab785a0b14305927dae1e4e8d)

(From OE-Core rev: 0666c66b2719f59e556c12d5875dea696006ed0b)

Signed-off-by: Samuli Piippo <samuli.piippo@qt.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-16 10:37:57 +00:00
Robert Yang
698bbcad68 oe/copy_buildsystem.py: dereference symlink
When there is a relative symlink in the layer, for example:
symA -> ../out/of/layer/file

symA will be invalid fater copied, it would be invalid from build time
if it points to a relative path, and would be invalid after extracted
the sdk if it points to a absolute py. Dereference symlink when copy
will fix the problem.

Use tar rather than shutil.copytree() to copy is because:
1) shutil.copytree(symlinks=Fasle) has bugs when dereference symlinks:
   https://bugs.python.org/issue21697
   And Ubunutu 1404 doesn't upgrade python3 to fix the problem.

2) shutil.copytree(symlinks=False) raises errors when there is a invalid
   symlink, and tar just prints a warning, tar is preferred here since
   the real world is unpredicatable

3) tar is faster than shutil.copytree() as said by oe.path.copytree()

So use tar to copy.

(From OE-Core rev: f4d70bb0882eec4fb46cd942f2796fad57c72982)

(From OE-Core rev: 51d3cab8aab593481be16cadaca6fcddbb64bc52)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-16 10:37:56 +00:00
Joshua Lock
751cb2c894 Remove RM_OLD_IMAGE, it's no longer useful
Since the move to put image deployment under sstate control in
d54339d4b1a7e884de636f6325ca60409ebd95ff old images are automatically
removed before a new image is deployed (the default behaviour of the
sstate logic).

RM_OLD_IMAGE is therefore no longer required to provide this
behaviour, remove the variable and its users.

(From OE-Core rev: 93631befe8b962bf99524746b49f4ebca336175c)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-15 10:01:42 +01:00
Ross Burton
84198174bf lib/oe/qa: add ELF machine to string function
Add a function (and test suite) to turn the ELF machine field (e_machine) into a
string, so we can tell the user "x86-64" instead of 0x3E.

(From OE-Core rev: 72336003741fb16a7ecdd6b753eae56310413ff7)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-11 22:19:21 +01:00
Richard Purdie
78c01995e3 oeqa/sshcontrol: Handle interrupted system call error
Deal with an interrupted system call gracefully:

|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-qa-systemd/build/meta/lib/oeqa/utils/sshcontrol.py", line 55, in _run
|     if select.select([self.process.stdout], [], [], 5)[0] != []:
| InterruptedError: [Errno 4] Interrupted system call

(From OE-Core rev: 556125e4004fb7ac5169b59f51dc151f18c1806a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-07 16:43:58 +01:00
Ed Bartosh
09a9de45e1 wic: selftest: add test for sdimage-bootpart
Test creation of sdimage-bootpart image

(From OE-Core rev: 8b18568591f66aa9770798bc61123898de92b8a9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-07 16:43:58 +01:00
Ed Bartosh
bed4a13440 wic: selftest: add test for systemd-bootdisk
Test creation of systemd-bootdisk image.

(From OE-Core rev: b8eef88536b944ea35664b9a6a9496fb0a3f7988)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-07 16:43:58 +01:00
Peter Kjellerstedt
d067b0e1e4 package_manager.py: Allow multiple regexps in PACKAGE_EXCLUDE_COMPLEMENTARY
The PACKAGE_EXCLUDE_COMPLEMENTARY variable can currently only contain
one regular expression. This makes it hard to add to it from different
configuration files and recipes.

Allowing it to contain multiple, whitespace separated regular
expressions should be backwards compatible as it is assumed that
whitespace is not used in package names and thus is not used in any
existing instances of the variable.

After this change, the following three examples should be equivalent:

  PACKAGE_EXCLUDE_COMPLEMENTARY = "foo|bar"

  PACKAGE_EXCLUDE_COMPLEMENTARY = "foo bar"

  PACKAGE_EXCLUDE_COMPLEMENTARY = "foo"
  PACKAGE_EXCLUDE_COMPLEMENTARY += "bar"

(From OE-Core rev: a5f7e98a94e96d40b1276c85249619aa8d7be847)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-06 07:51:00 +01:00
Peter Kjellerstedt
2334201960 package_manager.py: Allow a leading - in PACKAGE_EXCLUDE_COMPLEMENTARY
This allows a regular expression specified in
PACKAGE_EXCLUDE_COMPLEMENTARY to have a leading dash. Without this,
the dash was treated by oe-pkgdata-util as the beginning of a command
line argument. E.g., if PACKAGE_EXCLUDE_COMPLEMENTARY = "-foo$", it
resulted in an error like:

  ERROR: <imagename>-1.0-r0 do_populate_sdk: Could not compute
  complementary packages list. Command '<topdir>/scripts/oe-pkgdata-util -p
  <builddir>/tmp/sysroots/<machine>/pkgdata glob
  <workdir>/installed_pkgs.txt *-dev *-dbg -x -foo$' returned 2:
  ERROR: argument -x/--exclude: expected one argument
  usage: oe-pkgdata-util glob [-h] [-x EXCLUDE] pkglistfile glob [glob ...]

(From OE-Core rev: ac4ca41d3a27356d46c0c39053e74d3519b24c44)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-06 07:51:00 +01:00
Markus Lehtonen
b16001f3e2 oeqa.buildperf: measure apparent size instead of real disk usage
This change aligns disk usage measurements of the eSDK test with the old
build-perf-test.sh script. And thus, also makes the results between the
old and the new script comparable.

(From OE-Core rev: dadb84936b3672dcf07e5ab8226158136762801f)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-06 07:51:00 +01:00