Commit Graph

2563 Commits

Author SHA1 Message Date
Mikko Rapeli
2e09dc1be7 scripts: Fix return value checks from subprocess.call()'s
Python function subprocess.call() returns the return value of the
executed process. If return values are not checked, errors may
go unnoticed and bad things can happen.

Change all callers of subprocess.call() which do not check for
the return value to use subprocess.check_call() which raises
CalledProcessError if the subprocess returns with non-zero value.

https://docs.python.org/2/library/subprocess.html#using-the-subprocess-module

All users of the function were found with:

$ git grep "subprocess\.call" | \
  egrep -v 'if.*subprocess\.call|=\ +subprocess\.call|return.*subprocess\.call'

Tested similar patch on top of yocto jethro. Only compile tested
core-image-minimal on poky master branch.

(From OE-Core rev: 031cf9c7834cd1cba8b03832673a3e3cfcbfae7c)

Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-03 23:46:06 +01:00
Ed Bartosh
684534198f isoimage-isohybrid: don't use TRANSLATED_TARGET_ARCH
isoimage-isohybrid plugin fails with this error when
trying to find initrd image:
    initrd = glob.glob('%s/%s*%s.%s' % (initrd_dir, image_name, target_arch, image_type))[0]
    IndexError: list index out of range
as it uses TRANSLATED_TARGET_ARCH value as part of the image.
This approach stopped to work due to changes in oe core code.

initrd file name is made using MACHINE variable. wic can't get value
of this variable as it's not included into bitbake -e output.

Used basename of deploy dir as MACHINE value to fix the breakage.

(From OE-Core rev: cfbb3cc1279ea88ca3e2867f8a409c5120aa1f05)

(From OE-Core rev: e26fa1c34c33ffdc678f8073dade0126aff08b40)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-03 23:46:06 +01:00
Daniel Lublin
693878a3e9 devtool: deploy-target: Let script continue even if rm fails
The remote script is run with `set -e`, so doing rm without -f on a
$file that is already gone will exit the whole script, failing any
redeployment. Assume a use case where packages sometimes produces
certain test binaries stored on volatile media (tmpfs), and where the
system is occasionally rebooted.

(From OE-Core rev: db54c9a22a9b66c673df8e836de5e47fc9edda0b)

Signed-off-by: Daniel Lublin <daniel@lublin.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-03 23:46:06 +01:00
Ola x Nilsson
1deb4a00ba devtool: Compare abspath of both B and S
Either both or none of the paths must be passed through
os.path.abspath or things like 'A//B', 'A/./B/', and 'A/B/' in S will
cause unintentional mismatches even when B = "${S}".

Using os.path.abspath for both seems more likely to be correct as that
will also handle the case where ${B} != ${S} but the abspaths are
equal.

(From OE-Core rev: 061f2aab40fecbfe0dcb928baa95d6b3a6b45eed)

Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-03 23:46:06 +01:00
Daniel Lublin
dd2f423525 devtool: deploy-target: Don't use find -exec
find may be provided by busybox, which might be compiled without support
for -exec.

(From OE-Core rev: 404e8e3661469175e1ea087ebfaf3a7867bf4df2)

Signed-off-by: Daniel Lublin <daniel@lublin.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-03 23:46:06 +01:00
Leonardo Sandoval
60ade6074e oeqa/core/context: Include a _pre_run method
This pre runner will serve to allow Test components executes code
previously of the run a suite.

(From OE-Core rev: bafb7e221d40d7a87a02cec8a97d98eec7c23438)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-02 13:36:14 +01:00
Ed Bartosh
e680f23c8b runqemu: output qemu-system errors
Included error output from qemu-system into the runqemu error message.
Made error output more visible by printing new line before it.

[YOCTO #11542]

(From OE-Core rev: 7fe5f5c29ca271ab718bbd1383e596f2ae61554c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-29 15:15:20 +01:00
Li Zhou
1c0ce38ba0 python3: Move config-*/Makefile from misc package to dev package
Move config-*/Makefile in libdir from misc package to dev package for
python3, because it is only needed in development process.

(From OE-Core rev: d715dc422ce1723c8d05af7ad4183eeeb36bc2ec)

Signed-off-by: Li Zhou <li.zhou@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-29 15:15:20 +01:00
Saul Wold
ab0747e7c6 devtool/standard.py: Enable perf to be buildable
Perf is a tool build from the kernel source, which is normally available
in /work-shared/..., but when devtool is used to modify the kernel
source code, perf is not buildable since it gets an error about being unable
to add a depends to a non-exisit task do_patch.

This patch removes do_patch from the SRCTREECOVEREDTASKS and creates an empty
do_patch task to enable the VarFlags code to have someplace to attach depends
information to.

[YOCT #11120]

(From OE-Core rev: 86c793595e560e7bc52e3cd2a2752746e6adcb76)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-25 23:59:32 +01:00
Paul Eggleton
415fbfb0cd scriptutils: fix fetch_uri() to work with RSS
Since recipe-specific sysroots were implemented, devtool add and devtool
upgrade operations that fetch from a URL that requires native sysroot
dependencies will fail to work as there is no recipe-specific sysroot
set up for them during fetching. An example was any URL pointing to a
tarball compressed with xz, e.g. devtool upgrade on gnutls.

The most expedient way to fix this is to set up a dummy recipe-specific
sysroot to use for the fetch/unpack operations. We do this in the same
manner as bitbake -b does, so we're just taking all of the sysroot
components available and creating a sysroot from those rather than
ensuring the correct dependencies are there - this means that we're
still going to have problems if e.g. xz-native hasn't been built yet,
but that issue will be trickier to solve and is tracked separately.

Fixes [YOCTO #11474].

(From OE-Core rev: 559151e783759af78b5cdd76cdbb9ce325a391e6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-25 23:59:31 +01:00
Markus Lehtonen
35c3e57686 scripts/buildstats-diff: support optimized rusage values
Buildstats from oe-build-perf-test results have been optimized to not
have child rusage values at all. There, rusage is the sum of parent and
child rusage values. This patch makes buildstats-diff compatible with
this format.

[YOCTO #11355]

(From OE-Core rev: 496a9dc179fe9dc370c940f4a2f7bcab869a804f)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-23 17:45:37 +01:00
Markus Lehtonen
d6c437ff6c oe-build-perf-report: implement --dump-buildstats
For dumping buildstats from the test runs being reported. The output
directory where buildstats are copied is 'oe-build-perf-buildstats/'.
Buildstats can be then further analyzed with buildstats-diff script, for
example.

[YOCTO #11355]

(From OE-Core rev: e06266798d975bd6bebdb6bfdbd3d21be1c44ffd)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-23 17:45:36 +01:00
Markus Lehtonen
1c46fbd628 oe-build-perf-report: two verbosity levels for --list
(From OE-Core rev: a77066751c81f27332cc16c565dff6a45c173b6c)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-23 17:45:36 +01:00
Markus Lehtonen
e1f84ea823 oe-build-perf-report: use correct x-axis max value in html charts
(From OE-Core rev: 1d99b018c2b6a44b9db2f2ede113e64dae751eee)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-23 17:45:36 +01:00
Paul Eggleton
a7c686cfe6 recipetool: create: extract name of package from a repository
For git repositories in the absence of any other indicator, it's not an
unreasonable assumption that the name of the repository is the name of
the software package it contains, so use that as PN if we don't have
anything else.

(From OE-Core rev: ef73fa70f0955912b0da140922465a3c817424e9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-23 17:45:36 +01:00
Paul Eggleton
4f152bdce9 recipetool: create: skip values extracted from spec files containing macros
If a value we extract from a spec file contains an unexpanded macro
(e.g. %{macroname}) then we should discard it since we're not seeing the
actual value and we don't have an easy way of expanding it at the
moment.

This fixes for example getting %{name} as the recipe name when running
the following:

recipetool create https://github.com/gavincarr/mod_auth_tkt.git

(From OE-Core rev: eee56a19cda051da6267f808cd3a04a4c644acb3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-23 17:45:36 +01:00
Ed Bartosh
f7c2ec9f87 create-pull-request: support format-patch options
Added possibility to specify extra format-patch options
in the create-pull-request command line:
   create-pull-request -u contrib -r master -- -v3

(From OE-Core rev: 19e51f05011e827a34890e0c098f3bfa66559e1a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-23 17:45:35 +01:00
Ed Bartosh
7e23b174e5 Revert "create-pull-request: add "-t in-reply-to" option"
Rasons:
 - It breaks the script if script is used without -t
 - Its functionality covered by the next patch

This reverts commit 3ad3fda6c5.

(From OE-Core rev: 9be68e6f058e2ab149e5c6f045479413cf0db69b)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-23 17:45:35 +01:00
Andreas J. Reichel
db26c84474 wic: Add missing text to usage and help strings
Add missing parameters -c to and remove non-existent
parameter -i from usage string for 'wic create'.

(From OE-Core rev: da937e0e2478152968d5fd5eec7dd8cc2b922afb)

Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-23 17:45:35 +01:00
Andreas J. Reichel
05aa049e2c wic: Use argparse instead of optparse
* optparse is deprecated and will not be developed further
    (see: https://docs.python.org/2/library/optparse.html)
* argparse supports subcommands, which simplifies definition of
    arguments and options
* reimplement help mechanism through sub-subcommands

[YOCTO #9636]

(From OE-Core rev: 4fe9635c72a6ee1becc47d832ab54724176ee811)

Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-23 17:45:35 +01:00
Markus Lehtonen
4fdb6aa28d build-perf-test-wrapper.sh: support uploading test reports
Implement new '-R' command line option for specifying an rsync
destination where to upload test reports.

[YOCTO #5049]

(From OE-Core rev: ebc37ede5b247347483128f091b505fe33356591)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-18 14:01:47 +01:00
Markus Lehtonen
5403db7f40 oe-build-perf-report: allow slashes in {branch} field in tag names
The target branch name might contain slashes.

(From OE-Core rev: 3ddde7d5bcffdd855dae0da6ba5feec752cbacec)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-18 14:01:47 +01:00
Markus Lehtonen
cdb02b7d93 oe-build-perf-report: accept parenthesis in tag names
(From OE-Core rev: b1feae0e23300ea3894d14d2e7b1c1f8b419146e)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-18 14:01:47 +01:00
Martin Kelly
5b649b4bbd scripts/runqemu.README: fix typo
(From OE-Core rev: 35d2cb0c7945d5d115b8e5a320b6bd769cb61a27)

Signed-off-by: Martin Kelly <mkelly@xevo.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-18 14:01:47 +01:00
Alistair Francis
7243c21bf8 runqemu: Automatically add a TFTP directory for slirp boot
When booting QEMU with slirp networking we want to use QEMUs TFTP server
to make the images in deploy accessible to the guest.

(From OE-Core rev: a6bef2fa065f8bb74d0084e44dd0ca47d7859113)

Signed-off-by: Alistair Francis <alistair.francis@xilinx.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-16 14:08:28 +01:00
Jose Lamego
3ad3fda6c5 create-pull-request: add "-t in-reply-to" option
The create-patch-request script creates patches as replies to a cover
letter, in the form of an email thread. If further revisions are sent to
the mailing list without referencing to the first revision, these new
revisions are not identified at the mailing list as part of the original
thread, but as a new thread instead.

This change adds the "[-t in_reply_to]" option, where "in_reply_to" is
the original cover letter's Message-Id, so this reference is added
to the new cover letter to ensure the thread continuity.

[YOCTO #11294]

(From OE-Core rev: 8a3879a8ca71db7fb313417d86b3ac7904cb0f0e)

Signed-off-by: Jose Lamego <jose.a.lamego@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-11 16:59:19 +01:00
Patrick Ohly
1fb309447f runqemu: support virtio drive type
Setting QB_DRIVE_TYPE=/dev/vd selects virtio without triggering any
warnings. Previously, that was only possible by setting an unknown
value and relying on the fallback to virtio, which caused some
warnings to be printed.

(From OE-Core rev: 5cbf102662dde6f706a19fa133cfd1e7475eb8c1)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-29 11:17:22 +01:00
Kristian Amlie
e3e78df1b2 wic: Fix typo in help screen.
This was overlooked when f6a064d969 was merged.

(From OE-Core rev: 29a209822488ab687abdb1ceffdd9c7af5b3db68)

Signed-off-by: Kristian Amlie <kristian.amlie@mender.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-29 11:17:22 +01:00
Robert Yang
52493384aa oe-find-native-sysroot: don't clear OECORE_NATIVE_SYSROOT
The OECORE_NATIVE_SYSROOT may come from environment vars, so don't clear
it, otherwise it broke runqemu-extract-sdk on sdk.

[YOCTO #11409]

(From OE-Core rev: 91faa9132821db4e7010b79943b2cfb804d6f4c5)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-29 11:17:22 +01:00
Martin Jansa
9784ad79a4 runqemu-gen-tapdevs: fix runqemu-ifup script call
The tunctl binary is here:

OE @ /OE/openembedded-core # find /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/qemu-helper-native/
/OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/qemu-helper-native/
/OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/qemu-helper-native/sysroot-providers
/OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/qemu-helper-native/sysroot-providers/qemu-helper-native
/OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/qemu-helper-native/usr
/OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/qemu-helper-native/usr/bin
/OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/qemu-helper-native/usr/bin/tunctl

But the script still complains that it cannot find tunctl:

OE @ /OE/openembedded-core # ./scripts/runqemu-gen-tapdevs 1026 1026 4 /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/qemu-helper-native/
Note: Destroying pre-existing tap interface tap0...
TUNSETIFF: Device or resource busy
Creating 4 tap devices for UID: 1026 GID: 1026...
Creating tap0
Error running tunctl: Error: Unable to find tunctl binary in '/OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/qemu-helper-native/', please bitbake qemu-helper-native

The message is actually from runqemu-ifup, which is called from runqemu-gen-tapdevs as:
++ ./scripts/runqemu-ifup 1026 1026 /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/qemu-helper-native/

But runqemu-ifup expects 3rd parameter to be STAGING_BINDIR_NATIVE directly not just SYSROOT dir
STAGING_BINDIR_NATIVE=$3
because tunctl is then used as:
TUNCTL=$STAGING_BINDIR_NATIVE/tunctl

It looks like it got broken by:
commit cc5513bf7a6114e14bb307acb88a44e9cf0aed8a
Author: Ed Bartosh <ed.bartosh@linux.intel.com>
Date:   Wed Apr 12 23:40:59 2017 +0300

    runqemu: use bindir_native property to run ifup/down scripts

    Used self.bindir_native to point out to the native sysroot
    when running runqemu-ifup and runqemu-ifdown scripts.

    [YOCTO #11266]
    [YOCTO #11193]

    Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

Seemingly obvious fix would be to call runqemu-gen-tapdevs with path to STAGING_BINDIR_NATIVE in 4th parameter as well, but that won't work, because runqemu-gen-tapdevs checks for TUNCTL=$SYSROOT/usr/bin/tunctl

OE @ /OE/openembedded-core # ./scripts/runqemu-gen-tapdevs 1026 1026 4 /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/qemu-helper-native/usr/bin/
Error: /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/qemu-helper-native/usr/bin//usr/bin/tunctl is not an executable

I've tested that with this change it can call tunctl:
OE @ /OE/openembedded-core # ./scripts/runqemu-gen-tapdevs 1026 1026 4
/OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/qemu-helper-native/usr/bin
Note: Destroying pre-existing tap interface tap0...
TUNSETIFF: Device or resource busy
Creating 4 tap devices for UID: 1026 GID: 1026...
Creating tap0
Creating tap1
Creating tap2
Creating tap3
Note: For systems running NetworkManager, it's recommended
Note: that the tap devices be set as unmanaged in the
Note: NetworkManager.conf file. Add the following lines to
Note: /etc/NetworkManager/NetworkManager.conf
[keyfile]
unmanaged-devices=interface-name:tap*

but runqemu itself still doesn't work for me:
OE qemux86@ ~/build/oe-core $ runqemu
runqemu - INFO - Running MACHINE=qemux86 bitbake -e...
runqemu - INFO - Running ls -t /OE/build/oe-core/tmp-glibc/deploy/images/qemux86/*.qemuboot.conf...
runqemu - INFO - CONFFILE: /OE/build/oe-core/tmp-glibc/deploy/images/qemux86/core-image-sato-qemux86-20170427174052.qemuboot.conf
runqemu - INFO - Overriding conf file setting of STAGING_DIR_NATIVE to /OE/build/oe-core/tmp-glibc/work/i586-oe-linux/defaultpkgname/1.0-r0/recipe-sysroot-native from Bitbake environment
runqemu - INFO - Continuing with the following parameters:

KERNEL: [tmp-glibc/deploy/images/qemux86/bzImage--4.10.9+git0+ad2e885015_fe0fb8da3d-r0.2-qemux86-20170427085800.bin]
MACHINE: [qemux86]
FSTYPE: [ext4]
ROOTFS: [tmp-glibc/deploy/images/qemux86/core-image-sato-qemux86-20170427174052.rootfs.ext4]
CONFFILE: [/OE/build/oe-core/tmp-glibc/deploy/images/qemux86/core-image-sato-qemux86-20170427174052.qemuboot.conf]

runqemu - INFO - Running /bin/ip link...
runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap0.lock...
runqemu - INFO - Using preconfigured tap device tap0
runqemu - INFO - If this is not intended, touch /tmp/qemu-tap-locks/tap0.skip to make runqemu skip tap0.
runqemu - INFO - Network configuration: 192.168.7.2::192.168.7.1:255.255.255.0
runqemu - INFO - Running ldd tmp-glibc/work/x86_64-linux/qemu-helper-native/1.0-r1/recipe-sysroot-native/usr/bin//qemu-system-i386...
runqemu - INFO - Running tmp-glibc/work/x86_64-linux/qemu-helper-native/1.0-r1/recipe-sysroot-native/usr/bin//qemu-system-i386 -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:02 -netdev tap,id=net0,ifname=tap0,script=no,downscript=no -drive file=tmp-glibc/deploy/images/qemux86/core-image-sato-qemux86-20170427174052.rootfs.ext4,if=virtio,format=raw -vga vmware -show-cursor -usb -usbdevice tablet -device virtio-rng-pci   -cpu qemu32 -m 256 -serial mon:vc -serial null -kernel tmp-glibc/deploy/images/qemux86/bzImage--4.10.9+git0+ad2e885015_fe0fb8da3d-r0.2-qemux86-20170427085800.bin -append 'root=/dev/vda rw highres=off  mem=256M ip=192.168.7.2::192.168.7.1:255.255.255.0 vga=0 uvesafb.mode_option=640x480-32 oprofile.timer=1 uvesafb.task_timeout=-1 '
qemu-system-i386: -netdev tap,id=net0,ifname=tap0,script=no,downscript=no: could not configure /dev/net/tun (tap0): Device or resource busy
runqemu - INFO - Releasing lockfile for tap device 'tap0'
Traceback (most recent call last):
  File "/OE/build/oe-core/openembedded-core/scripts/runqemu", line 1235, in <module>
    ret = main()
  File "/OE/build/oe-core/openembedded-core/scripts/runqemu", line 1228, in main
    config.start_qemu()
  File "/OE/build/oe-core/openembedded-core/scripts/runqemu", line 1139, in start_qemu
    raise Exception('Failed to run %s' % cmd)
Exception: Failed to run tmp-glibc/work/x86_64-linux/qemu-helper-native/1.0-r1/recipe-sysroot-native/usr/bin//qemu-system-i386 -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:02 -netdev tap,id=net0,ifname=tap0,script=no,downscript=no -drive file=tmp-glibc/deploy/images/qemux86/core-image-sato-qemux86-20170427174052.rootfs.ext4,if=virtio,format=raw -vga vmware -show-cursor -usb -usbdevice tablet -device virtio-rng-pci   -cpu qemu32 -m 256 -serial mon:vc -serial null -kernel tmp-glibc/deploy/images/qemux86/bzImage--4.10.9+git0+ad2e885015_fe0fb8da3d-r0.2-qemux86-20170427085800.bin -append 'root=/dev/vda rw highres=off  mem=256M ip=192.168.7.2::192.168.7.1:255.255.255.0 vga=0 uvesafb.mode_option=640x480-32 oprofile.timer=1 uvesafb.task_timeout=-1 '

(From OE-Core rev: a31b1434c5f1edbd4e8faca813b4f084297c061d)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-29 11:17:22 +01:00
Paul Eggleton
6ffec971e8 runqemu: fix incorrect calls to get variable values
We were specifying a default parameter; the get() function defined here
does not take such a parameter. I appears this code had not been tested.
This fixes runqemu erroring out immediately when used within the eSDK.

(From OE-Core rev: e4548531112c824653ae42b9bcc335a7ca8588e0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-19 10:38:36 +01:00
Robert Yang
1fa1a7f174 oe-run-native: print more error messages
Fixed:
$ bitbake bmap-tools-native -ccleansstate && bitbake bmap-tools-native && oe-run-native bmap-tools-native bmaptool --help
[snip]
Error: Unable to find '' in <PATH>
[snip]

Note the blank '' word, it was because "tools" was overrided, now fix it.

And also check whether the recipe is a native one or not.

(From OE-Core rev: ba2884f6ad3a4e746fc80cbd707f83fa8abd4210)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-14 23:25:54 +01:00
Paul Eggleton
bce0b509bf devtool: extract: drop erroneous bb.event.TaskStarted
This is a non-existent event - we already have the actual
bb.build.TaskSucceeded further down in the list hence why it wasn't
noticed earlier.

(From OE-Core rev: 4e059a5ceb6f44401154e89e37f56de1d664a7cb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-14 09:47:08 +01:00
Paul Eggleton
02c39d3fad devtool: extract: fix handling of failed tasks
If a task such as do_fetch fails when we're extracting source for a
recipe (within devtool modify / upgrade / extract / sync) then we should
naturally stop processing instead of blundering on; in order to do that
we need to be listening for the TaskFailed event. Thanks to Richard
Purdie for noticing and fixing this.

(From OE-Core rev: 9174b845bf6a6be7753bf6b921959b1f3f2dcbc0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-14 09:47:08 +01:00
Paul Eggleton
2338032d0b devtool: modify: add --keep-temp option for debugging
Most of the other extract-based commands have this option but oddly I
left it out for modify - I guess because if I was debugging an issue here
I just used devtool extract to do so, but there's no reason why we can't
have it here and it is useful.

(From OE-Core rev: 98fbc46e1a51237213bd7825a922389d3ab2ad9b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-14 09:47:08 +01:00
brian avery
314df06b81 oe-run-native: explicitly use bash
This script sources another script (oe-find-native-sysroot) with
arguments.  It was using /bin/sh. Sourcing with arguments works only in
bash so it was failing in dash.  This commit makes it dash proof.

(From OE-Core rev: 334020a800434d20e7c3312890a2baca295c41c7)

Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:11 +01:00
brian avery
c8b67a463f oe-find-native-sysroot: add appopriate suggestion
right now, if it fails, the script tells the user to run bitbake foo
-caddto_recipe_sysroot. This works for native recipes but not things
like meta-ide. This patch checks whether the recipe is native and gives
out the appopriate warning.

(From OE-Core rev: fc61211efd57d1858954e5cd241fce58dee9d01b)

Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:11 +01:00
Robert Yang
4a0b6af446 oe-find-native-sysroot: work with RSS
The generic STAGING_DIR_NATIVE is gone since RSS, so when find
OECORE_NATIVE_SYSROOT, the user has to specify which recipe's
STAGING_DIR_NATIVE will be used as OECORE_NATIVE_SYSROOT.

* The usage is changed from ". oe-find-native-sysroot" to
  ". oe-find-native-sysroot <recipe>".
* The oe-run-native's usage has changed from
  "oe-run-native tool" to "oe-run-native native-recipe tool".

(From OE-Core rev: e2f6d937bd897083779507ecb9ecd15513b35f1f)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Ed Bartosh
5897c5edc4 Revert "scripts: change way we find native tools (pseudo)"
This reverts commit f200f37699.

This reverts the patch that fixed runqemu-extract-sdk. It failed
to fix other issues in the script/tools that were introduced by RSS. The
following patch from Robert Yang fixes both.  Therefore, reverting this
patch in favor of his.

(From OE-Core rev: 33263b69e6b470b4f02172dabe6815df8ab0cd0a)

Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Ed Bartosh
4393cc550b runqemu: use bindir_native property to run ifup/down scripts
Used self.bindir_native to point out to the native sysroot
when running runqemu-ifup and runqemu-ifdown scripts.

[YOCTO #11266]
[YOCTO #11193]

(From OE-Core rev: cc5513bf7a6114e14bb307acb88a44e9cf0aed8a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Ed Bartosh
4d3bececf6 runqemu: add bindir_native property
Isolated logic of getting path to native bin directory in
new bindir_native property method.

This property is going to be used to obtain location of
qemu-sytem and tunctl.

(From OE-Core rev: 26e97f7ebb7e3302e3d3c6646fb58baf395d62be)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Ed Bartosh
7ffcdd0a07 runqemu: get qemu from qemu-helper-native sysroot
If rm_work is enabled image native sysroot can be removed.
This makes runqemu to fail trying to find qemu binary.

Used native sysroot of qemu-helper-native to find system qemu
binary.

(From OE-Core rev: d42c02caaa4d6fb47681aa7ffe8b27fa38141e6a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Markus Lehtonen
11747280b5 oe-build-perf-report-email.py: use pwd for getting user name
Use pwd data instead of os.getlogin() to get the username for fallback
email address. os.getlogin() basically returns the name of the user
logged in on the controlling terminal of the process and raises an
exception on some systems if the process does not have a controlling
terminal - when executed from a cron script, for example.

(From OE-Core rev: 66b7f495bb000c043ae24176dcdec0cb087eeecf)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Richard Purdie
ffabe5d1b9 oe-selftest: Error if known problem variables are set
Setting SANITY_TESTED_DISTROS or PRSERV_HOST are known to break oe-selftest.
Rather than have the user experience this, refuse to execute unless the
environment is correct.

Ideally we'd try and unset these but that is a more invasive change and this
at least makes people aware of the problem.

[YOCTO #11292]

(From OE-Core rev: 0c9b981e88c76da316e76f17e6da3a03b87c5008)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Patrick Ohly
68079d0f87 yocto-compat-layer: better handling of per-machine world build breakage
It is fairly common that BSP layers enable recipes when choosing
machines from that layer without checking whether the recipe actually
builds in the current distro. That breaks "bitbake world", retrieving
signatures and thus the test_machine_signatures test.

It's better to let that test continue with the signatures that can be
retrieved and report the broken world build separately. Right now, the
new test_machine_world iterates over all machines. More elegant and
useful in combination with a (currently missing) selection of which
tests to run would be to generate one test instance per machine. But that
is not straightforward and has to wait.

The "-k" argument alone was not enough to proceed despite failures,
because bitbake then still returns a non-zero exit code. The existance
of the output file is taken as sign that the bitbake execution managed
was not fatally broken.

(From OE-Core rev: 02f5d7836b726e40fef82b50b8145acc839b360b)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Patrick Ohly
de76d1cc94 yocto-compat-layer: test signature differences when setting MACHINE
Selecting a machine is only allowed to affect the signature of tasks
that are specific to that machine. In other words, when MACHINE=A and
MACHINE=B share a recipe foo and the output of foo, then both machine
configurations must build foo in exactly the same way. Otherwise it is
not possible to use both machines in the same distribution.

This criteria can only be tested by testing different machines in combination,
i.e. one main layer, potentially several additional BSP layers and an explicit
choice of machines:
yocto-compat-layer --additional-layers .../meta-intel --machines intel-corei7-64 imx6slevk -- .../meta-freescale

To simplify the analysis and limit the amount of output, mismatches
are sorted by task order such that tasks that run first are also
reported first. Following tasks for the same recipe and set of
machines then get pruned, because they are likely to be different
because of the underlying task (same approach as in
test_signatures). The difference here is that we get information about
all machines. The task order in the base configuration serves as
heuristic for sorting that merged list.

The test has already found issues in go-cross (depended on
tune-specific libgcc) and gdb-cross (had a tune-specific path
unnecessarily), so it is also useful to uncover issues that are not
caused by the BSP layer itself.

(From OE-Core rev: cb0d3de4540e412cfcb7804b4b1689141c80e3a1)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Paul Eggleton
40d1771944 recipetool: create: hide missing npm error when called from devtool
If devtool is called with a URL to a source repository containing a
node.js module, we don't know that until recipetool has fetched it, and
due to the structure of the code we have to exit with a special code in
order to let devtool know it needs to build nodejs-native. We also want
to suppress the error message that recipetool would normally print under
these circumstances; there is already a mechanism for this but it wasn't
operative in the case where we're pointed to a source repository rather
than an npm:// URL, so create some plumbing so that we know to hide the
message.

(From OE-Core rev: 0c2d0fbb1c6c5b82183799eb7ef80074f86bcfc4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Paul Eggleton
48f8d1201a devtool: add: prevent repeatedly running recipetool
If recipetool returns with exit code 14 this means devtool needs to
build nodejs-native and then call it again. If recipetool returns exit
code 14 again then clearly something has gone wrong and we should just
quit with an error.

(From OE-Core rev: 8d7cced6e06d7c2037f5ab75ac859f501129532e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Paul Eggleton
bb8f141d0c devtool: add: fix node.js/npm handling with recipe specific sysroots
The change over to recipe specific sysroots means that we can no longer
get a known location simply from configuration for the npm binary - we
need to get the recipe sysroot for nodejs-native, look there for npm if
we need to check it's present, and add that to PATH when calling out to
npm. Unfortunately this means anywhere we need to get that path we have
to have parsed all recipes, otherwise we have no reliable way of
resolving nodejs-native. Thus we have to change recipetool create to
always parse all recipes (the structure of the code does not allow us to
do this conditionally).

In the worst case, if npm hasn't already been added to its own sysroot
and we are fetching from a source repository rather than an npm
registry, this gets a bit ugly because we end up parsing recipes three
times:
1) recipetool startup, which then fetches the code and determines it's
   a node.js module, finds that npm isn't available and then exits with
   a specific error to tell devtool it needs to build npm
2) when we invoke bitbake -c addto_recipe_sysroot nodejs-native
3) when we re-invoke recipetool

This code is badly in need of refactoring, but now is unfortunately not
the time to do that, so we're going to have to live with this ugliness
for now.

Fixes [YOCTO #10992].

(From OE-Core rev: acfdbd796c99882b8586023c8c6b848716105c8d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Paul Eggleton
0440cbccda recipetool: create: fix for regression in npm license handling
OE-Core commit c0cfd9b1d54b05ad048f444d6fe248aa0500159e added handling
for AND / OR in license strings coming from npm, but made the assumption
that an & would always be present in the license value. Check if it's
there first so we don't fail if it isn't.

(From OE-Core rev: abe2955df2dc558de6068d9373dfcb47d690704b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:09 +01:00