Commit Graph

3187 Commits

Author SHA1 Message Date
Khem Raj
aa88e318f2 multilib_header_wrapper: Drop using __MHWORDSIZE
This is not needed as __WORDSIZE already represents same value and is
directly defined in wordsize.h, this simplifies the multlib headers

(From OE-Core rev: 3a6ba47ede9b1d66f6cb58a776255047a5d069ab)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-18 06:33:36 +01:00
Khem Raj
c131b23cdb multilib_header: Fall back to worsize form libc for bpf target
Setting bpf to use 64bit for wordlength is not right, it happens to
work perhaps becuase the targets its being run on are 64bit inherently

(From OE-Core rev: 6d8c7e3426a74b1bc4752d508f68548480955853)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-18 06:33:36 +01:00
Khem Raj
55b6734be9 multilib_header_wrapper.h: Remove pragma once
This was a bandaid to avoid include recursion caused by multilibbed
wordsize.h

(From OE-Core rev: 7496c8fd4b2d9a69ee764f196abaec6172be2f02)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-18 06:33:36 +01:00
Alexander Kanavin
cfa1424b3c runqemu: force software x11 renderer for SDL when GL is not in use
If libsdl is configured with opengl enabled, it will try to use GL
renderers to open a window even if qemu doesn't use virgl. Setting
the environment variable forces a software X11 renderer, which is the
same one used when libsdl has opengl switched off. This avoids
needlessly loading the GL libraries and drivers from the host.

(From OE-Core rev: 73b75f5ff6c0b5333898fa340f4d9e2e5111fdef)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-12 15:20:42 +01:00
jan
a073a3e2d3 scripts/tiny/ksize: Fix for more recent kernels
In the past kernel built object files were named 'built-in.o'.
Nowadays it is 'built-in.a'.

The script is modified to work with both.  I expect
it will not happen that there are built-in.a and built-in.o
files in the same kernel.

(From OE-Core rev: facc8a5859d95c416aff8313bec3f3a6d5ef5fec)

Signed-off-by: Jan Vermaete <jan.vermaete@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-09 18:57:21 +01:00
Alexander Kanavin
1ec9103116 runqemu: set host DRI driver path for all instances of virgl usage
Falling through to host GL stack via epoxy chrpath hack was
found to be problematic in case of SDL particularly (because it does
not actually use epoxy, and does its own dlopen()).

So let's just use the mesa-native libraries everywhere, but instruct
them to load DRI drivers from the host.

NVidia's proprietary blob users have to take care of themselves, I'm afraid.

(From OE-Core rev: 2125920828eba7bab1afc977d5e9bccb124078e5)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-07 12:16:00 +01:00
Ricardo Ribalda Delgado
31b14186e9 wic: rootfs: Combine path_validation in one function
Combine all the common path validation in a function to avoid code
duplication.

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: ebd12b10d17db0b4176b0188407d7e9b8420eab1)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Ricardo Ribalda Delgado
a293c76c53 wic: root: Add an opt. destination on include-path
Allow specifying an optional destination to include-path and make the
option aware of permissions and owners.

It is very useful for making a partition that contains the rootfs for a
host and a target Eg:

/ -> Roofs for the host
/export/ -> Rootfs for the target (which will netboot)

Although today we support making a partition for "/export" this might
not be compatible with some upgrade systems, or we might be limited by
the number of partitions.

With this patch we can use something like:

part / --source rootfs --fstype=ext4 --include-path core-image-minimal-mtdutils export/ --include-path hello

on the .wks file.

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: e8c21c6ebaebde88151697381bdb2452f1171090)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Ricardo Ribalda Delgado
fc2589384b wic: misc: Do not find for executables in ASSUME_PROVIDED
Executables like tar won't be available on the native sysroot, as they
are part of the ASSUME_PROVIDED variable.

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: 2f574d535f8665b26dab65c14668cf8fc7b751c0)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Ricardo Ribalda Delgado
c81d5c6243 wic: Avoid creating invalid pseudo directory
If the source of the rootfs is not a bitbake cooked image, or it is not
pointing to the root of one, we call pseudo again, which will produce
a new pseudo folder at rootfs/../pseudo

Eg:

part /etc --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/home --fstype=ext4

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: 799a24ae78655f7a3eda7456b1a0ffaf3e43ec16)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Ricardo Ribalda Delgado
19e778fe1b wic: Continue if excluded_path does not exist
If an excuded path does not exist, continue without an error.
This allows to seamleasly reuse .wks among different projects.

Eg:

part / --source rootfs --fstype=ext4 --exclude-path=opt/private_keys

Where /opt/private_keys in only populated by some of the image.bb files.

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: 2dbd692b8e563cf991fb4ae1ef6129fda0d7e3c4)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Ricardo Ribalda Delgado
d19004da2f wic: Add --change-directory argument
This option allows to specify which part of a rootfs is going to be
included, the same way the -C argument on tar.

Thanks to this option we can make sure the permissions and usernames
on the target partition are respected, and also simplify the creation of
splitted partitons, not neeting to invoke external vars or using .wks.in
files. Eg:

part / --source rootfs --ondisk sda --fstype=ext4 --exclude-path=etc/   
part /etc --source rootfs --fstype=ext4 --change-directory=etc

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: 2265d089a58e1f78f26d623ee667c420cb1c3bd4)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Ricardo Ribalda Delgado
c58711f0ea wic: Fix multi images .wks with bitbake
In order to support .wks files with multiple images inside bitbake we
need to explicitly set the pseudo database in use.

Eg: If we try this .mks:
part / --source rootfs --ondisk sda --fstype=ext4
part /export --source rootfs --rootfs=core-image-minimal-mtdutils --fstype=ext4

The username for all the files under /export will be set to the runner
of bitbake (usually UID 1000).

Before we run wic, we need to make sure that the pseudo database will be
flushed, and contains all the data needed.

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: dde90a5dd2b22a539095d1bac82acc15c6380ac8)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Ricardo Ribalda Delgado
90164f0f3d wic: Fix permissions when using exclude or include path
When parameters include_path or exclude_path are passed to the rootfs
plugin, it will copy the partition content into a folder and make all
the modifications there.

This is done using copyhardlinktree(), which does not take into
consideration the content of the pseudo folder, which contains the
information about the right permissions and ownership of the folders.

This results in a rootfs owned by the user that is running the wic
command (usually UID 1000), which makes some rootfs unbootable.

This bug can be easily triggerd with the following .wks

part / --source rootfs --fstype=ext4 --exclude-path=home

And this sequence:

$ wic create test-permissions -e core-image-minimal -o test/
$ sudo mount test/test-permissions-202004080823-sda.direct.p1 /mnt
$ ls -la /mnt/etc/shadow

To fix this we copy the content of the pseudo folders to the new folder
and modify the pseudo database using the "pseudo -B" command.

If the rootfs is not a rootfs generated by bitbake a warning is shown
making the user aware that the permissions on the target might not match
what he expects.

WARNING: /tmp/test/../pseudo folder does not exist. Usernames and permissions will be invalid

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: fd739c15cdba221d9d497d3402b7d929c0be2ca4)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Tim Orling
b8ea59d595 scripts/install-buildtools: bump to 3.1 release by default
By default, use the extended buildtools installer from the
Yocto Project 3.1 "dunfell" release.

(From OE-Core rev: abd9bf4428e024f4fbcabd75235965769c03f2db)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Lee Chee Yang
df77ee4723 wic: use Filesystem UUID when expand swap partition
part.get("uuid") return the Partition UUID from the partition
table instead of Filesystem UUID. This lead to swap partition
UUID not match/change when wic write expand swap partition.

change it to read the filesystem UUID using blkid. The output
from blkid should looks like this:
wic-partvzhiwq3s: LABEL="swap1" UUID="04e55c19-3f3f-4491-9e32-44eea8daa827" VERSION="1" TYPE="swap" USAGE="other"

[YOCTO #13313]

(From OE-Core rev: 683297eefebe83f848daad9927871242ab28ef91)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Trevor Gamblin
691f8c5fc8 buildall-qemu: automate build testing for qemu MACHINEs
buildall-qemu simplifies the process of build testing an upgraded or
patched recipe by cycling through the build steps for each available qemu
target, with desired LIBC specified by the user as an option (defaulting
to both glibc and musl if no option is provided). While building, a log
file with the name "<recipe>-buildall.log" is written, containing a PASS
or FAIL line upon completion of the build for each architecture. For now,
qemu targets are not selectable (i.e. the recipe is built for all qemu
targets found in meta/conf/machine), but this functionality can be added
in the future along with other useful options.

The log file created by buildall-qemu also includes some basic system
info (e.g. build start time, hostname, host OS, host kernel). Since it is
not guaranteed that tools such as lsb_release will be available on the
host (it isn't by default on my Fedora machine), this information is
collected manually. Additionally, the previous run's log is retained for
comparison by renaming it in the format <recipe>-buildall.log.old once a
new set of builds is triggered for the same recipe.

We've seen multiple variations of this concept in use as one-liners and
as bash functions in dotfiles, so it seemed appropriate to submit it in
script form to oe-core for everyone to use.

Sample log output:

BUILDALL-QEMU LOG FOR aspell
START TIME: 2020-02-11_19:50:02
HOSTNAME: yow-tgamblin-fedora2
HOST OS: Fedora 31 (Server Edition)
HOST KERNEL: 5.4.10-200.fc31.x86_64
===============
BUILD RESULTS:
[glibc]
PASS: qemuarmv5
PASS: qemux86
PASS: qemuppc
PASS: qemumips64
PASS: qemux86-64
PASS: qemumips
PASS: qemuarm
PASS: qemuarm64
PASS: qemuriscv64
[musl]
PASS: qemuarmv5
PASS: qemux86
PASS: qemuppc
PASS: qemumips64
PASS: qemux86-64
FAIL: qemumips
FAIL: qemuarm
FAIL: qemuarm64
FAIL: qemuriscv64
===============
PASSED: 14
FAILED: 4

(From OE-Core rev: ee64b7db6651df4ed419e3d2649b4fb2f2faf830)

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-24 14:10:08 +01:00
Konrad Weihmann
8361b47d7f runqemu: add QB_ROOTFS_EXTRA_OPT parameter
Content of the optional parameter will be appended to the rootfs-device
in the qemu configuration, in case QB_ROOTFS_OPT is not specified.
By default this is empty.

Example use cases are:

Defining 'readonly' when using squashfs, so multiple instances of qemu
can share the same base image, something that cannot be done by just
specifying 'snapshot'.

Defining 'bootindex=0' which helps to get past the EFI shell in
ovmf-binary. This also enables the use case of running WIC images with
EFI bootloader through the testimage.bbclass.

(From OE-Core rev: e9b8c194636cb5505774a2a71bf54450580dd5b8)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-24 14:10:08 +01:00
Richard Purdie
f4e0b26568 Revert "wic: Fix permissions when using exclude or include path"
This reverts commit 36993eea89d1c011397b7692b9b8d61b499d0171.

After discussion on the mailing list it was felt these changes
were not ready yet.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-07 21:57:16 +01:00
Richard Purdie
157a12d499 Revert "wic: Add --embed-rootfs argument"
This reverts commit efdcf94801f6abe8e4099e324d9a3deccd8d4384.

After discussion on the mailing list it was felt these
changes were not ready yet.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-07 21:57:16 +01:00
Adrian Freihofer
f15a1f2b7f runqemu-extract-sdk: fix install debugfs on rootfs
At least with my current setup and acc. the manual, the images are named
like image-dbg.rootfs.tar.bz2.

The filter has two bugs:
- expects something like -dbg.tar
- tar without compression suffix is not allowed

(From OE-Core rev: e5fb903db308c508fc44bada89fd0210810301a9)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-06 16:45:11 +01:00
Ricardo Ribalda Delgado
242412656b wic: Add --embed-rootfs argument
This option adds the content of a rootfs on a specific location on the
rootfs.

It is very useful for making a partition that contains the rootfs for a
host and a target Eg:

/ -> Roofs for the host
/export/ -> Rootfs for the target (which will netboot)

Although today we support making a partition for "/export" this might
not be compatible with some upgrade systems, or we might be limited by
the number of partitions.

With this patch we can use something like:

part / --source rootfs --embed-rootfs target-image /export --embed-rootfs target-image2 /export2

on the .wks file.

(From OE-Core rev: efdcf94801f6abe8e4099e324d9a3deccd8d4384)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-06 16:45:11 +01:00
Ricardo Ribalda Delgado
6bac089383 wic: Fix permissions when using exclude or include path
When parameters include_path or exclude_path are passed to the rootfs
plugin, it will copy the partition content into a folder and make all
the modifications there.

This is done using copyhardlinktree(), which does not take into
consideration the content of the pseudo folder, which contains the
information about the right permissions and ownership of the folders.

This results in a rootfs owned by the user that is running the wic
command (usually UID 1000), which makes some rootfs unbootable.

To fix this we copy the content of the pseudo folders to the new folder
and modify the pseudo database using the "pseudo -B" command.

(From OE-Core rev: 36993eea89d1c011397b7692b9b8d61b499d0171)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-06 16:45:11 +01:00
Tim Orling
58796dc18f scripts/install-buildtools: refactor for Python 3.4
Our least common denominator supported distro is debian-8
which has python 3.4. The whole point of the install-buildtools
script is to make it easier on the user to install buildtools
tarball. So it needs to run on Python 3.4.

The way we checked if the install was successful in the prior
version of the script was not workable in python 3.4. Since
the environment-setup-... script is currently just exporting
environment variables, use os.environ to do the equivalent from
values gleaned via regex from the environment-setup-... file.

Corrected a couple minor whitespace errors

NOTE: License changed to GPL-2.0-only due to inclusion of code
copied directly from bitbake/lib/bb/utils.py. This avoids the
need to depend on bitbake, which is now Python 3.5+ only.

(From OE-Core rev: 869020dac889e9ed79a294f308a87cfd946a68bd)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-05 11:46:38 +01:00
Lee Chee Yang
b5a3625e59 wic: align layer plugin path
wic look for external layer source plugin under scripts/lib/
while other tools look for lib/.

allow wic to check for source plugin at both scripts/lib/ and
lib/ to align with other tools while avoid breaking any existing
source plugin in external layer.

[YOCTO #13056]

(From OE-Core rev: 986baff26bd96a6265f5fe2d631818fff9f66374)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-02 15:24:59 +01:00
Richard Purdie
02eacce410 scripts/oe-buildenv-internal: Add pointer to scripts/install-buildtools
If the minimum python version isn't met, show a pointer to the new
install-buildtools script.

(From OE-Core rev: 38e3d5bd3d05ed00a2fc55e3729cb8a6d4e4132f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-01 11:44:24 +01:00
Tim Orling
4787c97874 install-buildtools: bump default to yocto-3.1_M3, fixes
Add ability to check md5sum (yocto-3.1_M2 and before) or sha256
(yocto-3.1_M3 and beyond).

Make regex for path in checksum file optional, since
for yocto-3.1_M3 the format is <checksum>  <filename>,
but prior releases was <checksum>  <path><filename>

(From OE-Core rev: cb1c98f38755b8340140125064c21e407f39db74)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-01 11:44:24 +01:00
Tim Orling
1834a0ce44 oe-buildenv-internal: python 3.5 as min version
Python 3.4 is EOL:
https://www.python.org/downloads/release/python-3410/

The last supported distro was probably CentOS-7, which has python36 available
from epel-7 or scl (as rh-python36) [1]

[1] https://www.softwarecollections.org/en/scls/rhscl/rh-python36/

(From OE-Core rev: 20cdec283a2ee39daa2ffaace11daa764ba40606)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-01 11:44:23 +01:00
Tim Orling
420c926f8a scripts/install-buildtools: improvements
* Install directory defaults to scripts/../buildtools
  e.g. --directory is set by default
  This avoids the user having to type in their sudo password
  to install in /opt/poky/<installer-version>

* Use "." rather than "source" for sourcing the environment script
  as not all distros (e.g. Debian) have "source" by default.

* Add buildtools/ to .gitignore

* Fix typos in example usage (--install-version -> --installer-version)

[YOCTO #13832]

(From OE-Core rev: c6c3a58dbf0ca6c4a41df7ff50fa56d39d7ee23f)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-01 11:44:23 +01:00
Alex Kiernan
e3037b54ea systemd: Reinstate systemd-hwdb-update.service
systemd supports a distribution hwdb.bin in /usr/lib/udev/hwdb.bin,
which is used if /etc/udev/hwdb.bin is not present. When generating the
install time hwdb, for systemd, ensure that we put it in /usr/lib/udev,
which then ensures that at boot time we do not regenerate it, unless the
system is marked for update.

This allows fragments dropped into /etc/udev/hwdb.d to be processed
correctly, but without requiring a first boot time build:

root@qemumips:~# systemctl status systemd-hwdb-update.service
* systemd-hwdb-update.service - Rebuild Hardware Database
     Loaded: loaded (/usr/lib/systemd/system/systemd-hwdb-update.service; static; vendor preset: disabled)
     Active: inactive (dead)
  Condition: start condition failed at Wed 2020-03-04 15:18:11 UTC; 44s ago
             |- ConditionPathExists=|!/usr/lib/udev/hwdb.bin was not met
             |- ConditionPathExists=|/etc/udev/hwdb.bin was not met
             `- ConditionDirectoryNotEmpty=|/etc/udev/hwdb.d was not met
       Docs: man:hwdb(7)
             man:systemd-hwdb(8)

(From OE-Core rev: 78ff610e571e4d739326520c391a37ee1f1290ae)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Acked-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-31 13:58:01 +01:00
rpjday@crashcourse.ca
27b30835a4 yocto-check-layer: simple grammatical fixes
(From OE-Core rev: 15ad048ffa5a80e81b4e04bf98e14e8e23e409f9)

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-30 12:33:29 +01:00
He Zhe
2b8dd4ab6d runqemu: Change nfs to mount via TCP
Since kernel commit b24ee6c64ca7 ("NFS: allow deprecation of NFS UDP protocol"),
NFS UDP has been disabled by default due to the potential data corruption caused
by fragmentation during high loads. So now we cannot boot up with nfs mode and
default kernel.

We'd better turn to use TCP accordingly.

(From OE-Core rev: 73a54595d8ddf237d685ea9cb46df0cecac0280d)

Signed-off-by: He Zhe <zhe.he@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-29 20:04:10 +01:00
Tim Orling
03d54a42a3 scripts/install-buildtools: add helper script to install buildtools
For distros such as CentOS-7 where the default buildtools are too
old we need to make it easy for users to install a pre-built SDK
with all of "build-essentials" included.

Other uses may include building older Yocto Project releases with
a distro where buildtools are too new.

For convenience, the standard buildtools installation is also
supported.

NOTE: extended buildtools is the default, e.g.
      --with-extended-buildtools is on by default

Example usage (extended buildtools from milestone):
  (1) using --url and --filename
      $ install-buildtools \
        --url http://downloads.yoctoproject.org/releases/yocto/milestones/yocto-3.1_M2/buildtools \
        --filename x86_64-buildtools-extended-nativesdk-standalone-3.0+snapshot-20200122.sh
  (2) using --base-url, --release, --installer-version and --build-date
      $ install-buildtools \
        --base-url http://downloads.yoctoproject.org/releases/yocto \
        --release yocto-3.1_M2 \
        --install-version 3.0+snapshot
        --build-date 202000122

Example usage (standard buildtools from release):
  (3) using --url and --filename
      $ install-buildtools --without-extended-buildtools \
        --url http://downloads.yoctoproject.org/releases/yocto/yocto-3.0.2/buildtools \
        --filename x86_64-buildtools-nativesdk-standalone-3.0.2.sh
  (4) using --base-url, --release and --installer-version
      $ install-buildtools --without-extended-buildtools \
        --base-url http://downloads.yoctoproject.org/releases/yocto \
        --release yocto-3.0.2 \
        --install-version 3.0.2

[YOCTO #13832]

(From OE-Core rev: 2d0aea6a73c427ce6aa17dc71e0783977a52bb2b)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-29 12:05:31 +01:00
Peter Kjellerstedt
325b820dae buildstats-plot.sh: Show spaces and underscores correctly in titles/keys
Spaces were previously converted to dashes, and underscores caused the
next character in the title/key to be printed using subscript due to the
enhanced string support in gnuplot.

(From OE-Core rev: 1719f7062988889b7e24b871dc2f0e50c7bff07e)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-29 12:05:31 +01:00
Luis Martins
719c14c984 buildstats*.sh: fix spacing indentation in the files
Fix indentation in buildstats.sh and buildstats-plot.sh
to convert every line to 4 spaces indentation.

(From OE-Core rev: 3971572042ae9bc8226eb92892bb71010287e0a8)

Signed-off-by: Luis Martins <luis.martins@criticaltechworks.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-24 16:39:40 +00:00
Luis Martins
41b145b2b1 buildstats*.sh: add recipe parameter to scripts
Add option to filter the stats results for a single recipe, allowing
to easilly visuality the task resource consumption per recipe.
This is specially useful when debugging the performance of
the overall bitbake build system.

(From OE-Core rev: b4bf898d53af60f5d69b50ec5cdd7f9b3870f2dd)

Signed-off-by: Luis Martins <luis.martins@criticaltechworks.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-24 16:39:40 +00:00
Luis Martins
3bd3e23304 buildstats*.sh: add accumulate parameter to scripts
Add option to accumulate stats values per recipe, allowing
to sum related values such as memory (main process + childs).
This is specially useful when debugging the performance of
the overall bitbake build system.

(From OE-Core rev: e6f1aea6260343cb5194f7f8ab70213b705ab441)

Signed-off-by: Luis Martins <luis.martins@criticaltechworks.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-24 16:39:40 +00:00
Luis Martins
fe1e4a4771 buildstats-plot.sh: filter by task
Extend buildstats-plot.sh script to also accept the name
of the tasks as parameter.
This value will be passed directly to buildstats.sh is
already provides this option.

(From OE-Core rev: 495b6d3d85d2d14d54e324d8da43311a23fdfca6)

Signed-off-by: Luis Martins <luis.martins@criticaltechworks.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-24 16:39:40 +00:00
Luis Martins
5ea725cd5a buildstats-plot.sh: fix invoking buildstats.sh
Fix the internal call to buildstats.sh by also providing
the buildstats folder location in the system, which might
differ from the default location.

(From OE-Core rev: 9f4ee2ee8712528ac51ef4ecd17ccde737e8b21d)

Signed-off-by: Luis Martins <luis.martins@criticaltechworks.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-24 16:39:40 +00:00
Chee Yang Lee
590555e3b8 wic/direct: reserve 2 sector for extended partition
extended partition should reserve at least 2 unallocated sectors
between the start of the extended partition and the start of a
logical partition.

[YOCTO #13658]

(From OE-Core rev: 884c8a0e50a49bdca5e048197a9dc0ff37cc8d1a)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-24 16:39:40 +00:00
Adrian Freihofer
2f6a47c4d4 runqemu: support multiple NICs
Emulating more than one network interface with runqemu is a bit tricky,
but possible. For example, the following leads to an emulated device with
eth0 and eth1:

QB_NETWORK_DEVICE_prepend = " \
    -device virtio-net-device,mac=52:54:00:12:34:03 \
"

or

QB_NETWORK_DEVICE_append = " \
    -device virtio-net-pci,mac=52:54:00:12:34:03 \
"

When booting Qemu with two NICs, the kernel does not know which
interface the specified ip=192.168.7.... command line argument
should be applied. This delays the boot process for a very long
time and a guest wihtout IP configuration.

This add two new configuraton parameters to runqemu:
QB_CMDLINE_IP_SLIRP and QB_CMDLINE_IP_TAP to explicitely specify the ip=
kernel command line arguments for tap and slirp mode.

Note: Simply adding "::eth0" broke some builds on the Yocto autobuilder.

(From OE-Core rev: 59bfdc331c1494c05ab38804b281878a1f571f6d)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-21 22:39:00 +00:00
Richard Purdie
9194c6c249 scripts/pybootchartgui: Fix to work with python 3.8
time.clock() was removed in python 3.8, use one of its recommended replacements
to fix failures on python 3.8 systems.

(From OE-Core rev: f1d4ecce7d3968c5e08c268e3329041d4b647cf6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-15 22:23:19 +00:00
jan
7e9fdd755d scritpts/tiny/ksize: Cleanup some python2 leftover in ksize.py
Although the script was running with Python3, almost no detailed
output was generated.

(From OE-Core rev: bd17a7e60264bb40df340fa38c425a7d7372516e)

Signed-off-by: Jan Vermaete <jan.vermaete@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-13 11:32:10 +00:00
Richard Purdie
2150c6cb70 scripts/oe-buildenv-internal: Add BB_LOGCONFIG
We should allow the logging configurations to be specificed from the
environment, for example for autobuilder setups.

(From OE-Core rev: 66c9234b71791d2aab87068496aebdbd4eb1f5e2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-12 22:53:40 +00:00
Armin Kuster
26525f2f4f wic/engine: lets display an error not a traceback
If the requested partition does not exist in this request "wic ls {path}:pnum"
display a nice message not a trackback

Also fix displaying the pnum and not "%s"

(From OE-Core rev: 15d1722950a22649905cf8a5789d3cfe48a2a892)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-02 21:00:54 +00:00
Pierre-Jean Texier
0c8892fa1d wic/engine: dump the partition table in JSON format only
since commit 03154d2cf25c1d5ce908da9e72d324004b8d5722 ("sfdisk: add -J between mutually exclusive options")
in util-linux [1], the '-J' can't be used with the '-d' option anymore.

So, since we're using json format output, drop the '-d' option.

Fixes:

ERROR: _exec_cmd: <snip>/usr/sbin/sfdisk -dJ <snip>/core-image-minimal-qemux86-64.wic returned '1' instead of 0
output: sfdisk: mutually exclusive arguments: --list-free --json --dump

[1] - 03154d2cf2 (diff-31d5cc325380dbdafe64d0e019846f12)

(From OE-Core rev: 49e7f3e8e3a775510d8b0e2826a06d000934706b)

Signed-off-by: Pierre-Jean Texier <pjtexier@koncepto.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-01 10:40:11 +00:00
Richard Purdie
1f5736dbfb scripts/combo-layer: Fix deprecation warning
Resolve:
combo-layer:83: DeprecationWarning: This method will be removed in future versions.  Use 'parser.read_file()' instead.

(From OE-Core rev: 7b43e04424985cf71b9263969830c9e4e9d72e01)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-21 09:39:00 +00:00
Lori Hikichi
be53b56b0c rpm2cpio.sh: fix to handle newline character
This script is attempting to read binary data from an rpm file.
If any of the bytes it is attempt to read is the binary value 0xa
(i.e. a newline) then the script does not properly deal with this
special case. Due to the behavior of command substitution, instead of
fetching the value 0xa, the script makes an error and get the value 0
instead.

The fix and the following explantion was taken from this post...

https://unix.stackexchange.com/questions/10801/
how-to-use-bash-script-to-read-binary-file-content

Command substitution $(…) strips final newlines in the command output.
There's a fairly easy workaround.  Make sure the output ends in a
character other than a newline, then strip that one character.

(From OE-Core rev: a40a93e81766513cf710e713093ab74c6ec936c3)

Signed-off-by: Lori Hikichi <lori.hikichi@broadcom.com>
Signed-off-by: Scott Branden <scott.branden@broadcom.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-19 11:24:15 +00:00
Robert P. J. Day
e1a2a5a206 fix various, innocuous typos
Corrections:

  - environment
  - accommodate
  - conversion
  - compatible

(From OE-Core rev: 9797d3b45b4e1b9d77f0f2ee299c17b48d8d3cf6)

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18 23:53:54 +00:00
Alexander Kanavin
e88fe83014 qemux86: do not add vga=0 to kernel parameters
This was added ages ago to enable GL passthrough with
vmware driver, and is no longer relevant, as std or virgl is used
instead nowadays.

Original commit:

commit 072545b111
Author: Richard Purdie <rpurdie@linux.intel.com>
Date:   Wed Jan 21 17:40:51 2009 +0000

    scripts/poky-qemu-internal: Add support for GL passthrough in qemux86 images

(From OE-Core rev: 857078ba8eda153f4a097683db551a7d310ecc01)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18 23:53:54 +00:00