Commit Graph

3215 Commits

Author SHA1 Message Date
Paul Barker
7655375e15 wic: Add workdir argument
This allows the path for the temporary workdir used by wic to be set
when running wic from bitbake or directly from the command line.

(From OE-Core rev: 447643eebff00715bd9717ebcb75edb20fc69778)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2e7314ac3a3cab89de93c932e8efc75de0feb0a7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
(cherry picked from commit a2308800b7eb5215982db7269d74d6292075f19f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-03 14:13:54 +00:00
Peter Kjellerstedt
951aa2a088 wic: Pass canonicalized paths in PSEUDO_IGNORE_PATHS
(From OE-Core rev: 4117e497650a0d8e7709439385c926d58a7d9aca)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 55618cbd58d6784a82e773f323723be6f722232f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-03 14:13:53 +00:00
Richard Purdie
18b5662c22 wic: Handle new PSEUDO_IGNORE_PATHS variable
Adjust wic to correctly handle the new PSEUDO_IGNORE_PATH variable and avoid
inode corruption issues.

(From OE-Core rev: 16fbe45f2e0e7621139ae26ace59889a91fceda2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 13500f5234361385c365c7c35e83f99435500481)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-03 14:13:53 +00:00
Ricardo Ribalda Delgado
a8d85a74bd wic: Avoid creating invalid pseudo directory
If the source of the rootfs is not a bitbake cooked image, or it is not
pointing to the root of one, we call pseudo again, which will produce
a new pseudo folder at rootfs/../pseudo

Eg:

part /etc --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/home --fstype=ext4

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: b4393f56069801c014e736100dcdeab77e549f08)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 799a24ae78655f7a3eda7456b1a0ffaf3e43ec16)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-03 14:13:53 +00:00
Ricardo Ribalda Delgado
30fff836b9 wic: Fix multi images .wks with bitbake
In order to support .wks files with multiple images inside bitbake we
need to explicitly set the pseudo database in use.

Eg: If we try this .mks:
part / --source rootfs --ondisk sda --fstype=ext4
part /export --source rootfs --rootfs=core-image-minimal-mtdutils --fstype=ext4

The username for all the files under /export will be set to the runner
of bitbake (usually UID 1000).

Before we run wic, we need to make sure that the pseudo database will be
flushed, and contains all the data needed.

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: 63bebeaf60757967b47cae1e24aa61ec5bcbd61b)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit dde90a5dd2b22a539095d1bac82acc15c6380ac8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-03 14:13:53 +00:00
Ricardo Ribalda Delgado
3b06f25a80 wic: Fix permissions when using exclude or include path
When parameters include_path or exclude_path are passed to the rootfs
plugin, it will copy the partition content into a folder and make all
the modifications there.

This is done using copyhardlinktree(), which does not take into
consideration the content of the pseudo folder, which contains the
information about the right permissions and ownership of the folders.

This results in a rootfs owned by the user that is running the wic
command (usually UID 1000), which makes some rootfs unbootable.

This bug can be easily triggerd with the following .wks

part / --source rootfs --fstype=ext4 --exclude-path=home

And this sequence:

$ wic create test-permissions -e core-image-minimal -o test/
$ sudo mount test/test-permissions-202004080823-sda.direct.p1 /mnt
$ ls -la /mnt/etc/shadow

To fix this we copy the content of the pseudo folders to the new folder
and modify the pseudo database using the "pseudo -B" command.

If the rootfs is not a rootfs generated by bitbake a warning is shown
making the user aware that the permissions on the target might not match
what he expects.

WARNING: /tmp/test/../pseudo folder does not exist. Usernames and permissions will be invalid

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: 9ea7162e9efc29beacdf1c8f4ec98f4895dd5df6)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit fd739c15cdba221d9d497d3402b7d929c0be2ca4)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-03 14:13:53 +00:00
Richard Purdie
4e7c21160b base/bitbake.conf: Enable pseudo path filtering
This is a pretty big change to the way pseudo operates when used in OpenEmbedded.
Normally, pseudo monitors and logs (adds to its database) any file created or
modified whilst in a fakeroot environment. There are large numbers of files
we simply don't care about the permissions of whilst in that fakeroot context,
for example ${S}, ${B}, ${T}, ${SSTATE_DIR}, the central sstate control
directories,

This change uses new functionality in pseudo to ignore these directory trees,
resulting in a cleaner database with less chance of "stray" mismatches if files
are modified outside pseudo context. It also should reduce some overhead from
pseudo as the interprocess round trip to the server is avoided.

There is a possible complication where some existing recipe may break, for
example, we found a recipe which was writing to "${B}/install" for
"make install" in do_install and since we listed ${B} as not to be tracked,
there were errors trying to chown root for files in this location.

This patch fixes a few corner cases in OE-Core when used with this new
ignore list:

* The archiver directory matched a "${WORKDIR}/deploy*" pattern so was renamed
  to something else since that directory does need its root permissions
* The ${S} and ${B} ignoring is conditional on them being different to ${WORKDIR}
* package_write_* task output (the debs/rpms/ipks) are now owned by the build
  user so we don't want the file ownership information in the hashequiv outhash
  calculation even if they are built under pseudo.
* The fontcache postinstall intercept is run under qemu outside of pseudo context
  so delete files it may delete up front where pseudo can see this.
* SSTATE_DIR is in PSEUDO_PATHS_IGNORE, which is in FAKEROOTENV which is cached
  by bitbake. We therefore need to trigger reparsing if this changes, which means
  SSTATE_DIR can be in BB_HASHBASE_WHITELIST but not BB_HASHCONFIG_WHITELIST.
  Rework the variables to handle this. This otherwise breaks some of our sstate
  tests in oe-selftest.
* Ignore the temp directory wic uses for rebuilding rootfs.

(From OE-Core rev: 61f8cf0940f8b05e9399a062eddb8055ea69bc5e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ad8f5532ffaead9a5ad13e1034fe9e5e1b7979f4)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-03 14:13:53 +00:00
Adrian Herrera
809ca20ce1 scripts: oe-run-native, fix *-native directories
This fixes a crash with "find" when running a native tool and *-native
directories do not exist under the binary directory in the sysroot.
This happened because the directory wildcard was passed as part of the
root directory.
The directory wildcard is now passed by "-name", which returns an empty
result if no matching directory.

(From OE-Core rev: d5cb96e3b5501b7349d075c47636902c9ce1c49c)

Signed-off-by: Adrian Herrera <adrian.herrera@arm.com>
Change-Id: Iba7acd8bbd7e0beb4d25c984f6af7a4fd21486e6
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f6c90ed0ad24b7d4f892e22e088b1578824eb1d3)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-21 23:08:16 +00:00
Daniel Ammann
42fe4790dd wic: fix typo
(From OE-Core rev: 42f2312ec723c8e6559cd344e4afee4bbdaf9f41)

Signed-off-by: Daniel Ammann <daniel.ammann@bytesatwork.ch>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9ab4c4571d899d2eefcd2fc53af0851863f29008)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-13 08:57:57 +00:00
Lee Chee Yang
f00be81c5b wic/direct/kparser: ensure fsuuid for vfat and msdos align with format
vfat/msdos filesystem should have fsuuid in format 0xYYYYYYYY where "0x"
in front follow with 8 hexadecimal number in uppercase. In wic, when using
custom fsuuid for vfat/msdos partition in wks, it is able to set the value
in any length, with or without leading "0x". This can cause fsuuid
missaligned when fstab updates, fstab expect exactly 10 character
fsuuid for vfat/msdos partition and all in uppercase.

if custom fsuuid for vfat/msdos is set, check the length and format,
error if it exceed the format size. Amend it so it is align with format
0xYYYYYYYY. This is done before image create and fstab update to ensure the
fsuuid are same in all followup process. if custom fsuuid length less than
expected, fill in "0".

[YOCTO #14161]

(From OE-Core rev: 974e09f3460a27c85a599d8269e3dea66df5ddd2)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d9686ae511ef10a504becfd81bfe296b788e1456)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-09 09:17:17 +00:00
Ross Burton
5d93870049 devtool: remove unused variable
This variable isn't used at all.

(From OE-Core rev: 7317bc23d55dfd0068910e3d8524a284cc2bacd5)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9676ee24d190b144cb50be6ebde3990644a6df2a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-18 14:20:19 +00:00
Mingli Yu
7153e26724 update_udev_hwdb: clean hwdb.bin
Steps to reproduce:
echo "IMAGE_INSTALL_append = \" udev-hwdb lib32-udev-hwdb\"" >> conf/local.conf

When install both udev-hwdb and lib32-udev-hwdb as above,
there comes below do_populate_sdk error:
 $ bitbake core-image-sato  -c populate_sdk
 ERROR: Task (/path/core-image-sato.bb:do_populate_sdk) failed with exit code '134'
 NOTE: Tasks Summary: Attempted 5554 tasks of which 0 didn't need to be rerun and 1 failed.

 $ cat /path/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r5/pseudo/pseudo.log
 [snip]
 inode mismatch: '/path/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r5/sdk/image/usr/local/oecore-x86_64/sysroots/core2-64-poky-linux/lib/udev/hwdb.bin' ino 427383040 in db, 427383042 in request.
 [snip]

It is because both udev-hwdb and lib32-udev-hwdb will generate
${SDK_OUTPUT}/${SDKTARGETSYSROOT}/lib/udev/hwdb.bin during do_populate_sdk
and it triggers pseudo error.

So clean hwdb.bin before generate hwdb.bin to avoid conflict to
fix the above do_populate_sdk error.

(From OE-Core rev: 10413cf013ae5a92ef12f33dab8e67f0bfb0ae25)

Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c7472925feb53ce92c1799feba2b7a9104e3f38f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-12 13:07:52 +00:00
Richard Purdie
7a2bbca3da oe-build-perf-report: Ensure correct data is shown for multiple branch options
If --branch and --branch2 are the same, there is extra confusing data plotted
onto the graph. Only do that if the branches are actually different.

(From OE-Core rev: e2240d0a93a9a2932506a356315ad6702ecee9b6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c263d810b7eb47ee90f2adeb5ab6decf7332bec2)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-22 12:16:42 +01:00
Richard Purdie
97ac0a2f7c scripts/oe-build-perf-report: Allow operation with no buildstats
If buildstats is missing, avoid a backtrace.

(From OE-Core rev: a518443e6799d56b899536d853282182eff4822a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 23c4b21f947c7aae1303ca6526fc5aaaa9fc7bb8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-22 12:16:41 +01:00
Joshua Watt
181eeebd95 wic: Add 512 Byte alignment to --offset
Allows the --offset argument to use the "s" or "S" suffix to specify
that it is reporting the number of 512 byte sectors.

This is required for some SoCs where the mask ROM looks for an item at a
sector that isn't aligned to a 1KB boundary.

(From OE-Core rev: b9296bdeaacc1dce97aac9c9bf0d70555bb36646)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 938595d1dc4abaf5f7f3a7900add3f0492b805d0)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-06 14:15:22 +01:00
Joshua Watt
3c4f891402 wic: Fix error message when reporting invalid offset
The error message was reporting the calculated offset instead of the
current offset, which made it confusing.

(From OE-Core rev: 0ff25797126772f40e357a2f3cf81eccf659adaf)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2be775cfe1b49ce3889b5dc326e2b67a9667f18a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-06 14:15:22 +01:00
Joshua Watt
c52e34c2bf wic: Fix --extra-space argument handling
467f84e12b ("wic: Add --offset argument for partitions") broke the
--extra-space argument handling in wic. Fix the option and add a unit
test for the argument.

(From OE-Core rev: 62a7a10c31bcf133cbd99d4de928f15a30e45ab1)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 87722a92c18f94917c8f70afc8cd0763462a5c25)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-06 14:15:22 +01:00
Joshua Watt
343a7e2c61 wic: Add --offset argument for partitions
Add support for an --offset argument when defining a partition. Many
SoCs require that boot partitions be located at specific offsets. Prior
to this argument, most WKS files were using the --align attribute to
specify the location of these fixed partitions but this is not ideal
because in the event that the partition couldn't be placed in the
specified location, wic would move it to the next sector with that
alignment, often preventing the device from booting. Unlike the --align
argument, wic will fail if a partition cannot be placed at the exact
offset specified with --offset.

Changes in V2:
* Fixed a small typo that prevented test_fixed_size_error from passing

(From OE-Core rev: 897aaff8961f7fe83634a3b0b94e19b43aea5857)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 467f84e12b96bc977d57575023517dd6f8ef7f29)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-06 14:15:22 +01:00
Richard Purdie
5ad5949578 scripts/oe-build-perf-report: Use python3 from the environment
This means ot matches the rest of the system rather than causing confusing errors
where multiple python versions are mixed.

(From OE-Core rev: 95d5005828b0d32d610874986fcc298b12347079)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-23 12:47:59 +01:00
Adrian Freihofer
98905dd827 oe-publish-sdk: fix layers init via ssh
Escaping does not work in my use case. It must be escaped for
python, ssh and shell as well as for different versions of echo.
Let's try it a little less elegant, but hopefully more reliable.

(From OE-Core rev: cdbc9bf63b3e1354800032d0dd530949bafb7032)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5cc1ae332eb6b05d83802c8d64ab2767c7079412)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-17 12:51:24 +01:00
Richard Purdie
54ef7e8750 runqemu: Add a hook to allow it to renice
We have an issue where qemu is being starved of resources on our autobuilders.
We can't raise its priority without special capacilties, therefore add a hook
which if present can allow this to happen using an executable
"~/runqemu-renice".

(From OE-Core rev: 829443b3fafa000e330c06ebb144641bd2d20ad1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 141a3c9ce93bc3d526303021ecf0460c6e9fea8a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-17 12:51:24 +01:00
Michael Tretter
dfd88e61b2 devtool: deploy-target: Fix size calculation for hard links
If a package contains hard links to a file, the file size is added for
each hard link instead of once for the file. Therefore, the calculated
size may be much larger than the actual package size.

For example, the mesa-megadriver package contains several hard links to
the same library.

Keep track of the inode numbers when listing the files that are
installed and use the actual size only for the first occurrence of an
inode. All further hard links to the same inode are added to the file
list, but accounted with size 0.

All file names need to be added to the file list, because the list is
used for preserving the files/hard links on the target.

(From OE-Core rev: 85b1e835572d184cfff86a87fced8673d1d40a7c)

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 17e92572278980d1a7f06de9d72c68baf57698f1)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-10 19:07:40 +01:00
Vijai Kumar K
5e01f32330 wic: misc: Add /bin to the list of searchpaths
/bin is also a valid path where one can find executables. Add
that to the search path.

(From OE-Core rev: 2427c872baf1be5ab9e9fcc0a17ee3b267db0edd)

Signed-off-by: Vijai Kumar K <vijaikumar.kanagarajan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ca0a6025351cb2135e87cecf828633cf12aa34c6)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-10 19:07:40 +01:00
Martin Jansa
fb739bf016 devtool: expand SRC_URI when guessing recipe update mode
* I have recipes which use variable inside SRC_URI, e.g.:
  ROS_BRANCH ?= "branch=release/melodic/swri_nodelet"
  SRC_URI = "git://github.com/swri-robotics-gbp/marti_common-release;${ROS_BRANCH};protocol=https"

  and devtool modify works fine, but devtool finish fails with:

  $ devtool finish --force-patch-refresh swri-nodelet meta-ros/meta-ros1-melodic/
  ...
  Traceback (most recent call last):
    File "/OE/openembedded-core/scripts/devtool", line 334, in <module>
      ret = main()
    File "/OE/openembedded-core/scripts/devtool", line 321, in main
      ret = args.func(args, config, basepath, workspace)
    File "/OE/openembedded-core/scripts/lib/devtool/standard.py", line 2082, in finish
      updated, appendfile, removed = _update_recipe(args.recipename, workspace, rd, args.mode, appendlayerdir, wildcard_version=True, no_remove=False, no_report_remove=removing_original, initial_rev=args.initial_rev, dry_run_outdir=dry_run_outdir, no_overrides=args.no_overrides, force_patch_refresh=args.force_patch_refresh)
    File "/OE/openembedded-core/scripts/lib/devtool/standard.py", line 1737, in _update_recipe
      mode = _guess_recipe_update_mode(srctree, rd)
    File "/OE/openembedded-core/scripts/lib/devtool/standard.py", line 1721, in _guess_recipe_update_mode
      params = bb.fetch.decodeurl(uri)[5]
    File "/OE/bitbake/lib/bb/fetch2/__init__.py", line 390, in decodeurl
      raise MalformedUrl(url, "The URL: '%s' is invalid: parameter %s does not specify a value (missing '=')" % (url, s))
  bb.fetch2.MalformedUrl: The URL: 'git://github.com/swri-robotics-gbp/marti_common-release;${ROS_BRANCH};protocol=https' is invalid: parameter ${ROS_BRANCH} does not specify a value (missing '=')

  let it expand the SRC_URI before trying to decode it.

(From OE-Core rev: 2de84e6e767d4c3e517d7413a37700bf1991a460)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3535cfdbf3d77f550b804276f957acf859da484f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-10 19:07:40 +01:00
Richard Purdie
8554139871 runqemu: Show an error for conflicting graphics options
The autobuilder managed to pass the nographic option with "sdl gl" due
to a problem elsewhere. It would have been useful for runqemu to have
errored rather than passing conflicting options to qemu. Add an
error for this invalid usecase.

(From OE-Core rev: e62c8207fa47ff4e325529ea84e0d51710deaa3c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 377a6ffbcba0d3c5ede470c989756c4d1636873f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-02 16:04:37 +01:00
Vasyl Vavrychuk
4fe9a766d6 runqemu: Check gtk or sdl option is passed together with gl or gl-es options.
runqemu help reports that gtk or sdl option is needed with gl or gl-es
option. But if user forgot to add gtk or sdl option, then gl or gl-es
options were silently skipped.

(From OE-Core rev: bec0a45393d968251059f5075add2cf633aecd1a)

Signed-off-by: Vasyl Vavrychuk <vvavrychuk@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4d1e93d4bf013bb0c48032bfda43f77c5aba9ecf)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-27 08:25:50 +01:00
Lee Chee Yang
75fdc94ecb checklayer: check layer in BBLAYERS before test
layer under test should absent from BBLAYERS when running
yocto-check-layer. This allow to get signatures before layer
under test. There are existing steps to add the layer under
test to BBLAYERS after getting initial signatures.

add steps to check for layer under test in BBLAYERS before
running any test, skip test for the layer if the layer under
test exist in BBLAYERS.

[YOCTO #13176]

(From OE-Core rev: a10ff925584406e563edfb4042b44c3e8ea57ce3)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit be02e8dbfb0d1decce125322f9f1e11a649756c0)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-07 21:20:21 +01:00
Kevin Hao
716fe312f5 wic/filemap: Fall back to standard copy when no way to get the block map
For some filesystems, such as aufs which may be used by docker container,
don't support either the SEEK_DATA/HOLE or FIEMAP to get the block
map. So add a FileNobmap class to fall back to standard copy when there
is no way to get the block map.

[Yocto #12988]

(From OE-Core rev: 66f9db48bb9d59f08492f0515bc08b6b039aa03f)

Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 7934ed49179242f15b413c0275040a3bb6b68876)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-31 22:02:52 +01:00
Kevin Hao
1a24aed905 wic/filemap: Drop the unused get_unmapped_ranges()
This method is not used by any code, so drop it.

(From OE-Core rev: 19078f9540c455ccddb6f2986b96562a8186ec35)

Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6513fd9302b9989f97fc9d95e76e06ad5d266774)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-31 22:02:52 +01:00
Kevin Hao
1d0fbc64c8 wic/filemap: Drop the unused block_is_unmapped()
This method is not used by any code, so drop it.

(From OE-Core rev: 14cbfc04f4a9da5baba6267daefd54a12a04d56e)

Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit fb62a15349597ee026c67a0bb0a6ca2cc9bfe420)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-31 22:02:52 +01:00
Tim Orling
abe66c9a39 scripts/lib/recipetool/create.py: fix regex strings
Python now expects regex strings to be prepended with r.
Silence pylint/autopep8 and similar warnings by identifying
these regex patterns as... regex patterns.

(From OE-Core rev: f8a5db7a6072ddb1be96405fc8b44f595275206d)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0eae98a369d80340e48dc690d09a1364cde97973)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-22 22:46:37 +01:00
Bjarne Michelsen
2f954473bb devtool: default to empty string, if LIC_FILES_CHKSUM is not available
[Bug 13971] https://bugzilla.yoctoproject.org/show_bug.cgi?id=13971

A recipe using LICENSE=CLOSED would normally not have LIC_FILES_CHKSUM
set.
If LIC_FILES_CHKSUM was not set, it would cause `devtool upgrade` to fail.

This fixes the issue by defaulting to an empty string if
LIC_FILES_CHKSUM is not found.

[YOCTO #13971]

(From OE-Core rev: 9c7f3281884c50dbd05e2966e2409255c127ebfa)

Signed-off-by: Bjarne Michelsen <bjarne_michelsen@hotmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 95e112f0f465868e98285509ea0d1a624a439ac2)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-16 22:28:40 +01:00
Nicolas Dechesne
a225c162e6 checklayer: parse LAYERDEPENDS with bb.utils.explode_dep_versions2()
LAYERDEPENDS is a string of this format:
"DEPEND1 (optional version) DEPEND2 (optional version) ..."

However when we parse LAYERDEPENDS in _get_layer_collections() we
parse it as a simple string, and if any optional versions are there the
'depends' field is wrong. For example, running yocto-check-layer
might result in such errors:

ERROR: Layer meta-python depends on (>= and isn't found.
ERROR: Layer meta-python depends on 12) and isn't found.

Let's use bb.utils.explode_dep_versions2() to parse LAYERDEPENDS, and
create a string that contains all dependencies, effectively
skipping/ignoring any optional versions.

[YOCTO #13957]

(From OE-Core rev: 819f41906197bb712af37349c0865002bfbd7c9b)

Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f81f07afc200fe06c5c06ea81a4f8a3a43436faf)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-02 16:12:37 +01:00
Richard Purdie
c2b1198b79 scripts/install-buildtools: Handle new format checksum files
Autobuilder generated checksum files only have a single space between the sum and the
filename, tweak it to account for this.

(From OE-Core rev: 877a082b5f3f5ccf8191a2438e7707b419174ea9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d1e71bf5b399372166eb40bb0d99c8fb52231600)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-02 16:12:37 +01:00
Richard Purdie
bceae9e7de scripts/install-buildtools: Update to 3.2 M1 buildtools
This fixes issues with openssl certs not working properly which meant error
reporting to an error report server was failing.

Also, all our downloads are now standarised on "sha256sum" so adjust for that.

(From OE-Core rev: bef009c91e8acfc7c1ba3cfd3183f000a02e002b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit bc8b44e19a05f499f5cef049eedbed1fede2e765)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-02 16:12:37 +01:00
Konrad Weihmann
81c46e46a8 runqemu: add QB_ROOTFS_EXTRA_OPT parameter
Content of the optional parameter will be appended to the rootfs-device
in the qemu configuration, in case QB_ROOTFS_OPT is not specified.
By default this is empty.

Example use cases are:

Defining 'readonly' when using squashfs, so multiple instances of qemu
can share the same base image, something that cannot be done by just
specifying 'snapshot'.

Defining 'bootindex=0' which helps to get past the EFI shell in
ovmf-binary. This also enables the use case of running WIC images with
EFI bootloader through the testimage.bbclass.

(From OE-Core rev: 1a5cb1bd56be2b784208516a7c73c49906c7022f)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e9b8c194636cb5505774a2a71bf54450580dd5b8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-02 16:12:36 +01:00
Tuomas Salokanto
ea688fb506 recipetool: create: fix SRCBRANCH not being passed to params
When explicitly passing a branch using --srcbranch in 'devtool add' or
'recipetool create', the branch name is not included in the params of
bb.fetch2.encodeurl and default 'master' branch is used instead.

(From OE-Core rev: 50789b582908f78d9bb2b5a05418433ad8074825)

Signed-off-by: Tuomas Salokanto <tuomas.salokanto@gmail.com>
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0424df825f1e509faf6cd44403c0736bb91b57c3)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-26 18:26:49 +01:00
Daniel McGregor
4e2428464b buildhistory-collect-srcrevs: sort directories
In order to allow consistent output of buildhistory-collect-srcrevs sort
the list of directories returned by os.walk. Otherwise the list of
SRCREVs is returned in an unspecified order.

We save the output of this command on build, so it creates smaller diffs
between builds.

(From OE-Core rev: 9ca9c0fa320caeb2a1e7a60161c4db66dc0f2030)

Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3d7cb207ad4ec3cd5a3064147d3c9b5a1730d0fb)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-26 18:26:49 +01:00
Ross Burton
a3ee59032a install-buildtools: remove hardcoded x86-64 architecture
Remove all instances of the hardcoded 'x86_64' and replace with the current
host platform.

(From OE-Core rev: cd2c54d8ab545f39f23c5167ba5ca50f732f7cfa)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 52dc6f671ff67a1149be7ef4c65126ea3c907a3d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-26 18:26:49 +01:00
Ross Burton
95b86ef779 install-buildtools: fail if an error occurs
Several failure paths were displaying an error message but not returning,
so the install process continued and failed further.

(From OE-Core rev: deed12d01fa656ee0cf81a6b7b9ed74278e48c50)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b00e28735b64a781707441ec6187dd7f9240d97a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-26 18:26:49 +01:00
Alex Kiernan
5d71ab4210 recipetool: Fix list concatenation when using edit
If there are multiple appends, ensure we concatenate compatible things:

Traceback (most recent call last):
  File "/home/akiernan/poky/scripts/recipetool", line 111, in <module>
    ret = main()
  File "/home/akiernan/poky/scripts/recipetool", line 100, in main
    ret = args.func(args)
  File "/home/akiernan/poky/scripts/lib/recipetool/edit.py", line 38, in edit
    return scriptutils.run_editor([recipe_path] + appends, logger)
TypeError: can only concatenate list (not "tuple") to list

(From OE-Core rev: 9a45c7fb2d4491d2d34500acef8ea6dcd4f5d9d4)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4c1e74bdf4922519d168434afd69c9bebcb9bd82)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-26 18:26:48 +01:00
Richard Purdie
53559bb82d resulttool/log: Add ability to dump ltp logs as well as ptest
Currently only ptest logs are accessible with the log command, this
adds support so the ltp logs can be extracted too.

(From OE-Core rev: 0b513274a0ae722065cf1a605090000e854e2f81)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 64a2121a875ce128959ee0a62e310d5f91f87b0d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-05 21:36:30 +01:00
Richard Purdie
212c92482c resulttool/report: Remove leftover debugging
I've long since wondered why there was some odd output in result reports,
remove the leftover debug which was causing it.

(From OE-Core rev: 10d1d2ffa0906561d65886caee44652242139913)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 66e96bf70753933714ff8edcc13a1f35a052656f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-05 21:36:30 +01:00
Joshua Watt
5b65a822d6 checklayer: Skip layers without a collection
As in other places in the file, skip layers that don't define a
collection when searching for a layer to resolve a dependency. Fixes
KeyError exceptions when attempting to access the layer collections
later

(From OE-Core rev: ae65adf471a9ad04c6a44bf020a28f1006db106a)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 26090a2861ebe21224aaf89d7be0c0a89ca58e48)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-05 21:36:30 +01:00
jan
33763b8878 scripts/tiny/ksize: Fix for more recent kernels
In the past kernel built object files were named 'built-in.o'.
Nowadays it is 'built-in.a'.

The script is modified to work with both.  I expect
it will not happen that there are built-in.a and built-in.o
files in the same kernel.

(From OE-Core rev: 8a883c3b0773960908491c03c46e7ed320e41dc5)

Signed-off-by: Jan Vermaete <jan.vermaete@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-22 16:23:24 +01:00
Tim Orling
1246049541 scripts/install-buildtools: bump to 3.1 release by default
By default, use the extended buildtools installer from the
Yocto Project 3.1 "dunfell" release.

(From OE-Core rev: f1d4da322d607a17de6d9c291562b5fd1128fbc6)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-07 13:03:30 +01:00
Richard Purdie
f4e0b26568 Revert "wic: Fix permissions when using exclude or include path"
This reverts commit 36993eea89d1c011397b7692b9b8d61b499d0171.

After discussion on the mailing list it was felt these changes
were not ready yet.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-07 21:57:16 +01:00
Richard Purdie
157a12d499 Revert "wic: Add --embed-rootfs argument"
This reverts commit efdcf94801f6abe8e4099e324d9a3deccd8d4384.

After discussion on the mailing list it was felt these
changes were not ready yet.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-07 21:57:16 +01:00
Adrian Freihofer
f15a1f2b7f runqemu-extract-sdk: fix install debugfs on rootfs
At least with my current setup and acc. the manual, the images are named
like image-dbg.rootfs.tar.bz2.

The filter has two bugs:
- expects something like -dbg.tar
- tar without compression suffix is not allowed

(From OE-Core rev: e5fb903db308c508fc44bada89fd0210810301a9)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-06 16:45:11 +01:00
Ricardo Ribalda Delgado
242412656b wic: Add --embed-rootfs argument
This option adds the content of a rootfs on a specific location on the
rootfs.

It is very useful for making a partition that contains the rootfs for a
host and a target Eg:

/ -> Roofs for the host
/export/ -> Rootfs for the target (which will netboot)

Although today we support making a partition for "/export" this might
not be compatible with some upgrade systems, or we might be limited by
the number of partitions.

With this patch we can use something like:

part / --source rootfs --embed-rootfs target-image /export --embed-rootfs target-image2 /export2

on the .wks file.

(From OE-Core rev: efdcf94801f6abe8e4099e324d9a3deccd8d4384)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-06 16:45:11 +01:00