Commit Graph

1472 Commits

Author SHA1 Message Date
Daniel Ammann
42fe4790dd wic: fix typo
(From OE-Core rev: 42f2312ec723c8e6559cd344e4afee4bbdaf9f41)

Signed-off-by: Daniel Ammann <daniel.ammann@bytesatwork.ch>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9ab4c4571d899d2eefcd2fc53af0851863f29008)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-13 08:57:57 +00:00
Lee Chee Yang
f00be81c5b wic/direct/kparser: ensure fsuuid for vfat and msdos align with format
vfat/msdos filesystem should have fsuuid in format 0xYYYYYYYY where "0x"
in front follow with 8 hexadecimal number in uppercase. In wic, when using
custom fsuuid for vfat/msdos partition in wks, it is able to set the value
in any length, with or without leading "0x". This can cause fsuuid
missaligned when fstab updates, fstab expect exactly 10 character
fsuuid for vfat/msdos partition and all in uppercase.

if custom fsuuid for vfat/msdos is set, check the length and format,
error if it exceed the format size. Amend it so it is align with format
0xYYYYYYYY. This is done before image create and fstab update to ensure the
fsuuid are same in all followup process. if custom fsuuid length less than
expected, fill in "0".

[YOCTO #14161]

(From OE-Core rev: 974e09f3460a27c85a599d8269e3dea66df5ddd2)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d9686ae511ef10a504becfd81bfe296b788e1456)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-09 09:17:17 +00:00
Ross Burton
5d93870049 devtool: remove unused variable
This variable isn't used at all.

(From OE-Core rev: 7317bc23d55dfd0068910e3d8524a284cc2bacd5)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9676ee24d190b144cb50be6ebde3990644a6df2a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-18 14:20:19 +00:00
Joshua Watt
181eeebd95 wic: Add 512 Byte alignment to --offset
Allows the --offset argument to use the "s" or "S" suffix to specify
that it is reporting the number of 512 byte sectors.

This is required for some SoCs where the mask ROM looks for an item at a
sector that isn't aligned to a 1KB boundary.

(From OE-Core rev: b9296bdeaacc1dce97aac9c9bf0d70555bb36646)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 938595d1dc4abaf5f7f3a7900add3f0492b805d0)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-06 14:15:22 +01:00
Joshua Watt
3c4f891402 wic: Fix error message when reporting invalid offset
The error message was reporting the calculated offset instead of the
current offset, which made it confusing.

(From OE-Core rev: 0ff25797126772f40e357a2f3cf81eccf659adaf)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2be775cfe1b49ce3889b5dc326e2b67a9667f18a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-06 14:15:22 +01:00
Joshua Watt
c52e34c2bf wic: Fix --extra-space argument handling
467f84e12b ("wic: Add --offset argument for partitions") broke the
--extra-space argument handling in wic. Fix the option and add a unit
test for the argument.

(From OE-Core rev: 62a7a10c31bcf133cbd99d4de928f15a30e45ab1)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 87722a92c18f94917c8f70afc8cd0763462a5c25)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-06 14:15:22 +01:00
Joshua Watt
343a7e2c61 wic: Add --offset argument for partitions
Add support for an --offset argument when defining a partition. Many
SoCs require that boot partitions be located at specific offsets. Prior
to this argument, most WKS files were using the --align attribute to
specify the location of these fixed partitions but this is not ideal
because in the event that the partition couldn't be placed in the
specified location, wic would move it to the next sector with that
alignment, often preventing the device from booting. Unlike the --align
argument, wic will fail if a partition cannot be placed at the exact
offset specified with --offset.

Changes in V2:
* Fixed a small typo that prevented test_fixed_size_error from passing

(From OE-Core rev: 897aaff8961f7fe83634a3b0b94e19b43aea5857)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 467f84e12b96bc977d57575023517dd6f8ef7f29)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-06 14:15:22 +01:00
Michael Tretter
dfd88e61b2 devtool: deploy-target: Fix size calculation for hard links
If a package contains hard links to a file, the file size is added for
each hard link instead of once for the file. Therefore, the calculated
size may be much larger than the actual package size.

For example, the mesa-megadriver package contains several hard links to
the same library.

Keep track of the inode numbers when listing the files that are
installed and use the actual size only for the first occurrence of an
inode. All further hard links to the same inode are added to the file
list, but accounted with size 0.

All file names need to be added to the file list, because the list is
used for preserving the files/hard links on the target.

(From OE-Core rev: 85b1e835572d184cfff86a87fced8673d1d40a7c)

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 17e92572278980d1a7f06de9d72c68baf57698f1)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-10 19:07:40 +01:00
Vijai Kumar K
5e01f32330 wic: misc: Add /bin to the list of searchpaths
/bin is also a valid path where one can find executables. Add
that to the search path.

(From OE-Core rev: 2427c872baf1be5ab9e9fcc0a17ee3b267db0edd)

Signed-off-by: Vijai Kumar K <vijaikumar.kanagarajan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ca0a6025351cb2135e87cecf828633cf12aa34c6)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-10 19:07:40 +01:00
Martin Jansa
fb739bf016 devtool: expand SRC_URI when guessing recipe update mode
* I have recipes which use variable inside SRC_URI, e.g.:
  ROS_BRANCH ?= "branch=release/melodic/swri_nodelet"
  SRC_URI = "git://github.com/swri-robotics-gbp/marti_common-release;${ROS_BRANCH};protocol=https"

  and devtool modify works fine, but devtool finish fails with:

  $ devtool finish --force-patch-refresh swri-nodelet meta-ros/meta-ros1-melodic/
  ...
  Traceback (most recent call last):
    File "/OE/openembedded-core/scripts/devtool", line 334, in <module>
      ret = main()
    File "/OE/openembedded-core/scripts/devtool", line 321, in main
      ret = args.func(args, config, basepath, workspace)
    File "/OE/openembedded-core/scripts/lib/devtool/standard.py", line 2082, in finish
      updated, appendfile, removed = _update_recipe(args.recipename, workspace, rd, args.mode, appendlayerdir, wildcard_version=True, no_remove=False, no_report_remove=removing_original, initial_rev=args.initial_rev, dry_run_outdir=dry_run_outdir, no_overrides=args.no_overrides, force_patch_refresh=args.force_patch_refresh)
    File "/OE/openembedded-core/scripts/lib/devtool/standard.py", line 1737, in _update_recipe
      mode = _guess_recipe_update_mode(srctree, rd)
    File "/OE/openembedded-core/scripts/lib/devtool/standard.py", line 1721, in _guess_recipe_update_mode
      params = bb.fetch.decodeurl(uri)[5]
    File "/OE/bitbake/lib/bb/fetch2/__init__.py", line 390, in decodeurl
      raise MalformedUrl(url, "The URL: '%s' is invalid: parameter %s does not specify a value (missing '=')" % (url, s))
  bb.fetch2.MalformedUrl: The URL: 'git://github.com/swri-robotics-gbp/marti_common-release;${ROS_BRANCH};protocol=https' is invalid: parameter ${ROS_BRANCH} does not specify a value (missing '=')

  let it expand the SRC_URI before trying to decode it.

(From OE-Core rev: 2de84e6e767d4c3e517d7413a37700bf1991a460)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3535cfdbf3d77f550b804276f957acf859da484f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-10 19:07:40 +01:00
Lee Chee Yang
75fdc94ecb checklayer: check layer in BBLAYERS before test
layer under test should absent from BBLAYERS when running
yocto-check-layer. This allow to get signatures before layer
under test. There are existing steps to add the layer under
test to BBLAYERS after getting initial signatures.

add steps to check for layer under test in BBLAYERS before
running any test, skip test for the layer if the layer under
test exist in BBLAYERS.

[YOCTO #13176]

(From OE-Core rev: a10ff925584406e563edfb4042b44c3e8ea57ce3)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit be02e8dbfb0d1decce125322f9f1e11a649756c0)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-07 21:20:21 +01:00
Kevin Hao
716fe312f5 wic/filemap: Fall back to standard copy when no way to get the block map
For some filesystems, such as aufs which may be used by docker container,
don't support either the SEEK_DATA/HOLE or FIEMAP to get the block
map. So add a FileNobmap class to fall back to standard copy when there
is no way to get the block map.

[Yocto #12988]

(From OE-Core rev: 66f9db48bb9d59f08492f0515bc08b6b039aa03f)

Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 7934ed49179242f15b413c0275040a3bb6b68876)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-31 22:02:52 +01:00
Kevin Hao
1a24aed905 wic/filemap: Drop the unused get_unmapped_ranges()
This method is not used by any code, so drop it.

(From OE-Core rev: 19078f9540c455ccddb6f2986b96562a8186ec35)

Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6513fd9302b9989f97fc9d95e76e06ad5d266774)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-31 22:02:52 +01:00
Kevin Hao
1d0fbc64c8 wic/filemap: Drop the unused block_is_unmapped()
This method is not used by any code, so drop it.

(From OE-Core rev: 14cbfc04f4a9da5baba6267daefd54a12a04d56e)

Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit fb62a15349597ee026c67a0bb0a6ca2cc9bfe420)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-31 22:02:52 +01:00
Tim Orling
abe66c9a39 scripts/lib/recipetool/create.py: fix regex strings
Python now expects regex strings to be prepended with r.
Silence pylint/autopep8 and similar warnings by identifying
these regex patterns as... regex patterns.

(From OE-Core rev: f8a5db7a6072ddb1be96405fc8b44f595275206d)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0eae98a369d80340e48dc690d09a1364cde97973)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-22 22:46:37 +01:00
Bjarne Michelsen
2f954473bb devtool: default to empty string, if LIC_FILES_CHKSUM is not available
[Bug 13971] https://bugzilla.yoctoproject.org/show_bug.cgi?id=13971

A recipe using LICENSE=CLOSED would normally not have LIC_FILES_CHKSUM
set.
If LIC_FILES_CHKSUM was not set, it would cause `devtool upgrade` to fail.

This fixes the issue by defaulting to an empty string if
LIC_FILES_CHKSUM is not found.

[YOCTO #13971]

(From OE-Core rev: 9c7f3281884c50dbd05e2966e2409255c127ebfa)

Signed-off-by: Bjarne Michelsen <bjarne_michelsen@hotmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 95e112f0f465868e98285509ea0d1a624a439ac2)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-16 22:28:40 +01:00
Nicolas Dechesne
a225c162e6 checklayer: parse LAYERDEPENDS with bb.utils.explode_dep_versions2()
LAYERDEPENDS is a string of this format:
"DEPEND1 (optional version) DEPEND2 (optional version) ..."

However when we parse LAYERDEPENDS in _get_layer_collections() we
parse it as a simple string, and if any optional versions are there the
'depends' field is wrong. For example, running yocto-check-layer
might result in such errors:

ERROR: Layer meta-python depends on (>= and isn't found.
ERROR: Layer meta-python depends on 12) and isn't found.

Let's use bb.utils.explode_dep_versions2() to parse LAYERDEPENDS, and
create a string that contains all dependencies, effectively
skipping/ignoring any optional versions.

[YOCTO #13957]

(From OE-Core rev: 819f41906197bb712af37349c0865002bfbd7c9b)

Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f81f07afc200fe06c5c06ea81a4f8a3a43436faf)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-02 16:12:37 +01:00
Tuomas Salokanto
ea688fb506 recipetool: create: fix SRCBRANCH not being passed to params
When explicitly passing a branch using --srcbranch in 'devtool add' or
'recipetool create', the branch name is not included in the params of
bb.fetch2.encodeurl and default 'master' branch is used instead.

(From OE-Core rev: 50789b582908f78d9bb2b5a05418433ad8074825)

Signed-off-by: Tuomas Salokanto <tuomas.salokanto@gmail.com>
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0424df825f1e509faf6cd44403c0736bb91b57c3)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-26 18:26:49 +01:00
Alex Kiernan
5d71ab4210 recipetool: Fix list concatenation when using edit
If there are multiple appends, ensure we concatenate compatible things:

Traceback (most recent call last):
  File "/home/akiernan/poky/scripts/recipetool", line 111, in <module>
    ret = main()
  File "/home/akiernan/poky/scripts/recipetool", line 100, in main
    ret = args.func(args)
  File "/home/akiernan/poky/scripts/lib/recipetool/edit.py", line 38, in edit
    return scriptutils.run_editor([recipe_path] + appends, logger)
TypeError: can only concatenate list (not "tuple") to list

(From OE-Core rev: 9a45c7fb2d4491d2d34500acef8ea6dcd4f5d9d4)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4c1e74bdf4922519d168434afd69c9bebcb9bd82)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-26 18:26:48 +01:00
Richard Purdie
53559bb82d resulttool/log: Add ability to dump ltp logs as well as ptest
Currently only ptest logs are accessible with the log command, this
adds support so the ltp logs can be extracted too.

(From OE-Core rev: 0b513274a0ae722065cf1a605090000e854e2f81)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 64a2121a875ce128959ee0a62e310d5f91f87b0d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-05 21:36:30 +01:00
Richard Purdie
212c92482c resulttool/report: Remove leftover debugging
I've long since wondered why there was some odd output in result reports,
remove the leftover debug which was causing it.

(From OE-Core rev: 10d1d2ffa0906561d65886caee44652242139913)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 66e96bf70753933714ff8edcc13a1f35a052656f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-05 21:36:30 +01:00
Joshua Watt
5b65a822d6 checklayer: Skip layers without a collection
As in other places in the file, skip layers that don't define a
collection when searching for a layer to resolve a dependency. Fixes
KeyError exceptions when attempting to access the layer collections
later

(From OE-Core rev: ae65adf471a9ad04c6a44bf020a28f1006db106a)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 26090a2861ebe21224aaf89d7be0c0a89ca58e48)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-05 21:36:30 +01:00
Richard Purdie
f4e0b26568 Revert "wic: Fix permissions when using exclude or include path"
This reverts commit 36993eea89d1c011397b7692b9b8d61b499d0171.

After discussion on the mailing list it was felt these changes
were not ready yet.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-07 21:57:16 +01:00
Richard Purdie
157a12d499 Revert "wic: Add --embed-rootfs argument"
This reverts commit efdcf94801f6abe8e4099e324d9a3deccd8d4384.

After discussion on the mailing list it was felt these
changes were not ready yet.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-07 21:57:16 +01:00
Ricardo Ribalda Delgado
242412656b wic: Add --embed-rootfs argument
This option adds the content of a rootfs on a specific location on the
rootfs.

It is very useful for making a partition that contains the rootfs for a
host and a target Eg:

/ -> Roofs for the host
/export/ -> Rootfs for the target (which will netboot)

Although today we support making a partition for "/export" this might
not be compatible with some upgrade systems, or we might be limited by
the number of partitions.

With this patch we can use something like:

part / --source rootfs --embed-rootfs target-image /export --embed-rootfs target-image2 /export2

on the .wks file.

(From OE-Core rev: efdcf94801f6abe8e4099e324d9a3deccd8d4384)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-06 16:45:11 +01:00
Ricardo Ribalda Delgado
6bac089383 wic: Fix permissions when using exclude or include path
When parameters include_path or exclude_path are passed to the rootfs
plugin, it will copy the partition content into a folder and make all
the modifications there.

This is done using copyhardlinktree(), which does not take into
consideration the content of the pseudo folder, which contains the
information about the right permissions and ownership of the folders.

This results in a rootfs owned by the user that is running the wic
command (usually UID 1000), which makes some rootfs unbootable.

To fix this we copy the content of the pseudo folders to the new folder
and modify the pseudo database using the "pseudo -B" command.

(From OE-Core rev: 36993eea89d1c011397b7692b9b8d61b499d0171)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-06 16:45:11 +01:00
Lee Chee Yang
b5a3625e59 wic: align layer plugin path
wic look for external layer source plugin under scripts/lib/
while other tools look for lib/.

allow wic to check for source plugin at both scripts/lib/ and
lib/ to align with other tools while avoid breaking any existing
source plugin in external layer.

[YOCTO #13056]

(From OE-Core rev: 986baff26bd96a6265f5fe2d631818fff9f66374)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-02 15:24:59 +01:00
Chee Yang Lee
590555e3b8 wic/direct: reserve 2 sector for extended partition
extended partition should reserve at least 2 unallocated sectors
between the start of the extended partition and the start of a
logical partition.

[YOCTO #13658]

(From OE-Core rev: 884c8a0e50a49bdca5e048197a9dc0ff37cc8d1a)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-24 16:39:40 +00:00
Armin Kuster
26525f2f4f wic/engine: lets display an error not a traceback
If the requested partition does not exist in this request "wic ls {path}:pnum"
display a nice message not a trackback

Also fix displaying the pnum and not "%s"

(From OE-Core rev: 15d1722950a22649905cf8a5789d3cfe48a2a892)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-02 21:00:54 +00:00
Pierre-Jean Texier
0c8892fa1d wic/engine: dump the partition table in JSON format only
since commit 03154d2cf25c1d5ce908da9e72d324004b8d5722 ("sfdisk: add -J between mutually exclusive options")
in util-linux [1], the '-J' can't be used with the '-d' option anymore.

So, since we're using json format output, drop the '-d' option.

Fixes:

ERROR: _exec_cmd: <snip>/usr/sbin/sfdisk -dJ <snip>/core-image-minimal-qemux86-64.wic returned '1' instead of 0
output: sfdisk: mutually exclusive arguments: --list-free --json --dump

[1] - 03154d2cf2 (diff-31d5cc325380dbdafe64d0e019846f12)

(From OE-Core rev: 49e7f3e8e3a775510d8b0e2826a06d000934706b)

Signed-off-by: Pierre-Jean Texier <pjtexier@koncepto.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-01 10:40:11 +00:00
Alexander Kanavin
e88fe83014 qemux86: do not add vga=0 to kernel parameters
This was added ages ago to enable GL passthrough with
vmware driver, and is no longer relevant, as std or virgl is used
instead nowadays.

Original commit:

commit 072545b111
Author: Richard Purdie <rpurdie@linux.intel.com>
Date:   Wed Jan 21 17:40:51 2009 +0000

    scripts/poky-qemu-internal: Add support for GL passthrough in qemux86 images

(From OE-Core rev: 857078ba8eda153f4a097683db551a7d310ecc01)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18 23:53:54 +00:00
Kevin Hao
dd372f09fc wic: Honor APPEND
APPEND is used to add additional parameters to kernel by features, such
as read-only-rootfs. So we should honor this variable when we compose
the kernel parameter in wic. I know we also can resolve this kind of
issue by using the .wks.in template introduced by commit 42e870c5ed
("image_types.bbclass: support template .wks.in files for wic"), but the
APPEND is needed by all the wks, it would seems pretty ridiculous to me
to change all the .wks to .wks.in and then foist the APPEND into them.
So the APPEND is definitely deserved to export to the wic directly.

[Yocto #12809]

(From OE-Core rev: 18981b8a457104391dfd94938c247eac04e4ed50)

Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-15 22:42:14 +00:00
Richard Purdie
51e9c2fd73 resulttool/resultutils: Fix unicode error handling
This error handling didn't work as expected since upon failure it would
inject bytestreams back into the code leading to tracebacks.

Instead, ignore the decode errors. Fixes:

Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 78, in <module>
    sys.exit(main())
  File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 72, in main
    ret = args.func(args, logger)
  File "/home/pokybuild/yocto-worker/a-full/build/scripts/lib/resulttool/store.py", line 70, in store
    resultutils.save_resultsdata(results, tempdir, ptestlogs=True)
  File "/home/pokybuild/yocto-worker/a-full/build/scripts/lib/resulttool/resultutils.py", line 178, in save_resultsdata
    f.write(sectionlog)
TypeError: write() argument must be str, not bytes

(From OE-Core rev: 3aa0d0fd0bbf8c9c13719251a2fd1e85a22c7336)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-15 17:26:01 +00:00
Yeoh Ee Peng
0b032d39fa scripts/lib/resulttool/report: Enable report selected test case result
Enable reporting selected test case result given the user provided
the selected test case id. If both test result id and test case id
were provided, report the selected test case result from the
selected test result id.

(From OE-Core rev: 7161310ce32d6e0c397d0132808d556bdc80d183)

Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-13 12:19:14 +00:00
Tom Hochstein
e209b9f0bf devtool/standard.py: Allow recipe to disable menuconfig logic
u-boot.inc supports u-boot recipes with or without menuconfig [1].
However, running devtool on a u-boot recipe that does not support menuconfig
results in an error:

cp: cannot stat '/home/r60874/upstream/fsl-xwayland/tmp/work/imx8mmevk-fsl-linux/u-boot-imx/2018.03-r0/u-boot-imx-2018.03//.config': No such file or directory

The problem is the devtool logic assumes that any recipe with a do_menuconfig task
will generate a .config in do_configure().

Fix the problem by removing the assumption with a flag that the recipe can control,
like this:

do_configure() {
    if [ menuconfig-supported ]; then
        ...
    else
        DEVTOOL_DISABLE_MENUCONFIG=true
    fi
}

[1] 11278e3b2c

(From OE-Core rev: 6cc17e658c3dbadc772353f0f200561d2d988658)

Signed-off-by: Tom Hochstein <tom.hochstein@nxp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-02 16:57:21 +00:00
Jean-Marie LEMETAYER
4bb2d97e7f recipetool/create: npm: remove the 'noverify' url parameter
This commit removes the 'noverify' parameter which was added to the url
to fix warnings with the shrinkwrap / lockdown file generation. This is
not needed anymore with the new npm fetcher.

(From OE-Core rev: 401a9818c633373854d3c7dec032b9f455f2b2b4)

Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27 16:48:09 +00:00
Jean-Marie LEMETAYER
1f3b0e2f89 devtool/standard: npm: update the append file
When creating a recipe using devtool, a workspace is created to store
the new recipe, the recipe source and some append files. These append
files are used by devtool to build the recipe using externalsrc (to use
the source which are in the workspace). They can also have some
additional actions according to the class of the recipe.

This commit updates the append file for the npm recipes. The
devtool / externalsrc files are removed in the npm package directory
instead of the install directory.

(From OE-Core rev: a213a76e56e9bc215a61399ae35a3602863f5850)

Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27 16:48:09 +00:00
Jean-Marie LEMETAYER
cfa5544005 recipetool/create_npm: handle the licenses of the dependencies
As usual the 'LICENSE' and the 'LIC_FILES_CHKSUM' values reflects all
the license files discovered in the source tree (including the
dependencies).

For npm recipes the 'LIC_FILES_CHKSUM' value contains also the status of
the 'package.json' file of every packages as it contains license
informations.

Finally each package has a separate 'LICENSE_${PN}-package-name' value
which describes its license.

(From OE-Core rev: 9a70d4996c84b277f423eda5aac4acbe344599f4)

Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27 16:48:09 +00:00
Jean-Marie LEMETAYER
6fd9cebc98 recipetool/create_npm: refactor the npm recipe creation handler
This commit refactors the npm recipe creation handler to use the new npm
behavior. The process is kept as simple as possible and only generates
the shrinkwrap file.

To avoid naming issues the recipe name is now extracted from the npm
package name and not directly mapped.

(From OE-Core rev: 1deccb0f0c204cd02fb8606f180d8a13df9f31db)

Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27 16:48:09 +00:00
Jean-Marie LEMETAYER
3a8a675825 devtool: npm: rename npm command line options
This commit renames the '--fetch-dev' option into '--npm-dev' as it is a
npm only option.

(From OE-Core rev: 2b75cc848ceebee4067788a621299bfd5fb62231)

Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27 16:48:09 +00:00
Maxim Uvarov
8290a50fe2 wic: fix images build in parallel
OE wic plugins create temporary file with the index of the line
tmp file name. This causes race in case several builds run in time.
If source_params['file'] is an absolute path, the cr_workdir prefix
is not applied by os.path.join(). So instead it writes to a ".1"
file next to the original image - this is outside the WORKDIR
and at risk of collision.

(From OE-Core rev: c68d8a37ba4348fe1c0e75c63b5668187d326ec2)

Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
Suggested-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Suggested-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-19 13:24:39 +00:00
Paul Barker
2c81446248 wic: Add --include-path argument
This option adds the contents of the given path to a partition built
with the rootfs source plugin. The path is relative to the directory in
which wic is running not the rootfs itself so use of an absolute path
is recommended. This option is most useful when multiple copies of the
rootfs are added to an image and it is required to add extra content to
only one of these copies. This option only has an effect with the
rootfs source plugin.

(From OE-Core rev: d4cd27a9837426e809190548a83c6c7c76505114)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:22 +00:00
Paul Barker
ab6e8dae1e wic: Preserve intermediate files
When debugging issues or trying to add new features in wic it's useful
to examine the intermediate files created by the tool such as images of
the individual partitions. Previously these files were created in a
subdirectory of IMGDEPLOYDIR and, deleted by wic itself and then the
working subdirectory completely removed at the end of IMAGE_CMD_wic.
Instead we should work in a staging directory named build-wic and leave
behind intermediate files for analysis if necessary.

This has the added benefit of allowing individual partition images to
be preserved by adding commands to IMAGE_CMD_wic_append() in an image
recipe if needed. This is useful where an update system requires exact
sized and pre-formatted partition images for deployment.

(From OE-Core rev: 07b6b284204e912b5d08bdd0ca51b766ab7dd9c1)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:22 +00:00
Kalle Lampila
b14588a778 wic/filemap: If FIGETBSZ iotctl fail, failback to os.stat
Some file systems don't support fetching the block size (notably the
file system Docker uses for containers), so if iotctl() fail, try to use
failback via os.stat() to get block size.

(From OE-Core rev: e219f5175177a640dd62833082ea19adc1c13d42)

Signed-off-by: Kalle lampila <kalle.lampila@lempea.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-02 16:39:13 +00:00
Alexander Kanavin
ecda76eb1a recipetool: always use python 3 in recipe creation
(From OE-Core rev: 59e63a2d22988ef11fe36200692b967b67ca8641)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-30 08:47:12 +00:00
Chen Qi
50ebd252ac scriptutils: remove the useless import line
imp was replaced by importlib as it's deprecated, but the import
line was left, so remove this useless line.

(From OE-Core rev: 95574a7d18a8acfbed1b956677e72044bdfa7ebc)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-06 14:41:28 +00:00
Chee Yang Lee
89288705c6 wic: 'wic cp' to copy from image
currently 'wic cp' only works for copy file from local storage to
wic image.

enhance 'wic cp' to copy file/directory from wic image to local storage.

include selftest and 'wic help' updates.

[YOCTO#12169]

(From OE-Core rev: bd669c1809a378f93580eb9e0679a26ec6746cb8)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-27 13:25:18 +00:00
Paul Eggleton
ea01bd31c3 devtool: fix devtool upgrade with reproducible_builds class
If the reproducible_build class is inherited then there may be a
"source-date-epoch" subdirectory in a fetched source tree; devtool
upgrade was not expecting that in the upgraded source. Take a small
snippet of code from recipetool create which already handles this,
and make it a shared function that can be used in both places.

Additionally, fix an assumption that the source is always in a
subdirectory in the cleanup code that blocked debugging this.

[YOCTO #13635]

(From OE-Core rev: 0d642861cd9cf034b8d4951433980addc215d4fd)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21 23:08:20 +00:00
Alexander Kanavin
cd09d79602 Revert "devtool/standard.py: Not filtering devtool workspace for devtool finish"
This reverts commit 41d225f4a3.

Unfortunately this change broke 'devtool upgrade' functionality,
causing 'devtool finish' to write out an upgraded recipe that no
longer includes the original upstream source in SRC_URI.

(From OE-Core rev: 2d6e55192dba0bf7f6e23e5ab5b3dbc68835bb28)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21 23:08:20 +00:00
Richard Purdie
d05b78b245 recipetool/create: Fix to work with reproducible_builds
The extra diretory created by reproducibile_builds was confusing recipetool
and hence devtool. Ignore this directory to restore the correct behaviour and
allow it to work with the class.

(From OE-Core rev: 188ba30ef290efab6d5a9b95fba8e991b779ab33)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21 23:08:19 +00:00