Commit Graph

90 Commits

Author SHA1 Message Date
Alexander Kanavin
05d23378c2 meta/lib/oe/recipeutils.py: handle fetcher errors when checking for new commits
Recent freedesktop instabilities are causing 'devtool check-upgrade-status' to
fail with:

bb.fetch2.FetchError: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; git -c gc.autoDetach=false -c core.pager=cat -c safe.bareRepository=all -c clone.defaultRemoteName=origin ls-remote https://gitlab.>
fatal: unable to access 'https://gitlab.freedesktop.org/mesa/kmscube/': The requested URL returned error: 502

and not print any results for this one or any unrelated recipes included in the check.

This change handles the error, so that if some upstream server isn't working
properly, latest upstream revision for that is marked as unknown, a
warning is printed and upstream version check for other recipes
isn't thwarted:

WARNING: Unable to obtain latest revision: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; git -c gc.autoDetach=false -c core.pager=cat -c safe.bareRepository=all -c clone.defaultRemoteName=origin ls-remote https://gitlab.freedesktop.org/mesa/piglit.git  failed with exit code 128, output:
remote: GitLab is not responding
fatal: unable to access 'https://gitlab.freedesktop.org/mesa/piglit.git/': The requested URL returned error: 502

piglit                    1.0             UNKNOWN_BROKEN  Ross Burton <ross.burton@arm.com>

(From OE-Core rev: c1056293f7cb32ee2bdf31441cc0b59d9ccfe556)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-19 09:48:14 +00:00
Alexander Kanavin
3d98aafc43 recipeutils/get_recipe_upgrade_status: group recipes when they need to be upgraded together
This will allow 'lockstep upgrades' of such recipes, improving success
rates in automated version updating process.

devtool check-upgrade-status now prints:

These recipes need to be upgraded together {
glib-2.0                  2.80.2          2.80.4          Anuj Mittal <anuj.mittal@intel.com>
glib-2.0-initial          2.80.2          2.80.4          Anuj Mittal <anuj.mittal@intel.com>
}
These recipes need to be upgraded together {
util-linux                2.39.3          2.40.2          Chen Qi <Qi.Chen@windriver.com>
util-linux-libuuid        2.39.3          2.40.2          Chen Qi <Qi.Chen@windriver.com>
}
These recipes need to be upgraded together {
cmake                     3.29.3          3.30.0          Unassigned <unassigned@yoctoproject.org>
cmake-native              3.29.3          3.30.0          Unassigned <unassigned@yoctoproject.org>
}

etc.

(From OE-Core rev: 7874aea5c62be3e8dbd19e04fce5389c5ed7aab6)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-22 16:53:06 +01:00
Alexander Kanavin
bd18497110 lib/recipeutils: add a function to determine recipes with shared include files
This functionality is needed for 'lockstep version upgrades' where several
recipes need to be upgraded at the same time to produce a buildable
outcome.

The function itself obtains BBINCLUDED for each recipe and then massages
the data until it takes the form of a list of sets:

[{'cmake','cmake-native'},
 {'qemu','qemu-native','qemu-system-native'},
... ]

There's also a selftest that checks for the above.

Unfortunately this won't detect mutually exclusive recipes like mesa and mesa-gl
as they're chosen with PREFERRED_PROVIDER and can't be enabled in the same build
at the same time. ('devtool upgrade' will also accept just one of them but not the other)

(From OE-Core rev: 2400920f8b84cca9d6c1f6a2e850630554fe00fa)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-22 16:53:06 +01:00
Alexander Kanavin
597b87a468 lib/oe/recipeutils: return a dict in get_recipe_upgrade_status() instead of a tuple
Putting various things in a tuple is an anti-pattern of sorts, as the consumers
have to unpack it into local variables for readability, or access items directly
with indexes, which makes code pretty much unreadable.

(From OE-Core rev: e86aa26d209eb9809198f6dd40cd058366318e3d)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-22 16:53:06 +01:00
Richard Purdie
71c6db8e65 recipes: Start WORKDIR -> UNPACKDIR transition
Replace references of WORKDIR with UNPACKDIR where it makes sense to do
so in preparation for changing the default value of UNPACKDIR.

(From OE-Core rev: 1f18b9a512800860d5153d89eb82b56388efad6f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-21 12:08:04 +01:00
Peter Kjellerstedt
3ef9ea88f1 lib/oe/recipeutils: Avoid wrapping any SRC_URI[sha*sum] variables
Before, a variable such as SRC_URI[sha512sum] would end up as:

SRC_URI[sha512sum] = "45ff3abce4dab24a8090409e6d7bb26afa7fa7812a51e067 \
28c2aa47d5b4de610d97ba4609cf13d9173087bd909fdf377235eee988a6fdcf52abb7 \
0341c40b5b"

when updated by patch_recipe_lines().

(From OE-Core rev: a67e2feed1420739504d2a59d018dff7e6e17e04)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-08 16:58:34 +00:00
Julien Stephan
0ae9cf2373 recipeutils: bbappend_recipe: allow to patch the recipe itself
Add a new parameter update_original_recipe to allow to patch a recipe
instead of creating/updating a bbappend

(From OE-Core rev: 2f68ab2464bfad1b377df44a7b51203df59d66ce)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
165626f7b9 recipeutils: bbappend_recipe: remove old srcuri entry if parameters are different
Currently we do not add a new src_ury entry if the entry already exists
AND the parameters are the same.

I believe that when an entry already exist with different parameters,
we should remove it and add the new entry otherwise we end up with two
entries with different parameters

(From OE-Core rev: a4628fffcfecb5cd95dc2558dfd39ebd71121eab)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
11d4d437d5 recipeutils: bbappend_recipe: add a way to specify the name of the file to add
bbappend_recipe can take a dict of source files to add to SRC_URI where
the key is the full path to the file to be added and the value is a dict

Add a new optionnal entry "newname" to specify the name of the newly added file

(From OE-Core rev: e7bc09e5c9d7a0f4f8f4eba40730b68857b00677)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
b45cab4e1c recipeutils: bbappend_recipe: fix docstring
Add missing patchdir key in docstring description for srcfiles
parameters.

Also fix typo: value --> key in srcfile docstring description

(From OE-Core rev: 302dccaef46418d0e0731870a882b54a525739ed)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
eb0b664c8c recipeutils: bbappend_recipe: fix undefined variable
In the case get_bbappend_path returns None (could not find the layer
containing the recipe) the error message tries to print the recipefile,
but it is not defined. Fix it.

(From OE-Core rev: 234111fb67ffbcc5492cb0cd96db25ed8f5acea0)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
89f1662484 devtool: add support for git submodules
Adding the support of submodules required a lot of changes on the
internal data structures:
* initial_rev/startcommit used as a starting point for looking at new
  / updated commits was replaced by a dictionary where the keys are the
  submodule name ("." for main repo) and the values are the
  initial_rev/startcommit

* the extractPatches function now extracts patch for the main repo and
  for all submodules and stores them in a hierarchical way describing the
    submodule path

* store initial_rev/commit also for all submodules inside the recipe
  bbappend file

* _export_patches now returns dictionaries that contains the 'patchdir'
  parameter (if any). This parameter is used to add the correct
  'patchdir=' parameter on the recipe

Also, recipe can extract a secondary git tree inside the workdir.

By default, at the end of the do_patch function, there is a hook in
devtool that commits everything that was modified to have a clean
repository. It uses the command: "git add .; git commit ..."

The issue here is that, it adds the secondary git tree as a submodule
but in a wrong way. Doing "git add <git dir>" declares a submodule but do
not adds a url associated to it, and all following "git submodule foreach"
commands will fail.

So detect that a git tree was extracted inside S and correctly add it
using "git submodule add <url> <path>", so that it will be considered as a
regular git submodule

(From OE-Core rev: 900129cbdf25297a42ab5dbd02d1adbea405c935)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-01 11:48:25 +00:00
Julien Stephan
c7ed37b5ff lib/oe/recipeutils.py: remove trailing white-spaces
Remove useless trailing white-spaces

(From OE-Core rev: 4aa28d5ea74016087979afd1faea478fda253cc3)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-30 08:43:04 +00:00
Alexander Kanavin
d3e4f25993 lib/oe/recipeutils.py: accommodate SRCPV being optional and deprecated in version check regex
(From OE-Core rev: 84794b59940cb08ec44e5ee7718a464d24136812)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-30 09:56:12 +01:00
Richard Purdie
65318019cd recipes/classes/scripts: Drop SRCPV usage in OE-Core
Now that SRCPV isn't needed we can simplify things in a few places...

(From OE-Core rev: 843f82a246a535c353e08072f252d1dc78217872)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-24 16:50:24 +01:00
Michael Opdenacker
a26970fa3a recipes: remove unused AUTHOR variable
No longer used in generating packages
Also creates a possible confusion with the recipe maintainer
name.

(From OE-Core rev: 9d5edd124b7dddb995ceddd79f8a7fc8cf44badf)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-10 09:18:53 +01:00
Paul Eggleton
da71c928cd devtool: finish: handle patching when S points to subdir of a git repo
If devtool finish needs to create a patch and have it applied to the
sources for a recipe where S points to a subdirectory of the sources,
then the patch needs to be applied at the root of the repo i.e. we need
to add a patchdir= parameter to the SRC_URI entry.

(From OE-Core rev: ad3736d9ca14cac14a7da22c1cfdeda219665e6f)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-14 23:22:09 +01:00
Richard Purdie
eb72e3c381 recipeutils: Add missing get_srcrev() call
Updates within bitbake now require this call to be triggered somewhere with
floating git source revs. Add the missing call. Issue can be reproduced with:

devtool check-upgrade-status dbus-wait

(From OE-Core rev: d7088b2b5012f9e386914a6837685f5869379cc5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-18 11:37:34 +00:00
Richard Purdie
65133015b1 meta: Manual override fixes
The automated conversion of OE-Core to use the new override sytax isn't
perfect. This patches some mis-converted lines and some lines which were missed
by the automation.

(From OE-Core rev: 4e9a06b64b43131b731fb59a0305f78a98e27fbd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02 15:44:10 +01:00
Richard Purdie
bb6ddc3691 Convert to new override syntax
This is the result of automated script conversion:

scripts/contrib/convert-overrides.py <oe-core directory>

converting the metadata to use ":" as the override character instead of "_".

(From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02 15:44:10 +01:00
Andrei Gherzan
cbb794f9dd oe/recipeutils: Fix copying patches when BBLAYERS entries are not normalised
`devtool` uses `copy_recipe_files` for the upgrade operation when
creating the new, workspace recipe. Before handling the copy operations,
the function checks the entry in `SRC_URI` against `FILE` while in turn
uses absolute paths. When BBLAYERS contains entries that are not
normalised, this check will fail resulting in having the recipe in the
workspace without the initial patches.

Signef-off-by: Robert Drab <robert.drab@huawei.com>
(From OE-Core rev: 777fc24ab2c332954b56dac28cd9b3032808828c)

Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-19 11:17:11 +00:00
Naveen Saini
3204982fd7 lib/oe/recipeutils.py: add support for BBFILES_DYNAMIC
Instead of relying on value of BBFILES from bitbake, devtool
parses the layer.conf because the layer might not be in bblayers.conf.
And it currently does not consider the value of BBFILES_DYNAMIC because
of which recipes, in paths defined by BBFILES_DYNAMIC, upgraded
using devtool end up in wrong location.

Include the code from bitbake to append values to BBFILES based on
what is in BBFILES_DYNAMIC too.

(From OE-Core rev: c7bbb98ea8ccd3568dd8bded6e404e2f781e6841)

Signed-off-by: Naveen Saini <naveen.kumar.saini@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-27 08:28:03 +01:00
Tim Orling
c9ba6e7bfe lib/oe/recipeutils.py: add AUTHOR; BBCLASSEXTEND
If you try to create a plugin for recipetool that
adds the AUTHOR field, it is impossible to put it
in the recommended position [1] without adding to the
recipe_progression variable.

While we are at it, also add BBCLASSEXTEND at the
end, as also recommended by [1].

[1] http://www.openembedded.org/wiki/Styleguide

(From OE-Core rev: d687c5b7b10b3decdd80d5c2fd61072a87f061f2)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-12 22:39:41 +01:00
Konrad Weihmann
9ac51e479d lib/oe/recipeutils.py: passthrough of FETCHCMD
let FETCHCMD_* settings be passed to upstream check functions from
get_recipe_upgrade_status. It enables using different values for
the fetching tools, as otherwise always the defaults are used.
E.g. use different timeout or retry values for wget.
This is especially useful for remote server that do request-limiting (429)
or that are simply unavailable (500, 504) and can speedup upgrade-check
for a larger set of recipe significantly

(From OE-Core rev: f69347744241afea19f155d9ff5115c675e9043d)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-19 22:57:26 +01:00
Richard Purdie
4bf1c0401a oe/recipeutils: Drop obsolete GITDIR reference
The fetcher GITDIR variable wase dropped a while back, drop the
obsolete reference to it (thanks Robert Day).

(From OE-Core rev: de035e687e26cef96e9b737c47bfc291bdfbea48)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-29 12:05:31 +01:00
Alexander Kanavin
40f28d15c5 recipeutils: set SRCPV for devupstream variant
Otherwise bitbake is unable to expand it if the main
variant is not using a SCM fetcher.

(From OE-Core rev: 24e59094de42b4b0bed3584ca8571c4c19080428)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-28 23:25:42 +00:00
Hongxu Jia
c113a83dd5 lib/oe/reciputils.py: support character `+' in git pv
While pv contains char `+' (such as ncurses 6.1+20181013),
it was incorrectly converted to `6.1'

In this commit:
- Convert [^\+]* to .*?
  Support pv to contain char `+' and not greedy match
  (Such as ncurses 6.1+20181013)

- Add [^\+] to sfx match
  Support sfx contains extra chars between `+' and `git'
  (such as asciidoc "8.6.9+py3-gitAUTOINC+618f6e6f6b")

- Make sfx and rev greedy match

Run `devtool check-upgrade-status --all' in poky and compare results,
only one difference on ncurses version:
Without the commit:
INFO: ncurses                   6.1             6.1+20181013    Hongxu Jia <hongxu.jia@windriver.com> 7a97a7f937762ba342d5b2fd7cd090885a809835

With the commit:
INFO: ncurses                   6.1+20181013    MATCH           Hongxu Jia <hongxu.jia@windriver.com> 7a97a7f937762ba342d5b2fd7cd090885a809835

(From OE-Core rev: 8049bd34b89e710f7bb20883813ba3f929d9e997)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-04 09:09:42 +01:00
Richard Purdie
ffae400179 meta/lib+scripts: Convert to SPDX license headers
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.

The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).

More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.

The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.

(From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09 16:31:55 +01:00
Jaewon Lee
206ca252c6 devtool: Support kmeta directory usage with devtool modify/finish
When using Kmeta directories, devtool finish will add every single file
in the directory to the bbappend. This is because in the current
implementation, the get_recipe_local_files function treats the kmeta
directory like a file. Modifying the function to loop through the
provided directories and return all included files instead of just the
top level directory. This will enable correct file to file comparison
when determing which files are new/changed and need to be added to the
bbappend.

Adding an extra check in devtool-source.bbclass to not copy the cfg file
if its already included somewhere in the kmeta directory

Also during 'modify', when moving necessary files in the kmeta directory
from the workdir to oe-local-files, the dangling parent directories are
left behind.  This in itself is not an issue as the temporary devtool
workspace is automatically deleted, but this causes an incorrect include
directory to be added in kernel-yocto.bbclass.  Changing the order of
the if statements to catch the correct conditional. This is safe to do
as when not in the devtool context, there will be no oe-local-files
directory.

(From OE-Core rev: 4671011b5b02d0989aa0fdcb50e18cb385a0e95e)

Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Acked-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-06 10:39:25 +00:00
Richard Purdie
e1e38c9bd1 lib/oe,oeqa/selftest: Fix DeprecationWarning: invalid escape sequence
Fix another load of regex escape sequence warnings for newer
python versions.

(From OE-Core rev: bd2c125bb9c362b6122e99dfdf4e1cfe12c26a90)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-16 08:19:34 +00:00
Alexander Kanavin
411e2b7047 lib/oe/reciputils.py: parallelize upstream version checks
Previously this was done via bitbake tasks, and when this
was rewritten to a for loop, performance sufered significantly:
from 90 seconds to about 12 minutes for oe-core. This change
restores the previous run time, and makes it possible
to perform such checks with command line utilities in an
interactive way.

Implementation note: we have to create a copy of the recipe
data, as Tinfoil API can't be used from multiple threads
and only allows one process to access the data at a time.

(From OE-Core rev: b1d01911fa2a0a4945da071d66fb50e9f14ded81)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-26 13:39:37 +00:00
Richard Purdie
13940a81da lib/oe/recipeutils: Add a new function to mimic do_checkpkg
The code in distrodata.bbclass related to the do_checkpkg task is rather
dated, has holes in it (ignoring some recipes) and has horrible locking
and csv related issues.

We should use modern APIs such as tinfoil to make the calls we need directly
against bitbake, cutting out the middleman and clarifing the code.

This change imports the bits of distrodata.bbclass that are needed by the
automated upgrade helper (AUH) into a standalone function which uses the
tinfoil API. This can then be used by AUH and by the tests in
oeqa/selftest/distrodata as well as by any other standalone script that needs
this functionality. Its likely it can be further improved from here but this is a
good start and appears to function as before, with slightly wider recipe
coverage as some things skipped by distrodata are not skipped here (images,
pieces of gcc, nativesdk only recipes).

(From OE-Core rev: 92e33277b1b7892bae9cc0801ab379bd1c57c0f0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-15 11:48:07 +00:00
Paul Eggleton
1c0a6cdf42 lib/oe/recipeutils: drop obsolete functions
These date from the time before Tinfoil's API covered this functionality
(back when you could actually access cooker from a tinfoil-based
script).

(From OE-Core rev: fc8098756e8d1d995e3c68e4addc1a5e3596d2ae)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-03 12:20:00 +00:00
Paul Eggleton
f1ca6f73eb lib/oe/recipeutils: patch_recipe(): fix replacing varflag values
The code here wasn't correctly getting the variable history for
varflags, so for example if you did a devtool upgrade on a recipe where
the SRC_URI checksums were in the .inc file (typical for python recipes
in order to support both python 2 and 3) then after the upgrade the
new values would be set in the recipe and the old values were left in
the .inc, which is not right. Teach the code here how to get the history
for varflags so it works properly.

(From OE-Core rev: f077ab3ad67b2f3eb4aa8882fe2e7aef2d09a26c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-03 12:20:00 +00:00
Paul Eggleton
15fb7acb77 lib/oe/recipeutils: patch_recipe(): fix handling of values across includes/classes
If we were setting a variable and part of the variable's value was being
set in a class or a .inc file, we were still just setting the value
outright on the assumption that it was too hard to do otherwise. With
some careful use of the variable history we can do better for certain
situations i.e. when the recipe does not currently set the value
outright.

Additionally, correctly remove _appends for variables we are changing if
we're trying to remove the value added in the _append.

Fixes [YOCTO #12623] and partially fixes [YOCTO #9360].

(From OE-Core rev: f798d963db77db66a2a5a4b8d4c157d46ede417f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-03 12:20:00 +00:00
Paul Eggleton
2f48cb606b lib/oe/recipeutils: add .txz extension to archive list
Prompted by bitbake commit 2ba8a6b25ccc12e7b543e8450121e5311c7a701d,
add .txz to the list of archives used within get_recipe_local_files()
here as well.

(From OE-Core rev: 28466586f4777f11f0a29ed5ebe9734a4e15920e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-01 22:13:56 +00:00
Alexander Kanavin
c599921025 distrodata.bbclass: make upstream version check more useful for git upstreams
Specifically:

1) remove +git${SRCPV} stuff from comparison and output; it's just
unnecessary clutter;

2) write the commit id of the latest version tag into the output;
this saves quite a bit of trouble of manually checking what that
commit id is when doing version updates;

3) when UPSTREAM_CHECK_COMMITS is set, ignore the tags altogether;
instead check if the latest commit is different to the one we use,
and if so, report that the recipe can be updated to said commit
(which is also written into the output, as in 2). Multiple
recipes are failing the upstream check because they never
issue tags, now we can fix them.

(From OE-Core rev: 591d57877d3d3e659d78c0ed33f4c515e3f6f8fb)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-30 10:49:22 +00:00
Paul Eggleton
9a80078e4b devtool: finish: add dry-run option
If you're not sure what changes devtool finish is going to make, or
you're not sure you're finished with your modifications, it is useful to
be able to see what devtool finish is going to do beforehand, so add
a -N/--dry-run option to make that possible.

(It's also very useful for debugging devtool finish itself.)

(From OE-Core rev: 05f2d5d2ce00c53825ccea5cd9c2262f9d27a638)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-11 12:14:27 +00:00
Paul Eggleton
b5c72fe584 devtool: upgrade: handle recipes that use named SRC_URI checksums
devtool upgrade did not properly handle setting SRC_URI checksums for
recipes that use named SRC_URI entries and also use those names in the
SRC_URI checksums. A further complication was where the name contained
an expression that changed with the version e.g. ${PV} (probably quite
rare, but the dnsmasq recipe in meta-networking is currently one such
recipe.) All of these are now handled properly.

Additionally, drop the _get_checksums() function that wasn't being
called from anywhere in the code.

Note that this now turns nowrap_vars in recipeutils.py to be a list of
regexes, hence things such as [ and ] need to be appropriately escaped.

(From OE-Core rev: c914a5e1ad6d96e316746222e5d42f2ba9110060)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-11 12:14:27 +00:00
Paul Eggleton
202f9af816 lib/oe/recipeutils: fix line splitting in patch_recipe_*
If a value was split over multiple lines (e.g. as SRC_URI usually is)
then we were inserting the value as one item in the lines list with
newlines between each line. There's nothing wrong with this if you're
writing the list out to a file, but if you want to generate a patch (as
patch_recipe_file() will do if the patch parameter is set to True) then
the diff output looks a bit odd. Split the value before adding it to the
lines list to resolve this.

(From OE-Core rev: 44f4c97234623cbd770fbc86eabb04e7e0c91061)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-11 12:14:26 +00:00
Paul Eggleton
b575564ee3 lib/oe/recipeutils: fix find_layerdir() to return absolute paths
find_layerdir() should really return absolute paths, so make it do so.
This fixes devtool finish not deleting files it should do after devtool
upgrade if the specified path is relative, since the devtool finish code
was assuming that find_layerdir() was returning an absolute path.

Fixes [YOCTO #12318].

(From OE-Core rev: 5e3fe00a0233d563781849a44f53885b4e924a9c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-11 12:14:26 +00:00
Paul Eggleton
69d50eb9ec devtool: upgrade: workaround for recipes which apply patches conditional upon class
If we're upgrading a recipe that appends additional patches for, say,
class-native, and we're just upgrading the target variant, then when we
copied the recipe into the workspace we skipped copying the additional patches
for the native variant. This caused warnings because the workspace
recipe is preferred. Look at SRC_URI for all variants when copying files
to work around this.

More work is needed to make it easier to work with recipes that use
BBCLASSEXTEND where you need to build more than one variant at once, but
this at least fixes the immediate ugliness.

(From OE-Core rev: 56bf5e93358187e31160d7893f57906bb3dc7ad7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 23:30:02 +01:00
Paul Eggleton
34580ac287 devtool: upgrade: fix handling of non-absolute paths
If your BBLAYERS has non-absolute paths in it (e.g.
"${COREBASE}/../something") then none of the paths matched in
copy_recipe_files() with the result that no files got copied and you
ended up with an error later on because the recipe file couldn't be
found at the destination. Fix this as well as adding an explicit check
to see if no files got copied - error out earlier if so.

Fixes [YOCTO #10981].

(From OE-Core rev: 3861486ad06f90c8644ebab119bbc5ddb9e693ca)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 23:30:02 +01:00
Joshua Lock
c4e2c59088 meta: remove True option to getVar calls
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.

Search made with the following regex: getVar ?\(( ?[^,()]*), True\)

(From OE-Core rev: 7c552996597faaee2fbee185b250c0ee30ea3b5f)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Paul Eggleton
078ef4361d lib/oe/recipeutils: drop parse_recipe_simple()
This was intended to be used with tinfoil, but tinfoil now has its own
parse_recipe() method to do this which works properly in the memres
case.

(From OE-Core rev: cdfc6173cb06ca374b7d927442a0fdde8373ba48)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton
275b3fe5f4 classes/patch: move several functions to oe.patch
Move patch_path(), src_patches() and should_apply() to oe.patch, making
them easier to call from elsewhere (particularly across the
UI/server boundary).

(From OE-Core rev: 2724511e18810cc8082c1b028e3b7c8a8b5def56)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00
Paul Eggleton
557382ab42 lib/oe/recipeutils: use cooker function instead of bb.providers
We now have a function in cooker itself that can do this lookup;
additionally, the rewritten tinfoil's cooker adapter has its own
implementation that can work remotely, so if we use it then this
function can work in that scenario as well.

(From OE-Core rev: 0a6a4be99c1e4ef3c0da53d63f18ad579545d6a8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton
e3193b76e0 lib/oe/recipeutils: ignore archives by default in get_recipe_local_files()
By default, have get_recipe_local_files() not return any archive
files. This prevents a local tarball from being erroneously removed
from SRC_URI if you run "devtool modify" on a recipe followed by
"devtool update-recipe". It doesn't actually help you to directly
update the contents of such tarballs, but at least now it won't break
the recipe.

(From OE-Core rev: e9c418d4704c1bed4c5880e176e5288485f4f5a6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:13 +00:00
Paul Eggleton
55a157f4e6 devtool: update-recipe: fix handling of compressed local patches
It is possible to use gzip or bzip2 to compress patches and still refer
to them in compressed form in the SRC_URI value within a recipe. If you
run "devtool modify" on such a recipe, make changes to the commit for
the patch and then run devtool update-recipe, we need to correctly
associate the commit back to the compressed patch file and re-compress
the patch, neither of which we were doing previously.

Additionally, add an oe-selftest test to ensure this doesn't regress in
future.

Fixes [YOCTO #8278].

(From OE-Core rev: e47d21624dfec6f71742b837e91da553f18a28c5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:13 +00:00
Paul Eggleton
385f108377 lib/oe/recipeutils: print just filename in bbappend_recipe() if in temp dir
If you use devtool update-recipe with the --append option, and a "local"
(in oe-local-files) has been modified we copy it into the specified
destination layer. With the way the devtool update-recipe code works now
the source is always a temp directory, and printing paths from within
that is just confusing, so if the path starts with the temp directory
then just print the file name alone.

(From OE-Core rev: 61475f0267d40c618ebf36023d0b6414a25975cb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-06 23:35:36 +00:00