If we output extra blank lines (because of some automated editing) then
it makes the output recipe look a bit untidy. You could argue that we
should simply have the editing code not do that, but sometimes we don't
have enough context there for that to be practical. It's simple enough
to just filter out the extra blank lines when writing the file, so just
do it that way.
(From OE-Core rev: cbebc9a2edf7d7a422ee5c71219e79e3b349de3b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you have your own node.js application you may not publish it (or at
least not immediately) in an npm registry - it might just be in a
repository on github or on your local machine. Add support to recipetool
create for creating recipes to build such applications - extract their
dependencies, fetch them, and add corresponding npm:// URLs to SRC_URI,
and ensure that LICENSE / LIC_FILES_CHKSUM are updated to match. For
example, you can now run:
recipetool create https://github.com/diversario/node-ssdp
(I had to borrow some code from bitbake/lib/bb/fetch2/npm.py to
implement this functionality; this should be refactored out but now
isn't the time to do that refactoring.)
Part of the fix for [YOCTO #9537].
(From OE-Core rev: 4fb8b399c05a1b66986fc76e13525f6c5e0d9b58)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you make adjustments to the source tree (as create_npm.py will be)
then you will need to re-run the license variable handling code at the
end so that we get all of the files that should go into
LIC_FILES_CHKSUM if nothing else. Split out the license variable
handling to a separate function in order to allow this.
(From OE-Core rev: f0d6f4b7e87ea781ac0dffcc8d0310570975811b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For debugging it's useful to be able to tell recipetool to keep the
temporary directory.
(From OE-Core rev: 480a6b745a85b2881e5cc1a0bbb572e3235ca008)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure we fetch submodules and set SRC_URI correctly when pointing to a
git repository that contains submodules.
(From OE-Core rev: 65d5cc62d4ecfc78ce4b37b3886a7fe5aa05a75e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When trying to map python module dependencies to the packages that
provide them, if we're looking for .so files that satisfy
dependencies then we need to exclude files found under the .debug
directory, otherwise the dependency will get mapped to the python-dbg
package which isn't correct.
For example, this fixes creating a recipe for pyserial and not getting
python-fcntl in RDEPENDS_${PN}, leading to errors when trying to use the
serial module on the target.
(From OE-Core rev: 46a068ca35975988a8e9c0310f71fdcee55937a4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If AX_PKG_SWIG is found in configure.ac, then what's being looked for is
the swig binary, not swig for the target - so fix the dependency
accordingly.
(From OE-Core rev: 2600cd6f6c63ecf79804e2bc6eb6f198a012d5d6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's rare but there are recipes that have individual files (as opposed
to archives) in SRC_URI using subdir= to put them under the source tree,
the examples in OE-Core being bzip2 and openssl. This broke devtool
update-recipe (and devtool finish) because the file wasn't unpacked into
the oe-local-files directory and thus when it came time to update the
recipe, the file was assumed to have been deleted by the user and thus
the file was erroneously removed. Add logic to handle these properly so
that this doesn't happen.
(We still have another potential problem in that these files become part
of the initial commit from upstream, which could be confusing because
they didn't come from there - but that's a separate issue and not one
that is trivially solved.)
(From OE-Core rev: 9069fef5dad5a873c8a8f720f7bcbc7625556309)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The --log-file command line argument was slightly broken as {out_dir}
string replacement was not working as expected.
(From OE-Core rev: fc62f54e3d788cc79fd27664f05db7efccef23ab)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Not specifying -C caused oe-build-perf-test to try to commit results to
the build directory.
(From OE-Core rev: 6f4786f5522c366a7fd92f630c3f32629a9f9471)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should always shut down tinfoil when we're finished with it, either
by explicitly calling the shutdown() method or by using it as a
context manager ("with ...").
(From OE-Core rev: 5ec6d9ef309b841cdcbf1d14ac678d106d5d888a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The multiconfig changes altered some of the functions being called here,
so update the calls. Make use of the new Tinfoil.parse_recipe_file()
function to make parsing easier.
(From OE-Core rev: 95b6ceffd947271f315d8a7660797ab371adfbb9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using PATCHTOOL = "git", the user of the system is not really the
committer - it's the build system itself. Thus, specify "dummy" values
for username and email instead of using the user's configured values.
Various parts of the devtool code that need to make commits have also
been updated to use the same logic.
This allows PATCHTOOL = "git" and devtool to be used on systems where
git user.name / user.email has not been set (on versions of git where
it doesn't default a value under this circumstance).
If you want to return to the old behaviour where the externally
configured user name / email are used, set the following in your
local.conf:
PATCH_GIT_USER_NAME = ""
PATCH_GIT_USER_EMAIL = ""
Fixes [YOCTO #8703].
(From OE-Core rev: 765a9017eaf77ea3204fb10afb8181629680bd82)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rename the (main) log file of the oe-build-perf-test script from
'output.log' to 'oe-build-perf-test.log'. Also, add a new command line
option --log-file which makes it possible to use an alternative log file
name/path, if needed. Note that the file name/path is relative to the
output directory.
(From OE-Core rev: 4909fae1a6d1d068b33252088b41b8d82d1a836c)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Write globalres log file and commit results to Git even if some tests
failed. Now that tests do not depend on each other there should be no
risk of bogus results caused by test failures.
(From OE-Core rev: 8036975b268fe209476e230555006facd3cbda71)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New command line argument '-w' may be used to specify work dir other
than the default <GIT_DIR>/build-perf-test.
(From OE-Core rev: 824284895f25146520a624b7b97f7475d0135814)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add new command line argument '-a' that can be used to define the
directory where results (tarballs) are archived. Giving an empty string
disables archiving which makes sense if you store results in Git.
(From OE-Core rev: d53cf92847aa80724be4412801c993948a09cd27)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use getopts for parsing the command line. This changes the usage so that
if a commit (to-be-tested) is defined it must be given by using '-c',
instead of a positional argument.
(From OE-Core rev: b1f77ba41033397a2b25977963682b86f2f76471)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Makes it possible to create easily sortable tags. Also, the default tag
format is updated to use the new keyword.
(From OE-Core rev: e3161654d75dfc3b059c519205b38b26e3ffb215)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes it possible to create numbered tags, where the "basename" of
the tag is the same and the only difference is an (automatically)
increasing index number. This is useful if you do multiple test runs on
the same commit. For example, using:
--commit-results-tag {tester_host}/{git_commit}/{tag_num}
would give you tags something like:
myhost/decb3119dffd3fd38b800bebc1e510f9217a152e/0
myhost/decb3119dffd3fd38b800bebc1e510f9217a152e/1
...
The default tag format is updated to use this new keyword in order to
prevent unintentional tag name clashes.
(From OE-Core rev: cf2aba16338a147f81802f48d2e24a96c7133548)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a Git tag when committing results to a Git repository. This patch
also implements --commit-results-tag command line option for controlling
the tag name. The value
is a format string where the following fields may be used:
- {git_branch} - target branch being tested
- {git_commit} - target commit being tested
- {tester_host} - hostname of the tester machine
Tagging can be disabled by giving an empty string to
--commit-results-tag. The option has no effect if --commit-results is
not defined.
(From OE-Core rev: 60059ff5b81d6ba9ba344161d51d1290559ac2df)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Do a pre-check on the path that is specified with --commit-results
before running any tests. The script will create and/or initialize a
fresh Git repository if the given directory does not exist or if it is
an empty directory. It fails if it finds a non-empty directory that is
not a Git repository.
(From OE-Core rev: 759357a3bdbe75a3409b9e58979ab8b45d9b6ae8)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A new command line option for defining the branch where results are
commited. The value is actually a format string accepting two field
names:
- {git_branch} expands to the name of the target branch being tested
- {tester_host} expands to the hostname of the tester machine
The option has no effect if --commit-results is not used.
(From OE-Core rev: b54b63395ec632748a57a702812c8a9a07af35ab)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implement a new command line option '--commit-results' which commits the
test results data into a Git repository. The given path must be an
existing initialized local Git repository.
(From OE-Core rev: b6f635513ca971402e7a970acc2168fb5d4a9476)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Makes it possible to run only a subset of tests.
NOTE: The tests currently have (unwritten) dependencies on each other so
use this option with care. Mainly for debugging.
(From OE-Core rev: be4373be54e5b84f951771b0e75140f212838020)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding argparse module from Python's standard library. This allow use
argparse without installing all python-misc modules. For compatibility,
add python3-argparse as RDEPENDS to python3-misc.
(From OE-Core rev: f2b96001e074d26f5eb8711c2217a695fb02de4c)
Signed-off-by: Fabio Berton <fabio.berton@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The --wilcard-version flag was only used in the srcrev variant of the
update-recipe command.
(From OE-Core rev: d3057cba0b01484712fcee3c52373c143608a436)
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
check_patch function opens patch file in text mode. This causes
python3 to throw exception when calling readline():
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa7 in position
NNNN: invalid start byte
Opening file in binary mode and using binary type instead of strings
should fix this.
(From OE-Core rev: a7f1435c4c26237cdb55066c9f5408b4fdf016aa)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
$ python3
>>> import signal
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/path/to/sdk/sysroots/x86_64-pokysdk-linux/usr/lib/python3.5/signal.py", line 4, in <module>
from enum import IntEnum as _IntEnum
ImportError: No module named 'enum'
(From OE-Core rev: 6306dc8351c19059c4c2a8e75bb5733e64532732)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous attempt on this was a bit erroneous, dropping time stamps
completely although only the timestamp format should've been changed.
(From OE-Core rev: bafcff95e2b5e0b9a8c76ce46a62667bf6f49b00)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately to implenent multiconfig support in bitbake some APIs
had to change. This updates code in OE to match the changes in bitbake.
Its mostly periperhal changes around devtool/recipetool
[Will need a bitbake version requirement bump which I'll make when merging]
(From OE-Core rev: 041212fa37bb83acac5ce4ceb9b7b77ad172c5c3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With some recent changes in the kern tools, we can drop some changes in
the yocto-bsp and yocto-kernel tools that ensured proper patching and
BSP inheritance.
In particular, we no longer need to signify the start of patching, and
we must instruct the tools that we only want configuration fragments
via inheritance .. no patches (since they are already applied).
(From meta-yocto rev: 34ed5eebd0b5baab98b6b2d7b3f06ca40932b37d)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove timestamps from the stderr log in order to make the console
output more readable, i.e. more in line with the output from unittest
runner.
(From OE-Core rev: d28eeeabde9b4b7160a273445023a44fd50e29ab)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prevent logger from writing to stderr when the tests are being run by
the TestRunner. During this time the logger output is only written to
the log file. This way the console output from the script is cleaner and
not mixed with possible logger records.
(From OE-Core rev: 36f58b5172d4e2e182aa447fb3ec4d1ac9f6820d)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Convert scripts/oe-build-perf-test to be compatible with the new Python
unittest based buildperf test framework.
(From OE-Core rev: 249d99cd7ec00b3227c194eb4b9b21ea4dcb7315)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new class is derived from unittest.TextTestResult class. It is
actually implemented by modifying the old BuildPerfTestRunner class
which, in turn, is replaced by a totally new simple implementation
derived from unittest.TestRunner.
(From OE-Core rev: 89eb37ef1ef8d5deb87fd55c9ea7b2cfa2681b07)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds the following packages: python3-enum (needed by python3-git),
python3-selectors (needed by python3-subprocess), python3-signal (needed by python3-subprocess),
and it also fixes the following ones with missing dependencies: python3-subprocess,
python3-compression, python3-datetime
[YOCTO #10127] [YOCTO #10124] [YOCTO #10122]
(From OE-Core rev: 0575e8c9fb52a7b594025fd20445a2edd06e3c69)
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the dot parser to the new networkx API (using pydotplus to parse).
Also, switch the path display to output the paths as they are found instead of
collecting them into a list, so output appears sooner.
(From OE-Core rev: c91898b07465fdd5f3629babb7ff9226454de24e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you set up a local mirror in SSTATE_MIRRORS then you can end up with
symlinks in SSTATE_DIR rather than real files. We don't want these
symlinks in the sstate-cache prodcued by gen-lockedsig-cache, so
dereference any symlinks before copying.
(From OE-Core rev: d65a6ee9e7a9c63b9a16bdb5025af8a7c6433c4f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you check out OE-Core and then run oe-init-build-env you get an error
about not having bitbake checked out in a "bitbake" subdirectory,
however it's possible to specify the bitbake path on the
oe-init-build-env command line, so hint at that in the error message
rather than implying it has to be in the default location.
(From OE-Core rev: 5a1efa91a418e3206b047564d0fd6d5bac22a8d3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Set cover letter's subject automatically as the patch's subject when
there is only one patch.
[YOCTO #9410]
(From OE-Core rev: 162b80f8a4670befaf6ffd2c178671cf7370b767)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
So that we don't have specify "-u <contrib>" everytime, and
CPR_CONTRIB_REMOTE can be overrided by -u.
[YOCTO #9409]
(From OE-Core rev: 81c58fd33e725ce7dba693763646f4c30747bbd5)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before this patch, we need two steps to create PULL:
* Step 1, create branch:
$ git push <contrib> <local_branch>:<remote_branch>
* Step 2, create PULL:
$ create-pull-request -u <contrib> -l <local_branch> -b <remote_branch> -r <local_branch>~<n>
We can see that the args used in step 1 are in step 2, so we can use
"create-pull-request -a" or set CPR_CONTRIB_AUTO_PUSH in to create the
branch to simplify the steps.
[YOCTO #9408]
(From OE-Core rev: a569bec9219394703d1c1d9b28dd19bf5b058e7f)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
