The parent code currently assumed that any parent branch is locally
checked out which may not be the case.
Use the local branch by default but fall back to the origin. This
also means removing the later saftey check as the branch may not exist
locally.
This fixes the autobuilder resulttool test pushing code.
(From OE-Core rev: 0ee636bc89d93282cb3d3cd747b3dd046f6e5a9c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a combined patch of the various tweaks and improvements I
made to resulttool:
* Avoid subprocess.run() as its a python 3.6 feature and we
have autobuilder workers with 3.5.
* Avoid python keywords as variable names
* Simplify dict accesses using .get()
* Rename resultsutils -> resultutils to match the resultstool ->
resulttool rename
* Formalised the handling of "file_name" to "TESTSERIES" which the code
will now add into the json configuration data if its not present, based
on the directory name.
* When we don't have failed test cases, print something saying so
instead of an empty table
* Tweak the table headers in the report to be more readable (reference
"Test Series" instead if file_id and ID instead of results_id)
* Improve/simplify the max string length handling
* Merge the counts and percentage data into one table in the report
since printing two reports of the same data confuses the user
* Removed the confusing header in the regression report
* Show matches, then regressions, then unmatched runs in the regression
report, also remove chatting unneeded output
* Try harder to "pair" up matching configurations to reduce noise in
the regressions report
* Abstracted the "mapping" table concept used to pairing in the
regression code to general code in resultutils
* Created multiple mappings for results analysis, results storage and
'flattening' results data in a merge
* Simplify the merge command to take a source and a destination,
letting the destination be a directory or a file, removing the need for
an output directory parameter
* Add the 'IMAGE_PKGTYPE' and 'DISTRO' config options to the regression
mappings
* Have the store command place the testresults files in a layout from
the mapping, making commits into the git repo for results storage more
useful for simple comparison purposes
* Set the oe-git-archive tag format appropriately for oeqa results
storage (and simplify the commit messages closer to their defaults)
* Fix oe-git-archive to use the commit/branch data from the results file
* Cleaned up the command option help to match other changes
* Follow the model of git branch/tag processing used by oe-build-perf-report
and use that to read the data using git show to avoid branch change
* Add ptest summary to the report command
* Update the tests to match the above changes
(From OE-Core rev: ff2c029b568f70aa9960dde04ddd207829812ea0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Integrated “manualexecution” operation to resulttool scripts.
Manual execution script is a helper script to execute all manual
test cases in baseline command, which consists of user guideline
steps and the expected results. The last step will ask user to
provide their input to execute result. The input options are
passed/failed/blocked/skipped status. The result given will be
written in testresults.json including log error from the user
input and configuration if there is any.The output test result
for json file is created by using OEQA library.
The configuration part is manually key-in by the user. The system
allow user to specify how many configuration they want to add and
they need to define the required configuration name and value pair.
In QA perspective, "configuration" means the test environments and
parameters used during QA setup before testing can be carry out.
Example of configurations: image used for boot up, host machine
distro used, poky configurations, etc.
The purpose of adding the configuration is to standardize the
output test result format between automation and manual execution.
To use these scripts, first source oe environment, then run the
entry point script to look for help.
$ resulttool
To execute manual test cases, execute the below
$ resulttool manualexecution <manualjsonfile>
By default testresults.json store in <build_dir>/tmp/log/manual/
[YOCTO #12651]
(From OE-Core rev: f24dc9e87085a8fe5410feee10c7a3591fe9d816)
Signed-off-by: Mazliana <mazliana.mohamad@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OEQA outputs test results into json files and these files were
archived by Autobuilder during QA releases. Example: each oe-selftest
run by Autobuilder for different host distro generate a
testresults.json file.
These scripts were developed as a test result tools to manage
these testresults.json file.
Using the "store" operation, user can store multiple testresults.json
files as well as the pre-configured directories used to hold those files.
Using the "merge" operation, user can merge multiple testresults.json
files to a target file.
Using the "report" operation, user can view the test result summary
for all available testresults.json files inside a ordinary directory
or a git repository.
Using the "regression-file" operation, user can perform regression
analysis on testresults.json files specified. Using the "regression-dir"
and "regression-git" operations, user can perform regression analysis
on directory and git accordingly.
These resulttool operations expect the testresults.json file to use
the json format below.
{
"<testresult_1>": {
"configuration": {
"<config_name_1>": "<config_value_1>",
"<config_name_2>": "<config_value_2>",
...
"<config_name_n>": "<config_value_n>",
},
"result": {
"<testcase_namespace_1>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
"<testcase_namespace_2>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
...
"<testcase_namespace_n>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
}
},
...
"<testresult_n>": {
"configuration": {
"<config_name_1>": "<config_value_1>",
"<config_name_2>": "<config_value_2>",
...
"<config_name_n>": "<config_value_n>",
},
"result": {
"<testcase_namespace_1>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
"<testcase_namespace_2>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
...
"<testcase_namespace_n>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
}
},
}
To use these scripts, first source oe environment, then run the
entry point script to look for help.
$ resulttool
To store test result from oeqa automated tests, execute the below
$ resulttool store <source_dir> <git_branch>
To merge multiple testresults.json files, execute the below
$ resulttool merge <base_result_file> <target_result_file>
To report test report, execute the below
$ resulttool report <source_dir>
To perform regression file analysis, execute the below
$ resulttool regression-file <base_result_file> <target_result_file>
To perform regression dir analysis, execute the below
$ resulttool regression-dir <base_result_dir> <target_result_dir>
To perform regression git analysis, execute the below
$ resulttool regression-git <source_dir> <base_branch> <target_branch>
[YOCTO# 13012]
[YOCTO# 12654]
(From OE-Core rev: 78a322d7be402a5b9b5abf26ad35670a8535408a)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These functions can be reused by the resulttool code so move to the common
function library for this purpose.
(From OE-Core rev: c66f848938c04e133259c5b6903dc592866ab385)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The license checksum changed due to a change in copyright years only.
(From OE-Core rev: f45e890838544bcb77cf2e8503994ef894d3071e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The license checksum changed due to copyright year changes.
(From OE-Core rev: a50aa22a229e8c13d36528a9e5a776688a5133a7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The following two patches are removed as they are not needed anymore.
0001-Unset-need_charset_alias-when-building-for-musl.patch
0001-explicitly-disable-replacing-getopt.patch
(From OE-Core rev: cc8c9b36258d0bb1635b2d4644d7fc64357c5110)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Meant to fix this pre merge, the git revision is the PV so no need to
add SRCPV.
(From OE-Core rev: 3233096f0dbada97d9d71c98044150a641bfb0a5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove the musl specific do_install, as it's not suitable for this
version.
(From OE-Core rev: 348a96a5b4016a7615f8d22c03ec1ced60367c3b)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
During the test logrotate.LogrotateTest.test_1_logrotate_setup,
there is below logic:
# mkdir $HOME/logrotate_dir
# sed -i "s#wtmp {#wtmp {\n olddir $HOME/logrotate_dir#" /etc/logrotate.d/wtmp
After all logrotate.LogrotateTest finished, only cleanup
$HOME/logrotate_dir as below, but don't restore
the config file /etc/logrotate.d/wtmp.
[snip]
def tearDownClass(cls):
cls.tc.target.run('rm -rf $HOME/logrotate_dir')
[snip]
That's to say, there is one additional line added
to /etc/logrotate.d/wtmp and will make the logrotate
service start failed when run systemd.SystemdBasicTests.test_systemd_failed
Take an example as below when run test as root:
# cat /etc/logrotate.d/wtmp
# no packages own wtmp -- we'll rotate it here
/var/log/wtmp {
olddir /root/logrotate_dir
missingok
monthly
create 0664 root utmp
minsize 1M
rotate 1
}
# ls /root/logrotate_dir
ls: cannot access '/root/logrotate_dir': No such file or directory
# systemctl start logrotate
Job for logrotate.service failed because the control process exited with error code.
See "systemctl status logrotate.service" and "journalctl -xe" for details.
# systemctl status logrotate
logrotate.service - Rotate log files
Loaded: loaded (/lib/systemd/system/logrotate.service; static; vendor preset>
Active: failed (Result: exit-code) since Wed 2019-02-13 03:35:19 UTC; 7s ago
Docs: man:logrotate(8)
man:logrotate.conf(5)
Process: 540 ExecStart=/usr/sbin/logrotate /etc/logrotate.conf (code=exited, status=1/FAILURE)
Main PID: 540 (code=exited, status=1/FAILURE)
Feb 13 03:35:18 qemumips systemd[1]: Starting Rotate log files...
Feb 13 03:35:19 qemumips logrotate[540]: error: wtmp:9 error verifying olddir path /root/logrotate_dir: No such file or directory
Feb 13 03:35:19 qemumips logrotate[540]: error: found error in file wtmp, skipping
Feb 13 03:35:19 qemumips systemd[1]: logrotate.service: Main process exited, code=exited, status=1/FAILURE
Feb 13 03:35:19 qemumips systemd[1]: logrotate.service: Failed with result 'exit-code'.
Feb 13 03:35:19 qemumips systemd[1]: Failed to start Rotate log files.
Add the logic to restore /etc/logrotate.d/wtmp to
make the cleanup complete to fix the above issue.
(From OE-Core rev: a2db9320d97d12d87524ff16a329f9c38a8da33f)
Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for gobject-introspection.
Add v2 API fixes to the gtkdoc/gi disabling patch.
Add a dependency on glib-native to allow use of glib-mkenums.
Disable v2 API/library for now as libdnf is not compatible with it,
yet is erroneously linked against it (if it's enabled).
(From OE-Core rev: 3db1883a109b520909b12e91028ce8a79923e8f5)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Backport the CVE patch from the upstream
commit 3f635fb43389b54f682fc9ed2acc0b2aaf4a923d
(From OE-Core rev: 518be39ac82593c539144ac83acc459a45b7a81d)
Signed-off-by: Zhixiong Chi <zhixiong.chi@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
recipe glade do_configure failed after python upgrade
to 3.7.2, it will do runtime check of python headers
with native python3 under recipe-sysroot-native, it will check under
path INCLDIRSTOMAKE, but this value is '/usr/include /usr/include
/usr/include/python3.7m /usr/include/python3.7m', which will cause
below error:
ERROR: This autoconf log indicates errors, it looked at host include and/or
library paths while determining system capabilities.
fix it by replace /usr/include to STAGING_INCDIR
(From OE-Core rev: 4975a283478ec3cbde813a1b27ab318de4aa8051)
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
