mirrors/poky
1
0
Fork 0
mirror of https://git.yoctoproject.org/poky synced 2026-01-29 21:08:42 +01:00
Code Issues Packages Projects Releases Wiki Activity

scripts/resulttool: enable manual execution and result creation

Browse Source 
Integrated “manualexecution” operation to resulttool scripts.
Manual execution script is a helper script to execute all manual
test cases in baseline command, which consists of user guideline
steps and the expected results. The last step will ask user to
provide their input to execute result. The input options are
passed/failed/blocked/skipped status. The result given will be
written in testresults.json including log error from the user
input and configuration if there is any.The output test result
for json file is created by using OEQA library.

The configuration part is manually key-in by the user. The system
allow user to specify how many configuration they want to add and
they need to define the required configuration name and value pair.
In QA perspective, "configuration" means the test environments and
parameters used during QA setup before testing can be carry out.
Example of configurations: image used for boot up, host machine
distro used, poky configurations, etc.

The purpose of adding the configuration is to standardize the
output test result format between automation and manual execution.

To use these scripts, first source oe environment, then run the
entry point script to look for help.
        $ resulttool

To execute manual test cases, execute the below
        $ resulttool manualexecution <manualjsonfile>

By default testresults.json store in <build_dir>/tmp/log/manual/

[YOCTO #12651]

(From OE-Core rev: f24dc9e87085a8fe5410feee10c7a3591fe9d816)

Signed-off-by: Mazliana <mazliana.mohamad@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit is contained in:
Mazliana
2019-02-14 13:50:38 +08:00
committed by Richard Purdie
parent 1fd5ebdb06
commit beed7523b6
2 changed files with 145 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
scripts/resulttool
View File

@@ -17,6 +17,11 @@
# To perform regression file analysis, execute the below
# $ resulttool regression-file <base_result_file> <target_result_file>
#
# To execute manual test cases, execute the below
# $ resulttool manualexecution <manualjsonfile>
#
# By default testresults.json for manualexecution store in <build>/tmp/log/manual/
#
# Copyright (c) 2019, Intel Corporation.
#
# This program is free software; you can redistribute it and/or modify it
@@ -42,6 +47,7 @@ import resulttool.merge
import resulttool.store
import resulttool.regression
import resulttool.report
import resulttool.manualexecution
logger = scriptutils.logger_create('resulttool')
def _validate_user_input_arguments(args):
@@ -58,6 +64,8 @@ def main():
parser.add_argument('-q', '--quiet', help='print only errors', action='store_true')
subparsers = parser.add_subparsers(dest="subparser_name", title='subcommands', metavar='<subcommand>')
subparsers.required = True
subparsers.add_subparser_group('manualexecution', 'manual testcases', 300)
resulttool.manualexecution.register_commands(subparsers)
subparsers.add_subparser_group('setup', 'setup', 200)
resulttool.merge.register_commands(subparsers)
resulttool.store.register_commands(subparsers)