Uses TEST_SERIALCONTROL_CMD to open a serial connection to the target
and execute commands. This is a drop in replacement for the ssh target,
fully supporting the same API. Supported with testexport.
To use, set the following in local.conf:
- TEST_TARGET to "serial"
- TEST_SERIALCONTROL_CMD to a shell command or script which connects to
the serial console of the target and forwards that connection to
standard input/output.
- TEST_SERIALCONTROL_EXTRA_ARGS (optional) any parameters that must be
passed to the serial control command.
- TEST_SERIALCONTROL_PS1 (optional) A regex string representing an empty
prompt on the target terminal. Example: "root@target:.*# ". This is
used to find an empty shell after each command is run. This field is
optional and will default to "root@{MACHINE}:.*# " if no other value is
given.
- TEST_SERIALCONTROL_CONNECT_TIMEOUT (optional) Specifies the timeout in
seconds for the initial connection to the target. Defaults to 10 if no
other value is given.
The serial target does have some additional limitations over the ssh
target.
1. Only supports one "run" command at a time. If two threads attempt to
call "run", one will block until it finishes. This is a limitation of
the serial link, since two connections cannot be opened at once.
2. For file transfer, the target needs a shell and the base32 program.
The file transfer implementation was chosen to be as generic as
possible, so it could support as many targets as possible.
3. Transferring files is significantly slower. On a 115200 baud serial
connection, the fastest observed speed was 30kbps. This is due to
overhead in the implementation due to decisions documented in #2
above.
(From OE-Core rev: d817b27d73d29ba2beffa2e0a4e31a14dbe0f1bf)
Signed-off-by: Andrew Oppelt <andrew.j.oppelt@boeing.com>
Signed-off-by: Matthew Weber <matthew.l.weber3@boeing.com>
Signed-off-by: Chuck Wolber <chuck.wolber@boeing.com>
--
Tested with core-image-sato on real hardware. TEST_SERIALCONTROL_CMD
was set to a bash script which connected with telnet to the target.
Additionally tested with QEMU by setting TEST_SERIALCONTROL_CMD to
"ssh -o StrictHostKeyChecking=no root@192.168.7.2". This imitates
a serial connection to the QEMU instance.
Steps:
1) Set the following in local.conf:
- IMAGE_CLASSES += "testexport"
- TEST_TARGET = "serial"
- TEST_SERIALCONTROL_CMD="ssh -o StrictHostKeyChecking=no root@192.168.7.2"
2) Build an image
- bitbake core-image-sato
3) Run the test export
- bitbake -c testexport core-image-sato
4) Run the image in qemu
- runqemu nographic core-image-sato
5) Navigate to the test export directory
6) Run the exported tests with target-type set to serial
- ./oe-test runtime --test-data-file ./data/testdata.json --packages-manifest ./data/manifest --debug --target-type serial
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The intent behind these functions was to dump the system state when issues occured
but it has never really worked as we'd planned. Regular monitoring as the build
runs has largely replaced this as that allows a trend to be seen rather than a spot
value which was never really useful. The code is bitrotting and not functioning
correctly so drop it.
[YOCTO #13872]
RP: Reword commit message
(From OE-Core rev: dea37ba49a236029da73d5cfbfc069bffc38b508)
Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Providing ssh port number is supported too with
"--target-ip 192.168.0.10:22".
(From OE-Core rev: 637919b9df0abc06da5b2f9b389cf25376bd6b7c)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
sys.path can contain duplicate entries for each layer, which means that
the search in add_controller_list() will find the same name twice and
abort.
As duplicate directories should be harmless, remove any duplicates before
iterating through the entries.
(From OE-Core rev: e478381ac1cccc5f882198fd11c8757db7e3741a)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The controller module loading code needs to be told what directories
to search for modules via the target_modules_path keyword argument, which
is set to BBPATH.
However, as the actual module loading is done via importlib this relies
on the paths being on sys.path, which it is as base.bbclass puts each
layer's lib/ in sys.path.
Simplify the code by removing this indirection, and simply search
sys.path directly.
(From OE-Core rev: 570a19581f582f77e04d6892adb647cd649a6943)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Set the json result output dir in the oeqe runtime context to create
testresults.json file by default for exported runtime test runs.
Use current datetime for the json result property name (not DATETIME
from build) to allow multiple result entries.
(From OE-Core rev: e6c73ed8d3c5d45f387cab619ca73c21e850582f)
Signed-off-by: Stefan Kral <sk@typedivision.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Similar to the existing possibility to specify a port in
TEST_TARGET_IP, allow TEST_SERVER_IP to also contain a
port.
The intention is for this port to be passed into e.g.
the http server from the apt / dnf / opkg tests, or
any other (custom) tests that might need the target to
connect to a service spawned by bitbake / oeqa, where
bitbake is actually running inside a docker container.
(From OE-Core rev: 9f401cd8b010ca613a151d0323b251f2243f399f)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When loading controllers as (external) modules, the code currently
tries to load all files ending with .py. This is a problem when
during development using an editor that creates a lock-file
in the same directory as the .py file, as the lock file is
typically called '.#xxxx.py'.
Python will try to load the lock file and fail miserably with
an exception:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:do_testimage(d)
0003:
File: 'poky/meta/classes/testimage.bbclass', lineno: 114, function: do_testimage
0110: netstat -an
0111:}
0112:
0113:python do_testimage() {
*** 0114: testimage_main(d)
0115:}
0116:
0117:addtask testimage
0118:do_testimage[nostamp] = "1"
File: 'poky/meta/classes/testimage.bbclass', lineno: 294, function: testimage_main
0290:
0291: # the robot dance
0292: target = OERuntimeTestContextExecutor.getTarget(
0293: d.getVar("TEST_TARGET"), logger, d.getVar("TEST_TARGET_IP"),
*** 0294: d.getVar("TEST_SERVER_IP"), **target_kwargs)
0295:
0296: # test context
0297: tc = OERuntimeTestContext(td, logger, target, host_dumper,
0298: image_packages, extract_dir)
File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 116, function: getTarget
0112: # XXX: Don't base your targets on this code it will be refactored
0113: # in the near future.
0114: # Custom target module loading
0115: target_modules_path = kwargs.get('target_modules_path', '')
*** 0116: controller = OERuntimeTestContextExecutor.getControllerModule(target_type, target_modules_path)
0117: target = controller(logger, target_ip, server_ip, **kwargs)
0118:
0119: return target
0120:
File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 128, function: getControllerModule
0124: # ImportError raised if a provided module can not be imported.
0125: @staticmethod
0126: def getControllerModule(target, target_modules_path):
0127: controllerslist = OERuntimeTestContextExecutor._getControllerModulenames(target_modules_path)
*** 0128: controller = OERuntimeTestContextExecutor._loadControllerFromName(target, controllerslist)
0129: return controller
0130:
0131: # Return a list of all python modules in lib/oeqa/controllers for each
0132: # layer in bbpath
File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 163, function: _loadControllerFromName
0159: # Raise ImportError if a provided module can not be imported
0160: @staticmethod
0161: def _loadControllerFromName(target, modulenames):
0162: for name in modulenames:
*** 0163: obj = OERuntimeTestContextExecutor._loadControllerFromModule(target, name)
0164: if obj:
0165: return obj
0166: raise AttributeError("Unable to load {0} from available modules: {1}".format(target, str(modulenames)))
0167:
File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 173, function: _loadControllerFromModule
0169: @staticmethod
0170: def _loadControllerFromModule(target, modulename):
0171: obj = None
0172: # import module, allowing it to raise import exception
*** 0173: module = __import__(modulename, globals(), locals(), [target])
0174: # look for target class in the module, catching any exceptions as it
0175: # is valid that a module may not have the target class.
0176: try:
0177: obj = getattr(module, target)
Exception: ImportError: No module named 'oeqa.controllers.'
Simply ignore those when collecting the list of files to try
to load.
(From OE-Core rev: 682f223cf2e2dabe8cf60634b6779bb2d5e359bd)
Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.
The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).
More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.
The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.
(From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are failures when multiple users run oe-selftest on the same
host:
PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved-
tests/201812250324_qemu'
This is because /tmp/oe-saved-tests was created by user A, while user B tries
to write data in it, then the error will happen. This patch can fix the
problem.
Move the dumped data to ${LOG_DIR}/runtime-hostdump/ rather than
/tmp/oe-saved-tests/ to fix the problem.
(From OE-Core rev: e219fe5329599cd6c3682f521eaee3852a2c8980)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Qemu do not use the ip input from external. It will
retrieve ip from QemuRunner instance and assign
ip value.
(From OE-Core rev: 14d99dc6c39c963ba3e0d9a30274846bd5369210)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There were a lot of assumptions in the controller import
code of context.py which were not true anymore. These
assumptions reflected themselves by catching exceptions and
re-raising them with unhelpful error messages in other parts
of the code.
This commit does not fix the classes controller classes that
became broken after the refactor but at least it allows for
the exceptions to be thrown where the imports fail, thus
actually showing what exactly went wrong with the import.
An example of such an improvement is that before if the
controller class failed during it's init contructor the
controller would just be skipped and the task would just
complain it could not find the controller. Now for example,
if there is a NamerError due to a variable not being
declared, the user will get that report.
(From OE-Core rev: 719b0e88b875b067551067874778061f99b8c4b4)
Signed-off-by: Paulo Neves <ptsneves@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The idea on getTarget is to use kwargs to send custom variables
to different targets, instead of this, a new variable was added
(just used for custom targets) and this broke testexport. So
in order to fix it, just add the custom variable to kwargs.
This fixes the use of getTarget() in testexport class that was
introduced in 1dc8010afd71fe46fb28bb86fb7c07a5fbd3d7cf.
(From OE-Core rev: cf138029a1f18f991fc7a28c81d85884942e9d56)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch implements custom target loading for testimage, currently
missing due to major changes to the test framework.
Custom targets can be defined in various meta-layers, so we
need an extra path information in order to find them.
Any other additional info is retrieved as usual via the variables
TEST_TARGET and TEST_SUITES
(From OE-Core rev: 1dc8010afd71fe46fb28bb86fb7c07a5fbd3d7cf)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds default values to OERuntimeTestContextExecutor class in
order to make easier the execution of exported test that were
generated with testexport class.
[YOCTO #10686]
(From OE-Core rev: c78aeaac3b75610bada62b138c9670815a07ee80)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows to use the package install feature with
the new OEQA framework.
[YOCTO #10234]
(From OE-Core rev: 077dc19445574457769eb4f231de97e8059cb75e)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current targets (ssh and qemu) require a logger in their
constructors, so in order to get a new target we need
to provide the logger.
[YOCTO #10686]
(From OE-Core rev: 26bae3c255bc1e1cc8d81db0cffc809de0182a43)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit will prepare context to use qemu and not just
simpleremote.
(From OE-Core rev: 7485a08c967916fb6edff4cc573d9314ec577031)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As part of the refactor we require to move the files used
in runtime testing to the new directory. This also adds
the path to the runtime test context.
[YOCTO #10234]
(From OE-Core rev: f80c21b46f69c42ff0853bf8abd01cf0c082c346)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds OERuntimeTestCase, OERuntimeTestContext, and OERuntimeTestLoader
to be used for runtime testing.
As expected there are some changes in runtime context:
- Adds the target to be used for runtime testing, the default
is a SSH connection to the device under test running a OE image.
- Runtime context requires image manifest because several
tests are skipped if a package is missing or installed.
- Several tests require the output of the ps command and it changes
its output and arguments if busybox o procps is installed, so the
case must use the correct ps command.
[YOCTO #10234]
(From OE-Core rev: f995f178de79d6d11422cd879d06371811f50651)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
