diff --git a/documentation/dev-manual/packages.rst b/documentation/dev-manual/packages.rst index 33cf78747d..4ba2dcae3a 100644 --- a/documentation/dev-manual/packages.rst +++ b/documentation/dev-manual/packages.rst @@ -16,7 +16,7 @@ This section describes a few tasks that involve packages: - :ref:`dev-manual/packages:generating and using signed packages` - :ref:`Setting up and running package test - (ptest) ` + (ptest) ` - :ref:`dev-manual/packages:creating node package manager (npm) packages` @@ -882,114 +882,8 @@ related to signed package feeds are available: Testing Packages With ptest =========================== -A Package Test (ptest) runs tests against packages built by the -OpenEmbedded build system on the target machine. A ptest contains at -least two items: the actual test, and a shell script (``run-ptest``) -that starts the test. The shell script that starts the test must not -contain the actual test --- the script only starts the test. On the other -hand, the test can be anything from a simple shell script that runs a -binary and checks the output to an elaborate system of test binaries and -data files. - -The test generates output in the format used by Automake:: - - result: testname - -where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and -the testname can be any identifying string. - -For a list of Yocto Project recipes that are already enabled with ptest, -see the :yocto_wiki:`Ptest ` wiki page. - -.. note:: - - A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` - class. - -Adding ptest to Your Build --------------------------- - -To add package testing to your build, add the :term:`DISTRO_FEATURES` and -:term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which -is found in the :term:`Build Directory`:: - - DISTRO_FEATURES:append = " ptest" - EXTRA_IMAGE_FEATURES += "ptest-pkgs" - -Once your build is complete, the ptest files are installed into the -``/usr/lib/package/ptest`` directory within the image, where ``package`` -is the name of the package. - -Running ptest -------------- - -The ``ptest-runner`` package installs a shell script that loops through -all installed ptest test suites and runs them in sequence. Consequently, -you might want to add this package to your image. - -Getting Your Package Ready --------------------------- - -In order to enable a recipe to run installed ptests on target hardware, -you need to prepare the recipes that build the packages you want to -test. Here is what you have to do for each recipe: - -- *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* - Include the following line in each recipe:: - - inherit ptest - -- *Create run-ptest:* This script starts your test. Locate the - script where you will refer to it using - :term:`SRC_URI`. Here is an - example that starts a test for ``dbus``:: - - #!/bin/sh - cd test - make -k runtest-TESTS - -- *Ensure dependencies are met:* If the test adds build or runtime - dependencies that normally do not exist for the package (such as - requiring "make" to run the test suite), use the - :term:`DEPENDS` and - :term:`RDEPENDS` variables in - your recipe in order for the package to meet the dependencies. Here - is an example where the package has a runtime dependency on "make":: - - RDEPENDS:${PN}-ptest += "make" - -- *Add a function to build the test suite:* Not many packages support - cross-compilation of their test suites. Consequently, you usually - need to add a cross-compilation function to the package. - - Many packages based on Automake compile and run the test suite by - using a single command such as ``make check``. However, the host - ``make check`` builds and runs on the same computer, while - cross-compiling requires that the package is built on the host but - executed for the target architecture (though often, as in the case - for ptest, the execution occurs on the host). The built version of - Automake that ships with the Yocto Project includes a patch that - separates building and execution. Consequently, packages that use the - unaltered, patched version of ``make check`` automatically - cross-compiles. - - Regardless, you still must add a ``do_compile_ptest`` function to - build the test suite. Add a function similar to the following to your - recipe:: - - do_compile_ptest() { - oe_runmake buildtest-TESTS - } - -- *Ensure special configurations are set:* If the package requires - special configurations prior to compiling the test code, you must - insert a ``do_configure_ptest`` function into the recipe. - -- *Install the test suite:* The :ref:`ref-classes-ptest` class - automatically copies the file ``run-ptest`` to the target and then runs make - ``install-ptest`` to run the tests. If this is not enough, you need - to create a ``do_install_ptest`` function and make sure it gets - called after the "make install-ptest" completes. +See the :ref:`test-manual/ptest:Testing Packages With ptest` section of the +Yocto Project Test Environment Manual. Creating Node Package Manager (NPM) Packages ============================================ diff --git a/documentation/migration-guides/migration-1.6.rst b/documentation/migration-guides/migration-1.6.rst index 916169e836..b052a43a31 100644 --- a/documentation/migration-guides/migration-1.6.rst +++ b/documentation/migration-guides/migration-1.6.rst @@ -221,7 +221,7 @@ Package Test (ptest) Package Tests (ptest) are built but not installed by default. For information on using Package Tests, see the -":ref:`dev-manual/packages:testing packages with ptest`" section in the +":ref:`test-manual/ptest:testing packages with ptest`" section in the Yocto Project Development Tasks Manual. See also the ":ref:`ref-classes-ptest`" section. diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst index 8c3cc0140f..f416a6d623 100644 --- a/documentation/ref-manual/classes.rst +++ b/documentation/ref-manual/classes.rst @@ -2615,7 +2615,7 @@ runtime tests for recipes that build software that provides these tests. This class is intended to be inherited by individual recipes. However, the class' functionality is largely disabled unless "ptest" appears in :term:`DISTRO_FEATURES`. See the -":ref:`dev-manual/packages:testing packages with ptest`" +":ref:`test-manual/ptest:testing packages with ptest`" section in the Yocto Project Development Tasks Manual for more information on ptest. @@ -2639,7 +2639,7 @@ Enables package tests (ptests) specifically for GNOME packages, which have tests intended to be executed with ``gnome-desktop-testing``. For information on setting up and running ptests, see the -":ref:`dev-manual/packages:testing packages with ptest`" +":ref:`test-manual/ptest:testing packages with ptest`" section in the Yocto Project Development Tasks Manual. .. _ref-classes-python3-dir: diff --git a/documentation/ref-manual/features.rst b/documentation/ref-manual/features.rst index b2ba731bb2..face6c1062 100644 --- a/documentation/ref-manual/features.rst +++ b/documentation/ref-manual/features.rst @@ -207,7 +207,7 @@ metadata, as extra layers can define their own: - *ptest:* Enables building the package tests where supported by individual recipes. For more information on package tests, see the - ":ref:`dev-manual/packages:testing packages with ptest`" section + ":ref:`test-manual/ptest:testing packages with ptest`" section in the Yocto Project Development Tasks Manual. - *pulseaudio:* Include support for diff --git a/documentation/ref-manual/qa-checks.rst b/documentation/ref-manual/qa-checks.rst index 1b83834df1..27d46de3fd 100644 --- a/documentation/ref-manual/qa-checks.rst +++ b/documentation/ref-manual/qa-checks.rst @@ -791,7 +791,7 @@ Errors and Warnings This check will detect if the source of the package contains some upstream-provided tests and, if so, that ptests are implemented for this - recipe. See the ":ref:`dev-manual/packages:testing packages with ptest`" + recipe. See the ":ref:`test-manual/ptest:testing packages with ptest`" section in the Yocto Project Development Tasks Manual. See also the ":ref:`ref-classes-ptest`" section. diff --git a/documentation/ref-manual/release-process.rst b/documentation/ref-manual/release-process.rst index 691e7d3ed9..3383a4e1b0 100644 --- a/documentation/ref-manual/release-process.rst +++ b/documentation/ref-manual/release-process.rst @@ -175,7 +175,7 @@ consists of the following pieces: operation and functions. However, the test can also use the IP address of a machine to test. -- :ref:`ptest `: +- :ref:`ptest `: Runs tests against packages produced during the build for a given piece of software. The test allows the packages to be run within a target image. diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst index 869100ce34..200def084f 100644 --- a/documentation/ref-manual/variables.rst +++ b/documentation/ref-manual/variables.rst @@ -6828,7 +6828,7 @@ system and gives an overview of their function and contents. :term:`PTEST_ENABLED` Specifies whether or not :ref:`Package - Test ` (ptest) + Test ` (ptest) functionality is enabled when building a recipe. You should not set this variable directly. Enabling and disabling building Package Tests at build time should be done by adding "ptest" to (or removing it diff --git a/documentation/test-manual/index.rst b/documentation/test-manual/index.rst index 86a2f436ea..ad71f37910 100644 --- a/documentation/test-manual/index.rst +++ b/documentation/test-manual/index.rst @@ -12,6 +12,7 @@ Yocto Project Test Environment Manual intro test-process + ptest understand-autobuilder reproducible-builds yocto-project-compatible diff --git a/documentation/test-manual/intro.rst b/documentation/test-manual/intro.rst index c31fd11c7a..7bbd11295f 100644 --- a/documentation/test-manual/intro.rst +++ b/documentation/test-manual/intro.rst @@ -140,7 +140,7 @@ the following types of tests: - *Package Testing:* A Package Test (ptest) runs tests against packages built by the OpenEmbedded build system on the target machine. See the :ref:`Testing Packages With - ptest ` section + ptest ` section in the Yocto Project Development Tasks Manual and the ":yocto_wiki:`Ptest `" Wiki page for more information on Ptest. diff --git a/documentation/test-manual/ptest.rst b/documentation/test-manual/ptest.rst new file mode 100644 index 0000000000..dea1bad23b --- /dev/null +++ b/documentation/test-manual/ptest.rst @@ -0,0 +1,114 @@ +.. SPDX-License-Identifier: CC-BY-SA-2.0-UK + +*************************** +Testing Packages With ptest +*************************** + +A Package Test (ptest) runs tests against packages built by the +OpenEmbedded build system on the target machine. A ptest contains at +least two items: the actual test, and a shell script (``run-ptest``) +that starts the test. The shell script that starts the test must not +contain the actual test --- the script only starts the test. On the other +hand, the test can be anything from a simple shell script that runs a +binary and checks the output to an elaborate system of test binaries and +data files. + +The test generates output in the format used by Automake:: + + result: testname + +where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and +the testname can be any identifying string. + +For a list of Yocto Project recipes that are already enabled with ptest, +see the :yocto_wiki:`Ptest ` wiki page. + +.. note:: + + A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest` + class. + +Adding ptest to Your Build +========================== + +To add package testing to your build, add the :term:`DISTRO_FEATURES` and +:term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which +is found in the :term:`Build Directory`:: + + DISTRO_FEATURES:append = " ptest" + EXTRA_IMAGE_FEATURES += "ptest-pkgs" + +Once your build is complete, the ptest files are installed into the +``/usr/lib/package/ptest`` directory within the image, where ``package`` +is the name of the package. + +Running ptest +============= + +The ``ptest-runner`` package installs a shell script that loops through +all installed ptest test suites and runs them in sequence. Consequently, +you might want to add this package to your image. + +Getting Your Package Ready +========================== + +In order to enable a recipe to run installed ptests on target hardware, +you need to prepare the recipes that build the packages you want to +test. Here is what you have to do for each recipe: + +- *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:* + Include the following line in each recipe:: + + inherit ptest + +- *Create run-ptest:* This script starts your test. Locate the + script where you will refer to it using + :term:`SRC_URI`. Here is an + example that starts a test for ``dbus``:: + + #!/bin/sh + cd test + make -k runtest-TESTS + +- *Ensure dependencies are met:* If the test adds build or runtime + dependencies that normally do not exist for the package (such as + requiring "make" to run the test suite), use the + :term:`DEPENDS` and + :term:`RDEPENDS` variables in + your recipe in order for the package to meet the dependencies. Here + is an example where the package has a runtime dependency on "make":: + + RDEPENDS:${PN}-ptest += "make" + +- *Add a function to build the test suite:* Not many packages support + cross-compilation of their test suites. Consequently, you usually + need to add a cross-compilation function to the package. + + Many packages based on Automake compile and run the test suite by + using a single command such as ``make check``. However, the host + ``make check`` builds and runs on the same computer, while + cross-compiling requires that the package is built on the host but + executed for the target architecture (though often, as in the case + for ptest, the execution occurs on the host). The built version of + Automake that ships with the Yocto Project includes a patch that + separates building and execution. Consequently, packages that use the + unaltered, patched version of ``make check`` automatically + cross-compiles. + + Regardless, you still must add a ``do_compile_ptest`` function to + build the test suite. Add a function similar to the following to your + recipe:: + + do_compile_ptest() { + oe_runmake buildtest-TESTS + } + +- *Ensure special configurations are set:* If the package requires + special configurations prior to compiling the test code, you must + insert a ``do_configure_ptest`` function into the recipe. + +- *Install the test suite:* The :ref:`ref-classes-ptest` class + automatically copies the file ``run-ptest`` to the target and then runs make + ``install-ptest`` to run the tests. If this is not enough, you need + to create a ``do_install_ptest`` function and make sure it gets + called after the "make install-ptest" completes.