Some of the autobuilder workers have python 3.11 or in the future 3.12.
Add these to allow toaster tests to work on those platforms.
(Bitbake rev: 04d5edd81f5e924370090d68fd4a254abaacbfe4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Cleanup an extraneous # at the end of the hash bang line.
* Cleanup some trailing whitespace errors.
* Fix typo inf dunfull -> dunfell.
(Bitbake rev: 3938b53f40960ff4a96a55fe5ef35b69737538a3)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 99305c25fafa61064a1bb0849616e4627e3453d7.
After discussion we found various versions of selenium don't work
with this change.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Test that the followings link are visible and clickable:
- OpenEmbedded
- BitBake
- Yocto Project
- Read the Toaster manual
- Contribute to Toaster
- Fixed typo: using instead of usign line:97
- Remove whitespace
(Bitbake rev: d4c35eaff3c152ee864a886aac8ad5476f7f29f7)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- check that the documentation link is visible
- check browser open new tab toaster manual when clicking on the documentation link
(Bitbake rev: befe970fcc7742cf266e03849be36c733bb09de6)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check the information icon is visible and clickable
Also if popup appearing when clicked and contain "Toaster version information"
(Bitbake rev: 0508fd74ebd43359473a9718edf0d146002041ad)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The main reason for using pytest is to be able to generate a positive test report
using the pytest-html plugin.
Integrating Pytest with Tox is a straightforward process, this can be
done using tox.ini instead of pytest.ini used to configure pytest, that
is another reason for using pytest. Tox is a tool that automates testing
across different virtual environments, it can help ensure application
will be tested against multiple Python versions and environments.
https://github.com/pytest-dev/pytest/blob/main/tox.ini
Generated reports create a historical record of test results over time.
This can help track the progress of the application's stability and quality
Documentation and Transparency: Test reports provide us a clear and detailed
documentation of the test results. They show what tests were executed,
which ones passed, and which ones failed. This transparency is critical
for understanding the current state of the application and its test coverage.
Communication: Test reports are an effective means of communication among community
to understand the testing progress and results.
Debugging, Troubleshooting Historical Tracking and Regression Testing:
In case of test failures, a detailed test report can be invaluable for debugging.
It provides information about the specific test case that failed,
the input data used, and any error messages.
(Bitbake rev: 6ba046b8d0d821e304c14b78ef6b00945e0ab453)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As mentionned in earlier commit, pytest-django takes a conservative approach
to enabling database access. By default our tests will fail if they try to access
the database, Only if we explicitly request database access will this be allowed,
using pytest marks to tell pytest-django our test needs database access.
A side effect of pytest mark, is test_case method marked is execute out of
scope of its module class, which create an inconsistance sequence and make fails
followings tests.
The scope of the ordering is global per default, e.g. tests with lower ordinal
numbers are always executed before tests with higher numbers in the same test session,
regardless of the module and class they reside in. This can be changed by using
the --order-scope option from module pytest-order.
To fix that i added execution order to tests suite using pytest-order.
(Bitbake rev: 8bf32459ab6d8ce7632fca0a4ce0f3aecc7495c2)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
toastergui-unittest-data.xml fixture contains a release objects with
name="master" or an release with same name seem existing
- Change release name and fix bitbakeversion instead of bitbake_version
(Bitbake rev: 36b48905ea26e5173b5ab2369b90952bc13e9c0d)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove load and create build environment from tests/functional/functional_helpers
- Testcases in the FunctionalTestCase do not require a build process,
- Also marked setUpClass or class with pytest django_db, db ... not
working, as declared above of file functional_helpers.py, The database access
process runs as an external process, separate from the test case process
and outside the context of pytest.
(Bitbake rev: 35c02c26541c05a8d147f41d0855e09d8fba1a83)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The pytest framework makes it easy to write small, readable tests, and can scale
to support complex functional testing for applications and libraries.
- Pytest-html is a plugin for pytest that generates a HTML report for test results.
- Pytest-django allows us to test Django project/applications with the pytest testing tool.
- Pytest-env is a pytest plugin that enables us to set environment variables in a pytest.ini or pyproject.toml file
- Pytest-metadata is a plugin for pytest that provides access to test session metadata, required by pytest-html plugin
to provides metadata information in the HTML report like below:
Python 3.10.12
Platform Linux-6.2.0-35-generic-x86_64-with-glibc2.35
Packages:
- pytest: 7.4.2
- pluggy: 1.3.0
Plugins :
- django: 4.5.2
- metadata: 3.0.0
- order: 1.1.0
- html: 4.0.2
- env: 1.1.0
To make difference between bitbake and toaster python requirements,
a dedicated requirements file is created for toaster in root of toaster dir.
(Bitbake rev: 804d17df22391b2d18a68d8cb05a04841d4e551b)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updated tox file to fix enviroments variables.
to run tox use the following command:
tox -e env_name -c /path/to/file/tox.ini
See tox cli https://tox.wiki/en/latest/cli_interface.html
The variable env_name is linked to the python versions (py38, py39, py310).
If env_name option is not set, the test suite will run for all indicated versions.
It was also modified the webdriver options for chrome to run selenium test in console because of the error detected in the previous test run
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome failed to start: exited normally.
(session not created: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
(Bitbake rev: 11dfd7c05642269b7a8bff2918667348d041ec79)
Signed-off-by: Marlon Rodriguez Garcia <marlon.rodriguez-garcia@savoirfairelinux.com>
Changes in v2
- Updated variable path BUILDDIR and EVENTREPLAY_DIR to use enviroment value
- Updated toaster-requirements.txt file path
- Removed flake8
- Added webdriver options to chrome
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes "2efb14648 toaster: Monitoring - implement Django logging system" when
running in a container.
When running in a container, the previous approach of using BASE_DIR
is not a writable path. Also, we really do not want to be writing logs into
the source tree, as the BASE_DIR was resolving to bitbake/lib/toaster/logs
Since Toaster is only ever running in an environment where oe-init-buildenv
or similar has been sourced, we should instead write the logs to BUILDDIR.
Using BUILDDIR to logs make path writable but django-log-viewer does'nt manage
to write logs using an absolute path as BUILDDIR, where the existing toaster_ui.log
was already being written.
Also drop the /logs/ directory, as it has not been created which also breaks
in a container environment
To handle the constraints linked to django-log-viewer and /logs/, we've updated
bitbake/bin/toaster to create a toaster_logs/ directory in BUILDDIR if it doesn't exist,
when toaster starts up.
Also manage to set BUILDDIR/toaster_logs/ as default location for toaster logs.
(Bitbake rev: efbd9d54f57be7a7a10f0b56e7e62c25974e99e6)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Tested-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To test custom image creation, a file for base_recipe should exists otherwise test fail,
User we need to build a base image first, that will create a file for base_recipe, which take a while.
To avoid test dependencies between test and run build, i create a tmp file for base recipe.
Signed-off-by Alassane Yattara <alassane.yattara@savoirfairelinux.com>
(Bitbake rev: 99c24df73831a72f311090b3aebcf3cc4e86851a)
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Call wait_until_visible before send_keys to be sure inputs is visible otherwise
we get followings error: selenium.common.exceptions.WebDriverException -> Message: element not interactable
Signed-off-by Alassane Yattara <alassane.yattara@savoirfairelinux.com>
(Bitbake rev: 8d671e50036acb0e65c90a07f757e066c0463113)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updated functional test file to fix url access and added waiting time after click operations
to allow following elements to be available for tests, this will needed to be revisited
when autobuilder is ready
(Bitbake rev: fa30af75b3b7d37215f2615aa4b458525fce509e)
Signed-off-by: Marlon Rodriguez Garcia <marlon.rodriguez-garcia@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update bootstrap version to 3.4.1 to fix incompatibility issues and security vulnerabilities
Files base, base_specific and js-unit-test were updated to match new versions of files
(Bitbake rev: 046bbe22f7144efa02fc33fec9c933c2e97e0ba7)
Signed-off-by: Marlon Rodriguez Garcia <marlon.rodriguez-garcia@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updated selenium version to latest 4.13.0, changed selenum specific version syntax elements to accomplish test success
(Bitbake rev: 868c88a249ef4b9fe5a891e76e25e054e4fcd994)
Signed-off-by: Marlon Rodriguez Garcia <marlon.rodriguez-garcia@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you import a build directory, Toaster still adds openembedded-core,
meta-poky and meta-yocto-bsp to the newly created project. Toaster
should only be including in the project the layers that it imported.
[YOCTO #13764]
(Bitbake rev: e73c4d7685a3bd6b806a8f1a3600a3a86266f0b6)
Signed-off-by: Kieran McNulty <Kieran.McNulty@windriver.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After updating gen_fixtures.py for 'mickledore' 4.2,
run './gen_fixutures.py --all' to update oe-core.xml
and poky.xml.
Drop langdale as support is ending soon.
(Bitbake rev: 282a3d618b565a60b8734ca0fb710db93023b5c8)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update for 'mickledore' 4.2.0 release and bitbake 2.4 branching.
Drop 'langdale' as support will be ending soon.
Update stable branches to latest patch release.
(Bitbake rev: b01d369f6d6cccf77b23017ff343820d800d5621)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Update to latest langdale 4.1.2 and kirkstone 4.0.6
* Re-instate dunfell and update to 3.1.22
- drop comments about bitbake crash
(Bitbake rev: f90c45df5cf4640e0714fde13e311db3327f9fd4)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After updating gen_fixtures.py, run ./gen_fixtures.py --all
This includes the latest stable/supported releases.
Currently excludes 'dunfell' as it seems to crash.
(Bitbake rev: 58e29fb15cd65795d6cf65a4db2b87f34649cb1f)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
