After the fixture changes, the tests need to be tweaked unfotunately.
(Bitbake rev: 708abd1a8060684127acc7ce4142f05865005750)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the exception handling in the cancel builds function.
This involved adding WebDriverException which sometimes occurs but
also correcting the other exception handlers to continue to increment
the timeout to avoid test hangs.
(Bitbake rev: e111a2bd4f7a8a4dc2c63e94e91ac6cacca95af8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switch the clickable() calls to use the new element_clickable() function
which accepts a finder labmda function. This means if the element doesn't
yet exist, the code can rebuild the query and try again once a small
amount of time has elapsed.
There were a ton of timing related races around these element interactions
and this seemed to be the most robust way to address the issues.
The change also makes some of the elements slightly more specific so
the code can work effectively.
(Bitbake rev: 38643aadbb5a960004b886cf7709beaf2fc96652)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the element never exists, the timeout is never incremented and the test
hangs indefinitely. Fix the exception handling to avoid that and allow
the timeout to happen.
(Bitbake rev: 9eabe923d457bbce65227da4cd71c275c32108e6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code modifying the build object requires direct database access
which is potentially problematic. It was being run on a database which
was being reset after changes anyway so the code effectively did nothing.
Remove it as it is unneeded.
(Bitbake rev: 2e3fa624eb3972aef91d877337479eeb0f3dd365)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop the poll parameters and make the waits much more specific for
the requirements of the tests. This includes looping waiting for
a list of layer elements as that code was previously particularly
fragile.
(Bitbake rev: cf6b8e8aa5484110a41377ba42b3fdd9d6efd877)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switch this test module to use the common project creation code which contains
race fixes. That code requires the database access wrapper be dropped and
we no longer have ordering constraints.
There is one test that does require database access. Move this to a separate class
and allow database access there. Use ordering constraints to allow them to run
after the main code. They depend on the project creation from the other class which
isn't ideal but good enough for now.
(Bitbake rev: fa10ba2a8749415d8f06cfc15c228c6eb7df1bcf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These changes:
* Drop the poll parameters
* Add waits where needed or make them more specific
* Add calls to close notification boxes since they can obscure
other page elements
* In one case wait for a "Loading" message to be replaced
(Bitbake rev: 2bde969486a4761fa0e6f9590c9fa08d385666b1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The project creation test is well covered in another module so just
drop this one as there isn't any extra value added here.
(Bitbake rev: d26414f191c73fb5330d10d97e240150a3fa1ee8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switch this test module to use the common project creation code which contains
race fixes. That code requires the database access wrapper be dropped and
we no longer have ordering constraints.
There are two tests that do require database access. Move these to a separate class
and allow database access there. Use ordering constraints to allow them to run
after the main code. They depend on the project creation from the other class which
isn't ideal but good enough for now.
(Bitbake rev: e441bfe98ac41d48692ffbaeec90a9c780337fa4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop poll calls within the module, adding wait calls for elements of the
page where testing showed it was needed.
(Bitbake rev: 26746eae31f126ff87e678da66dcc976e412c89b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The project page selection code is race prone. Create a common function to resolve
the race issue and use it from all the call sites rather than duplicate code.
(Bitbake rev: f2bd615b97a6ff3944fa9c1d89a0ea996a12943d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop the remaining poll values and the sleep within the main wait function
as the races this was working around now seem to be resolved with the
preceeding patches.
(Bitbake rev: 91e54cbacc57316a4f2ca766afac81e7f3a54718)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The tests would race and potentially fail as the elements on the
page being waited for were not specific enough.
Add suitable elements to wait for and drop the remainder of the poll
parameters since these are no longer needed since we no longer need
to sleep for things to appear.
(Bitbake rev: 9db2fd8534610ad8db90a7a696fa7b748b497d15)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 'fail' calls are useful for find() failures but not for wrapping existing asserts
where they hide more useful error messages. Rework the code flow to give more readable
error reporting.
(Bitbake rev: 1a7a9af418874d3112aa6f148b30e5a4d3d176f3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switch this test module to use the common project creation code which contains
race fixes. That code requires the database access wrapper be dropped and
we no longer have ordering constraints.
(Bitbake rev: 653233a1c798d71800b41d3321203a48aa0337d8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switch this test module to use the common project creation code which contains
race fixes. That code requires the database access wrapper be dropped and
we no longer have ordering constraints.
(Bitbake rev: f015d96e28d89fd0aef76ff127ebde8613125a66)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tweak the function name and return the created project ID.
Allow the release_name test to be skipped with a value of None.
Add an extra wait call to ensure the create project button is ready
before clicking it.
(Bitbake rev: d20c6e3c67298c5db33b220a2e4a554e6f2c4418)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test project creation is needed by mutliple tests. Move it to the
base class as a helper function for reuse.
(Bitbake rev: abe483b98b38e95b4ad42629a67fb5d243c94094)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mixing database access and access via a running server is fraught with
danger and problems. The "django_db" marker means the transactions are
dropped at the end of the test but the transactions made via the webapi
remain so the database ends up confused at best.
Drop the database accesses and use the server API. This means slightly
abusing the typeahead to get lists of projects in the database.
Add code to delete a project if it already exists. This allows tests
to re-run against an existing database. Deletion is done using the
server API but this means handling CSRF tokens.
Add requests module requirement to requirements file since the project
creation code now uses requests.
(Bitbake rev: 738270c53a08ddc95400de70f3dd8c08b2940182)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
assetTrue is a poor choice for "x in y" since assertIn gives much more
useful output upon failure.
Change such inserts to assertIn or assertEqual to make errors easier
to debug.
(Bitbake rev: dde78e0ff8af872fdc5cdf5354174fc713141102)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
THe test fails on faster systems as we need to wait. Fix that and
a typo.
(Bitbake rev: 749b561196b0b1b388051b345fee2efb6ce68ff7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using a hardcoded database ID for a recipe is a bad idea and causes tests
to fail when we change data/versions.
Remove magic numbers.
(Bitbake rev: 6e3a958666ae4821cfa1d43c064a2294e6881638)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Consistently use meta-poky as the test target, instead of a magic number
ID which can vary.
Fix incorrect comments.
Also ensure the viewport is scrolled to the top when changing tabs.
This breaks if other layers are selected like openembedded-core due
to the number of recipes.
(Bitbake rev: 0bcb7b4a8d8e89647aa6179cc61e5eb55d433585)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"False is not True" is not a helpful error messgae. "XXX not in YYY" is
much more helpful. Tweak the test to give more meaninful failure messages
using assertIn rather than assertTrue.
(Bitbake rev: 6292b5a81ce1f3cf35980d0f669a1b52f3c56695)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the fixtures were updated for the styhead release, the test cases
were not updated to match. Fix this.
(Bitbake rev: d40bd9d35fa3850e77a1e349dab2308a54d70d60)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test_single_layer_page test case consistently fails. It is not obvious why
but if we change the argument in the following from 8 to 7 it passes.
url = reverse("layerdetails", args=(TestProjectPage.project_id, 8))
E selenium.common.exceptions.TimeoutException: Message: An element matching "#change-notification" should be visible
=========================== short test summary info ============================
FAILED ../bitbake/lib/toaster/tests/functional/test_project_page.py::TestProjectPage::test_single_layer_page
(Bitbake rev: c7e12145d8ea641925e3c06ba4f11c2dae66288a)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In line with changes in gen_fixtures.py:
* Add projectscarthgap
- Add Scarthgap to slot 1.
* Move Kirkstone down to slot 4
* Drop projectdunfell
- Drop EOL Dunfell from slot 5
(Bitbake rev: a4ae788f95d8e54713528374a9171c636aa747c5)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check some rows are visible in table instead of compare table row to
row_to_show, because recipe image table sometime doesn't display full avaiblable
images
(Bitbake rev: 1e2e5927ef7a8adfd3d0a3be1c75b4aa410d9908)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The errors causing faileds on functional_basic are dû to the delay between actions,
increase between driver actions.
(Bitbake rev: e8f8f6203b63c46249673e80872fea40475f6875)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
tests/functional/test_project_config.py::TestProjectConfig::test_set_download_dir
/home/pokybuild/yocto-worker/toaster/build/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.11/unittest/case.py:678: DeprecationWarning: It is deprecated to return a value that is not None from a test case (<bound method TestProjectConfig.test_set_download_dir of <toaster.tests.functional.test_project_config.TestProjectConfig testMethod=test_set_download_dir>>)
return self.run(*args, **kwds)
tests/functional/test_project_config.py::TestProjectConfig::test_set_sstate_dir
/home/pokybuild/yocto-worker/toaster/build/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.11/unittest/case.py:678: DeprecationWarning: It is deprecated to return a value that is not None from a test case (<bound method TestProjectConfig.test_set_sstate_dir of <toaster.tests.functional.test_project_config.TestProjectConfig testMethod=test_set_sstate_dir>>)
return self.run(*args, **kwds)
(Bitbake rev: 938cba3e80f26589ccbe34483c79e17056346fde)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
All issues and failures stemmed from a specific test case:
test_project_config_tab_right_section in the file
bitbake/lib/toaster/tests/functional/test_project_page_tab_config.py.
This test was designed to verify whether the "Most built recipes"
section on the project page correctly displays the latest and oldest
recipes built by the user, irrespective of the build outcome (failed,
cancelled, succeeded, or errored).
The errors and failures arose because the build process did not
terminate as expected, particularly when attempting to build recipe
images such as "core-image-minimal" or "bash." It was discovered that
building a real recipe/image was unnecessary for the test's purpose.
Instead, building a fake recipe like "foo" provided a reliable way to
ensure the build would fail or be interrupted.
(Bitbake rev: 5162db5305826235c09d9fcd38b5fb48ded31622)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Functional tests sometimes do not properly wait for previous tests to
close their instance of Toaster before launching their own, which
causes failures.
Track test created Toaster processes globally and wait for them to be
exited before executing further tests which need a Toaster instance to
fix this.
Additionally, quit Toaster in the teardown using the stop command with
a fallback of manually killing the processes in case of documented
stalling problem.
(Bitbake rev: 16aad11ce8eadd93b4b00dc65826329ff5526c84)
Signed-off-by: Alexander Lussier-Cullen <alexander.lussier-cullen@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Functional tests can sometimes fail to initialize toaster.
Most often this is due to a conflict on port 8000.
Add command information about whichever other process is running on
that port to better describe the initialization failure.
(Bitbake rev: da7f91d6dbe8703fb12d58ec95f077349d0005c8)
Signed-off-by: Alexander Lussier-Cullen <alexander.lussier-cullen@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster currently uses qemux86 as the default, update to match the
local.conf default changes, i.e. qemux86-64.
(Bitbake rev: 27fbba9ee15994a69284a7f8579c22d85e0ce863)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
../bitbake/lib/toaster/tests/functional/functional_helpers.py:66
/home/pokybuild/yocto-worker/toaster/build/bitbake/lib/toaster/tests/functional/functional_helpers.py:66: DeprecationWarning: invalid escape sequence '\s'
project_url=re.search("(projectPageUrl\s:\s\")(.*)(\",)",rc)
Traceback (most recent call last):
File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/home/pokybuild/yocto-worker/toaster/build/bitbake/lib/toaster/tests/commands/test_runbuilds.py", line 39, in run
os.kill(int(pid), signal.SIGTERM)
ProcessLookupError: [Errno 3] No such process
(Bitbake rev: 5a4732d5e4437cfc366c6b034868903ad6f0088c)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- Split testcases from test_project_page_tab_config into tow files
- Added new testcases in test_project_config
- Test changing distro variable
- Test setting IMAGE_INSTALL:append variable
- Test setting PACKAGE_CLASSES variable
- Test creating new bitbake variable
(Bitbake rev: 649218c648b79a89b0e91aa80d8c9bf8fa2de645)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
