For better long-term maintainability, use ToasterTable instead
of Django template and view code to display the all builds page.
NB the builds.html template has been left in, as this will
otherwise cause conflicts when merging the new theme.
[YOCTO #8738]
(Bitbake rev: e0590fc8103afeb4c5e613a826057555c8193d59)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The algorithm for finding the suffix for image files produced by
the build doesn't reference a list of known file suffixes, so
could be prone to error.
Modify how file suffixes are parsed from the file path so that
they are compared against a list of known types; if this fails,
use the part of the basename of the file path after the first
'.' character.
Also rationalise the places in the views code where we
extract the file name extensions for builds, so they both use
the same algorithm (before, the same code was duplicated in
two places).
[YOCTO #8417]
(Bitbake rev: dd1c509696b8ab5e593cc64637060a58e95fcd1f)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Image file suffixes are used in the project configuration page to
show a list of available image file types. This list is stored
as a function in the views code.
However, this list is also needed when parsing image file paths,
so that the suffixes can be shown in the "all builds" and "project
builds" tables.
Move the list of valid image file suffixes to the Target_Image_File
class to make is accessible in other places where it may be needed.
[YOCTO #8738]
(Bitbake rev: c2f20232077917552623fd0726d0820e50b04cae)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed 'database is locked' issue by monkey patching django QuerySet
methods.
The actual patching places were found by bisecting Django codebase.
This commit should be removed after Django is fixed if it's fixed
at all.
(Bitbake rev: 175411bf05423b1892c7928c2b928843b39645f0)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When sqlite can not cope with a stream of 'insert' queries it throws
'database is locked' exception.
Wrapping model.save in transaction.atomic context and repeating the call
should solve this issue.
(Bitbake rev: eb305308ca8f6228c6f52dac1bd941f29c7e5eb6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Printing id as a number breaks toaster if object is not saved
yet as id/pk is None in this case.
Changed format spec to print id as a string in __str__
and __unicode__ methods of Build, Task and LayerVersion models.
(Bitbake rev: d5ca2bae623db214764324da5c9e4dac6beef760)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove compatible_layerversions() and replace calls to it with
calls to get_all_compatible_layer_versions().
The sorting done in compatible_layerversions() is not relevant,
and the code can be simplified by using get_all_compatible_layer_versions().
[YOCTO #8364]
(Bitbake rev: 7bca51277314f7c0b6ee0e0d470327dfd60d37fe)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Renamed get_query_set to get_queryset as get_queryset causes
deprecation warning in Django 1.7 and is deprecated in Django 1.8
(Bitbake rev: ef8b399dccd413070e32ce05e013337ea01fa64d)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than maintain data as part of the migrations (as was
done for the default project previously), create the default
(cli builds) project on demand as a by-product of getting
it from the database.
[YOCTO #8364]
(Bitbake rev: 5fd8e90ab9b81d1bd0d301bc1c91228ecbbea74b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
South has been deprecated in favour of Django's own migration framework,
so remove the old South migrations and replace them with Django ones.
[YOCTO #8364]
(Bitbake rev: 427d8bc02d1aa00a19057602d592d58334514804)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The count of a project's builds should not include those which are
currently in progress.
(Bitbake rev: a981700701c41c7bbb6a9778e95f691278c5c294)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Log messages can be quite long so use a TextField rather than a char
field with max length of 240. mySQL is especially picky about field
lengths and will cause an exception if the log is too long.
[YOCTO #8475]
(Bitbake rev: 349119f44cdad240dda7f4db45992e2539e1416f)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fatal build errors were displayed as exceptions, and highlighted
with less severity than they deserved.
Roll back to treating Toaster exceptions as errors by removing
the toaster_exceptions member on Build objects and displaying
EXCEPTION events in the errors section on the dashboard.
[YOCTO #8320]
(Bitbake rev: 59197320bdcefddf06084e871f1b5b21b21cbb63)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Critical errors (where a build failed for reasons of
misconfiguration, such as a machine being specified which is not
in a project's layers) were being ignored (only log records
up to ERROR level were being logged to Toaster's db). This meant that
the build would fail but would not correctly report why.
Add support for CRITICAL error levels to the LogMessage model,
include errors at this level in the errors property for a build,
and show errors at this level in the build dashboard.
[YOCTO #8320]
(Bitbake rev: b6eacbca9cacb607de864ab7d093deb296da8226)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Builds initiated from the command line don't have a buildrequest
associated with them. The build.buildrequest association is
only added if a build is triggered from toaster.
Some of the code for displaying the status of a build refers
to build.buildrequest without checking whether it has been set,
which causes an error to be thrown.
Add a guard to check whether the buildrequest has been set.
[YOCTO #8277]
(Bitbake rev: af33409612139ab2347baf6b847b23faea19752d)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modified get_all_compatible_recipes function to exclude recipes
with empty names from the result queryset. This should stop UI
to show recipes with empty names in compatible recipes page.
[YOCTO #7969]
(Bitbake rev: f7c1bd49e2e28d12c6604f5ae54bc96e1e7b6946)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to Recipe model constraints some recipes can't be
saved. However, they still can create incomplete records in
the database. This causes all sorts of errors when Toaster
operates with those objects. Removing them should fix those
issues.
[YOCTO #7969]
(Bitbake rev: 4d76a9e418fd98a7882aa29f974a7389f9689314)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Cleaned up and fixed orm tests. Removed test_build_layerversion as
it's not needed due to changed compatible_layer_versions API.
(Bitbake rev: 73ea29ed065bfaa80ee368b2a38c157e36fe1676)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implemented Layer_Version.get_alldeps API to recursively get
full list of dependencies for the layer. Dependencies that are
already in the project are filtered out from the result.
Result list of Layer_Version objects is sorted by layer name
for UI to look consistent.
This API is going to be used to show amount and list of
dependencies for the layer in the list of compatible layers
for the project.
(Bitbake rev: 7d853a3054a9ae3d18eb6f5bc13ba27d2795c31a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was accidentally returning the primary key of the ProjectLayer
rather than the ProjectLayer.Layer_Version.pk
(Bitbake rev: b20f3626148e89af0e9fcfca911a5a1e4e355a41)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix typo don't pass self in as a parameter, this evaluated to true
giving the wrong results meaning the machines typeahead did not return
valid results.
(Bitbake rev: 55ba889ef8900c95447861fa3985ca9cfe06afdf)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a copy of the built layer and the recipes associated with it.
This is so that the user can view the historical information about a
build. i.e. a snapshot of the layer version and artifacts produced at
that build.
(Bitbake rev: 0683d9a2b15e3234a94437aaebac84bfcca1420b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we do a build we update the last commit value that the layer was built at
However in future builds we do want to use the named reference rather
than the commit sha, e.g. master/fido
(Bitbake rev: a3d47eb923dd2cb5259f387c79549822807eca49)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously this layer relationship was done by trying to match path
information that came back to the buildinfohelper with trying to query for
data in toaster's layers table. This rarely matched due to the lose coupling.
(Bitbake rev: 838e77c7c3c4006abd1327343a004590ab652de9)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the openembedded-core layer is specified in the toasterconf we need
to treat it differently because we may also get this layer either from
the layerindex source or I can also be provided locally.
(Bitbake rev: 8e36b6848c14f2708e1f87d12fb533ca0b596c65)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This model lists custom image recipes for the project.
It is populated when new custom image is created.
It holds reference to the base recipe and list of packages
included into custom image.
For CustomImageRecipes the packages will be copied in and
therefore not associated with a build so Remove the requirement
for the package to have a Build.
co-author: Ed Bartosh <ed.bartosh@linux.intel.com>
(Bitbake rev: 33cbf4cd3b4ca47c4901501f5f1eafdfdfdae023)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For a new style indexes 'inherits' field is used to determine if
recipe is an image recipe.
As old style indexes don't have 'inherits' field this can be
guessed from recipe name. Let's consider recipe an image recipe
if recipe name contains '-image-'.
(Bitbake rev: d769d3f54faaf1a3091b1eb6ed1d17a8a108210a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Information about a task is not stored in Target objects.
This makes it impossible to correctly operate with the builds
where task is specified.
Storing taks name is an enabler for other fixes in UI and backend
related to restarting builds.
(Bitbake rev: 0a69a8a18075c976ed8681d9d75529f8c2f48514)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Command line builds are associated with a "default project"
(as we currently require a build to have a project). This
acts as a container for builds initiated outside Toaster.
Currently, this project is marked as the default by its ID
being 0. However, this doesn't work with MySQL, as MySQL
won't allow 0 in a foreign key which references an
autoincrement field.
Instead, use an is_default field to track the default Project
for builds initiated outside Toaster.
Add a method to fetch this default project, rather than fetching
a project with a magic ID.
Add this default project in a migration, rather than as a side
effect of a get_or_create() style method.
Also ensure that builds always have a project explicitly assigned
to avoid any magic with a build's project foreign key defaulting to
0 (as it no longer does).
[YOCTO #7932]
(Bitbake rev: 71b709a1bbc26d89d61873763b467d21e625b274)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Two issues prevent this migration from working correctly
with a MySQL back-end:
1. MySQL won't allow a default value to be set for an
AutoField, which is what the migration tries to do
for project_id ("ValueError: The database backend does not accept
0 as a value for AutoField.")
2. When migrations are applied to a MySQL back-end, Django
(via South) attempts a dry run of the migration first: it
applies the forward migration then rolls it back. However, this
migration raises an exception on roll back, which causes the
whole series of migrations to fail.
This commit fixes both issues.
[YOCTO #7932]
(Bitbake rev: 12f6278d56d7dec57308adc17411802f15d395d7)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In an environment with a proxy which requires authentication, e.g. with
$http_proxy = 'http://user:password@ip:port', the following error
occurs when running Toaster:
EE: Using proxy http://user:password@ip:port
EE: could not connect to 'url', skipping update: 'error message'
This prevents Toaster from fetching layer, recipe and machine information
from remote repositories.
This patch allows Toaster to use the proxy settings from the
environment for HTTP/HTTPS requests.
(Bitbake rev: e7a85031fd05a46ef60b380883da4cc372acf89b)
Signed-off-by: Bian Naimeng <biannm@cn.fujitsu.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prompted by issues discovered during running pylint on
the toaster sources, this patch fixes:
* missing import in toaster ui
* improper call of function in toaster ui (logger.debug)
* improper function definitions in bbcontroller
* invalid references to objects in bldcontrol.models
* proper initialization of object fields in ToasterTables
Also inhibiting specific pylint errors in files where
the problems are mis-identified.
(Bitbake rev: 1c71955c416fb68455f7f70669aba4202c411807)
Signed-off-by: Alexandru Damian <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The layer data update in Layer source needlessly stops
processing by throwing an Exception when bad layer data cannot
be inserted in the database (e.g. invalid dependencies).
This patch changes the behaviour to ignore bad data, and
continue with data import after detailed logging through the
logging system.
[YOCTO #7955]
(Bitbake rev: e0911e408db4fa07086738fe11121b173b4d1cf5)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switching debugging from using print statement to using
loggers, as it uses the logging infrastructure to process
the messages.
The messages are logged with the "toaster" logger which
is defined in the toastermain/settings.py.
(Bitbake rev: adf3bdcbe8b0b0bbe9de2800f2d20a53e8d88543)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As prompted by pylint, the object fields need to be
initialized properly, so this patch fixes that.
Also adds some casts to int, because, sometimes, the IntegerField
is not returning an int, but a string.
(Bitbake rev: cc9ac01d0726203fbe916cec51cc464eaeae9305)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch improves exception handling, fixing issues
as prompted by pylint, including exception renaming.
(Bitbake rev: 652fad64864703cb1bbeede8d5bfc4208974be68)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch fixes imports, default parameters to functions,
incorrect overloading and exception handling issues
highlighted by pylint. There are no functional changes.
(Bitbake rev: 066e096b80dcb0e93e1a088acf5d914184361769)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As ProjectLVSelectionTestCase was removed there is no point
to keep this function. Moved code back to setUp method.
(Bitbake rev: 8e9bd559c8ef0ebc9e8babbada06e710908bae08)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved code from ProjectLVSelectionTestCase.test_single_layersource to
LayerVersionEquivalenceTestCase.test_compatible_layerversions.
Removed ProjectLVSelectionTestCase.
(Bitbake rev: b322cb6f5fec0b979a646605ed0d7646764c07a7)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed test_dual_layersource method from ProjectLVSelectionTestCase
as identical method presents in LayerVersionEquivalenceTestCase.
(Bitbake rev: 5611c6b911e8a638f462e80e924ac9265a2199f5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
