With the resent change in rootfs creation code setVariable
rpc calls don't set variables for bitbake workers anymore.
Writen variables to toaster.conf should solve this issue.
[YOCTO #8910]
(Bitbake rev: d6dfe40320ff6ca420d9f9016b4d1d83d10f1d59)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used OneToOneField to reference BuildRequest in BRBitbake model.
Fixed django warning:
WARNINGS: Setting unique=True on a ForeignKey has the same effect
as using a OneToOneField.
(Bitbake rev: aaa4319ebbb06facb77b4ba936cf3aa2068ff238)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runbuilds previously had its methods decorated with
transaction.commit_on_success, which is now deprecated. transaction.atomic
is an (almost) drop-in replacement for this, so use this instead.
[YOCTO #8364]
(Bitbake rev: c4804b84eaaef6a81027bae5cf8bfe63d949c293)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
South has been deprecated in favour of Django's own migration framework,
so remove the old South migrations and replace them with Django ones.
[YOCTO #8364]
(Bitbake rev: 427d8bc02d1aa00a19057602d592d58334514804)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Triggered MetadataEvent "SetBRBE" when TOASTER_BRBE variable
is set on bitbake server. This should make buildinfohelper
aware of the build request id, which is used to properly
report build status and failures back to Toaster.
(Bitbake rev: 8b136fb093020bc912a7b21d5163e1cd5fb12124)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This API is not used anymore as toaster doesn't write variables
to configuration files anymore. It sets variables through its
connection to bitbake server.
(Bitbake rev: 6dda2916b75d688874d208192d5c7cdb302eec35)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Set project variables using connection to bitbake server
instead of writing them to toaster configuration files.
This is a way to avoid rewriting configuration files before
and after every build, triggered by toaster UI.
(Bitbake rev: 3f77c69b867735b5d27c6ab7c62e39ac8d5cb2b5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added new API to get value of bitbake variable from bitbake server.
The API will be used to update INHERIT variable instead of writing
it to the toaster configuration files.
(Bitbake rev: effc371e3a4a2262e53e9e560c7c08cffad195da)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed stopBBServer API from build controller as toaster
doesn't stop bitbake server anymore. It's reused for both
types of builds: triggered by UI and started manually.
(Bitbake rev: 0cad802da171d51814f22cc9383f496a63cd0c75)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rewritten LocalhostBEController.startBBServer to use
'toaster restart-bitbake' and read bitbake port number from
bitbake.lock.
Removed complicated logic of running oe-init-memres and looking for
bitbake port number in the logs.
(Bitbake rev: 4b065353c3454923a1ef88e9f0a8702e5626060e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This function is not needed as build environment is always created
because of the new way to run Toaster. It can be only sourced after
oe-init-build-env is sourced.
(Bitbake rev: 27a59b4373a449f17e46ab93e0eba15cf75967ae)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Made runbuilds to loop to avoid having a loop in shell code and
initialize heavy Django init machinery every second.
Ignored exceptions to prevent exiting a loop.
(Bitbake rev: e04da15556ca0936de652b8c085e4199e5551457)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of putting the DATABASE_URL as part of the command for launching
the bitbake observer process set it as part of environment.
This fixes two issues 1. Where the value isn't quoted and therefore will be
interpreted in the shell and 2. Anyone being able to see the value of
DATABASE_URL in the process tree.
[YOCTO #8669]
(Bitbake rev: 832a8523067606b180c02f0d1544e8a23219bb08)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As toaster makes use of clones of itself we have a mechanism to just
clone the current checkout and use that as a basis for the git clone for
the layers inside. For instance, in poky, if we're running in that directory,
we don't ever want to change the current checkout so we make a copy of
ourselves to do this work in.
This steps through cached_layers and previously-traversed directories
inside the current checkout to find any pre-existing checkouts.
This was ending up traversing too many directories down, so remove
the traverse and only deal with the current directory.
[YOCTO #8463]
(Bitbake rev: 9c40b3b6377ab8f5d6ac9b8e00585b71de00bf74)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster uses git url and branch to make a clone directory
name. Current code leaves '@' and '%' characters unchanged,
which can cause generation of wrong directory names.
Fixed this issue by replacing '@' and '%' with underscore.
(Bitbake rev: e076888a2120a37c388930073694750735d86507)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
removed layer dir questions and base off TOASTER_DIR
removed build dir questions and base off TOASTER_DIR
base configuration file off of TOASTER_CONF
fixed some pylint issues
[YOCTO #8217]
(Bitbake rev: d5811968b5f22093365f381fb7e75ab46e5269c2)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Artifact download links were broken because the function to
get the mimetype for the artifact was incorrectly using the
underlying mimetype library. The function was also attached
to the build environment controller, which was unnecessary, as
we only support local controllers anyway.
Remove the mimetype getter on the build environment and
use the one in the view code instead. This works correctly
and prevents the download error from occurring.
[YOCTO #8369]
(Bitbake rev: 805fb2a9388c728600596e9b845a5c7eeaebd99c)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default Toaster clones layers to the same level where
poky clone is. This can look messy if a lot of layers are
used for Toaster builds.
Moving them into _toaster_clones/ subdirectory should make
directory structure looking cleaner. It also safer to isolate
toaster clones from what user might create.
(Bitbake rev: 139e851831eea682aba0f9403dcc2eea7c1b05bd)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changed naming scheme for clone directory. Used full git url and branch to
make it unique. This should fix the issue with using the same git
repository, but different protocols, e.g. git://some.git.repo and
http://some.git.repo.
[YOCTO #8101]
(Bitbake rev: a9baf856a1ef7d84a47fa6615353f5b3fdb8a73b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
git reset --hard should be given either commit id or
origin/<ref name> to work properly. Without this fix git will
complain that origin/<commit id> does not exist.
[YOCTO #7505]
(Bitbake rev: ec05beff7d1b06e4df98199925c7102f5684f4e0)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replaced 'git checkout <ref> && git rebase' with 'git reset --hard' as
with git checkout repository ends up with detached HEAD. Rebase makes
things even worse as it can cause conflicts. git reset --hard resets
repository to the required state in a most straightforward and
reliable way.
[YOCTO #7505]
(Bitbake rev: f3641a8422e67890410594453b8baf362c27df2b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
git clone --single-branch works only with ref names. It fails if
commit sha1 is set in layer configuration with this error:
fatal: Remote branch <commit sha1> not found in upstream origin
Cloning repository without using --single-branch should work for
refs and commit sha1.
[YOCTO #7505]
(Bitbake rev: edac7b7533d50455baaaf05a46b7ace49bb24b95)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously this layer relationship was done by trying to match path
information that came back to the buildinfohelper with trying to query for
data in toaster's layers table. This rarely matched due to the lose coupling.
(Bitbake rev: 838e77c7c3c4006abd1327343a004590ab652de9)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When building customised recipes toaster creates custom layer directory
and puts layer.conf and custom recipes to it.
[YOCTO #8075]
(Bitbake rev: f81b48c30a548bee946d34c56aa1872785bcec30)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Information about a task is not stored in Target objects.
This makes it impossible to correctly operate with the builds
where task is specified.
Storing taks name is an enabler for other fixes in UI and backend
related to restarting builds.
(Bitbake rev: 0a69a8a18075c976ed8681d9d75529f8c2f48514)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The cooker log is copied from its original (bitbake) location
to an artifact directory chosen by the user (chosen when
checksettings.py runs as part of the managed mode;
it's one of the annoying questions you're asked at startup).
The copy happens as part of the runbuilds script run, which
is started in a loop from the toaster startup script
in managed mode.
When a user requests the log for a build via toaster, they
are getting the log which has been copied to the artifact
directory, not the original bitbake log.
This works for the managed case, where the runbuilds command is
running in a loop and copying log files for completed builds to
the artifact directory. However, in analysis mode, there are two
problems:
1. checksettings isn't run, so the artifacts directory isn't
set. toaster is then unable to figure out where the log files
should have been copied to.
2. The log files aren't copied to the artifacts directory
anyway, as runbuilds isn't running in analysis mode.
To fix this, just point the user to the local bitbake log file
in its original location. This avoids the copy step, and means
we can remove a whole question from the toaster startup sequence,
as we no longer need an artifact directory.
The only downside to this is that it's not appropriate for
remote bitbake servers. We will need to revisit this and
possibly reinstate the copy step once we have to reconcile
local and remote builds and make their logs available in
the same way.
[YOCTO #8209]
(Bitbake rev: 5697bbcc88edad85891f66d28b8803a9c9d27ff2)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Increased waiting time for toasterUI from 10 to 25 seconds. Bitbake
takes longer time to start, so toaster should wait longer.
[YOCTO #8240]
(Bitbake rev: 224ee3685fe20915b21d299ab80f7b289a916dca)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Improve the explanation we show to users during set up
when asking them to set the layers checkout directory.
The patch also makes minor changes in text formatting to
improve legibility.
[YOCTO #7740]
(Bitbake rev: dd9284944ae0f0feecb70adab880fed636f7cd59)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prompted by issues discovered during running pylint on
the toaster sources, this patch fixes:
* missing import in toaster ui
* improper call of function in toaster ui (logger.debug)
* improper function definitions in bbcontroller
* invalid references to objects in bldcontrol.models
* proper initialization of object fields in ToasterTables
Also inhibiting specific pylint errors in files where
the problems are mis-identified.
(Bitbake rev: 1c71955c416fb68455f7f70669aba4202c411807)
Signed-off-by: Alexandru Damian <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The patch to fix the original mistake is wrong, in the sense
that a constant from the BuildRequest class is used on the build
object.
Fixing the patch to bring in the correct constant in.
(Bitbake rev: a740c17efc0ccc9f89a428ead08f9739af3c6de4)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch replaces throwing Exceptions in toaster loadconf
command with proper explicit error messages. This allows
the user to properly understand and debug what went wrong when
loading a config file.
Additionally we change a bit the logic around auto-detecting relative
giturl handling so the user gets proper error messages when
trying to load an invalid toasterconf.json file.
[YOCTO #7945]
(Bitbake rev: 3d14cc033a855bf5b20e799438548b6d8f29d9b8)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Improving the TEMPLATECONF detection by verifying the return
code and dumping extra debug information in case of exception.
(Bitbake rev: bdc00755993aa37e7669c3859ef4ea9b1fc3e680)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch fixes imports, default parameters to functions,
incorrect overloading and exception handling issues
highlighted by pylint. There are no functional changes.
(Bitbake rev: 066e096b80dcb0e93e1a088acf5d914184361769)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removing an spurious 'it' when setting the build directory.
Thanks to Scott Rifenbark for finding the typo!
(Bitbake rev: dd644a2c87e651c3990520759bab55a56ba110d1)
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Uncommented debug and warning messages in _shellcmd method of
LocalhostBEController as they seem to be useful for debugging.
(Bitbake rev: 9446f02520a3bee4417908d8da2ab1848f4759c3)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure the error value is initialised and update the string match
that we have to identify bitbake's current startup status.
Patch contribution from Eduard Bartosh
(Bitbake rev: 915ba08a8a3013e9787e564f2ffd8698c948f433)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent refactoring moved this enum value to BuildRequests table
rather than Build.
(Bitbake rev: 4338ab03826df3353e1222f0bd825dfdd04dd933)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If toaster has previously cloned poky or a layer with a toasterconf
file we don't want this to be picked up as a choice as these clones are
"internal" to toaster and may have undesired effects on toaster's setup.
[YOCTO #7741]
(Bitbake rev: 2f74ff59f2873c1cf121f7a385f3ab971c793a9c)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We remove the "timespent", "errors_no" and "warnings_no" fields
in favor of computing the needed values at runtime. This prevents
inconsistencies in the UI.
Also removeing all references to BuildRequests from the interface -
all build details now display in the build dashboard.
Minor fixes related to data logging.
(Bitbake rev: 44f37394ed3e4ca02f940be172fe4395b0ee0f7d)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a set of fixes that repair the interface after
we switched from displaying BuildRequest data to Build data
in the formerly "managed" mode.
(Bitbake rev: 57f790b0c56297af8c83d5def8461bd5d61fe4af)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds logic to complete changing the interface
from showing BuildRequests to showing Build data.
The BuildRequest data is now transformed in Build data with
proper Toaster exceptions being recorded instead of listing
problems during startup as build errors.
(Bitbake rev: 51a41172d0b390370f9a38696b1ac65666ada4d2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We improve the buildenvironment API by reducing it to a single
command: triggerBuild.
This command is specifically implemented in each BE controller
type, so the runbuilds management command is only concerned
with scheduling the next build, and not with the details
of how a build is actually started.
(Bitbake rev: 7ee0f1da0a8fcac37419ffdddbe35a9268a1ded4)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes the unit tests for the bldcontrol it requires the
implementation of a new Exception type so that a known Exception can be
handled. Also fixed is the path to the toaster conf files so that the
test doesn't need to be run from the top level directory and the ability
to specify the values of TTS_SOURCE_DIR and TTS_BUILD_DIR and
TTS_TEST_ADDRESS used for testing.
Edited by Alex Damian to correct the rebasing of the localhostbecontroller.py
file.
(Bitbake rev: c17933271cd273a346115c2ee0b6695ff3f981ce)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch refactors the checksetting command to prevent
early return from the handle function.
It also adds a check that marks IN PROGRESS builds at startup time
as FAILED. Minor changes to BuildRequest and Build classes
ensure useful string representation for the objects.
(Bitbake rev: adf67dd79dbf6b585bf8cd54f99c389409b88ecd)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
