The 'toastergui' application had duplicate entries,
so this patch removes the duplicate ones.
Based on an original patch from Ravi Chintakunta <ravi.chintakunta@timesys.com>.
(Bitbake rev: 86d3a44e81aa3dbff947330dfaeff6040bbd625f)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds the initial api tests for toaster, using Django's unittest modules.
(Bitbake rev: daf9a61fbf69a46b7afd781a6175b05b05fd452f)
Signed-off-by: Andreea Proca <andreea.b.proca@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
THis is a large set of fixes for the generic table, Build and
Dashboard pages.
Among the fixes:
* the table remembers which columns to show across refreshes,
based on saving the settings in a cookie
* added column timespent for a build which is a denormalization
of the completed_on - started_on information due to limits in
computing datetime differences in the SQL engine
* fixed formatting of the time differences
* various sorting header links fixed
* correct error and warning CSS classes applied to the
respective rows
* fixes multiple divide-by-zero error in displaying duration
estimations
(Bitbake rev: 61e3dee55ac577fce1c0ae0fe7e0d3cf644e8ae6)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add the jquery cookie plugin to allow us to save
and read local client date in cookie.
jquery-cookie using version 1.4.0 under MIT licence.
(Bitbake rev: 8ce1668dd93269add878f8ea69deb8b53f3bca8b)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
No-op patch that adds vim modelines for all .py files
intended to be user-edited.
(Bitbake rev: 73271a7c6f1913c68a4b39ab86414f44acc04776)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch updates the task classification fields (outcome, etc)
as to
* Changes outcome names from SSTATE to CACHED and
from EXISTING to PREBUILT
* NoExec tasks now recorded as Not Executed / script type NA instead
of Executed / script type NOEXEC. Script type NOEXEC is deleted.
* SetScene tasks do not get order numbers
* New task method that returns a QuerySet for setscene tasks related
to this task: Task.get_related_setscene()
* New custom TaskManager that allows searching for setscene tasks
related to a certain task: Task.objects.related_setscene(task)
(Bitbake rev: a4164821a142f8b625a5fdc209adc6dc80874241)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the Toaster GUI was cloned from the Simple UI, we need
to update the URL pattern names in Simple UI to
prevent conflict when determining the reverse URL path.
(Bitbake rev: 6ed1a28eb0e52a6de83a37664ff6f4418ce84ee4)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I can't think of a reason we'd download zero sized files however there are
reasons zero length files can accidently make it onto source mirrors.
This check allows us to ignore the broken files and switch to another
mirror rather than fail with odd checksum failures.
(Bitbake rev: 300cba2e1a720dba4b83b0c76208ea93c608c1de)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to deference tags when trying to map them to commit IDs with
ls-remote. If we don't do this, a given commit might not show up
later in a specific branch. There appears to be no good reason not
to do this.
(Bitbake rev: 8ef24f4c834298348172b96ec0b855bf09552b09)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When specifying tags, they're searched for unanchored so foo/bar could
match:
refs/heads/abc/foo/bar
refs/heads/xyz/foo/bar
refs/heads/foo/bar
This change anchors the expressions so they are based against heads
or tags (or any other base level tree that has been created).
(Bitbake rev: df2e0972cd1db7abd5ec8b7cb295fb0c42e284a4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current message can be ambiguous, improve it (and also rename a
variable to clean up the rest of the function).
(Bitbake rev: 0c1bb7c0fce7b0f334311a2893ccb00385fa8d55)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a sanity check so that if some SRCREV is set and a rev parameter is given
to the url, the revision given should match.
Any tag parameter behaves the same as rev. If both are specified, error to
tell the user we're confused rather than do something which may or may not
be what they intended.
Also add some unittests for this.
(Bitbake rev: e82a4ab48991035866da9914c8b75a9bfbc9a7fc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently INVALID and None are checked as incorrect values under different
circumstances. This code standardises those checks to be consistent. We
should phase out the use of "INVALID".
(Bitbake rev: 86ef4e65ce18b71dc69643586bd2aa8f48703171)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The fetcher will try:
1) PREMIRROR
2) Upstream
3) MIRROR
If it fails to download from the Upstream, but succeeds from the MIRROR,
and ud.localpath != origud.localpath (for example, the git tarball),
then we will get the error (e.g.: xf86-video-omapfb):
ERROR: Function failed: Fetcher failure for URL: 'xxx'. Unable to fetch URL from any source.
ERROR: Logfile of failure stored in: /path/to/log.do_fetch.28024
It should not show the error and let the build go on since it succeeds.
(e.g.: xf86-video-omapfb)
[YOCTO #5686]
(Bitbake rev: c08ca1e4eeb04f78e1354780cf5a4c3855e49572)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake -S throws exception when 'noop' signature handler is used
[YOCTO #5738]
(Bitbake rev: 53352e8d388b7fc4da73f95b93dcc087e76d0426)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows FETCHCMD_git to override the fetcher command as the git fetcher does.
[YOCTO #5717]
(Bitbake rev: 23ab943be3a33077d6ad8be68bba53cd1e2270b4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* currently decode_url regexp parses branch=@foo as username so it ends like this:
- ('git', '', 'foo', 'git.openembedded.org/bitbake;branch=', '', {})
+ ('git', 'git.openembedded.org', '/bitbake', '', '', {'branch': '@foo'})
* http://hg.python.org/cpython/file/2.7/Lib/urlparse.py also assumes
that there is at least one '/' as separator between netloc and path,
params, so it looks reasonable to prevent including '/' in username
(Bitbake rev: 2c82742114091cb55055328b54223686816582f2)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fsync was added for belt and braces protection for things like sstate
on NFS. To be honest, it probably doesn't buy much, if the rename isn't
atomic, all bets are off anyway and there are bigger issues to worry about.
The issue is that at the end of every task, the dump_sig() code is triggered
to save out information about the task and this was triggering an fsync(fd).
Whilst it may select the file descriptor, on file systems like ext4, it will
require large parts of the journal to be written out so it can have
significant impact. latencytop showed an average fsync() call overhead of about
2s and if that happens for 5000 tasks, the time mounts up. This blocks the next
task execution by that time.
We therefore drop the fsync since in reality its causing problems and is unlikely
to buy much.
(Bitbake rev: 46fd841319479f6079d850b3813e64bd8c2680a3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch implements table searching, filtering and ordering, in a
generic mode reusable for all tables.
The search operates list of fields defined in the corresponding
class for each model, search_allowed_fields.
The search expression and filters are sent through GET requests
using a QuerySet-like input. The inputs are filtered and
validated before usage to prevent inadvertent or malicious use.
Filters and table headers are defined in the views for each table,
and rendered by generic code which is easily modified for various
tables.
The Build table and Configuration table are implemented using this
framework as an example of how it should be used.
[YOCTO #4249]
[YOCTO #4254]
[YOCTO #4255]
[YOCTO #4256]
[YOCTO #4257]
[YOCTO #4259]
[YOCTO #4260]
(Bitbake rev: 2ca15117e4bbda38cda07511d0ff317273f91528)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds the build dashboard page implementation,
which is the landing page for the Toaster GUI.
Also adds correct links from the main build page
to the various parts of the dashboard.
[YOCTO #4258]
(Bitbake rev: bf7fbf5c0ee39564d813f82e194242f9d4f73c47)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updating the general container pages to use the graphical
design and features from the design phase.
In the process of adapting the Simple UI to the designed
interface, we create all the pages and the navigation
structure for the Toaster GUI.
Views for each page have been added, and the url mapping
has been updated to reflect newly added pages.
The table page has been refactored to be component-oriented
instead of class-oriented in order to facilitate reusage.
Changes are made in different layers of the template
(base, basetable) in order to maximize code reuse among
different pages in the build.
(Bitbake rev: d31f039ae31b77023722c06e66542751536a1362)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the process of making the GUI more usable, we
bring in the static assets used during the design
phase into the application.
Licenses used:
Bootstrap 2.3.2 under Apache License 2.0
Font Awesome under SIL OFL 1.1 (GPL compatible) and MIT licence
GLYPHICONS Halflings released with Bootstrap, CC BY 3.0
jQuery 2.0.3 under MIT licence
prettify.js under Apache Licence 2.0
(Bitbake rev: 0b3d2000c0bcd2bb5a3af5d49a514a90f001a7d2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch clones the Simple UI to provide the base code for
the development of the Toaster GUI. The clone takes the place
of the application that was reserved for Javascript MVC code.
The templates used for Simple UI are renamed to start with
an "simple_" to prevent name resolution conflict with the
Toaster GUI templates.
Minor changes are made to the settings.py and urls.py in the
toaster main section to account for the newly enabled application.
(Bitbake rev: e2fde84f16da017ba0d71aef6a1fa8e2b9255db4)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This code clearly uses an earlier fetcher API. Update it to match master.
(Bitbake rev: e13acb4113ce75226664c3006a9776cc885e860d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This add a Git Annex backend which reuses the Git fetcher code; it
allows managing files with git, without checking the file contents
into git, being useful when dealing with files larger than git can
currently easily handle, whether due to limitations in memory, time,
or disk space.
(Bitbake rev: a61fc4db598e9d13c966712a6a0e4783e19448be)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For rebased git tree, some commits doesn't exist in any branch, and such commits are
valid in tag, the change is useful for such case.
(Bitbake rev: f594cb9f5a18dd0ab2342f96ffc6dba697b35f65)
Signed-off-by: Zhenhua Luo <zhenhua.luo@freescale.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The user manual said:
'cleandirs' - directories which should created before the task runs but should be empty
But it only removes the dir, doesn't create it
[YOCTO #5703]
(Bitbake rev: 0636797d75874ce4577f29011d69c56a4c6b9e89)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There was a problem:
$ bitbake xf86-video-omapfb -cfetch && bitbake xf86-video-omapfb -ccleanall
Everything should be removed, but the
0006-omapfb-port-to-new-xserver-video-API.patch.done still exists in the
DL_DIR, this is because the clean() in the fetch2/__init__.py skips
removing the local file, so that it will skip removing the .done.
The local file (file://) isn't needed to be removed since it is not
downloaded into DL_DIR, but the .done should be removed, this patch will
remove the .done, and it doesn't remove anything else since the clean()
in local.py does nothing.
[YOCTO #5687]
(Bitbake rev: 2bc99b9dfa532430a13c39fca4e5ef3a2206b3b8)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There was a problem:
$ bitbake xf86-video-omapfb -cfetch && bitbake xf86-video-omapfb -ccleanall
The git2_git.pingu.fi.xf86-video-omapfb.tar.gz has been removed from the
DL_DIR, but the git2_git.pingu.fi.xf86-video-omapfb.tar.gz.done still exists,
this is because the "open(ud.donestamp, 'w').close()" in try_mirror_url() will
create the git2_git.xxx.tar.gz.done, but no one removes it (the clean() in
fetch2/__init__.py removes the DL_DIR/git2/pkg.done)
This only happens on the git fetcher AFAIK.
[YOCTO #5688]
(Bitbake rev: fb2dc84875eb477661f421b21bc404d4805ce379)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A part of fetch2/__init__.py uses 3 spaces as the indent, I
think that they should be typos.
(Bitbake rev: abafd85e2fcf23cee872e0e9e468898101430f1f)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Because of the way we were handling this error, it was printed twice -
once via logger.error() (to avoid the log being printed) and a second
time when the exception gets wrapped in a FuncFailed at a higher level.
Call logger.error() earlier and change the text we send in the
exception to be more brief, so it more closely resembles the behaviour
when there is an invalid checksum.
(Bitbake rev: 46765369d7f76ec7f67b90430131a79eb6a66235)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were checking SRC_URI md5sum/sha256sum values against None here, so
if they were set to "" then no error was produced. Since the value is
still effectively unset in this case, this is not the right behaviour;
just check if the value doesn't evaluate to False instead.
(Bitbake rev: 040943a718795c64dc4e604abfcf08b26b7d00e6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The output when comparing siginfo files for dict_diff is reversed and shows
additions when things were removed and vice versa. This patch reverses the operation
so the changes are shown correctly and makes the output less confusing.
(Bitbake rev: 9b4142df36619099670740a5d3bc94e404ab2b56)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Based upon the list of difference starting points, we can use the siggen.find_siginfo()
function call and the difference printing code to provide a list of differences
between the current build target and whatever can be obtained from the sstate cache.
(Bitbake rev: 7a77861feb62750ef166d2d1e89ed1f444ca8dc7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the code prints all differences. If the task dependencies have changed hash,
we recurse into those and print those differences as well. This leads to a lot
of output. The reality is if the parents changed signature, we might as well just
say that and recurse with no other output since we're much more interested in how
the parents changed in nearly all cases. The changes in the parent are probably
the same ones we'd have printed at each level anyway.
By doing this we focus the output more carefully on the thing the user wants/needs
to see.
(Bitbake rev: 7a17fd6e51a76d3582c357b79f5ef86e1969650c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The way hash_deps was being generated was different to the way siggen generated
the data internally which lead to seemingly different sigdata/siginfo files
for the same checksum. The -S output correct but the files written during
builds contained superflous data which would look like a difference.
This patch removes the badly duplicated data and uses it from the source
which ensures its consistent.
(Bitbake rev: e6d5e925c402cd2cc7ee034e9de4cc6df8944a34)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its useful to understand where the delta starts against an existing sstate cache
for a given target. Adding this to the output of the -S option seems like a
natural fit.
We use the hashvalidate function to figure this out and assume it can find siginfo
files for more than just the setscene tasks.
(Bitbake rev: c18b8450640ebfd55a2b35b112959f9ea3e0a700)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The basecmd is initialized in urldata_init; there's no need redoing that
work.
(Bitbake rev: f8df6f746fb2e27f029a5449cee6c891b1f36f4f)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the fetcher doesn't distinguish between names that the fetcher
needs to resolve verses branch names that the user specified.
This meant that if you specify a tag and a branch, the fetcher broke. This
separates the two so that the branch name is preserved and can be used in
appropriate places.
(Bitbake rev: e85f39fe9d1b224414b5da0780da514f75c5df92)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently its near impossible to control task addition/deletion from
metadata context. This adds stong add/deltask API to bb.build
which is traditionally where it resided. The rather broken
remove_tasks function was removed, it didn't appear to do anything
useful or have any users.
This allows us to clean up hacks currently in use in metadata and use
standard API for it instead.
(Bitbake rev: bf7138dd38fc1f8efca80891198e3422fef64093)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Back in the depths of time we did support task removal. In the pre
AST days it was nearly impossible to continue supporting it, it wasn't
used so it was dropped. With the modern codebase we can easily now support
deltask and it would be very useful within the metadata since it can
massively simplify dependency trees.
As an example, a core-image-sato had 47703 inter task dependencies before
this patch and a patch to native.bbclass, afterwards with the noexec tasks
deleted, we had 29883. Such a significant simplification is worthwhile
and justifies adding a deltask operation to the system.
(Bitbake rev: acecbde6fb70ff3c96deab3cdf819d8442e87ed4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were accidentally using references to sets in the contains functionality
instead of creating a copy. This could cause data corruption and corruption
of the resulting sstate checksums.
This patch fixes this to make a copy of the set and resolved the corruption
issue.
(Bitbake rev: 8f4733257ad665aa7c7e7061c543379d5e4e3af2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On some machines, a hang has been noticed where the system sits in
the select call despite the task having completed.
The exact reasons for this as unknown but adding a timeout unblocked
the builds and resolved the hangs in question.
(Bitbake rev: 5223ffb5b6a46d8b3f6ac3362bd2672e2edf2691)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved the parseConfiguration method before obtaining the machines and
other configurations from bitbake. If not, Hob doesn't see the new machines
added by the new layer.
[YOCTO #5632]
(Bitbake rev: 8de14b2a481d61424eb32fd0234f7a38a961a75b)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
