If you load memory resident bitbake, then change bblayers.conf or other
configuration files, then execute your first command, bitbake will not
notice the changes. This is because it adds the file watches during
inital parsing, which happens at the time the first command is run.
To fix this, we move the addition of the file watches to earlier in
the process, so bitbake then does track them correctly. This also
avoids some issues Paul was seeing with tinfoil2.
(Bitbake rev: b9375c73e736003e66575969c9ea244403e47aeb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've had a long-standing bug where a legitimate error reading a file (IOError
or OSError) is always suppressed as though it was a 'file not found' case. As
a concrete example, if you do a `chmod 000 conf/local.conf`, it'll silently
not parse local.conf, rather than erroring to let the user know about the
problem.
Fix this by handling the ENOENT case specifically.
(Bitbake rev: e691312a3add222b04e7b2f52f8df6abcb9068bf)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tracebacks are of extremely limited usefulness in this context. The exceptions
carry the necessary context already, and the user doesn't care about the calls
in bitbake internals that led to an expansion or parse failure.
(Bitbake rev: 9b95fa94eaae452ac7814f1e67c2f7a6314c52f1)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added new field is_image to the Recipe model.
Made sure is_image is populated when layer sources are imported.
Made sure the change doesn't break loading old style layer sources.
Tested by loading data from layers.pythonanywhere.com and
from layers.openembedded.org.
[YOCTO: #7571]
(Bitbake rev: 3ac0ba73790b8f68567daa2de313c78053411fbb)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Uncommented debug and warning messages in _shellcmd method of
LocalhostBEController as they seem to be useful for debugging.
(Bitbake rev: 9446f02520a3bee4417908d8da2ab1848f4759c3)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The dash character was being stripped from the cache key when we sanitise
it for the memcache backend. This meant that we were getting a false hit
on the cache and returning non descending results which are indicated by
the dash prefix on the field name.
(Bitbake rev: 18743274a94966d2cd8d17d163e4dbab501a8a52)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch enables pylint running on the toaster files, showing
only the errors. This enables spotting common mistakes in the
toaster python files.
(Bitbake rev: d2e69721233ab3cbbc3bdd6a1c198fde8a1f287e)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As the tests are verifying different type of the functionality,
it is usually the case that a failing early test will completely
make the subsequent tests failing, e.g. if the system cannot
start due to a bug, there is little point in testing other
functions.
In order to prevent uneeded test runs, and to generate repeatable
test patterns, the test cases have now a numeric order in the
class name (e.g. Test01XXX). The tests are executed in this order,
and the first test failing will stop the test run.
(Bitbake rev: 639c46a08e524902018e28367fcb4e26362cd3e3)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Importing data from the layer index should be part of the setup
because otherwise we have no reliable database configuration.
This patch selects the first option for importing the database
configuration file.
(Bitbake rev: 86a69d294ace41bd109db97b753554ccc8f3dac0)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To allow the tests to run on different databases (in case of
manual runs), the HTML5 test will only delete the database
file if it was created in the setUp.
(Bitbake rev: 2b04165f6a70fad5beb8e4c7053a2b2053a51cde)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There may be leftover *.pyc files from previous code runs in the
current checkout directory, so we delete everything before
starting the tests, to make sure that we actually testing the
current checkout.
(Bitbake rev: db29ef9ab23c9f083e9aa175b28430f122f35a84)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch fixes the toaster startup and database cleanup
for managed mode. It is needed because the toaster script
thinks it has been called with "source" instead of being
executed as independent script if not called through bash.
(Bitbake rev: f08ce41f1137f268049ef8345462aa58b233e50b)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch improves logging in the urlcheck.py.
It allows properly running just HTML5 tests outside the
test suite.
(Bitbake rev: 91566d2ea8ca5a696ce742b9e5e3b7b6c10c200c)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We add debug information, and exception handling as to make
it easy to figure out what is going on when tests fail on
automatic start.
(Bitbake rev: d7c5989b795566f8611208b26851871abccf9578)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
gitignore the cache directory created by the http client
the log file for tests is already set up as an absolute path,
so no need to recompute the path
(Bitbake rev: 80f525e5cbe83e0407ecddf84401d68213c6d5cf)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch cleans up the list of URLs that are tested for
HTML5 compatibility using the w3c-validator. It removes
the URLs that are no longer in use, and the AJAX-only URLs
that return JSON content.
(Bitbake rev: c85a35e37150ee8be091bd142cac8a1885f1dff0)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch brings TTS to the pylint coding standards.
Pylint was run with some disables:
disable=logging-too-many-args,line-too-long,missing-docstring
and achieved
Your code has been rated at 10.00/10
There are no functional changes.
(Bitbake rev: 2b40b412ff6a7e3fd4cc32707bd3cd713bc09ddb)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
All the elements for the new build button need to be present for the
build button to work. Now that the template variables all have to have
values we have to make sure that the elements are still added to the dom
for the js to manipulate them.
(Bitbake rev: e2ab67eaf76da9ee2009e8420d5584c3daa97ac1)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To share the build-button mechanism from the pop over and project
topbar build button we now use a build-button class to reference them
and share the event handlers.
(Bitbake rev: e3c6a00b848a6f147e93c8da46b8a3516499fde8)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As the JSON response changed the duplicate layer functionality wasn't
wasn't working as the field names in the returned data changed. This
patch brings the field names back in sync with the response.
(Bitbake rev: bdea86ae8720ee5792a5617c6073d51922a224ff)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The machine change notification animation is no longer required in
favour of the global overlay notification.
(Bitbake rev: aef36f2674d3d82b1757c02ffbf8b72fd4f32a4e)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This brings in the new project page design and improved navigation. As
this also removes the dependency on Angular it also required that the
entry points to the project page such as machine-change notifications
are also updated.
[YOCTO #7329]
(Bitbake rev: 6489e6eb5c3b0d59063b6d60521fc33fe563e707)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we get a request to the project change api which either contains the
current configuration (i.e no change) or empty configuration handle this
gracefully.
(Bitbake rev: cc6c606c07baadc849c3290b23c56ad4e719fba2)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that we have a change notification that is global move this
functionality to libtoaster for shared use.
(Bitbake rev: 9d760462de99cb8729b7e0db773e77d4cd508733)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To be able to use the utility functions in libtoaster we need to have a
single property name for the value that represents the url for the
layer details in the layer object.
(Bitbake rev: 4d64fd0d1236d342b29537f601a68cd23a8255ea)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After API was changed we now need to pass the parameter "format=json" as
a get parameter when requesting a json payload. Otherwise we get the
html version of the page instead of the data we need.
(Bitbake rev: 4545c21b70998bd240c3a21de05e8d4642044119)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change the structure of the project page to include a navigation menu
and top tab navigation. Remove old breadcrumb method.
[YOCTO #7329]
(Bitbake rev: 66fa0dd988e01ec79e74be7a5697eaa3b4f017d8)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[\.-_] means "any character between . and _"
What was meant here is certainly "any character from the three characters .-_"
(Bitbake rev: af13eaba627f199f91c048c435b9dbe19c79527f)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the PR server or indeed any other child process takes some time to
exit (which it sometimes does when saving its database), it can end up
holding bitbake.lock after the UI exits, which led to errors if you ran
bitbake commands successively - we saw this when running the PR server
oe-selftest tests in OE-Core. The recent attempt to fix this wasn't
quite right and ended up breaking memory resident bitbake. This time we
close the lock file when cooker shuts down (inside the UI process)
instead of unlocking it, and this is done in the cooker code rather than
the actual UI code so it doesn't matter which UI is in use. Additionally
we report that we're waiting for the lock to be released, using lsof or
fuser if available to list the processes with the lock open.
The 'magic' in the locking is due to all spawned subprocesses of bitbake
holding an open file descriptor to the bitbake.lock. It is automatically
unlocked when all those fds close the file (as all the processes terminate).
We close the UI copy of the lock explicitly, then close the server process
copy, any remaining open copy is therefore some proess exiting.
(The reproducer for the problem is to set PRSERV_HOST = "localhost:0"
and add a call to time.sleep(20) after self.server_close() in
lib/prserv/serv.py, then run "bitbake -p; bitbake -p" ).
Cleanup work done by Paul Eggleton <paul.eggleton@linux.intel.com>.
This reverts bitbake commit 69ecd15aece54753154950c55d7af42f85ad8606 and
e97a9f1528d77503b5c93e48e3de9933fbb9f3cd.
(Bitbake rev: a29780bd43f74b7326fe788dbd65177b86806fcf)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since shell scripts run with 'set -e' the final exit at the end of the
script can only be returning 0. However, for correctness and to follow
the original intention of the 'cleanup' commands, let's fix the typo
and return the success of the last shell function rather than the
success of unhooking the exit trap handler.
(Bitbake rev: bef724057f1ea81571dd3ac5a9cf862f818434b5)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Calling record() for each override alteration is slow. Since we now expand
overrides dynamically we don't have to record the log data at each alteration,
we can instead print it directly from the existing data stores at variable
history print time using the exact same data stores.
This massively improves performance of the data store when parsing
with bitbake -e for example, it will improve memory overhead as well.
The only downside is that VariableHistory has to poke into the datastore
for some of its data but that seems an acceptable tradeoff rather than
double caching.
(Bitbake rev: 100b447a161ef20fa559e39516cd32fa78e38262)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As things stand now, bitbake -e (which turns on all the caller tracking)
of OE-Core generates around 9.5 million stat calls which is slow and the
largest single thing on the profile data.
This is because infer_caller_details() calls traceback.extract_stack()
which adds line contents to the traceback. This in turn calls python's
internal linecache code which calls stat on every file for every callback.
We don't even use that info. We only even want a single frame of the stack.
Instead, open code for the pieces of information we need. Also, only
obtain the stack once for both halves of the infer_caller_details()
code.
This reduces the number of stat calls to around 0.5 million and significantly
improves parsing with bitbake -e.
(Bitbake rev: 7be76d8f79ea92fd4bd36146eb9a4b86551b526d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should use the d.xxxVar syntax rather than the older function style.
Also replace 0/1 with the more pythonic True/False.
No functionality changes.
(Bitbake rev: 90fdd69cca951f8bd2ff634f3b42fccd4fc03095)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Minor formatting improvement in generated shell scripts.
Add a newline after the last export emitted by emit_func. Also, remove
chance of putting an extra newline in the middle of the exports, since
we never want to do that.
(Bitbake rev: 5117d9b5d32d5d81adf70fa3e3feac9cef654240)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
