We update the structure used to hold interm task data,
before it is written to the database, to lower the changes
of key collision.
This will also lead to a cleaner data structure and easier
inspection.
(Bitbake rev: 49cb9f543526a161bc4c097f94422ea08b491ef9)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This optimization is in support of the bug #5485. The function called
at the beginning of every build: store_dependency_information was taking
approximately 20sec and it was delaying the arrival of events from the
event queue. The change minimizes the calls to _save_a_task(),
reducing the time to half.
(Bitbake rev: b86fd2be40303d886fdb9ad3009355584d285acc)
Signed-off-by: Marius Avram <marius.avram@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to mark the package entries that are
created solely for dependency tracking purposes.
In order to avoid altering the database schema,
we mark the dependency targets with size = -1, since
this is not a valid size anyway and makes for easy
filtering.
[YOCTO #5803]
(Bitbake rev: d11ed273dd6c520b16e9ccfe79476f340006a55d)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to separate tasks with invalid states from the
no exec tasks, we add a new value OUTCOME_EMPTY for the tasks.
OUTCOME_EMPTY has the same value as OUTCOME_NA as to maintain
compatibility with already existing builds. New value for
OUTCOME_NA can be used to detect tasks with invalid states, i.e.
it should never appear after finishing a build.
Fixing noexec tasks outcomes.
[YOCTO #5763]
(Bitbake rev: 475643ad78796835bf2e731b9d0fa5794ec80dd1)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of the previous cleanups broke the downloadfilename functionality.
This change fixes the code to ensure the commandline is correctly built.
Thanks Kristof Robot <krirobo@gmail.com> for reporting the issue.
(Bitbake rev: e008d9bb07e5d1a3584cc04ca2cd3dd906fd5759)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
slef.self is clearly meant to be self, fix typo.
Otavio spotted and reported, thanks.
(Bitbake rev: 316daad7928a58cdfc42e27b20e739f4dd74a02a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On rare occasions it's useful to be able to exclude a part of a
variable's value from the variable's signature; for example if you want
to add an item to a list sometimes and not have the signature of the
variable change depending on whether the item is in the list or not. The
initial intended use case for this in OpenEmbedded is to allow adding a
function to SSTATEPOSTINSTFUNCS in buildhistory.bbclass and not have
that change any task signatures (so adding and removing
INHERIT += "buildhistory" won't lead to any rebuilds).
Part of the fix for [YOCTO #5897].
(Bitbake rev: f803bf8cfefafcbe212442e66b301ccd9c5aa2a5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The svk fetcher never appears to be used by anyone and the development
on svk appears to have stopped in 2010. We might as well drop support
for it.
(Bitbake rev: 8239264753977bd06ad5b1b574245d3842af489b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using the "execs" information in new code, it became clear that
the returned data was incorrect and there were missing exec'd functions.
This corrects the error and changes one of the test results to match
the correct behaviour.
(Bitbake rev: 8a24f2d3b735bbc59ca4a09670cabbadb1868c1a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These two functions have little in common, separate them out.
(Bitbake rev: 7413e0fa4dca9571ea98f32dab87d4fd60bc8de9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Start to clean up wget fetcher command construction to allow clearer
and more extensible code structure. Drops support for ${URI} and
${FILE} directly in the commands.
(Bitbake rev: 4e59fe45be2088996abc21e9a631a32b9a9642c9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These variables were dropped from OE-Core some time ago, drop their
usage from the fetcher as well.
(Bitbake rev: bd33e709ab65d6966b234010641861834d170e2b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Clean up some horrible old code and drop usage of the old style *COMMAND
variable and MKTEMPDIRCMD whilst in here. This means we don't need to touch
OVERRIDES either.
(Bitbake rev: c127bb3a9b7b1d2ab2c833ff73186b6ead0dc29c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Clean up some horrible old code and drop usage of the old style *COMMAND
variable and MKTEMPDIRCMD whilst in here. This means we don't need to touch
OVERRIDES either.
(Bitbake rev: 6b79789769da160d0e7fca0f9c6044dc1e11a107)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is other code which can want to run ls-remote style commands with
different parameters so split out the function.
(Bitbake rev: 13f1138f5504feee0ee8e8f3a0675d0bea490351)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use a newly created "git-submodule-test" repo on git.yoctoproject.org
which currently contains one submodule (the bitbake repository).
(Bitbake rev: a750c57242928c546a5aace632543e956ee908eb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Git versions older than 1.7.10 put absolute paths in configuration files
for the submodule repositories, leading to errors when the repository
checkout is moved. We move the repository as a matter of course in the
gitsm fetcher; the failure occurs in do_unpack). Change the absolute
paths to be relative during processing to fix this.
(At the time of writing, Ubuntu 12.04.4 LTS ships Git version 1.7.9.5,
hence the desire to fix this rather than just mandating a newer Git
version.)
Fixes [YOCTO #5525].
(Bitbake rev: e700d5a41deed4ee837465af526ed30c8a579933)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to remove hob-image.bb from meta-hob, a hob-image.bb should
be created somewhere in the build directory. I've saved it in build/recipes/images
directory, and moved the templates to recipes/images/custom (here are those
templates saved by the user).
The image is created when hob starts. Also it appends to BBFILES the directory
where it is created.
Removed images directory from meta-hob.
[YOCTO #5118]
(Bitbake rev: 4587297b51b7ca71d314bdb2c06f2061e7d4aa7d)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we have something similar to:
LOCKFILES = "${TMPDIR}/my.lock"
LOCKFILES_qemuall = ""
do_task[lockfiles] += "${LOCKFILES}"
when expanded, lockfiles will be empty for qemu, resulting in
File "/home/stefans/yocto/poky/bitbake/lib/bb/utils.py", line 630, in mkdirhier
raise e
OSError: [Errno 2] No such file or directory: ''
This should filter out the empty expansions.
(Bitbake rev: 7813e1bfd08cd48871f8c03cae2810265590105d)
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now bitbake has an API to run individual task for targets.
Hob can use this to build the sdk, instead of using the hob-
toolchain.
(Bitbake rev: 4a5009036e9cb38f6e0260a88278948931073bc6)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A typo was meaning that the mirror creation method wasn't being called
when it should have been. Fix the type to fix mirror tarball creation.
[YOCTO #5284]
(Bitbake rev: 348971d410bfd5d8b1757468d73e1d24ae78a594)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* exception like this keeps spinning quite quickly generating GBs of logs
better to kill it asap and show invalid pickle
(Bitbake rev: a69eb4c12c71bba9d742c4e5578f25c388d9f825)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using wildcard .bbappend files with anonymous methods in them,
bitbake/python fails to parse the generated code since the '%' is encoded
in the generated method name.
Fix this by including '%' in the convert-to-underscore list during
method name mangling.
While we're at it, move the method name mangling translation table
to a class variable, as suggested by Chris Larson.
[YOCTO #5864]
(Bitbake rev: 537f1f9bbe110acc9848ef95f43468c07d87af79)
Signed-off-by: Jacob Kroon <jacob.kroon@mikrodidakt.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tasks whose names contain underscores (such as do_populate_sdk in OE)
when converted to a task override do not function properly. If we
replace underscores with hyphens we can still have a working override
for these tasks.
(Bitbake rev: cf90bd6b2a0ab7dce922bffb500d6a2ff2ff10e2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It doesn't really make sense to set PN from .conf files, for example.
More concretely, this avoids the config hash changing unnecessarily
within Hob due to PN effectively changing (since bblayers.conf is
parsed first and then .hob.conf).
(Bitbake rev: 22e03ef7ac9bb6b7245250347ae9c10c19f1d74e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A previous commit of mine used the target variable for two different uses
resulting in a lot more sstate being installed than is needed.
Fix the variable to use two different names and unbreak the setscene
behaviour.
(Bitbake rev: f975ca2cf728561bd6317ed8f76303598546113a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you specify multiple targets on bitbake's commandline and some of them are
setscene tasks which are "masked" by other tasks they may not get run.
For example <image>:do_rootfs <kernel>:do_populate_sysroot
the rootfs tasks "masks" the populate_sysroot task so bitbake would currently
decide not to run it. In this case, we do really want it to be run.
The fix is not to skip anything which has been given as an explict target.
(Bitbake rev: 0753899d1e855795cc18671357609a86f169b379)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit c54e738e2b5dc0d8e6fd8e93b284ed96e7a83051 added in the idea of hard dependencies
such as the case a setscene has a hard dependency on pseudo-native and that
dependency wasn't available from sstate for some reason.
Unfortunately the implementation was a bit too enthusiastic, causing rebuilds
of things when it wasn't necessary. A test case was:
bitbake quilt-native
bitbake quilt-native -c clean
bitbake <some-image>
and then you'd watch quilt-native get rebuilt for no good reason.
The clue to the problem is in the for loop where it never depends on
the item being iterated over.
The fix is to include the exact list of hard dependencies rather than
guessing. With these changes, the use case above works, the one in
the original commit also works.
This patch also adds in or cleans up various pieces of logging to
allow issues like this to be more easily debugged in future.
(Bitbake rev: 81bd475585ff1b44b390036b1eca0feae7c149eb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the shell trap code was created in exec_func_shell(). Split
the function so that we can create the code separately.
Also, some whitespaces were automatically deleted by my editor. Since
this is not necessarily a bad thing, leave these changes too.
(Bitbake rev: c712e622d20c61a07c9c172b60e9dc6beae14197)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
pre/postfuncs were not being added to checksums. This meant that when reconfiguration
occurred, tasks were not always being rerun when they should. This include
sstate functions as well as systemd's do_install function in the OE metadata.
With the addition of postfuncs, its possible a shell task can have a python
pre/postfunc so we have to guard against this when generating shell output
in emit_func.
(Bitbake rev: b84d010144de687667cf855ddcb41c9b863c236e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Shared work directories work by assuming bitbake will not run
more than one task with a specific stamp name. Recent runqueue optimisations
accidentally broke this meaning there could be races. This fixes the code.
(Bitbake rev: b1628b1a260ddf43fc9985535b1ddcfcebbb1e5b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we set:
BB_SIGNATURE_HANDLER = "noop"
Then we would get the following errors:
[snip]
File "runqueue.py", line 876, in RunQueue._start_worker(fakeroot=False, rqexec=None):
"fakerootnoenv" : self.rqdata.dataCache.fakerootnoenv,
> "hashes" : bb.parse.siggen.taskhash,
"hash_deps" : bb.parse.siggen.runtaskdeps,
AttributeError: 'SignatureGenerator' object has no attribute 'taskhash'
[snip]
This patch fixes the problem.
[YOCTO #5741]
(Bitbake rev: 2bfcb751891cf3b4050e996b3c8e28678c3a8bf4)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hob doesn't read these variable from file and bitbake doesn't use them,
so they shouldn't be set in conf file.
(Bitbake rev: a8c9df86b96e27dc49028c2da42034d13988960c)
Signed-off-by: Irina Patru <irina.patru@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If enable_proxy is not activated, Hob should not save the proxy values
in the conf file when user hits save button.
[ HOB #5308 ]
(Bitbake rev: fbe0851221ecfcefea5bdd4b629a05ed4f5ac189)
Signed-off-by: Irina Patru <irina.patru@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When Hob receives a bb.command.CommandFailed event, it should check if
it's a log kind of information.
"Forced shutdown" and "Stopped build" are messages that show when a build
is not complete, but Hob considered them error.
[HOB #5609]
(Bitbake rev: ea1939f7ec8d8a71ce16a60c251c2413d7d91eb3)
Signed-off-by: Irina Patru <irina.patru@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The busy cursor would never change after pressing Stop button.
It should be set after the possible return inside machine_combo_changed_cb()
method.
(Bitbake rev: d440d3ad4b2d99bc20e06d2d5f5e76d07864dff3)
Signed-off-by: Irina Patru <irina.patru@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It must be checked first if parser has the attribute 'shutdown' when
user hits Stop button and the forceshutdown state is given.
[HOB #5579]
(Bitbake rev: 46943b442ea4fa778f70590b6dcce483595efaf8)
Signed-off-by: Irina Patru <irina.patru@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a bb.command.CommandFailed event is received by Hob, the error
message is stored inside event.error.
This information tells exactly why bitbake failed, so Hob should display
it instead of the current composed message.
(Bitbake rev: 24543ff6b45771712d624541ae35738d7d98f33c)
Signed-off-by: Irina Patru <irina.patru@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently its near enough impossible to tell bitbake to run target X,
task Y and target A, task B. We could hack various parts of the API
around but it would mean incompatibilities.
An alternative is to accept the syntax "<target>:do_<task>" as a target
name. The default task would be used where the task is unspecified.
This has the advantage that its neat/clean code and works from all
current APIs including the commandline.
(Bitbake rev: 55f6bee3114e582333a1784caeddb197b9163d02)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Under OE-Core, the name under which a package would
be installed in a target may have been different than the
name under it has been built or recorded in the dependencies
listings.
This patch addresses the way that Toaster records package
names, and adds the field of "installed_name" to save the
name under which a package have been installed in an image.
(Bitbake rev: 24e0367429b248108b104ab5a2af05efcf7a8c39)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
THis is a large set of fixes for the generic table, Build and
Dashboard pages.
Among the fixes:
* the table remembers which columns to show across refreshes,
based on saving the settings in a cookie
* added column timespent for a build which is a denormalization
of the completed_on - started_on information due to limits in
computing datetime differences in the SQL engine
* fixed formatting of the time differences
* various sorting header links fixed
* correct error and warning CSS classes applied to the
respective rows
* fixes multiple divide-by-zero error in displaying duration
estimations
(Bitbake rev: 61e3dee55ac577fce1c0ae0fe7e0d3cf644e8ae6)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch updates the task classification fields (outcome, etc)
as to
* Changes outcome names from SSTATE to CACHED and
from EXISTING to PREBUILT
* NoExec tasks now recorded as Not Executed / script type NA instead
of Executed / script type NOEXEC. Script type NOEXEC is deleted.
* SetScene tasks do not get order numbers
* New task method that returns a QuerySet for setscene tasks related
to this task: Task.get_related_setscene()
* New custom TaskManager that allows searching for setscene tasks
related to a certain task: Task.objects.related_setscene(task)
(Bitbake rev: a4164821a142f8b625a5fdc209adc6dc80874241)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I can't think of a reason we'd download zero sized files however there are
reasons zero length files can accidently make it onto source mirrors.
This check allows us to ignore the broken files and switch to another
mirror rather than fail with odd checksum failures.
(Bitbake rev: 300cba2e1a720dba4b83b0c76208ea93c608c1de)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to deference tags when trying to map them to commit IDs with
ls-remote. If we don't do this, a given commit might not show up
later in a specific branch. There appears to be no good reason not
to do this.
(Bitbake rev: 8ef24f4c834298348172b96ec0b855bf09552b09)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When specifying tags, they're searched for unanchored so foo/bar could
match:
refs/heads/abc/foo/bar
refs/heads/xyz/foo/bar
refs/heads/foo/bar
This change anchors the expressions so they are based against heads
or tags (or any other base level tree that has been created).
(Bitbake rev: df2e0972cd1db7abd5ec8b7cb295fb0c42e284a4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
