Commit Graph

5201 Commits

Author SHA1 Message Date
Mirela Rabulea
5a87d8c477 bitbake: Allow Hob to run images on a custom simulator, other than qemu
The current behavior of Hob is that there is a "Run Image" button which becomes visible only for qemu images.

My suggested change is:
- if an image is selected and it is qemu-compatible, let the "Run image" button be named "Run qemu image"
- if an image is selected and it is not qemu-compatible, let the same button show up with the name "Run custom image", and besides that, an option shows-up to allow the selection of the custom script (by default it points out to runqemu script) to be used for launching this custom image

Note: in case there is more than one toggled image (qemu runnable or deployable), when the user clicks the "Run custom image" button, a dialog will be presented, allowing to choose between any of the existing images.

[YOCTO #8940]

(Bitbake rev: cc4cfc2370297b8feb2dc39d4262e73adf06c09a)

Signed-off-by: Mirela Rabulea <mirela.rabulea@nxp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-18 07:41:15 +00:00
Christopher Larson
a25858982a bitbake: bb.ui.knotty: prefix task messages with recipe/task
An example prefix: `perl-5.22.1-r0 do_compile:`

(Bitbake rev: 792b759e59e31d2e43d525a6e50d866b4f51f072)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-16 09:04:23 +00:00
Christopher Larson
4bf8b21ab8 bitbake: Move bb.{debug,note,..} into their own logging domain
This lets us filter and use -l to show messages from that source specifically.

(Bitbake rev: 7946927156dec33364418988eb921ddb273660eb)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-16 09:04:23 +00:00
Richard Purdie
8b217208cf bitbake: knotty: Set exit failure code on runQueueTaskFailed events
If the worker segfaults, we may never see a TaskFailed event from it, only
a runQueueTaskFailed event. In this case, return_value isn't getting set
leading to an incorrect exit code from bitbake. Fix by setting return_value
in both places.

(Bitbake rev: e5dd50e0d95d532fe31dde61f8c6b1a7a72321e9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:48:12 +00:00
Richard Purdie
a9223e20d1 bitbake: taskdata: Fix traceback issue with missing provider
If there is a missing provider and we're using "-k" mode alongside "-w",
we could get a traceback since there was no provider. Add tests to avoid this.

(Bitbake rev: 90a4805e4e770a433b4394ea99792731e9a4b546)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:48:12 +00:00
Richard Purdie
7593756cbb bitbake: cooker: Improve cache handling
We're seeing issues where the self test, which uses tinfoil doesn't
notice the changed contents of include files. The issue is
cached_statements in the parser being reused when the files have changed.

Whilst looking at this, I realised there were some other issues:

* We need to also invalidate the mtime cache when cooker restarts
* We should pass full filenames to the file invalidation code
* We should process cached_statements as part of inotify invalidation

With these fixes, the caching is more reliable for memory resident
bitbake too. It does raise some questions about cache validation and
lifecycles and indicates bitbake does need more work in the area,
preferably with the removal of the globals. This at least highlights
and works around some of the current issues.

(Bitbake rev: 3f507ff8bc467fba936cf3f31bb8ea8e02f168e8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:48:12 +00:00
Richard Purdie
1c2d632a02 bitbake: Bump version post release to 1.29.0
(Bitbake rev: da1107a2c2331b3335c7bd46821edcb1933a0ff8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-12 13:44:22 +00:00
Ed Bartosh
f9f93ae856 bitbake: cooker: gracefully shutdown parsers
CookerParser.shutdown code doesn't do all required work to shutdown
parser processes. As a result bitbake hangs if interrupted during
parsing. Putting None into the parser_quit queue should fix this issue
as it makes parsers to quit main loop.

(Bitbake rev: f67307977e8f089ce6d208d3e9de2a6a1768757e)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 22:34:19 +00:00
Elliot Smith
1f7f077466 bitbake: buildinfohelper: unset brbe variable when build finishes
The brbe variable is set on the bitbake server when the Toaster
UI starts. This enables Toaster to associate events with the
correct build and build environment.

However, the brbe variable is also used when a build starts to
identify whether a new build needs to be created, or an existing
one looked up. This causes a bug for command-line builds which
happen after a Toaster-triggered build: because the brbe variable
is never unset on the server or the buildinfohelper, the new
command-line build events are treated as originating from the
previous build.

Ensure the brbe variable is reset when the buildinfohelper "closes"
a build, so that each build then either sets the brbe variable
(Toaster-triggered builds) or leaves it blank (command-line builds).

Also modify the localhostbecontroller so that the brbe variable
is not set on the server and not looked up from the server. This
ensures that it is only set when the triggerBuild() method is
called, and that it remains as None for command-line builds.

[YOCTO #9021]

(Bitbake rev: 4a6a8d0074f62208d843b06344be31ae73d9b745)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 22:34:19 +00:00
Richard Purdie
400f53003e bitbake: build: Improve python execution tracebacks
If something fails in a exec_func_python() the current stack trace shows
incorrect filenames and linenumbers. For example:

The stack trace of python calls that resulted in this exception/failure was:
File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 200, function: <module>
     0196:                chksum = bb.utils.sha256_file(fn)
     0197:                f.write('%s\t%s\n' % (chksum, os.path.relpath(fn, baseoutpath)))
     0198:
     0199:
 *** 0200:copy_buildsystem(d)
     0201:
File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 9, function: copy_buildsystem
     0005:IMAGE_FEATURES += "splash package-management x11-base x11-sato ssh-server-dropbear hwcodecs"
     0006:
     0007:LICENSE = "MIT"
     0008:
 *** 0009:inherit core-image
     0010:
     0011:IMAGE_INSTALL += "packagegroup-core-x11-sato-games"
File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call
     0531:    The arguments are the same as for the Popen constructor.  Example:
     0532:
     0533:    check_call(["ls", "-l"])
     0534:    """
 *** 0535:    retcode = call(*popenargs, **kwargs)
     0536:    if retcode:
     0537:        cmd = kwargs.get("args")
     0538:        if cmd is None:
     0539:            cmd = popenargs[0]

The problem is the use of "FILE" to obtain the current filename. Instead,
we therefore inject the function being executed into the methodpool which
allows us to correct its linenumber and filename information. We can then
clearly mark the initial piece as autogenerated and the rest of the linenumber
and filename information should be correct. Afterwards the trace starts:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:copy_buildsystem(d)
     0003:
File: '/media/build1/poky/meta/classes/populate_sdk_ext.bbclass', lineno: 66, function: copy_buildsystem
     0062:    import glob
     0063:    import oe.copy_buildsystem
     0064:    import subprocess
     0065:
 *** 0066:    subprocess.check_call("foo")
     0067:
     0068:    oe_init_env_script = d.getVar('OE_INIT_ENV_SCRIPT', True)
     0069:
     0070:    conf_bbpath = ''
File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call
     0531:    The arguments are the same as for the Popen constructor.  Example:
     0532:
     0533:    check_call(["ls", "-l"])
     0534:    """
 *** 0535:    retcode = call(*popenargs, **kwargs)
     0536:    if retcode:
     0537:        cmd = kwargs.get("args")
     0538:        if cmd is None:
     0539:            cmd = popenargs[0]

We can't inject into methodpool at parsing time, since there may be
_append or other override operations against the function before its
execution.

(Bitbake rev: fae153095d23157dd7e72c29f683f86149ee33a8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 15:47:11 +00:00
Richard Purdie
aece748762 bitbake: build/data: Don't expand python functions before execution [API change]
Right now, if you have some python code like:

X = "a"

def somefunction(d):
    d.setVar("X", "b")
    d.setVar("Y", "${X}")

then any sane person would expect that Y = "b" at the end of the
function. This is not the case, Y = "a".

This is due to the python function being expanded before execution, the
executed code would read d.setVar("Y", "a"). This understandably
confuses people, it also makes it near impossible to write ${} in a
python function without unintended things happening.

I think there is general agreement we should fix this and standardise
on non-expansion of python functions. We already don't expand anonymous
python (mostly).

I've checked OE-Core with buildhistory before and after this change and
there were a small number of issues this exposed which I've sent
patches for.

I propose we default to not expanding python code and then deal with
any consequences from that if/as/where identified. This will improve
new user understanding and usability of the system, it also allows
several long standing weird expansion issues to be fixed.

(Bitbake rev: 8bf33a8e92c0e188fa392030025756196c96fcbb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 15:47:11 +00:00
Richard Purdie
e39cfb1f9c bitbake: cooker: Don't expand python functions in variable dumps
We don't want to expand python functions since they aren't expanded
at execution time (e.g. anonymous python). They can also have side
effects.

This function is primarily used by toaster for variable dumps for later
display. The lack of expansion of python functions won't matter in this case
and actively helps some variable handling (e.g. SRCPV).

(Bitbake rev: 3f5520b4844a4bdd615046479ba08ed192bdc8cd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 15:47:11 +00:00
Richard Purdie
f652b6b9d0 bitbake: data: Don't expand python functions for variable dependencies
Expanding python functions for variable dependencies doesn't really make sense,
not least since this causes execution of any inline python, it also makes it
impossible to write expressions like d.expand("${X}") of d.setVar("X", "${Y}")
which may have the wrong values if expanded now.

This starts to standardise the approach across bitbake for handling python code.

(Bitbake rev: 765a2480dbe288f64562a9611dd93b6b6dd0a64e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 15:47:11 +00:00
Richard Purdie
d3e0c448ea bitbake: data_smart: Avoid expanding anonymous python functions
We don't expand anonymous python before execution, so nor should
we do this when calculating checksums for them.

(Bitbake rev: 5f10987edda35b08970a6dd6ccf9febad271ce3e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 15:47:11 +00:00
Richard Purdie
99184d7879 bitbake: BBHandler/ast: Merge handMethod and handleMethodFlags
The functionality overlap between these two functions is significant and
its clearer to handle both things together since they are intimately
linked. There should be no behaviour change, just clearer code.

(Bitbake rev: 391aa4afc91be90d8d3ee47e1bf797d6ebe61a71)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:32:46 +00:00
Richard Purdie
6ba69b4383 bitbake: utils: Drop datastore function inspection during exception
When we use functions from the data store, they now have correct line number
and filename information. This function would attempt to correct line numbers
which doesn't need correcting, leading to misleading messages to the user.
Therefore remove this code as being obsoleted.

(Bitbake rev: 918bec86bc8ee94feb82380ff410d9fdcbe9e720)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:32:46 +00:00
Schroeder, Henning
f8a44b160d bitbake: cooker: extended dot styling
Extended the dot styling of dependencies created by bitbake -g in dot syntax to differentiate between the various kinds.
  depends: solid
  rdepends: dashed
  rrecommends: dotted

The change observed is that depends get an explicit style which is the same as dot default behavior and the runtime recommends get
dotted while before they were dashed. This helps to distinguish them graphically as well as eases post processing by script.

(Bitbake rev: 86e78e0ca7aa5452411f35239942ecee3d8824ec)

Signed-off-by: Henning Schroeder <henning.schroeder@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:32:46 +00:00
Michael Wood
a8ab1c6b25 bitbake: toaster: buildinfohelper CustomImagePackage update dependency info
Instead of keeping the original dependency information for the pool of
CustomImagePackage reset it with each new build.

(Bitbake rev: a0b97ffc7a468bad081ce3276c74728bf6830250)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:21 +00:00
Michael Wood
a6e4f94b01 bitbake: toaster: buildinfohelper Add the concept of CustomImagePackage
This adds the concept of CustomImagePackage this is similar to the way
layers and recipes work in that we have a set of data which is part of
the build history and a set of data which is part of the configuration
data that toaster uses to guide people in configuring their project. We
create a set of  built_packages for every build but only create a package
for configuration purposes if we don't already have one, so that the
CustomImagePackage only ever contains a unique list of packages that are
available to be added and removed from a CustomImageRecipe.

(Bitbake rev: f81bb65883baa6c0f8a4d48a4de3291a10543992)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:17 +00:00
Aníbal Limón
d0d85a4d6b bitbake: bb/fetch2: Move export_proxies function from wget to utils.
In order to use in other modules since is a common function
when needs to get proxies working.

(Bitbake rev: 85c529044381895556d603a3974de22392646a22)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-08 10:52:40 +00:00
Ed Bartosh
916c73dfef bitbake: cooker: shutdown cooker parser on shutdown
Currently any not catched exception in cooker causes bitbake
to hang because of not terminated children of CookerParser.

Long term solution would be to reimplement Cooker as a context
manager and terminate parser children in its __exit__ method.

Partial fix is to call CookerParser.shutdown in Cooker.shutdown in
hope that all Cooker exceptions are caught and shutdown method is
called.

[YOCTO #8900]

(Bitbake rev: 3f67600dc3292bc8208644ce89e8bf7ab95cf2e7)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-06 23:13:47 +00:00
Richard Purdie
8857498ba4 bitbake: fetch2/osc: Clean up old variable syntax
Since people do copy and paste these things, clean up old syntax styles.

(Bitbake rev: 4fb028b0bd14d3e4b3fd7a89c643528728566476)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-06 23:13:47 +00:00
Richard Purdie
54da82972c bitbake: fetch2/osc: Remove hardcoded url
This shouldn't be in here, use a variable instead.

(Bitbake rev: 2e25d09a1ab62ccc3573d13114d59838cf4b07f2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-06 23:13:47 +00:00
Richard Purdie
6ae662acdc bitbake: parse/ast: Mark anonymous functions as python functions
Anonymous functions are python functions, set the variable
flags as such so we can detect them and avoid expansion where
needed.

(Bitbake rev: 1b303785c578bbae3a89be8d751d80fba860f62e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:47:49 +00:00
Richard Purdie
9913fd88d9 bitbake: codeparser: Improve handling of data.expand() dependencies
Currently bitbake doesn't parse into data.expand() expressions,
relying on high level expansion of python code to handle this.

One of the tests does however test this works.

We don't really want to be doing string expansion on python code,
so specifically parse into expand() function calls so that when
the high level behaviour is tweaked, the self tests continue to
pass and that we do continue to handle expand() function calls as
best we can.

(Bitbake rev: b12c17be5e4a74c9680876605c87f46501f78d28)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:47:49 +00:00
Richard Purdie
4628fe12e7 bitbake: lib/bb: Add expansion parameter to getVarFlag
This sets the scene for removing the default False for expansion from
getVarFlag. This would later allow True to become the expand default.

On the most part this is an automatic translation with:

sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g'  -i `grep -ril getVar *`

There should be no functional change from this patch.

(Bitbake rev: 7c3b99c6a716095af3ffce0b15110e91fb49c913)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:47:49 +00:00
Felipe F. Tonello
b98866d003 bitbake: fetch2/gitsm: Fix when repository change submodules
This fix a problem when checking out a commit that changes the submodules
previously checkout.

Example:
Recipe uses branch A and then it updates to use branch B, but branch B has
different submodules dependencies then what branch A previously had.

(Bitbake rev: 54a3864246f2be0b62761f639a1d5c9407aded4f)

Signed-off-by: Felipe F. Tonello <eu@felipetonello.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:47:49 +00:00
Richard Purdie
390c2c1268 bitbake: data_smart: Add missing expand parameter to getVar call
(Bitbake rev: 2876019e696ff2af164961d5d4c8e3ea9dfefc23)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04 23:47:49 +00:00
Phil Blundell
a4a5d1f323 bitbake: cooker, bitbake-worker: Fix spelling of "received"
I before E, except after C...

(Bitbake rev: 14c9593265f7469cb8a205a46f845ac7491246df)

Signed-off-by: Phil Blundell <pb@pbcl.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-01 11:20:42 +00:00
Richard Purdie
8f6b9c75a7 bitbake: cooker: Only start as many parse threads as we need
If we're only going to parse one recipe, no point in starting
a large number of threads.

(Bitbake rev: b977faf59dc08050a44a16032fe52d1bbb80f2a1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-01 11:20:42 +00:00
Richard Purdie
602da7c921 bitbake: knotty: Don't show errors for universe provider issues
When running universe builds, we don't expect an error exit code for
provider warnings. Change the error messages to warnings in this case.

This deals with errors causing problems on our autobuilders amongst
other issues.

(Bitbake rev: d4989fb0355476de172169f0698757f7360e9a1f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-01 11:20:42 +00:00
Sujith H
3072361f86 bitbake: bitbake: BBUIHelper: Remove function findServerDetails
findServerDetails function can be removed safely
from the source tree. Couldn't find any files
calling this function.

(Bitbake rev: 46871f769db13ccd36deedc5b6f3dbc0a3d31c4b)

Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:43:56 +00:00
Peter Kjellerstedt
28c041c1e7 bitbake: fetch2: Simplify logic in verify_checksum()
The recent change to verify_checksum() to only show checksum warnings
if no checksums are supplied made it possible to simplify the logic a
bit more.

(Bitbake rev: 1dc00b874acae44bbba9d8028d94f7bc97ddcd76)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:43:56 +00:00
Richard Purdie
5375e6431c bitbake: bitbake: Set process names to be meaninful
This means that when you view the process tree, the processes
have meaningful names, aiding debugging:

$ pstree -p 30021
bash(30021)───KnottyUI(115579)───Cooker(115590)─┬─PRServ(115592)───{PRServ Handler}(115593)
                                                ├─Worker(115630)───bash:sleep(115631)───run.do_sleep.11(115633)───sleep(115634)
                                                └─{ProcessEQueue}(115591)

$ pstree -p 30021
bash(30021)───KnottyUI(117319)───Cooker(117330)─┬─Cooker(117335)
                                                ├─PRServ(117332)───{PRServ Handler}(117333)
                                                ├─Parser-1:2(117336)
                                                └─{ProcessEQueue}(117331)

Applies to parse threads, PR Server, cooker, the workers and execution
threads, working within the 16 character limit as best we can.

Needed to tweak the bitbake-worker magic values to tell the
workers apart.

(Bitbake rev: 539726a3b2202249a3f148d99e08909cb61902a5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:43:56 +00:00
Richard Purdie
5b234d1539 bitbake: utils: Add ability to change the process name
Being able to tell the bitbake processes apart is useful for debugging.
Add a helper function which allows this without making it a hard
dependency. Errors are ignored, this is just nice to have.

(Bitbake rev: fd7f1a94d196b8a3c445e313d9e699b352b1da97)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:43:55 +00:00
Andre McCurdy
0b06924759 bitbake: data.py: avoid double newlines at the end of functions in emit_var()
A newline is always appended to the function body when it's written
out, so strip any trailing newlines which may be there already.

(Bitbake rev: 8a3f50936113e15d2f2822f6aee494204fa1c24f)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:43:55 +00:00
Andre McCurdy
68600aee2d bitbake: build.py: minor shell_trap_code() formatting tweaks
Fix quoting of $BASH_COMMAND and avoid wrapping at 80 columns (the
script which follows is likely to contain some very long lines, so
line wrapping in bb_exit_handler() looks somewhat out of place).

(Bitbake rev: 8e12c8f8441a7c6a03e603c5789d6037945704c1)

Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:43:55 +00:00
Peter Kjellerstedt
a948f5252a bitbake: cooker: Allow BBMASK to contain multiple regular expressions
Before, BBMASK was only permitted to contain one regular expression.
This made it hard to add to the BBMASK in multiple places as one was
supposed to separate the different regular expressions with a "|"
rather than with whitespace as is customary in BitBake variables.

Now one can specify any number of regular expressions in BBMASK. This
makes it possible to, e.g., mask out recipes in another layer from the
layer.conf file.

This also properly ignores any regular expressions that do not compile
(before an invalid regular expression would cause a ParseError in the
first bbappend file found stating that it was not a BitBake file...)

(Bitbake rev: 2c778ad50aceaffb855baf5f4aa0fed98c880870)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-29 17:03:53 +00:00
Ross Burton
9f22898e0f bitbake: fetch2/wget: fallback to GET if HEAD is rejected in checkstatus()
The core change here is to fall back to GET requests if HEAD is rejected in the
checkstatus() method, as you can't do a HEAD on Amazon S3 (used by Github
archives).  This meant removing the monkey patch that the default method was GET
and adding a fixed redirect handler that doesn't reset to GET.

Also, change the way the opener is constructed from an if/elif cluster to a
conditionally constructed list.

(Bitbake rev: 6ec70d5d2e330b41b932b0a655b838a5f37df01e)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 22:34:45 +00:00
Richard Purdie
4a5458f38f bitbake: fetch2: Don't show checksum warnings if a single checksum was supplied
If one checksum is supplied to a SRC_URI, we really don't want to show
warnings about the other type which isn't present as one checksum
is really good enough for most cases.

(Bitbake rev: 43358a9b595b2928458a5f463cf1949394160c3a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:42 +00:00
Paul Eggleton
76a281c870 bitbake: taskdata: add the ability to access world targets list
In certain circumstances it can be useful to get access to the world
targets list from a recipe in order to add dependencies on some or all
of the items in it. If a special function, 'calculate_extra_depends' is
defined in the recipe, and the recipe is to be built, then call it at
the right point before we calculate which tasks should be run. The
function can append items to the "deps" list in order to add
dependencies. This is not as tidy a solution as I would have liked, but
it does at least do the job.

As part of this change, the buildWorldTargets function was moved to
bb.providers to make it possible to call from taskdata.

Part of the implementation of [YOCTO #8600].

(Bitbake rev: aba0dce57c889495ec5c13919991a060aeff65d2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22 12:45:44 +00:00
Robert Yang
11a1f49630 bitbake: cache.py: check existence before add to cachedata.rproviders
The rprovides maybe contain duplicated lines when parse again, we need
check it before add to cachedata.rproviders, similar to what we had done
to cachedata.providers.

(Bitbake rev: 6c488afb0fe30a9655ec62a1d22f9f388365f012)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22 12:45:44 +00:00
Robert Yang
05c17750c3 bitbake: taskdata.py: add RuntimeProviders to close matches
This is useful for newbie, for example:
$ bitbake rpm-build
ERROR: Nothing PROVIDES 'rpm-build'. Close matches:
  pm-utils
  rpm RPROVIDES rpm-build

[YOCTO #8881]

(Bitbake rev: 4b59eb8cc2321fe72f2988b6c9c0fecd4883255b)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22 12:45:44 +00:00
Richard Purdie
cf9cb65eec bitbake: data_smart: Don't show exceptions for EOL literals
If variables are unset, the code simply doesn't expand them, there
aren't errors. If the code is a python expression, this can get a bit
messy, see the attached test case. The python expansion code sees the }
of the unexpanded value rather than the close of the python expression
and then raises a SyntaxError exception.

Ideally, we'd update the code to match pairs of brackets. I don't know
how to do that with the current regex and this is unfortunately a
performance sensitive piece of code. We also run the risk of breaking
existing code in OE-Core where there are "{" characters but not "}"
to close them (PKGE and PE).

Rather than raising the exception, matching the existing "just return
the expression" behaviour seems more consistent with the standard
variable behaviour.

This addresses an issue found in the recent image.bbclass code where
there are some variables we choose not to expand (TMPDIR/DATETIME).

This patch also adds a test case for this behaviour. It wouldn't preclude
improved bracket matching code in the future either.

(Bitbake rev: d80d39e73223a50fda0090784303d2c57167bb4c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22 12:01:24 +00:00
Olof Johansson
c886a786cf bitbake: tests/codeparser.py: Add filename/lineno flags to test variable
A recent change in bitbake added filename/lineno information to the
parameters of bb.data.build_dependencies(). The codeparser tests
required a little adaption to the changes, adding the flags to the FOO
variable used in the tests.

The error seen when running the tests is a TypeError exception raised
in bb.codeparser:

  TypeError: int() argument must be a string or a number, not 'NoneType'

(Bitbake rev: f1fe674397ac5cd355696d5b4cc90b7cfa6c867f)

Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:38:51 +00:00
Richard Purdie
f0d47a6d9a bitbake: Revert "runqueue.py: Ensure one setscene function doesn't mask out another which needs to run"
This reverts commit b22592af81.

That commit isn't entirely clear about why this change is needed but
I do have a usecase where this breaks things. If for example you run
"bitbake X -c packagedata" and that packagedata is in sstate, you'd
expect this to work.

If sstate doesn't contain a do_populate_sysroot for a dependency, you
would still expect the command above to succeed and you would not
expect
it to rebuild that dependency. With the current code, this isn't what
happens. The code finds the sstate for do_populate_sysroot missing,
this makes the task "uncovered" and this in turn makes it unskippable.

The example I found with this was avahi-ui, where it would trigger
a build of libdaemon to obtain its populate_sysroot.

Since this behaviour seems completely incorrect, revert the older patch
and we'll address any issues that crop up as a result.

(Bitbake rev: 36a9840a5da17cc14561881fdd6a4f2cb0a75e49)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:26:15 +00:00
Elliot Smith
68f3e1e067 bitbake: toasterui: log OSErrorException metadata events
OSErrors occurring in toaster.bbclass are converted to
OSErrorException metadata events. They were then being swallowed
as unprocessed events by toasterui, which made them difficult
to spot.

Explicitly catch OSErrorException events and log them so they
are easier to spot and debug.

(Bitbake rev: 69f2b2bc373ce114609600b59a6b6ccef20771c9)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 22:06:46 +00:00
Elliot Smith
fb94754e83 bitbake: toasterui: listen for bb.event.MetadataEvent
The event mask for toasterui doesn't include MetadataEvents.
This means that we're missing the ArtifactFileSize event
(among others), which is the one we use to populate the SDK
artifact table.

Add that event type to the toasterui event mask so we can
record SDK artifacts as they are created.

[YOCTO #7603]

(Bitbake rev: d0276a831bb8cffd42c8367895633eaa1fa1ed30)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-18 22:06:46 +00:00
Richard Purdie
f8508deb01 bitbake: Revert "fetch/git: Change to use clearer ssh url syntax for broken servers"
This reverts commit 4193e99adce8e88f12ac88d7578ad39575f7e346.

It seems the underlying issue was caused by ":" in the url which isn't
supported. The patch was therefore incorrect.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-15 15:02:40 +00:00
Richard Purdie
da43a56d35 bitbake: Revert "fetch2/local.py: avoid using PREMIRROR"
This reverts commit e130dca85bac82bd4d88f94a6bf9fe36e8ad4d7c.

This is in fact a valid use case, for example the sstate.bbclass code
sets up SSTATE_MIRRORS as PREMIRRORS. Its quite common to map those
file:// urls to remote http:// urls and with the above change, this
no longer works.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-13 09:47:28 +00:00