If we find an error in the layer configuration (such as an unsatisfied
item in LAYERDEPENDS) then exit by raising an exception at the end of
handleCollections() (without producing a backtrace).
(Bitbake rev: c7486a09310fe63b1aa1b7b0bb9450f306b6093b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit implements a new design for hob
Some of the new features:
- Friendly new designed GUI. Quick response to user actions.
- Two step builds support package generation and image generation.
- Support running GUI seprarately from bitbake server.
- Recipe/package selection and deselection.
- Accurate customization for image contents and size.
- Progress bars showing the parsing and build status.
- Load/save user configurations from/into templates.
(Bitbake rev: 4dacd29f9c957d20f4583330b51e5420f9c3338d)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Shane Wang <shane.wang@intel.com>
Signed-off-by: Liming An <limingx.l.an@intel.com>
Signed-off-by: Fengxia Hua <fengxia.hua@intel.com>
Designed-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For similar reasons as the nocheckout option, packages that need
enhanced control over the checkout and branch creation on a repository
may want a complete mirror/bareclone created of the repository when
performing the unpack.
This is useful/required when a local respository is being used, but
local tracking branches have not been created for all branches that
a given recipe needs to manipulate. The standard git clone operations
will create remote branches for the branches that are local to the
source repository, but branches that are remote do not translate to
the destination repository. Doing a mirror/bare clone of the source,
makes all branches available to the repository.
This is a particular use case, but the ability to do a bare clone
creates great flexibility in recipe space, with no impact to recipes
that don't need this functionality.
To implement this, a new option 'bareclone' is craeted which creates
a mirror copy of the repository and leaves it bare in the unpacking
phase. A recipe that uses this option must both checkout and debare
the repository itself.
(Bitbake rev: 82482aae6f311c994275fb0b6b32d954bbfc78c3)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current code prints a log when a setscene task starts, therefore
the progressbar in hob will not receive it. Use a sceneQueueTaskStarted
event instead.
Besides, change the sceneQueueTaskFailed event to inherit runQueueEvent
directly to avoid confusion to event receiver.
(Bitbake rev: 7c07cc93d6558d7d9c3144b13493901b7ebae050)
Signed-off-by: Shane Wang <shane.wang@intel.com>
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The changes include:
- Clean some events in event.py
- Fire essential events for Hob2 to handle with more information.
- knotty changes
(Bitbake rev: 9ede881620c501574f014e600cea6947ea908ac2)
Signed-off-by: Shane Wang <shane.wang@intel.com>
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we have generateTargetsTree API, which is used to get
dependency information. However in that tree, there will be
"virtual/xxx" in depends fields. Therefore we add the resolve option
to replace it with its real providers.
Besides, for packages that provided by multiple recipes, we will find
their preverred provider.
(Bitbake rev: 28501612efdfc6ee47576cc90deb6e897883e7f5)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This functions enables the client to request triggering specific event
from bitbake server.
(Bitbake rev: 45da6d709a69697158fae92e1c0c0a6ac8f30831)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new API in command.py to get the cpu count in order to set the appropriate default BB_NUMBER_THREADS and PARALLEL_MAKE variables.
(Bitbake rev: 335047b2e440e65713e88fabb24b47a9c82f939b)
Signed-off-by: Shane Wang <shane.wang@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In server/client split model, the client will bind to a specific address
and port. We need to pass the values to BitBakeServerConnection().
(Bitbake rev: c8e19c5c389efc06696084c6f9439ba75472c5b7)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When start bitbake as a server only process, we need to assign certain
interface to it.
(Bitbake rev: 95b97d2dc6466ea3d99371f5b5bd68f6f3c99074)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
RequestPackageInfo is triggered by GUI client to request the available
package information.
PackageInfo event is to pass package information back to GUI.
(Bitbake rev: 9020c2d4476766f63ff7e024bbd99043d06feefc)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously we use the file time stamp to judge if a cache is valid.
Here this commit introduce a new method, which calculates the total
hash value for a certain configuration's key/value paris, and tag
it into cache filename, for example, bb_cache.dat.xxxyyyzzz.
This mechanism also ensures the cache's correctness if user
dynamically setting variables from some frontend GUI, like HOB.
(Bitbake rev: 1c1df03a6c4717bfd5faab144c4f8bbfcbae0b57)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that a file and line number are reported for ParseError where
possible. This helps particularly in the case of inherit and require
which previously did not report either of these upon failure.
(Bitbake rev: f588ba69622a2df35417ced184e56c79ac1b40d5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes a regression introduced in BitBake rev
c9f58ef6b897d3fa5b0d23734b5f2cb3dabb057a which prevents errors during
parsing from being fully reported because BitBake shuts down before it
can print them. Move the error printing to before the shutdown in order
to fix it.
Also remove a couple of superfluous calls to sys.exit() since these
will never be called (as self.shutdown(clean=False) does this itself).
(Bitbake rev: 356c42ed00541dea2c382c57f768a4f431d22db0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The dataCache exposes much useful information to the singature generation code
for example its possible to figure out whether kernel dependencies are
important or not based on the RRECOMMENDS values.
We therefore pass this information to the handler and trust it to handle
any API changes in the dataCache structure.
For backwards compatibility, OE-Core is already updated to handle
the extra argument when it appears.
(Bitbake rev: 4ebc25155d5f96a7f63de22b52b18d045eb4dec7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent Ctrl+C patch was not fully tested; the variable it was
relying upon is set to non-zero under other circumstances and thus
bitbake was reporting that it was interrupted and returning a non-zero
exit code when it was not. Track this status in a separate variable
in order to fix the issue.
(Bitbake rev: 95a599067650902727ecb4a39d6dd003c5cfedf3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* otherwise it reports lots of changed checksums just because it compares different tasks
notice linux-libc-headers_3.1.bb.do_package_write:
Task dependency hash changed from 42acced29debf54d598802474c5e9cbb to f438a54f995df77620d0727d6f4b4ce5 (for linux-libc-headers_3.1.bb.do_package_write and linux-gta04_git.bb.do_deploy)
Task dependency hash changed from 61f8babe1d10c6e7fb1423112bb04e1e to 1b3e21ff106ecfcb7ddf76a1e29537bb (for linux-nokia900-meego_git.bb.do_deploy and linux-gta04_git.bb.do_package_write)
Task dependency hash changed from 512f9d6686d760b318d8b11c8b589226 to 42acced29debf54d598802474c5e9cbb (for linux-nokia900-meego_git.bb.do_package_write and linux-libc-headers_3.1.bb.do_package_write)
Task dependency hash changed from 153e91dfd1d2053fda7b98cc08d4b802 to 92a293bdd8ed234932b87a66025038c5 (for systemd-serialgetty.bb.do_package_write and systemd-serialgetty.bb.do_package_write)
Task dependency hash changed from 730abebf9954794bb440c2f3239f79fe to 413eaebaff27a2fd16f5cf68c1f4ff17 (for systemd_git.bb.do_package_write and systemd_git.bb.do_package_write)
with this patch:
Dependency on task linux-nokia900-meego_git.bb.do_package_write was added
Dependency on task linux-nokia900-meego_git.bb.do_deploy was added
Dependency on task linux-gta04_git.bb.do_deploy was removed
Dependency on task linux-gta04_git.bb.do_package_write was removed
Hash for dependent task systemd_git.bb.do_package_write changed from 730abebf9954794bb440c2f3239f79fe to 413eaebaff27a2fd16f5cf68c1f4ff17
Hash for dependent task systemd-serialgetty.bb.do_package_write changed from 153e91dfd1d2053fda7b98cc08d4b802 to 92a293bdd8ed234932b87a66025038c5
* added test if there is different task with same hash then we don't
show it as added/removed dependency, because bitbake doesn't care
(Bitbake rev: ca52bf32b479811bd7fed41648bedcc06b00430b)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
CommandFailed already sets the return code, so we print out the error
summary message, however we don't increase the error count so it is
usually zero in this case. As we are actually showing an ERROR message,
increment the error count so that the summary makes sense.
(Bitbake rev: aeb71c3ca65dc42015c29c0e4cac050bfbcb51a5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user uses Ctrl+C to exit out of the build, then ensure we return
a non-zero exit code since the entire build was not completed
successfully.
If nothing else, this is useful to avoid continuing in the bitbake
wrapper script in OE-Core if the initial pseudo build is interrupted.
(Bitbake rev: c11eab7fa230f1fd5cc33589f3555b94e95f202d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When building the universe target:
* Show a warning about universe likely producing errors
* Multiprovider errors are now shown as warnings
Also remove an unused "error" variable in runqueue.py.
Fixes [YOCTO #1936]
(Bitbake rev: 293c796e8a3d1f74ca1e51017b6dede261612281)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The number of threads is self.num_processes - 1 due to the range(1,
self.num_processes). Fixed by changing to range(0, self.num_processes).
(Bitbake rev: adc041fd9e3def29cdf9c1ae4849c5383bac46e5)
Signed-off-by: Caleb Crome <caleb@signalessence.com>
Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This ensures that when we are shutting down, we drain the queue of events from
the server. This ensures that the server never gets hung up on the join of the
feeder thread associated with the event queue, thereby avoiding a bitbake hang
issue.
Also, since the third keyboard interrupt now behaves the same as the second,
removed that message.
(Bitbake rev: ce9e0946b8a412616074809c76fba54f8bff36c4)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes the hang issue encountered with parse errors. The underlying issue
seems to have been the pool.terminate(). This sends SIGTERM to each of the
multiprocessing pool's processes, however, a python process terminating in
this fashion can corrupt any queues it's interacting with, causing a number of
problems for us (e.g. the queue that sends events to the UI).
So instead of using multiprocessing's pool, we roll our own, with the ability
to cancel the work. In the very long term, the python concurrent.futures
module introduced in python 3.2 could be used to resolve this as well.
(Bitbake rev: 7c39cfd8e060cca8753ac4114775447b18e13067)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without it, we get random hangs on parse failure. With it, some folks have
seen hangs even on successful cases. The former is clearly less problematic.
This is temporary, until I can finish investigating the root causes of both
issues.
(Bitbake rev: db689a99beffea1a285cdfc74a58fe73f1666987)
Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the addition of new cache domains, the summary, license and section
information is no longer requred in the core cache since its only used by
the hob UI. This patch removes the duplicated entries.
It also adds the DESCRIPTION field to the cache for the benefit of hob2.
(Bitbake rev: 33ffb2e99825cb643b148b3462c2d4cf33ff5f58)
Signed-off-by: Shane Wang <shane.wang@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To quote my report of this to the git mailing list:
"""
I have a problem with git clone commands using alternates failing by
mixing up different repositories. I have a situation where I could end
up with both:
/srv/mirrors/repo
/srv/mirrors/repo.git
as bare clones.
I then try cloning "repo" with alternates with the command:
$ git clone -s -n /srv/mirrors/repo /tmp/foo
Cloning into /tmp/foo...
done.
$ cat /tmp/foo/.git/objects/info/alternates
/srv/mirrors/repo.git/objects
Note how I'm now referencing repo.git, not repo. This doesn't work as
expected giving some very bizarre results when actually using the
repository.
I appreciate this is a rather bizarre corner case but its one that is
breaking the build system I work with. Ideally people would use a
consistent URL for the same repository but we have an example where they
haven't and this really shouldn't break like this.
Looking at the code, the cause seems to be
clone.c:get_repo_path():
static char *suffix[] = { "/.git", ".git", "" };
since its looking in order for:
repo/.git (fails)
repo.git (suceeds, incorrect)
repo (never looked at)
I'm not sure what would break if that order were to change, swapping the
last two options.
I can "force" the issue by running:
git clone -s -n /srv/mirrors/repo/ /tmp/foo
but this results in the slightly odd looking:
$ cat /tmp/foo/.git/objects/info/alternates
/srv/mirrors/repo//objects
which does at least work.
"""
This patch adds the trailing slash to ensure the correct repository is
referenced at the expense of some ugliness in the alternates file.
(Bitbake rev: d978e7b35550e3785c7c567ffe4c40a3c3947450)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #1942]
Ping the PRService only if PRSERV_HOST and PRSERV_PORT are set.
(Bitbake rev: 20f24de0bdafac21f5d8a58701f977efa7041288)
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make the following improvements to the show-overlayed subcommand:
* Show recipes that are overlayed when the version is higher or lower,
not just when it is the same. This gives a much better picture of the
influence each layer is having over the metadata used for building.
This can be disabled with the -s option if you just want to see
recipes with the same version as before.
* Default to showing name (PN), layer and version rather than the full
path and filename. The old style formatting can be used by specifying
the -f option.
* Mark skipped recipes as such in the output, and print them in the
correct sorted place in the list rather than at the end
* Prefix/suffix title line with === so it can be filtered out easily in
shell scripts if desired
(Bitbake rev: 43b473275d3cb2e60a14e4a52cdc4654b3f4e5e7)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is being added to resolve setscene race issues where we do have
particular dependencies required between setscene tasks. This allows
specific dependencies to be specified. This allows us to fix the races
in sstate with the useradd class in OE-Core.
Any tasks being depended upon have their reverse dependencies cleared to
ensure we don't have circular references.
(Bitbake rev: e1b157d26374a70e6274edcb4c0b9f3bc48f765c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When constructing the setscene inter-dependencies, we need to account for all task,
not just the last one found. This patch corrects this oversight and ensures all
dependencies are added, not just the first one found.
(Bitbake rev: b9b5b5129d066e1ff7d3effda116afc3c6657beb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* this is used in all other bitbake parts where sqlite3 is used, don't
know why it wasn't used here, but it fails e.g. on Gentoo
Traceback (most recent call last):
File "bin/bitbake", line 39, in <module>
from bb import cooker
File "lib/bb/cooker.py", line 39, in <module>
import prserv.serv
File "lib/prserv/serv.py", line 4, in <module>
import xmlrpclib,sqlite3
ImportError: No module named sqlite3
(Bitbake rev: 9a57ec705cf5c932d8c2a35852db7a4627c57937)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This means that custom signature handlers can override specific parts
of the code without having to reimplement whole functions allowing them
more flexibility.
(Bitbake rev: 164195c068a656733cfe7aa07369c5ed6ea62ca5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By calling init_parser which sets up the siggen code after the ConfigParsed
event is fired, we can allow the metadata to add siggen classes which
was always what the code intended.
(Bitbake rev: 69fd5ad4a8402fab5eb8348260966b337c5648ff)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A lot of our checks in the fetcher code assume that if the downloaded
file exists, the download finished successfully; this meant that if
BitBake was interrupted in the middle of a fetch then it would not
resume the fetch the next time, but instead attempt to use the
half-fetched file and usually fail as a result.
Since we're already writing a ".done" stamp file when a fetch completes,
just check for its existence and assume the download didn't complete
successfully if it isn't present.
(Bitbake rev: 721e986624529aedef96dd06c9fe6243f2897b10)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's unclear from the history but at some point we stopped logging the
"Tasks Summary" NOTE when tasks failed. Reinstate this for failure, and
also make the count of attempted tasks include the failed task.
(Bitbake rev: e44d5be98fc5d2589cd929ce143638395936b936)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The actual task names are discounted for comparison of dependent tasks, only
the actual hashes are used. This updates the comparison code to account for
this change, attempting heuristic matching for more user friendly output but
falling back to showing the changed hashes directly. This avoids some confusing
output to users where it looked like tasks had changed when they had not and
actually had the same hash but a different taskname.
(Bitbake rev: 7da7dff83ed765c9cde1d7f91ee1b65e49520481)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids some silent parser hangs we were seeing which were
near impossible to debug as no user feedback was given.
[RP: Tweak commit message]
(Bitbake rev: d104f29871c04a5a36600a35b2568b49e5b21ca0)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #1761]
Reconnect to the backend Sqlite DB in 'database is locked' exception so
the timeout can be leveraged in each time retry.
(Bitbake rev: b310382764367b573c84f33d847c6eb821266f9e)
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove the error logged within cooker summarising the list of failed
tasks, and instead print this in the UI (knotty) where it belongs. This
also adds the actual name of the task that failed as well as the
corresponding recipe file that was being shown previously.
In addition, reformat the summary messages more tidily - no extra breaks
between lines and use correct English singular/plurals, with some
allowance for future translation.
(Bitbake rev: cdf69913f99d28bc7f51067a60257701f952c6cb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Improves error output for matching problems when the -b / --buildfile
command line option is used.
Rename MultipleMatches exception to NoSpecificMatch (as it is also
raised when there are no matching recipes) and make it inherit from
BBHandledException so that it doesn't print a stack trace (we always log
an ERROR prior to raising it.)
In addition, improve the formatting of the error message - only call the
log function once rather than once for every match, and use a more
appropriate message if there are no matches.
Fixes [YOCTO #1141]
(Bitbake rev: 803550a5098ec878164245e71344c3d687310b72)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make the warning for the initial fetch failure a single line - we don't
need the full command and output here yet, but write it into the log in
full as a debug message. However, if fetching from mirrors fails as well
then print out the full details for the first error that occurred as an
ERROR rather than a WARNING.
Since this is logged as an ERROR, combined with an earlier patch it
suppresses the full log which does make the output much more readable
for any fetch error.
Fixes [YOCTO #1832].
(Bitbake rev: 6bbdc7d259c0cc041b62dbdb26cfc3ec6edcb6f3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a task has logged an ERROR then don't print the contents of the
task's log file in knotty (the default terminal UI).
As a side-effect we now also respect BBINCLUDELOGS in knotty; if it is
false we never print the log (but the pointer to the log file is always
printed).
(Bitbake rev: b9746b7e4d7aa5c34eba15a61427bfc6949af123)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The termination signal and exit code of the fetch process were not being
decoded correctly, resulting in bitbake reporting that the process
terminated with a signal of the exit code (if it was under 255). There
are functions in the Python os module to do this decoding correctly (for
Unix at least), so let's use them.
(Bitbake rev: 50aea9a76e40cf71cc3f1462c88298e4846a031c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Seeing the word "failed" alone without reading the whole context has
occasionally triggered an automatic assumption on the part of some users
(myself included) that something has gone wrong, even when this message
is telling you that "0 [tasks] failed". To avoid this let's just say
"all succeeded" in this case instead.
As a bonus this means you can now search the output for "fail" and not
find anything if all went well.
(Bitbake rev: b6f067af12d4661758a78788f1db472684b9aba8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
