Small tweak to ensure these items are printed into the log since there
is other logging code which looks for the header and this makes it clearer
the server did start but is slow somewhere in startup.
(Bitbake rev: e698ff1817e3536211f40af161c563d15e2ef3b5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if there are no setscene tasks, the disk monitor isn't started.
Move the startup code to somewhere to ensure it always is started. This
issue would partially explain occasional selftest failures.
(Bitbake rev: 9f72f5af7eb04c87be0e11a9c519a3941789bf7e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/main.py", line 464, in setup_bitbake
server_connection = bb.server.process.connectProcessServer(sockname, featureset)
File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/server/process.py", line 490, in connectProcessServer
if command_chan_recv:
UnboundLocalError: local variable 'command_chan_recv' referenced before assignment
(Bitbake rev: bdffac83bcbd12668d262867f3a6e329327ff1e4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updates for YOCTO #12891 allowed a user to have a directory
structure different to that of yocto (bitbake isn't inside
oe-core) whereas the definition of OE_ROOT in the main toaster
binary still assumes the same while checking for .templateconf
and hence we see an error on the cmdline in such cases:
bash: <repo-path>/bitbake/bin/../../.templateconf: No such file or directory
The change here now allows the user to provide OE_ROOT through
the environment in such cases and otherwise defaults to the older
mechanism to fix this issue.
(Bitbake rev: 79d01b9fec99cec19156e0ebe05b4f2040bcef05)
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before this fix it is assumed that the removal of the
remote can only fail because there is not remote to remove. This
is a false assumption. Example error which would be ignored:
git -c core.fsyncobjectfiles=0 remote rm origin failed with exit code 1, output:
Note: A branch outside the refs/remotes/ hierarchy was not removed;
to delete it, use:
git branch -d master
error: could not lock config file config
error: Could not remove config section 'remote.origin'
Due to the masking of this error a stranger error will be
presented to the user, because this time we do not mask the
exception:
git -c core.fsyncobjectfiles=0 remote add --mirror=fetch origin https://github.com/ptsneves/tl-wn722.git failed with exit code 128, output:
fatal: remote origin already exists.
The most likely reason that the remote cannot be removed nor
modified is that the DL_DIR/git2 does not have permissions
compatible with the user running bitbake.
This commit fixes:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12728
(Bitbake rev: 67189588a68b9bcb39421ef12103507b4c8820c3)
Signed-off-by: Paulo Neves <ptsneves@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The environment was being cleared before the UI imports occurred which
caused problems for graphical UIs like taskexp. The full environment was
intended to be available to UI clients and it was only meant to be cleared
for the server/cooker, so tweak the code order so this is the case.
This fixes problems reported for taskexp.
[YOCTO #12670]
(Bitbake rev: 66ac33b6888edb7077b6e94a4d3d1e1d1d53c0c5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit 15340edce23e63b060c75114d508e1f76757239c introduced a way which
allowed bitbake to be found from the PATH variable when directory
structures different from poky are used. This just drops a leftover
bitbake definition which made the earlier implementation meaningless
by reassigning the path relative to oe-core/meta.
[YOCTO #12942]
(Bitbake rev: 99321236c516277c9d423b1b355bbce1245d9923)
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use == instead of = when comparing task outcome to OUTCOME_FAILED.
Prior to this fix the recipe template would cause a TemplateSyntaxError
exception.
(Bitbake rev: a53ffec4ed3d0f9221bca398e20e8f480fb2b325)
(Bitbake rev: bef6ef1a79b2d5a3688ae845824f722c332d7f19)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The local layer info (provided through custom fixtures) should
not be hidden. It is better to handle it in the same manner
as an imported layer, otherwise the layer path and dependency
info is not shown. The layer editing fields are handled in the
html side of things appropriately so this does not harm that
implementation.
[YOCTO #12891]
(Bitbake rev: 75918405346abde6055658372c1a954a214001cc)
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In a case where the layer source is local only and the recipe
is not yet built, we can search for the path with layer's
local_source_dir, and if available that should be used rather
than just skipping the scenario.
[YOCTO #12891]
(Bitbake rev: 30f622dbd874b800c5bbdbeac992dd7783092928)
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change allows the CUSTOM_XML_ONLY toaster setting to be
provided through the environment so the user can do this without
mingling with the settings.xml, for scenarios where modifying
settings.xml is not achievable.
[YOCTO #12891]
(Bitbake rev: 87060a84e74125be10db062da3032e9b01f5dc96)
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The release drop down divs are not being closed
appropriately, which showed adverse reactions on
the UI that aligned the "Create project" button
with the left edge of the screen without any
margins. This fixes these divs which in turn
aligns the button appropriately.
[YOCTO #12891]
(Bitbake rev: bc6a137f4f8193cadfa8a8d0e43875991d47a818)
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There can be cases where the variables being used
to divide in build percentage expressions can be
zero. For example, a setup consisting of only local
repos will have repos_to_clone=0 and will generate
a divide by zero scenario.
Fix this by checking the divisor in such cases.
[YOCTO #12891]
(Bitbake rev: 30702f29928c3b088f199bf8b1609b2956f8c47a)
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
TOASTER_DIR is used for higher level toaster artifacts
such the SQL DB and creating toaster internal build
directories for projects. Prior to this change it was
evaluated as `dirname $BUILDDIR` and user had no control
over it. This change allows to override this variable
from the command line for more flexibility. The variable
defaults to its original setting if the optional argument
is not passed.
[YOCTO #12891]
(Bitbake rev: e073775d3b6980fc8004ae28a3ccc3c5bbf50fb2)
(Bitbake rev: 486e571b1caaf7f86f8f969c512566487bcd9841)
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current mechanism for finding the bitbake binary
assumes a directory structure which is identical to
poky, where oe-core's meta and bitbake directories are
at the same level. There can be a case where bitbake
is used from elsewhere and in such cases the above
mentioned assumption fails to hold, whereas this is
totally allowed by the oe-init-build-env script which
can take bitbakedir as an argument.
The better approach is to allow bitbake to be derived
from PATH, while keeping the older mechanism in place so
it can be removed after tests are done in various
environments. This makes more sense as toaster has
also been launched from the same bitbake instance
that is the one in PATH.
[YOCTO #12891]
(Bitbake rev: 15340edce23e63b060c75114d508e1f76757239c)
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster depends on pokydirname for identifying the location of
the oe-init-build-env script (and there might be other purposes
in the future). The problem with current approach is that it
only checks/sets the variable with git based repos, whereas
toaster provides mechanisms to allow having layers that are all
locally available. The evaluation of the variable fails in such
scenarios, so use a more flexible mechanism in this case and
try to locate poky in the local layers as well, if not already
set.
[YOCTO #12891]
(Bitbake rev: 971c728075af05e71edfd8e5212728c3dd0787b6)
Signed-off-by: Awais Belal <awais.belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case something goes tragically wrong, catch a request to checksum / and
refuse.
(Bitbake rev: 7444419b7fda34e14d653ba8470f5dfabb5da4f3)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Opening a file in binary mode and iterating it seems like the simple solution
but will still break on newlines, which for binary files isn't really useful as
the size of the chunks could be huge or tiny.
Instead, let's be a bit more clever: we'll be MD5ing lots of files, but we don't
want to fill up memory: use mmap() to open the file and read the file in 8k
blocks.
(Bitbake rev: 41e6161c8ce8cc90ebc93d72852673ae60fac923)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running bitbake command with Python 3.6.5 always result in
import error causing by the change of distutils module.
This patch replaces the method to search executable in PATH by
"/usr/bin/env <command>".
(Bitbake rev: eb3bdcc9e7cb273de4114367887d1d7df9f036b7)
Signed-off-by: Tzu Hsiang Lin <t9360341@ntut.org.tw>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prevent movefile from falsely setting the source file's owner and
permissions on the destination directory instead of the destination
file when moving between devices.
This bug caused the last file moved into a directory to dictate the
directory's owner and permissions.
(Bitbake rev: 82ea737a0b42a8b53e11c9cde141e9e9c0bd8c40)
Signed-off-by: Mattias Hansson <mattias.hansson@axis.com>
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [#YOCTO 12030]
Updated the "Task Checksums and Setscene" section to provide a bit
of user information around the bitbake-dumpsigs use that lets a
user examine signatures and inputs that determine if a do_compile
task is indeed supposed to be run.
Added more explanation of how a user can examine signatures used
to determine if a do_compile task is indeed supposed to be run.
(Bitbake rev: 103bbc6642261cd5da038ba2071621919ee01253)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Made sure that the terms "OpenEmbedded-Core" and "OE-Core"
are used as such throughout the manual.
(Bitbake rev: 677e58f8616a4bf58772e54d2313af3885a3b110)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We'd like layers to set this variable so that we know which layers are compatible
with which others, even if the branch is a generic un-updated "master" branch.
Start printing a warning to highlight this issue.
(Bitbake rev: cca81e33b58c390dcf5cc3a31555a43b79177166)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was removed in the bitbake repository back in 2010 (bitbake
revision 24857e2ceb405916b0b0b3e75c6c2375a909b9ba) - it's not clear
to me what happened, but for some reason the file remained in the poky
repository. However it does not appear that it has been used since then
and the reasons for removing even more valid now than they were then -
the code has moved on even further, we have memory resident bitbake
server, and if we were to re-implement this we would do it in a
different way. Drop the file and bring us back in sync with the bitbake
repo.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix typo in shutdown code to kill threads when "kill -0" is not enough.
Use the '--noreload' flag for 'runserver' so that there are no extra
and unaccounted threads.
[YOCTO #12555]
(Bitbake rev: 256990943075e89cb9aee2bc6488344b6783e07b)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Explicitly capture and ignore errors when trying to load the optional
'custom.xml' fixture file.
[YOCTO #12554]
(Bitbake rev: 132458939d3987ebc58685397714af3d6d5cd8fd)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before, explode_dep_versions2 would sort the OrderedDict before
returning. This function will still sort the OrderedDict by default, but
will now have the option to return the OrderedDict unsorted. This option will
allow us to check if the order of the package list has changed.
(Bitbake rev: 39d6a30a28f66c599e18beddbd847f40dcff623c)
Signed-off-by: Amanda Brindle <amanda.r.brindle@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The base configuration needs key expansion and anon python execution,
the parsed configurations do not. Fix this consistently, its been
broken and causing double key expansion for a while, only relised
when we started double anonymous python exeution too.
(Bitbake rev: 6138897de5ac6becf3bff56ce7a78f3ec208fcdf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The output of bitbake -e can differ from what actually is used due
to anonymous python making changes to the data store. Execute any
anonymous python added in the base configuration to make things
more consistent.
(Bitbake rev: bcdc2f73e3b4a10b1e479c2891f251d9507a9e30)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows us to call this code from other contexts without
duplicating it.
(Bitbake rev: c6be487f9bd5d95915f2495d555b9f539adb1d44)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a standalone tasks adds a dependency on X:do_build, the code in runqueue would
currently remove it if that do_build was part of an image recipe which uses
recrdeptask on do_build.
Such individual tasks shouldn't do this, therefore tweak the recursive reference code
to only process recurseive tasks, not all tasks.
(Bitbake rev: 4cfca360891e1ed876a9c19487b4f6210686af26)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added a new repo Fetcher section in the same spirit as the existing
sections for other supported fetchers. Changes included the new section,
removal of the bulleted item that mentioned this fetcher as an
"additional" fetcher, and the creation of a new variable in the glossary
named REPODIR.
(Bitbake rev: 1cf084cee506574b7bb6ed03344597b3b2f2fe08)
Signed-off-by: Nicolas Cornu <nicolac76@yahoo.fr>
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The '--all'' fetches all remotes, including the ones in $HOME/.gitconfig, which
causes the fetching very slow, so don't use "--all", git fetch should be
enough.
(Bitbake rev: 038a5f4b767df7944e73b2e859ac3af2d3a08bf1)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to allow users to manually populate the download directory with
valid content change the assumption that missing the donestamp file
means unfetched content.
This allows users to populate the download dir without needing to create
dummy .done files such that a user does not need a PREMIRROR when using
BB_NO_NETWORK to provide valid content files in the download directory.
To ensure the correct result this change also fails first if the
localpath does not exist. This prevents further parts of the function
attempting to calculating the checksum on non-existent files. This also
fixes some edge conditions around where if the donestamp exists but the
localpath does not it returns, and did not remove the donestamp.
Also added test cases to cover this use case and additional use cases
where for example the fetcher does not support checksums.
(Bitbake rev: a335dbbb65d5b56e71d98cf3e4fa9bfbec1dcde6)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At least the cli-color node module has dependencies that have
cyclic dependency among themselves. npm.py is prepared to deal
with such a case but the condition is handled only for downloading
or not a dependency again, but then it goes checking the its
dependency which causes an infinite loop in _getdependencies().
Make this function simply return when a dependency is already
downloaded and only download and check its dependencies when not.
(Bitbake rev: 545540420112992e53f4a83104af10452df168d0)
Signed-off-by: Zoltán Böszörményi <zboszor@pr.hu>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Have a simple tool to dump bb_cache.dat is useful for investigating and
studying bitbake cache. The old contrib/dump_cache.py can dump pn, pv and
packages for it, now enhance it dump everything.
Here is the usage:
$ /path/to/dump_cache.py --help
usage: dump_cache.py [-h] [-r RECIPE] [-m MEMBERS] [-s] cachefile
bb_cache.dat's dumper
positional arguments:
cachefile specify bb_cache.dat
optional arguments:
-h, --help show this help message and exit
-r RECIPE, --recipe RECIPE
specify the recipe, default: all recipes
-m MEMBERS, --members MEMBERS
specify the member, use comma as separator for
multiple ones, default: all members
-s, --skip skip skipped recipes
Use dump_cache.py --help to get help
(Bitbake rev: 104572438dfedf6025fbfd125aef1d56134012e7)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The resolve_file() calls mark_dependency(), so the one which calls
resolve_file() doesn't need call mark_dependency() again.
(Bitbake rev: 4682571107323a39b42cd9ec8ee67419e7f15acc)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Print message when cachefile is found/not can help debug.
* Update "Using cache in" to "Cache dir:" since it was the same as the debug
message of "codeparser & file checksum caches", which caused confusion. And
whether the cache file will be used or not is still unknown at that time, so
just print the cache dir.
(Bitbake rev: c8d3a2016f432e8ed9e99d9c28850149ab6fd6d8)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
