We already export http,ftp,https proxy environment variables. Some environments,
GNOME for instance, place the socks proxy in ALL_PROXY and all_proxy. Export it
as well.
Signed-off-by: Darren Hart <dvhart@linux.intel.com>
We catch any exception raised by the fetchers go() method and attempt to work
around it by trying any (post) mirrors which are configured. However, should
the mirrors fail the exception is lost and the fetch is assumed to have
completed successfully.
Instead, save the exception and if the local file does not exist after trying
the mirrors re-raise the exception.
Fixes [BUGID #362]
Signed-off-by: Joshua Lock <josh@linux.intel.com>
With "no_proxy" exported to bitbake, both internal and externel file mirror can
be used. "https_proxy" enable fetching "https://" file through proxy.
Signed-off-by: Zhai Edwin <edwin.zhai@intel.com>
Add a new method, try_premirror, to the Fetch object which checks to see
whether the file needs to be fetched from a premirror.
Override this in the Git fetcher to only require a pre-mirror fetch when the
clone directory does not exist.
Fixes [BUGID 290]
Signed-off-by: Joshua Lock <josh@linux.intel.com>
When setting localpath in the SRC_URI parameters the basename should be set
explicitly as the localpath() method is not called.
Fixes [BUGID #81]
Signed-off-by: Joshua Lock <josh@linux.intel.com>
If we fetch a tarball from a mirror it's entirely possible that the
mirror will have been from before the required tag/branch/etc was included
in the repository. To that end use forcefetch() as a way of testing whether
the repository is up to date and if not fetch updates.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
The git fetcher should force a fetch if the required tag is not present in
the local clone, or if the fullclone parameter is set.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
When pulling from a premirror we would prefer a local tarball even when the
caller had specified the forcefetch parameter.
Add an extra parameter 'force' to try_mirrors, defaulting to False. If set
the mirrors will be tested even if the file exists locally.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
* Fixes proxy support to honor standard proxy environment variables.
* Quote environment variables
(Bitbake rev: f84f382f340d6db15b9e5afb8c7c93969249a958)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Store database connection to persistent database in fetcher.
(Bitbake rev: 8a6876752b90efd81d92f0947bfc9527d8260969)
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
types.IntType -> int
types.StringType -> basestring
...
Also moves our ImmutableTypes tuple into our own namespace.
(Bitbake rev: 83674a3a5564ecb1f9d2c9b2d5b1eeb3c31272ab)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
We should add back in some proper unit testing. Doctest is rather limited,
and we haven't actually made use of it since the original implementation of
the datastore when the project started, as far as I'm aware.
(Bitbake rev: 3a11c2807972bbbddffde2fa67fc380d159da467)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Without this patch decoding a url of this kind file://dir/filename gives
path=/filename host=dir.
With the patch it decodes as path=/dir/filename host=""
Probably nobody stumbled on this issue yet because nobody used
file:// urls with directory names in the path.
Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com>
When not expanding PREMIRRORS, the functions fails/does not work correctly
when PREMIRRORS is not a plain string (e.g. contains ${...} or a ${@...}
statements).
(Bitbake rev: d612d22b073f68b8cf1bb7344e0487820040d80d)
Signed-off-by: Enrico Scholz <enrico.scholz@informatik.tu-chemnitz.de>
Signed-off-by: Chris Larson <clarson@kergoth.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
dict objects provide an __iter__ method for the iteration which gives you the
keys, so calling keys directly is unnecessary, and isn't really a best
practice. The only time you really need to call the keys is if there's a
danger of the dict changing out from underneith you, either due to external
forces or due to modification of the iterable in the loop. Iterations over
os.environ are apparently subject to such changes, so they must continue to
use keys().
As an aside, also switches a couple spots to using sorted() rather than
creating a temporary list with keys() and sorting that.
(Bitbake rev: 5b6ccb16c6e71e23dac6920cd2df994d67c2587b)
Signed-off-by: Chris Larson <clarson@mvista.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
The Git fetcher currently hardwires the git command to "git". Allow the
path and any additional wrappers to the Git command to be provided via
FETCHCMD functionality, as with some of the other fetchers.
If FETCHCMD_git is not define in bitbake.conf, the fetcher defaults to "git".
(Bitbake rev: f3afb79ecac30d973a3c62ff6baf28d8b7388a24)
Signed-off-by: Malcolm Crossley <malcolm.crossley@ge.com>
Signed-off-by: Martyn Welch <martyn.welch@ge.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
checkstatus() is used to ensure we can fetch a copy of each file, so it makes
sense to also test PREMIRRORS and MIRRORS in the method.
This patch adds calls to try_mirrors() to the Fetch.checkstatus() method and
changes the try_mirrors() method to take a check argument, which is False by
default. When check is True try_mirrors() will call a fetchers checkstatus()
with the replaced uri.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
The logic in try_mirrors() was buggy such that only the first entry in the
mirrors list was tried.
This patch fixes this. Thanks to Richard for the review.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
