Commit Graph

803 Commits

Author SHA1 Message Date
Richard Purdie
59a08262ab bitbake/fetch2/git: Write mirror tarballs if enabled and they don't exist, also set a default value for mirror tarball generation
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-10 15:24:31 +00:00
Richard Purdie
29ed2c6e9a bitbake/fetch2: Fix negated if check for BB_FETCH_PREMIRRORONLY
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-09 01:23:56 +00:00
Mark Hatle
906285ff00 fetch2: Add SRPM knowledge
Enable the fetcher to be able to unpack and SRPM.  By default the system will
unpack the contents of the SRPM into the WORKDIR.

A new syntax "unpack=file" was developed for the SRC_URI, to allow for a
recipe to extract a specific file within an SRPM.  An unpack operation will
then be executed on the extracted file.

In order to apply extracted patches (or unpack files not specified with
unpack), you must specify the path using WORKDIR, i.e.:

file://${WORKDIR}/mypatch.patch

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
2011-02-08 18:01:35 +00:00
Saul Wold
07088f7711 bitbake/utils.py: add glob name matching to remove
Signed-off-by: Saul Wold <sgw@linux.intel.com>
2011-02-08 17:35:49 +00:00
Richard Purdie
aa45760702 bitbake/fetch2: Ignore UnboundLocalError in exception handler in try_mirror
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-08 17:30:45 +00:00
Richard Purdie
ee1a9c0476 bitbake/fetch2: Move symlink handling into try_mirror where is belongs instead of the main download function
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-08 14:02:16 +00:00
Richard Purdie
c4e66a7fe0 bitbake/fetch2: When using BB_FETCH_PREMIRRORONLY, set BB_NO_NETWORK after premirrors as there could be data processing needed by the real fetcher
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-08 14:02:16 +00:00
Richard Purdie
91eeb9b0d3 bitbake/fetch2: Ensure we only remove files, not directories when fetch failures occur
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-08 14:02:16 +00:00
Richard Purdie
0713fb6b99 bitbake/fetch2: Ensure original ud is preserved in try_mirror
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-08 14:02:16 +00:00
Richard Purdie
a235c67479 bitbake/fetch2/git: Fix broken variable reference
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-08 12:05:35 +00:00
Richard Purdie
5920e85c56 bitbake/fetch2/git: Ensure unresolved branches are translated into revisions
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-08 12:05:35 +00:00
Saul Wold
1544aa8ab4 fetch2: add try/finally to ensure lockfile is unlocked on failure
Signed-off-by: Saul Wold <sgw@linux.intel.com>
2011-02-07 17:29:46 -08:00
Saul Wold
232b6f3c92 fetch2: add runfetchcmd to import for fetchers
Signed-off-by: Saul Wold <sgw@linux.intel.com>
2011-02-07 16:28:35 -08:00
Richard Purdie
500c66337c bitbake/fetch2:Fetch Make using the fn based cache optional
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 21:13:13 +00:00
Richard Purdie
d4b4b48add bitbake/fetch2: Fix pickling issues with fetcher exceptions
See the problems in http://bugs.python.org/issue1692335, need to set self.args
correctly.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 21:13:13 +00:00
Richard Purdie
94faffdaf6 bitbake/fetch2: Update mirror processing to ensure we look for mirror tarballs
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 19:48:21 +00:00
Richard Purdie
37624b9745 bitbake/fetch2: Update forcefetch and mirror handling to clean up, simplfy and bug fix the code
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 16:52:42 +00:00
Richard Purdie
7586adb360 bitbake/msg: Ensure lower level debug messages have DEBUG prefix and reuse log level values from formatter
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 14:50:27 +00:00
Richard Purdie
ded83ffc3b bitbake/fetch2: Match Fetcher log domain to that in bb.msg
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 14:50:27 +00:00
Richard Purdie
d7eebbe9db bitbake:__init__.py: Fix debug log level handling to correct debug output
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 14:50:27 +00:00
Richard Purdie
fb9c5a196e bitbake: event/build: Drop stdout specific handling for python messages as this is no longer needed with newer log handling
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 14:50:27 +00:00
Richard Purdie
681bcf4e6b bitbake/fetch2/git: use clonedir as ud.localfile too since the mirror tarball may not exist
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 10:13:29 +00:00
Richard Purdie
3308868267 bitbake/fetch2/git.py: Ensure that forcefetch operates in the correct directory for calling _contains_ref()
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:34:24 +00:00
Richard Purdie
917d3e9697 bitbake/fetch2/git: Fix localpath to point at the clone repo since we no longer always generate a mirror tarball and it isn't a good guide to fetcher success
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:13:44 +00:00
Richard Purdie
7202a77134 bitbake/fetch2: Use True instead of integer values
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:37 +00:00
Richard Purdie
8f2abf4a9f bitbake/fetch2: Drop name/override ordering backwards compatibility code
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:37 +00:00
Richard Purdie
ee6f34483a bitbake/fetch2/svn: The concept of using dates with svn is totally insane, drop support
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:37 +00:00
Richard Purdie
ca7adf7529 bitbake/fetch2: Define a new interface to the fetcher code though the 'Fetch' class
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:37 +00:00
Richard Purdie
08a9fef4fa bitbake/fetch2: Fix typo
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:37 +00:00
Richard Purdie
806d1c25f8 bitbake/fetch2: Drop legacy CVSDATE support
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:36 +00:00
Richard Purdie
f60d9d6bd4 bitbake/fetch2: Move getSRCDate to FetchData class where is more appropriate
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:36 +00:00
Richard Purdie
831c6c1d3c bitbake/fetch2: Make srcrev_internal_helper a normal function, doesn't belong in the FetchMethod class
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:36 +00:00
Richard Purdie
ab0dd13974 bitbake/fetch2: Rename Fetch class to FetchMethod
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:36 +00:00
Richard Purdie
74b71864fe bitbake/fetch2: Drop old md5 handling code
Drop some old md5 functions since we have improved functionality now which includes
sha256 checksum support. This stops each download being md5 checksumed twice.

Also change ".md5" stamp extentions to ".done" to better describe its use as a
download complete marker file and no longer write the md5 sum to the files.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:36 +00:00
Richard Purdie
d08397ba4d bitbake/fetch2: Rewrite and improve exception handling, reusing core functions for common operations where possible
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:36 +00:00
Richard Purdie
f6eefb3ca3 bitbake/fetch2: Have all fetcher exceptions derived from a common BBFetchException class
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:35 +00:00
Richard Purdie
8daab5b951 bitbake/fetch2: Drop unused parameter for localpath() and update comments
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:35 +00:00
Richard Purdie
7e4fbfc1c1 bitbake/fetch2: Simplfy setup_srcrevs
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:35 +00:00
Richard Purdie
49a022d25d bitbake/fetch2: Simplify localpath variable handling FetchData init
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:35 +00:00
Richard Purdie
5a52cb80d6 bitbake/fetch2: Define a sane localpath function and remove code duplication
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:35 +00:00
Richard Purdie
972eb5faba bitbake/fetch2: Move ud.localfile setup into urldata_init
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:35 +00:00
Richard Purdie
0e5404ceda bitbake/fetch2: Drop horrible SRCREV hack which is no longer needed
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:34 +00:00
Richard Purdie
e4c1e99f23 bitbake/fetch2/git: Switch to completely use bare clones
We no longer need index/workdir support in the mirror tree and it causes all
kind of reference naming problems.Simplifying the code to remove this and use
just bare clones addresses this problem.

We increase the "version" number on the mirror tarballs to reflect the change
and ensure older mirror tarballs are not used as they would break.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-06 11:44:12 +00:00
Saul Wold
192f631129 fetch2/git.py: Fix problem when fetching with no origin
When updating a git repo, its possible to not have an origin,
so fail gracefully with a debug message.

Signed-off-by: Saul Wold <sgw@linux.intel.com>
2011-02-05 14:14:21 -08:00
Richard Purdie
96810cade0 bitbake/fetch2/git: Ensure we prune origin to remove stale branches
Thanks to Bruce Ashfield for the tip.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-05 15:43:36 +00:00
Richard Purdie
dfc5faf717 bitbake/fetch2/git: Ensure deleted branches upstream are removed, all branch tags are forced to the upstream revisions and all revisions and tags are always fetched
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-03 20:18:52 +00:00
Richard Purdie
282a828f3d bitbake/fetch2: Remove old local file acceleration code
Since local mirror fetches are always symlinked from the download directory
directly, there is no need for this premirrors hack which doesn't cover
mirrors and also abuses the localpath variable with inconsistent results.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-03 18:24:25 +00:00
Richard Purdie
fa9fda05e1 bitbake/fetch2: Ensure that mirror fetches are symlinked from the download directory
When files are fetched from a mirror source that happens to be local,
ensure links are created for the file since subsequent fetch calls
can then follow the links to find files.

Any other approach such as the existing manipulations of localpath
internally to the fetcher are prone to errors, races and other issues.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-03 18:22:06 +00:00
Richard Purdie
7f99605562 bitbake/fetch2: When encoding a file:// url, drop user and host information
When processing a cvs SRC_URI to a file:// mirror, the user and host information
will break the mirror processing. This patch addresses it by only constructing
valid urls.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-03 18:19:23 +00:00
Richard Purdie
a55d86b405 bitbake/lib/bb/fetch2: Assign a default name in the names array as empty strings as names cause problems for python
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-03 00:16:23 +00:00