~jelmer/bzr-builddeb/multiple-upstream-tarballs

121 by James Westby
Add a function to repack a tarball or dir to a .tar.gz.
1
#    repack_tarball.py -- Repack files/dirs in to tarballs.
2
#    Copyright (C) 2007 James Westby <jw+debian@jameswestby.net>
3
#
4
#    This file is part of bzr-builddeb.
5
#
6
#    bzr-builddeb is free software; you can redistribute it and/or modify
7
#    it under the terms of the GNU General Public License as published by
8
#    the Free Software Foundation; either version 2 of the License, or
9
#    (at your option) any later version.
10
#
11
#    bzr-builddeb is distributed in the hope that it will be useful,
12
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
13
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
14
#    GNU General Public License for more details.
15
#
16
#    You should have received a copy of the GNU General Public License
17
#    along with bzr-builddeb; if not, write to the Free Software
18
#    Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
19
#
20
21
import gzip
22
import os
268 by James Westby
Add support for repacking zip files. Thanks Daniel Hahler.
23
from StringIO import StringIO
121 by James Westby
Add a function to repack a tarball or dir to a .tar.gz.
24
import tarfile
166 by Frederic Brin
Corrects repack from tar.bz2 to tar.gz
25
import bz2
192 by James Westby
* Don't complain when repacking the tarball if the target exists, but is the
26
import sha
268 by James Westby
Add support for repacking zip files. Thanks Daniel Hahler.
27
import zipfile
121 by James Westby
Add a function to repack a tarball or dir to a .tar.gz.
28
178.1.20 by James Westby
Remove some unused imports. Thanks pyflakes.
29
from bzrlib.errors import (
121 by James Westby
Add a function to repack a tarball or dir to a .tar.gz.
30
                           FileExists,
122 by James Westby
Raise an error on trying to repack an unkown format.
31
                           BzrCommandError,
123 by James Westby
Add a target_dir parameter to the repack_tarball function.
32
                           NotADirectory,
121 by James Westby
Add a function to repack a tarball or dir to a .tar.gz.
33
                           )
142 by James Westby
Support repacking over any transport.
34
from bzrlib.transport import get_transport
35
from bzrlib import urlutils
121 by James Westby
Add a function to repack a tarball or dir to a .tar.gz.
36
266.1.9 by James Westby
Handle remote files in repack check that checksums match when target exists.
37
def _get_file_from_location(location):
38
    base_dir, path = urlutils.split(location)
39
    transport = get_transport(base_dir)
40
    return transport.get(path)
121 by James Westby
Add a function to repack a tarball or dir to a .tar.gz.
41
123 by James Westby
Add a target_dir parameter to the repack_tarball function.
42
def repack_tarball(orig_name, new_name, target_dir=None):
121 by James Westby
Add a function to repack a tarball or dir to a .tar.gz.
43
  """Repack the file/dir named to a .tar.gz with the chosen name.
44
45
  This function takes a named file of either .tar.gz, .tar .tgz .tar.bz2 
46
  or .zip type, or a directory, and creates the file named in the second
47
  argument in .tar.gz format.
48
123 by James Westby
Add a target_dir parameter to the repack_tarball function.
49
  If target_dir is specified then that directory will be created if it
50
  doesn't exist, and the new_name will be interpreted relative to that
51
  directory.
52
  
121 by James Westby
Add a function to repack a tarball or dir to a .tar.gz.
53
  The source must exist, and the target cannot exist.
123 by James Westby
Add a target_dir parameter to the repack_tarball function.
54
121 by James Westby
Add a function to repack a tarball or dir to a .tar.gz.
55
  :param orig_name: the curent name of the file/dir
56
  :type orig_name: string
57
  :param new_name: the desired name of the tarball
58
  :type new_name: string
123 by James Westby
Add a target_dir parameter to the repack_tarball function.
59
  :keyword target_dir: the directory to consider new_name relative to, and
166 by Frederic Brin
Corrects repack from tar.bz2 to tar.gz
60
                       will be created if non-existant.
123 by James Westby
Add a target_dir parameter to the repack_tarball function.
61
  :type target_dir: string
121 by James Westby
Add a function to repack a tarball or dir to a .tar.gz.
62
  :return: None
124 by James Westby
Add some 'throws' documentation to the repack_tarball docstring.
63
  :throws NoSuchFile: if orig_name doesn't exist.
64
  :throws NotADirectory: if target_dir exists and is not a directory.
192 by James Westby
* Don't complain when repacking the tarball if the target exists, but is the
65
  :throws FileExists: if the target filename (after considering target_dir)
66
                      exists, and is not identical to the source.
226.8.4 by Daniel Hahler
Use system commands (unzip + tar) for zip repacking, since using zipfile/tarfile is more complex and does not keep file permissions etc.
67
  :throws BzrCommandError: if the source isn't supported for repacking.
121 by James Westby
Add a function to repack a tarball or dir to a .tar.gz.
68
  """
123 by James Westby
Add a target_dir parameter to the repack_tarball function.
69
  if target_dir is not None:
70
    if not os.path.exists(target_dir):
71
      os.mkdir(target_dir)
72
    else:
73
      if not os.path.isdir(target_dir):
74
        raise NotADirectory(target_dir)
75
    new_name = os.path.join(target_dir, new_name)
266.1.9 by James Westby
Handle remote files in repack check that checksums match when target exists.
76
  old_contents = None
77
  if isinstance(orig_name, unicode):
78
    orig_name = orig_name.encode('utf-8')
79
  if isinstance(new_name, unicode):
80
    new_name = new_name.encode('utf-8')
121 by James Westby
Add a function to repack a tarball or dir to a .tar.gz.
81
  if os.path.exists(new_name):
192 by James Westby
* Don't complain when repacking the tarball if the target exists, but is the
82
    if not orig_name.endswith('.tar.gz'):
83
      raise FileExists(new_name)
266.1.9 by James Westby
Handle remote files in repack check that checksums match when target exists.
84
    trans_file = _get_file_from_location(orig_name)
192 by James Westby
* Don't complain when repacking the tarball if the target exists, but is the
85
    try:
266.1.9 by James Westby
Handle remote files in repack check that checksums match when target exists.
86
      old_contents = trans_file.read()
192 by James Westby
* Don't complain when repacking the tarball if the target exists, but is the
87
    finally:
266.1.9 by James Westby
Handle remote files in repack check that checksums match when target exists.
88
      trans_file.close()
89
    orig_sha = sha.sha(old_contents).hexdigest()
192 by James Westby
* Don't complain when repacking the tarball if the target exists, but is the
90
    f = open(new_name)
91
    try:
92
      new_sha = sha.sha(f.read()).hexdigest()
93
    finally:
94
      f.close()
95
    if orig_sha != new_sha:
96
      raise FileExists(new_name)
97
    return
121 by James Westby
Add a function to repack a tarball or dir to a .tar.gz.
98
  if os.path.isdir(orig_name):
99
    tar = tarfile.open(new_name, 'w:gz')
100
    try:
101
      tar.add(orig_name, os.path.basename(orig_name))
102
    finally:
103
      tar.close()
104
  else:
266.1.9 by James Westby
Handle remote files in repack check that checksums match when target exists.
105
    if old_contents is None:
106
      trans_file = _get_file_from_location(orig_name)
107
      try:
108
        old_contents = trans_file.read()
109
      finally:
110
        trans_file.close()
142 by James Westby
Support repacking over any transport.
111
    base_dir, path = urlutils.split(orig_name)
112
    transport = get_transport(base_dir)
113
    trans_file = transport.get(path)
266.1.9 by James Westby
Handle remote files in repack check that checksums match when target exists.
114
    if orig_name.endswith('.tar.gz') or orig_name.endswith('.tgz'):
115
      dest = open(new_name, 'wb')
116
      try:
117
        dest.write(old_contents)
118
      finally:
119
        dest.close()
120
    elif orig_name.endswith('.tar'):
121
      gz = gzip.GzipFile(new_name, 'w')
122
      try:
123
        gz.write(old_contents)
124
      finally:
125
        gz.close()
126
    elif orig_name.endswith('.tar.bz2'):
127
      old_tar_content_decompressed = bz2.decompress(old_contents)
128
      gz = gzip.GzipFile(new_name, 'w')
129
      try:
130
        gz.write(old_tar_content_decompressed)
131
      finally:
132
        gz.close()
276 by James Westby
Merge some 2.0 work.
133
    elif orig_name.endswith('.zip') or zipfile.is_zipfile(orig_name):
134
      import time
135
      old_contents_f = StringIO(old_contents)
136
      zip = zipfile.ZipFile(old_contents_f, "r")
137
      try:
138
        tar = tarfile.open(new_name, 'w:gz')
139
        try:
140
          for info in zip.infolist():
141
            tarinfo = tarfile.TarInfo(info.filename)
142
            tarinfo.size = info.file_size
143
            tarinfo.mtime = time.mktime(info.date_time + (0, 1, -1))
144
            if info.filename.endswith("/"):
145
                tarinfo.mode = 0755
146
                tarinfo.type = tarfile.DIRTYPE
147
            else:
148
                tarinfo.mode = 0644
149
                tarinfo.type = tarfile.REGTYPE
150
            contents = StringIO(zip.read(info.filename))
151
            tar.addfile(tarinfo, contents)
152
        finally:
153
          tar.close()
154
      finally:
155
        zip.close()
266.1.9 by James Westby
Handle remote files in repack check that checksums match when target exists.
156
    else:
157
      raise BzrCommandError('Unsupported format for repack: %s' % orig_name)
121 by James Westby
Add a function to repack a tarball or dir to a .tar.gz.
158
178.1.29 by James Westby
Add support for incremental import dsc.
159
# vim: ts=2 sts=2 sw=2
192 by James Westby
* Don't complain when repacking the tarball if the target exists, but is the
160