~dkuhlman/python-training-materials/Materials

<h1>10.10. <a class="reference internal" href="#module-shutil" title="shutil: High-level file operations, including copying."><code class="xref py py-mod docutils literal">shutil</code></a> — High-level file operations<a class="headerlink" href="#module-shutil" title="Permalink to this headline">¶</a></h1>

Source code: <a class="reference external" href="https://hg.python.org/cpython/file/2.7/Lib/shutil.py">Lib/shutil.py</a>

The <a class="reference internal" href="#module-shutil" title="shutil: High-level file operations, including copying."><code class="xref py py-mod docutils literal">shutil</code></a> module offers a number of high-level operations on files and

collections of files. In particular, functions are provided which support file

copying and removal. For operations on individual files, see also the

<a class="reference internal" href="os.html#module-os" title="os: Miscellaneous operating system interfaces."><code class="xref py py-mod docutils literal">os</code></a> module.

Warning

Even the higher-level file copying functions (<a class="reference internal" href="#shutil.copy" title="shutil.copy"><code class="xref py py-func docutils literal">shutil.copy()</code></a>,

<a class="reference internal" href="#shutil.copy2" title="shutil.copy2"><code class="xref py py-func docutils literal">shutil.copy2()</code></a>) can’t copy all file metadata.

On POSIX platforms, this means that file owner and group are lost as well

as ACLs. On Mac OS, the resource fork and other metadata are not used.

This means that resources will be lost and file type and creator codes will

not be correct. On Windows, file owners, ACLs and alternate data streams

are not copied.

</div>

<h2>10.10.1. Directory and files operations<a class="headerlink" href="#directory-and-files-operations" title="Permalink to this headline">¶</a></h2>

<code class="descclassname">shutil.</code><code class="descname">copyfileobj</code>(fsrc, fdst[, length])<a class="headerlink" href="#shutil.copyfileobj" title="Permalink to this definition">¶</a></dt>

100

<dd>Copy the contents of the file-like object fsrc to the file-like object fdst.

101

The integer length, if given, is the buffer size. In particular, a negative

102

length value means to copy the data without looping over the source data in

103

chunks; by default the data is read in chunks to avoid uncontrolled memory

104

consumption. Note that if the current file position of the fsrc object is not

105

0, only the contents from the current file position to the end of the file will

106

be copied.

107

</dd></dl>

108

109

110

111

<code class="descclassname">shutil.</code><code class="descname">copyfile</code>(src, dst)<a class="headerlink" href="#shutil.copyfile" title="Permalink to this definition">¶</a></dt>

112

<dd>Copy the contents (no metadata) of the file named src to a file named

113

dst. dst must be the complete target file name; look at

114

<a class="reference internal" href="#shutil.copy" title="shutil.copy"><code class="xref py py-func docutils literal">shutil.copy()</code></a> for a copy that accepts a target directory path. If

115

src and dst are the same files, <a class="reference internal" href="#shutil.Error" title="shutil.Error"><code class="xref py py-exc docutils literal">Error</code></a> is raised.

116

The destination location must be writable; otherwise, an <a class="reference internal" href="exceptions.html#exceptions.IOError" title="exceptions.IOError"><code class="xref py py-exc docutils literal">IOError</code></a> exception

117

will be raised. If dst already exists, it will be replaced. Special files

118

such as character or block devices and pipes cannot be copied with this

119

function. src and dst are path names given as strings.

120

</dd></dl>

121

122

123

124

<code class="descclassname">shutil.</code><code class="descname">copymode</code>(src, dst)<a class="headerlink" href="#shutil.copymode" title="Permalink to this definition">¶</a></dt>

125

<dd>Copy the permission bits from src to dst. The file contents, owner, and

126

group are unaffected. src and dst are path names given as strings.

127

</dd></dl>

128

129

130

131

<code class="descclassname">shutil.</code><code class="descname">copystat</code>(src, dst)<a class="headerlink" href="#shutil.copystat" title="Permalink to this definition">¶</a></dt>

132

<dd>Copy the permission bits, last access time, last modification time, and flags

133

from src to dst. The file contents, owner, and group are unaffected. src

134

and dst are path names given as strings.

135

</dd></dl>

136

137

138

139

<code class="descclassname">shutil.</code><code class="descname">copy</code>(src, dst)<a class="headerlink" href="#shutil.copy" title="Permalink to this definition">¶</a></dt>

140

<dd>Copy the file src to the file or directory dst. If dst is a directory, a

141

file with the same basename as src is created (or overwritten) in the

142

directory specified. Permission bits are copied. src and dst are path

143

names given as strings.

144

</dd></dl>

145

146

147

148

<code class="descclassname">shutil.</code><code class="descname">copy2</code>(src, dst)<a class="headerlink" href="#shutil.copy2" title="Permalink to this definition">¶</a></dt>

149

<dd>Similar to <a class="reference internal" href="#shutil.copy" title="shutil.copy"><code class="xref py py-func docutils literal">shutil.copy()</code></a>, but metadata is copied as well – in fact,

150

this is just <a class="reference internal" href="#shutil.copy" title="shutil.copy"><code class="xref py py-func docutils literal">shutil.copy()</code></a> followed by <a class="reference internal" href="#shutil.copystat" title="shutil.copystat"><code class="xref py py-func docutils literal">copystat()</code></a>. This is

151

similar to the Unix command cp -p.

152

</dd></dl>

153

154

155

156

<code class="descclassname">shutil.</code><code class="descname">ignore_patterns</code>(*patterns)<a class="headerlink" href="#shutil.ignore_patterns" title="Permalink to this definition">¶</a></dt>

157

<dd>This factory function creates a function that can be used as a callable for

158

<a class="reference internal" href="#shutil.copytree" title="shutil.copytree"><code class="xref py py-func docutils literal">copytree()</code></a>‘s ignore argument, ignoring files and directories that

159

match one of the glob-style patterns provided. See the example below.

160

161

New in version 2.6.

162

</div>

163

</dd></dl>

164

165

166

167

<code class="descclassname">shutil.</code><code class="descname">copytree</code>(src, dst, symlinks=False, ignore=None)<a class="headerlink" href="#shutil.copytree" title="Permalink to this definition">¶</a></dt>

168

<dd>Recursively copy an entire directory tree rooted at src. The destination

169

directory, named by dst, must not already exist; it will be created as

170

well as missing parent directories. Permissions and times of directories

171

are copied with <a class="reference internal" href="#shutil.copystat" title="shutil.copystat"><code class="xref py py-func docutils literal">copystat()</code></a>, individual files are copied using

172

<a class="reference internal" href="#shutil.copy2" title="shutil.copy2"><code class="xref py py-func docutils literal">shutil.copy2()</code></a>.

173

If symlinks is true, symbolic links in the source tree are represented as

174

symbolic links in the new tree, but the metadata of the original links is NOT

175

copied; if false or omitted, the contents and metadata of the linked files

176

are copied to the new tree.

177

If ignore is given, it must be a callable that will receive as its

178

arguments the directory being visited by <a class="reference internal" href="#shutil.copytree" title="shutil.copytree"><code class="xref py py-func docutils literal">copytree()</code></a>, and a list of its

179

contents, as returned by <a class="reference internal" href="os.html#os.listdir" title="os.listdir"><code class="xref py py-func docutils literal">os.listdir()</code></a>. Since <a class="reference internal" href="#shutil.copytree" title="shutil.copytree"><code class="xref py py-func docutils literal">copytree()</code></a> is

180

called recursively, the ignore callable will be called once for each

181

directory that is copied. The callable must return a sequence of directory

182

and file names relative to the current directory (i.e. a subset of the items

183

in its second argument); these names will then be ignored in the copy

184

process. <a class="reference internal" href="#shutil.ignore_patterns" title="shutil.ignore_patterns"><code class="xref py py-func docutils literal">ignore_patterns()</code></a> can be used to create such a callable that

185

ignores names based on glob-style patterns.

186

If exception(s) occur, an <a class="reference internal" href="#shutil.Error" title="shutil.Error"><code class="xref py py-exc docutils literal">Error</code></a> is raised with a list of reasons.

187

The source code for this should be considered an example rather than the

188

ultimate tool.

189

190

Changed in version 2.3: <a class="reference internal" href="#shutil.Error" title="shutil.Error"><code class="xref py py-exc docutils literal">Error</code></a> is raised if any exceptions occur during copying, rather than

191

printing a message.

192

</div>

193

194

Changed in version 2.5: Create intermediate directories needed to create dst, rather than raising an

195

error. Copy permissions and times of directories using <a class="reference internal" href="#shutil.copystat" title="shutil.copystat"><code class="xref py py-func docutils literal">copystat()</code></a>.

196

</div>

197

198

Changed in version 2.6: Added the ignore argument to be able to influence what is being copied.

199

</div>

200

</dd></dl>

201

202

203

204

<code class="descclassname">shutil.</code><code class="descname">rmtree</code>(path[, ignore_errors[, onerror]])<a class="headerlink" href="#shutil.rmtree" title="Permalink to this definition">¶</a></dt>

205

<dd>Delete an entire directory tree; path must point to a directory (but not a

206

symbolic link to a directory). If ignore_errors is true, errors resulting

207

from failed removals will be ignored; if false or omitted, such errors are

208

handled by calling a handler specified by onerror or, if that is omitted,

209

they raise an exception.

210

If onerror is provided, it must be a callable that accepts three

211

parameters: function, path, and excinfo. The first parameter,

212

function, is the function which raised the exception; it will be

213

<a class="reference internal" href="os.path.html#os.path.islink" title="os.path.islink"><code class="xref py py-func docutils literal">os.path.islink()</code></a>, <a class="reference internal" href="os.html#os.listdir" title="os.listdir"><code class="xref py py-func docutils literal">os.listdir()</code></a>, <a class="reference internal" href="os.html#os.remove" title="os.remove"><code class="xref py py-func docutils literal">os.remove()</code></a> or

214

<a class="reference internal" href="os.html#os.rmdir" title="os.rmdir"><code class="xref py py-func docutils literal">os.rmdir()</code></a>. The second parameter, path, will be the path name passed

215

to function. The third parameter, excinfo, will be the exception

216

information return by <a class="reference internal" href="sys.html#sys.exc_info" title="sys.exc_info"><code class="xref py py-func docutils literal">sys.exc_info()</code></a>. Exceptions raised by onerror

217

will not be caught.

218

219

Changed in version 2.6: Explicitly check for path being a symbolic link and raise <a class="reference internal" href="exceptions.html#exceptions.OSError" title="exceptions.OSError"><code class="xref py py-exc docutils literal">OSError</code></a>

220

in that case.

221

</div>

222

</dd></dl>

223

224

225

226

<code class="descclassname">shutil.</code><code class="descname">move</code>(src, dst)<a class="headerlink" href="#shutil.move" title="Permalink to this definition">¶</a></dt>

227

<dd>Recursively move a file or directory (src) to another location (dst).

228

If the destination is an existing directory, then src is moved inside that

229

directory. If the destination already exists but is not a directory, it may

230

be overwritten depending on <a class="reference internal" href="os.html#os.rename" title="os.rename"><code class="xref py py-func docutils literal">os.rename()</code></a> semantics.

231

If the destination is on the current filesystem, then <a class="reference internal" href="os.html#os.rename" title="os.rename"><code class="xref py py-func docutils literal">os.rename()</code></a> is

232

used. Otherwise, src is copied (using <a class="reference internal" href="#shutil.copy2" title="shutil.copy2"><code class="xref py py-func docutils literal">shutil.copy2()</code></a>) to dst and

233

then removed.

234

235

New in version 2.3.

236

</div>

237

</dd></dl>

238

239

240

241

exception <code class="descclassname">shutil.</code><code class="descname">Error</code><a class="headerlink" href="#shutil.Error" title="Permalink to this definition">¶</a></dt>

242

<dd>This exception collects exceptions that are raised during a multi-file

243

operation. For <a class="reference internal" href="#shutil.copytree" title="shutil.copytree"><code class="xref py py-func docutils literal">copytree()</code></a>, the exception argument is a list of 3-tuples

244

(srcname, dstname, exception).

245

246

New in version 2.3.

247

</div>

248

</dd></dl>

249

250

251

<h3>10.10.1.1. copytree example<a class="headerlink" href="#copytree-example" title="Permalink to this headline">¶</a></h3>

252

This example is the implementation of the <a class="reference internal" href="#shutil.copytree" title="shutil.copytree"><code class="xref py py-func docutils literal">copytree()</code></a> function, described

253

above, with the docstring omitted. It demonstrates many of the other functions

254

provided by this module.

255

<div class="highlight-python"><div class="highlight"><pre>def copytree(src, dst, symlinks=False, ignore=None):

256

names = os.listdir(src)

257

if ignore is not None:

258

ignored_names = ignore(src, names)

259

else:

260

ignored_names = set()

261

262

os.makedirs(dst)

263

errors = []

264

for name in names:

265

if name in ignored_names:

266

continue

267

srcname = os.path.join(src, name)

268

dstname = os.path.join(dst, name)

269

try:

270

if symlinks and os.path.islink(srcname):

271

linkto = os.readlink(srcname)

272

os.symlink(linkto, dstname)

273

elif os.path.isdir(srcname):

274

copytree(srcname, dstname, symlinks, ignore)

275

else:

276

copy2(srcname, dstname)

277

# XXX What about devices, sockets etc.?

278

except (IOError, os.error) as why:

279

errors.append((srcname, dstname, str(why)))

280

# catch the Error from the recursive copytree so that we can

281

# continue with other files

282

except Error as err:

283

errors.extend(err.args[0])

284

try:

285

copystat(src, dst)

286

except WindowsError:

287

# can't copy file access times on Windows

288

pass

289

except OSError as why:

290

errors.extend((src, dst, str(why)))

291

if errors:

292

raise Error(errors)

293

</pre></div>

294

</div>

295

Another example that uses the <a class="reference internal" href="#shutil.ignore_patterns" title="shutil.ignore_patterns"><code class="xref py py-func docutils literal">ignore_patterns()</code></a> helper:

296

<div class="highlight-python"><div class="highlight"><pre>from shutil import copytree, ignore_patterns

297

298

copytree(source, destination, ignore=ignore_patterns('*.pyc', 'tmp*'))

299

</pre></div>

300

</div>

301

This will copy everything except <code class="docutils literal">.pyc</code> files and files or directories whose

302

name starts with <code class="docutils literal">tmp</code>.

303

Another example that uses the ignore argument to add a logging call:

304

<div class="highlight-python"><div class="highlight"><pre>from shutil import copytree

305

import logging

306

307

def _logpath(path, names):

308

logging.info('Working in %s' % path)

309

return [] # nothing will be ignored

310

311

312

</pre></div>

313

</div>

314

</div>

315

</div>

316

317

<h2>10.10.2. Archiving operations<a class="headerlink" href="#archiving-operations" title="Permalink to this headline">¶</a></h2>

318

High-level utilities to create and read compressed and archived files are also

319

provided. They rely on the <a class="reference internal" href="zipfile.html#module-zipfile" title="zipfile: Read and write ZIP-format archive files."><code class="xref py py-mod docutils literal">zipfile</code></a> and <a class="reference internal" href="tarfile.html#module-tarfile" title="tarfile: Read and write tar-format archive files."><code class="xref py py-mod docutils literal">tarfile</code></a> modules.

320

321

322

<code class="descclassname">shutil.</code><code class="descname">make_archive</code>(base_name, format[, root_dir[, base_dir[, verbose[, dry_run[, owner[, group[, logger]]]]]]])<a class="headerlink" href="#shutil.make_archive" title="Permalink to this definition">¶</a></dt>

323

<dd>Create an archive file (eg. zip or tar) and returns its name.

324

base_name is the name of the file to create, including the path, minus

325

any format-specific extension. format is the archive format: one of

326

“zip”, “tar”, “bztar” or “gztar”.

327

root_dir is a directory that will be the root directory of the

328

archive; ie. we typically chdir into root_dir before creating the

329

archive.

330

base_dir is the directory where we start archiving from;

331

ie. base_dir will be the common prefix of all files and

332

directories in the archive.

333

root_dir and base_dir both default to the current directory.

334

owner and group are used when creating a tar archive. By default,

335

uses the current owner and group.

336

logger must be an object compatible with <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0282">PEP 282</a>, usually an instance of

337

<a class="reference internal" href="logging.html#logging.Logger" title="logging.Logger"><code class="xref py py-class docutils literal">logging.Logger</code></a>.

338

339

New in version 2.7.

340

</div>

341

</dd></dl>

342

343

344

345

<code class="descclassname">shutil.</code><code class="descname">get_archive_formats</code>()<a class="headerlink" href="#shutil.get_archive_formats" title="Permalink to this definition">¶</a></dt>

346

<dd>Return a list of supported formats for archiving.

347

Each element of the returned sequence is a tuple <code class="docutils literal">(name, description)</code>.

348

By default <a class="reference internal" href="#module-shutil" title="shutil: High-level file operations, including copying."><code class="xref py py-mod docutils literal">shutil</code></a> provides these formats:

349

350

<li>gztar: gzip’ed tar-file</li>

351

<li>bztar: bzip2’ed tar-file</li>

352

<li>tar: uncompressed tar file</li>

353

354

</ul>

355

You can register new formats or provide your own archiver for any existing

356

formats, by using <a class="reference internal" href="#shutil.register_archive_format" title="shutil.register_archive_format"><code class="xref py py-func docutils literal">register_archive_format()</code></a>.

357

358

New in version 2.7.

359

</div>

360

</dd></dl>

361

362

363

364

<code class="descclassname">shutil.</code><code class="descname">register_archive_format</code>(name, function[, extra_args[, description]])<a class="headerlink" href="#shutil.register_archive_format" title="Permalink to this definition">¶</a></dt>

365

<dd>Register an archiver for the format name. function is a callable that

366

will be used to invoke the archiver.

367

If given, extra_args is a sequence of <code class="docutils literal">(name, value)</code> that will be

368

used as extra keywords arguments when the archiver callable is used.

369

description is used by <a class="reference internal" href="#shutil.get_archive_formats" title="shutil.get_archive_formats"><code class="xref py py-func docutils literal">get_archive_formats()</code></a> which returns the

370

list of archivers. Defaults to an empty list.

371

372

New in version 2.7.

373

</div>

374

</dd></dl>

375

376

377

378

<code class="descclassname">shutil.</code><code class="descname">unregister_archive_format</code>(name)<a class="headerlink" href="#shutil.unregister_archive_format" title="Permalink to this definition">¶</a></dt>

379

<dd>Remove the archive format name from the list of supported formats.

380

381

New in version 2.7.

382

</div>

383

</dd></dl>

384

385

386

<h3>10.10.2.1. Archiving example<a class="headerlink" href="#archiving-example" title="Permalink to this headline">¶</a></h3>

387

In this example, we create a gzip’ed tar-file archive containing all files

388

found in the <code class="file docutils literal">.ssh</code> directory of the user:

389

<div class="highlight-python"><div class="highlight"><pre>>>> from shutil import make_archive

390

>>> import os

391

>>> archive_name = os.path.expanduser(os.path.join('~', 'myarchive'))

392

>>> root_dir = os.path.expanduser(os.path.join('~', '.ssh'))

393

>>> make_archive(archive_name, 'gztar', root_dir)

394

'/Users/tarek/myarchive.tar.gz'

395

</pre></div>

396

</div>

397

The resulting archive contains:

398

<div class="highlight-shell-session"><div class="highlight"><pre>$ tar -tzvf /Users/tarek/myarchive.tar.gz

399

drwx------ tarek/staff 0 2010-02-01 16:23:40 ./

400

-rw-r--r-- tarek/staff 609 2008-06-09 13:26:54 ./authorized_keys

401

-rwxr-xr-x tarek/staff 65 2008-06-09 13:26:54 ./config

402

-rwx------ tarek/staff 668 2008-06-09 13:26:54 ./id_dsa

403

-rwxr-xr-x tarek/staff 609 2008-06-09 13:26:54 ./id_dsa.pub

404

-rw------- tarek/staff 1675 2008-06-09 13:26:54 ./id_rsa

405

-rw-r--r-- tarek/staff 397 2008-06-09 13:26:54 ./id_rsa.pub

406

-rw-r--r-- tarek/staff 37192 2010-02-06 18:23:10 ./known_hosts

407

</pre></div>

408

</div>

409

</div>

410

</div>

411

</div>

412

413

414

</div>

415

</div>

416

</div>

417

418

419

<h3><a href="../contents.html">Table Of Contents</a></h3>

420

<ul>

421

<li><a class="reference internal" href="#">10.10. <code class="docutils literal">shutil</code> — High-level file operations</a><ul>

422

<li><a class="reference internal" href="#directory-and-files-operations">10.10.1. Directory and files operations</a><ul>

423

<li><a class="reference internal" href="#copytree-example">10.10.1.1. copytree example</a></li>

424

</ul>

425

</li>

426

<li><a class="reference internal" href="#archiving-operations">10.10.2. Archiving operations</a><ul>

427

<li><a class="reference internal" href="#archiving-example">10.10.2.1. Archiving example</a></li>

428

</ul>

429

</li>

430

</ul>

431

</li>

432

</ul>

433

434

<h4>Previous topic</h4>

435

<a href="linecache.html"

436

title="previous chapter">10.9. <code class="docutils literal">linecache</code> — Random access to text lines</a>

437

<h4>Next topic</h4>

438

<a href="dircache.html"

439

title="next chapter">10.11. <code class="docutils literal">dircache</code> — Cached directory listings</a>

440

441

442

<li><a href="../bugs.html">Report a Bug</a></li>

443

<li><a href="../_sources/library/shutil.txt"

444

rel="nofollow">Show Source</a></li>

445

</ul>

446

447

448

<h3>Quick search</h3>

449

450

451

452

453

454

</form>

455

456

Enter search terms or a module, class or function name.

457

458

</div>

459

460

</div>

461

</div>

462

463

</div>

464

465

<h3>Navigation</h3>

466

<ul>

467

468

<a href="../genindex.html" title="General Index"

469

>index</a></li>

470

471

<a href="../py-modindex.html" title="Python Module Index"

472

>modules</a> |</li>

473

474

<a href="dircache.html" title="10.11. dircache — Cached directory listings"

475

>next</a> |</li>

476

477

<a href="linecache.html" title="10.9. linecache — Random access to text lines"

478

>previous</a> |</li>

479

<li><img src="../_static/py.png" alt=""

480

style="vertical-align: middle; margin-top: -1px"/></li>

481

<li><a href="https://www.python.org/">Python</a> »</li>

482

<li>

483

2.7.12

484

<a href="../index.html">Documentation</a> »

485

</li>

486

487

<li class="nav-item nav-item-1"><a href="index.html" >The Python Standard Library</a> »</li>

488

<li class="nav-item nav-item-2"><a href="filesys.html" >10. File and Directory Access</a> »</li>

489

</ul>

490

</div>

491

492

493

494

The Python Software Foundation is a non-profit corporation.

495

<a href="https://www.python.org/psf/donations/">Please donate.</a>

496

497

Last updated on Sep 20, 2016.

498

<a href="../bugs.html">Found a bug</a>?

499

500

Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.3.3.

501

</div>

502

503

</body>

504

</html>

b'\\ No newline at end of file'

Older »