1
Handling of file:// URIs:
3
This directory contains code to map basic boto connection, bucket, and key
4
operations onto files in the local filesystem, in support of file://
7
Bucket storage operations cannot be mapped completely onto a file system
8
because of the different naming semantics in these types of systems: the
9
former have a flat name space of objects within each named bucket; the
10
latter have a hierarchical name space of files, and nothing corresponding to
11
the notion of a bucket. The mapping we selected was guided by the desire
12
to achieve meaningful semantics for a useful subset of operations that can
13
be implemented polymorphically across both types of systems. We considered
14
several possibilities for mapping path names to bucket + object name:
16
1) bucket = the file system root or local directory (for absolute vs
17
relative file:// URIs, respectively) and object = remainder of path.
18
We discarded this choice because the get_all_keys() method doesn't make
19
sense under this approach: Enumerating all files under the root or current
20
directory could include more than the caller intended. For example,
21
StorageUri("file:///usr/bin/X11/vim").get_all_keys() would enumerate all
22
files in the file system.
24
2) bucket is treated mostly as an anonymous placeholder, with the object
25
name holding the URI path (minus the "file://" part). Two sub-options,
26
for object enumeration (the get_all_keys() call):
27
a) disallow get_all_keys(). This isn't great, as then the caller must
28
know the URI type before deciding whether to make this call.
29
b) return the single key for which this "bucket" was defined.
30
Note that this option means the app cannot use this API for listing
31
contents of the file system. While that makes the API less generally
32
useful, it avoids the potentially dangerous/unintended consequences
33
noted in option (1) above.
35
We selected 2b, resulting in a class hierarchy where StorageUri is an abstract
36
class, with FileStorageUri and BucketStorageUri subclasses.
38
Some additional notes:
40
BucketStorageUri and FileStorageUri each implement these methods:
41
- clone_replace_name() creates a same-type URI with a
42
different object name - which is useful for various enumeration cases
43
(e.g., implementing wildcarding in a command line utility).
44
- names_container() determines if the given URI names a container for
45
multiple objects/files - i.e., a bucket or directory.
46
- names_singleton() determines if the given URI names an individual object
48
- is_file_uri() and is_cloud_uri() determine if the given URI is a
49
FileStorageUri or BucketStorageUri, respectively
added
removed
boto/file/README
