14
14
with the possibility of parallel uploads of the segments.
16
16
----------------------------------
17
Using ``st`` for Segmented Objects
17
Using ``swift`` for Segmented Objects
18
18
----------------------------------
20
The quickest way to try out this feature is use the included ``st`` Swift Tool.
20
The quickest way to try out this feature is use the included ``swift`` Swift Tool.
21
21
You can use the ``-S`` option to specify the segment size to use when splitting
22
22
a large file. For example::
24
st upload test_container -S 1073741824 large_file
24
swift upload test_container -S 1073741824 large_file
26
26
This would split the large_file into 1G segments and begin uploading those
27
segments in parallel. Once all the segments have been uploaded, ``st`` will
27
segments in parallel. Once all the segments have been uploaded, ``swift`` will
28
28
then create the manifest file so the segments can be downloaded as one.
30
So now, the following ``st`` command would download the entire large object::
32
st download test_container large_file
34
``st`` uses a strict convention for its segmented object support. In the above
30
So now, the following ``swift`` command would download the entire large object::
32
swift download test_container large_file
34
``swift`` uses a strict convention for its segmented object support. In the above
35
35
example it will upload all the segments into a second container named
36
36
test_container_segments. These segments will have names like
37
37
large_file/1290206778.25/21474836480/00000000,
43
43
upload of a new file with the same name won't overwrite the contents of the
44
44
first until the last moment when the manifest file is updated.
46
``st`` will manage these segment files for you, deleting old segments on
46
``swift`` will manage these segment files for you, deleting old segments on
47
47
deletes and overwrites, etc. You can override this behavior with the
48
48
``--leave-segments`` option if desired; this is useful if you want to have
49
49
multiple versions of the same large object available.
55
55
You can also work with the segments and manifests directly with HTTP requests
56
instead of having ``st`` do that for you. You can just upload the segments like
56
instead of having ``swift`` do that for you. You can just upload the segments like
57
57
you would any other object and the manifest is just a zero-byte file with an
58
58
extra ``X-Object-Manifest`` header.
60
60
All the object segments need to be in the same container, have a common object
61
61
name prefix, and their names sort in the order they should be concatenated.
62
62
They don't have to be in the same container as the manifest file will be, which
63
is useful to keep container listings clean as explained above with ``st``.
63
is useful to keep container listings clean as explained above with ``swift``.
65
65
The manifest file is simply a zero-byte file with the extra
66
66
``X-Object-Manifest: <container>/<prefix>`` header, where ``<container>`` is
added
removed
doc/source/overview_large_objects.rst
