-
Committer:
Scott Moser
-
Date:
2015-09-22 20:28:53 UTC
-
mfrom:
(398.1.21 trunk.lp1487004)
-
Revision ID:
smoser@ubuntu.com-20150922202853-hxqmj6bpif5n62v9
provide insert_item with a contentsource that does checksumming
Previously, insert_item would receive a 'contentsource' (essentially
a file like object to be read as a stream). The user of that object
needed to calculate checksums and verify the data they read.
Now, instead the contentsource will do checksumming as read()
operations are done, and will raise a checksum error in any failure
case.
Thus to use this, the user now simply has to loop over reads
and catch the exception.
stream data is now expected to have valid checksums and size on
all items with a path. If the user is using a stream source that
does not have either size or checksum information, they have a few
options:
a.) [legacy/SRU only] set environment variable
SS_MISSING_ITEM_CHECKSUM_BEHAVIOR can be set to
silent: behave exactly as before. No checksumming is done,
no warnings are emitted. The consumer of the
contentsource must check checksums.
warn: log messages at WARN level (same as default/unset)
fail: the new behavior. raise an InvalidChecksum exception.
b.) instantiate the BasicMirrorWriter with config checksumming_reader=False
the default for that config setting is True, meaning that you
will get a reader that checksums content as it goes and
raises exception on bad data.
c.) fix their source to have a sha256sum and a size
The 'sstream-mirror' program now has a '--no-checksumming-reader' flag
that does 'b' for this mirror.