~smoser/simplestreams/trunk

  • Committer: Scott Moser
  • Date: 2015-09-22 20:28:53 UTC
  • mfrom: (398.1.21 trunk.lp1487004)
  • Revision ID: smoser@ubuntu.com-20150922202853-hxqmj6bpif5n62v9
provide insert_item with a contentsource that does checksumming

Previously, insert_item would receive a 'contentsource' (essentially
a file like object to be read as a stream).  The user of that object
needed to calculate checksums and verify the data they read.

Now, instead the contentsource will do checksumming as read() 
operations are done, and will raise a checksum error in any failure
case.

Thus to use this, the user now simply has to loop over reads
and catch the exception.

stream data is now expected to have valid checksums and size on
all items with a path.  If the user is using a stream source that
does not have either size or checksum information, they have a few
options:
 a.) [legacy/SRU only] set environment variable
     SS_MISSING_ITEM_CHECKSUM_BEHAVIOR can be set to
        silent: behave exactly as before.  No checksumming is done,
                no warnings are emitted.  The consumer of the
                contentsource must check checksums.
        warn:   log messages at WARN level (same as default/unset)
        fail:   the new behavior. raise an InvalidChecksum exception.

 b.) instantiate the BasicMirrorWriter with config checksumming_reader=False
     the default for that config setting is True, meaning that you
     will get a reader that checksums content as it goes and 
     raises exception on bad data.

 c.) fix their source to have a sha256sum and a size


The 'sstream-mirror' program now has a '--no-checksumming-reader' flag
that does 'b' for this mirror.
Filename Latest Rev Last Changed Committer Comment Size
..
bin 98 11 years ago Scott Moser add sstream-sync, somewhat functional client Diff
debian 177 11 years ago Scott Moser add initial debian packaging Diff
doc 8 11 years ago Scott Moser add initial doc/example. Diff
examples 117.1.1 11 years ago Scott Moser initial gpg keys added, examples moved from doc to Diff
simplestreams 15 11 years ago Scott Moser add initial simple streams and tests that pass Diff
tests 15 11 years ago Scott Moser add initial simple streams and tests that pass Diff
tools 1 11 years ago Scott Moser add initial tools/make-test-data Diff
.bzrignore 286 10 years ago Scott Moser Add a tests to test that bad signatures cause fail 71 bytes Diff Download File
.gitignore 286 10 years ago Scott Moser Add a tests to test that bad signatures cause fail 66 bytes Diff Download File
LICENSE 177 11 years ago Scott Moser add initial debian packaging 33.7 KB Diff Download File
Makefile 351.2.19 9 years ago Scott Moser switch pylint to pyflakes 1.8 KB Diff Download File
README.txt 249 11 years ago Scott Moser change '.js' extension to '.json' (.sjs -> .sjson) 1.8 KB Diff Download File
setup.py 390 8 years ago Scott Moser fix 'make pep8' on wily (1.6.2) newer pep8 starte 688 bytes Diff Download File