~abentley/simplestreams/bad-hoist : changes

~abentley/simplestreams/bad-hoist » Changes from revision 417

From Revision 401 to 382

Rev	Summary	Authors	Date
401	export checksummer in simplestreams.util export checksummer in simplestreams.util When moving the checksumming things out of checksum_util, I added names for CHECKSUMS, item_checksums and ALGORITHMS, but failed to get checksummer. Simply export checksummer in util, pointing it to checksum_util.util	Scott Moser	8 years ago
400	provide insert_item with a contentsource that does checksumm... provide insert_item with a contentsource that does checksumming Previously, insert_item would receive a 'contentsource' (essentially a file like object to be read as a stream). The user of that object needed to calculate checksums and verify the data they read. Now, instead the contentsource will do checksumming as read() operations are done, and will raise a checksum error in any failure case. Thus to use this, the user now simply has to loop over reads and catch the exception. stream data is now expected to have valid checksums and size on all items with a path. If the user is using a stream source that does not have either size or checksum information, they have a few options: a.) [legacy/SRU only] set environment variable SS_MISSING_ITEM_CHECKSUM_BEHAVIOR can be set to silent: behave exactly as before. No checksumming is done, no warnings are emitted. The consumer of the contentsource must check checksums. warn: log messages at WARN level (same as default/unset) fail: the new behavior. raise an InvalidChecksum exception. b.) instantiate the BasicMirrorWriter with config checksumming_reader=False the default for that config setting is True, meaning that you will get a reader that checksums content as it goes and raises exception on bad data. c.) fix their source to have a sha256sum and a size The 'sstream-mirror' program now has a '--no-checksumming-reader' flag that does 'b' for this mirror.	Scott Moser	8 years ago
399	glance.py: fix KeyError due to typo glance.py: fix KeyError due to typo adding the os_version checked for existance of version in the data but then referenced 'os_version'.	Scott Moser	8 years ago
398	provide 'os_name' and 'os_version' to glance if available provide 'os_name' and 'os_version' to glance if available we recently added 'os' and 'version' strings to the stream data for Ubuntu cloud images. This takes those if they're there and adds them to the correctly named property fields in glance. See for more information: http://docs.openstack.org/cli-reference/content/ chapter_cli-glance-property.html	Scott Moser	8 years ago
397	Map ubuntu/dpkg arch strings to those that libvirt understan... Map ubuntu/dpkg arch strings to those that libvirt understands. Ensure all Ubuntu architectures get correctly mapped into ones that OpenStack and libvirt understand.	Scott Moser	8 years ago
396	make-test-data: support skipping unsupported releases make-test-data: support skipping unsupported releases This adds support for skipping the publication of data for releases that have reached end of life. We came to do this work when bad data on cloud-images.ubuntu.com (described in bug 1493395) meant that many referenced urls were not present. The general idea seems sane though, so we'll add this code. The behavior can be controlled by environment variable 'SS_SKIP_UNSUPPORTED' which defaults ot '0'. Setting to a value other than '0' will mean that the unsupported releases are simply skipped. Doing so means we must have distro-info (as it is how we get the supported flag for a release).	Scott Moser	8 years ago
395	require distro-info unless environment variable says otherwi... require distro-info unless environment variable says otherwise This is in a move to get 'supported' information consistently available. For development purposes you can override this setting with: SS_REQUIRE_DISTRO_INFO=0 The default is to require.	Scott Moser	8 years ago
394	tools/hook-check-downloads: fix typo in help output tools/hook-check-downloads: fix typo in help output	Scott Moser	8 years ago
393	add 'make-test-data' to run-pep8 and fix pep8 issues in it. add 'make-test-data' to run-pep8 and fix pep8 issues in it.	Scott Moser	8 years ago
392	make-test-data: add mechanism for skipping older builds make-test-data: add mechanism for skipping older builds since make-test-data crawls so much data from cloud-images.ubuntu.com it can be slow and painful on its first run. Once that data is obtained, its cached in exdata-query/FILE_DATA_CACHE, but its slow the first time. This lets you do something like: SS_DEBUG_MIN_SERIAL=20150101 make exdata and then any serial number < 20150101 will just be skipped.	Scott Moser	8 years ago
391	make-test-data: add 'version' back to download data make-test-data: add 'version' back to download data version was inadvertently dropped from download data in revision 387. It had stayed in the generated ec2 data. This adds 'new_ubuntu_product_dict' and uses it in both places to create the initial product data. That should ensure that we dont make download and ec2 different unintentionally in the future. This also will end up adding 'release_title' and 'release_codename' to the ec2 data. And also will add 'supported' for the case where python-distroinfo is available during the build (tools/ubuntu_versions.py only gets that data if using distro info so we are not expected to keep up to date information here).	Scott Moser	8 years ago
390	fix 'make pep8' on wily (1.6.2) fix 'make pep8' on wily (1.6.2) newer pep8 started complaining about these things.	Scott Moser	8 years ago
389	Create a sha256 sum for a combined lxd/root tarball for the ... Create a sha256 sum for a combined lxd/root tarball for the LXD folks. lxd is currently using a -lxd.tar.xz file which contains metadata and a -root.tar.xz which contains the same payload as the -root.tar.gz but is xz compressed. They expect a 'combined_sha256' key that is essentially: cat file-lxd.tar.xz file-root.tar.xz \| sha256 This adds that field and the new ftypes.	Scott Moser	8 years ago
388	remove 'os_title'. remove 'os_title'. It wasn't clear what we were gaining from this. The reason is that we may have a product that is 'Ubuntu Server' and one that is 'Ubuntu Desktop'. Both are 'ubuntu' as the distro but the 'os_title' doesn't buy us anything here. Perhaps sometime in the future we'd add 'Ubuntu Server' as a product_title or something.	Scott Moser	8 years ago
387	tools/make-test-data: include additional info in stream outp... tools/make-test-data: include additional info in stream output This adds the following fields of information to each of the dictionaries in release data: os_title: literal "Ubuntu" release_codename: the full code name ('Saucy Salamander', 'Trusty Tahr') release_title: numeric portion + " LTS" if this is an lts '13.10', '14.04 LTS" Each of those fields are added to stream data. note, 'devel' and 'supported' are only present if it gets data from the distro_info module. we add additional fields of information for each of the data in REL2VER. Then, simplify the hard coded data we have to maintain.	Scott Moser	8 years ago
386	ubuntu_versions: fix bad data for wily ubuntu_versions: fix bad data for wily wily was labeled 'vivid'. this path would be taken only if distro_info was not available.	Scott Moser	8 years ago
385	glance mirror: fix stack trace on no existing entries glance mirror: fix stack trace on no existing entries if all entries are filtered out, the call to prune would remove the top level content_id also, causing problems.	Scott Moser	8 years ago
384	fix previous commit fix previous commit I hadn't added the util portion of this fix.	Scott Moser	8 years ago
383	add --no-verify argument to not gpg verify signed json files... add --no-verify argument to not gpg verify signed json files. Its possible that the user doesn't have the key in their keyring and is only testing or not concerned with checking content signatures. This provides '--no-verify' or '-U' for that case to sstream-mirror, sstream-query, sstream-sync	Scott Moser	8 years ago
382	README formatting fixes. README formatting fixes. The RFC URL used is now owned by a domain squatter; as I was changing that I figured I may as well do a little bit of clean-up as I read through.	Scott Moser	8 years ago

Older »