~cfuhrman/+junk/netbsd-othersrc-trunk : changes

~cfuhrman/+junk/netbsd-othersrc-trunk » Changes from revision 1922

From Revision 1922 to 1903

Rev	Summary	Authors	Date
1922	Break out the routines (from tnftp) to calculate and draw a ... Break out the routines (from tnftp) to calculate and draw a progress bar. The code was originally written by Luke Mewburn - all mangling and bugs courtesy of agc. A quick (hypothetical) example of how to use this progress bar: progress_t prog; uint64_t total; uint64_t done; total = 15768554; progress_init(&prog, "test update bar", total); for (done = 0 ; done < total ; done += (total / 20) + ((random() % 20) * 1024)) { progress_draw(&prog, done); sleep(1); } progress_complete(&prog, done);	agc	11 years ago
1921	Allow the bootstrap to install xbase.tgz, since that will un... Allow the bootstrap to install xbase.tgz, since that will unfortunately be needed for many binary packages. One of these days, this script should be cleaned up a bit, too.	riz	11 years ago
1920	Example scripts and supporting files for creating Amazon EC2... Example scripts and supporting files for creating Amazon EC2 AMIs (Amazon Machine Images) running NetBSD. Requires an Amazon EC2 account. Please test and extend these scripts! The original scripts were written by Jean-Yves Migeon, heavily modified From the README: Notes on using these scripts for creating EC2 AMIs -------------------------------------------------- Here are some assumptions made: - You have the EC2 API tools already set up. If you don't already, the easiest way to do so is to install the misc/ec2-api-tools package from pkgsrc. An easy way to check if they're set up properly is if you can run "ec2-describe-instances" without an error. - You are running this from a NetBSD system. It's certainly not impossible to update this for running on another platform, but it'll be necessary to adapt to using a NetBSD tools build for (at least) makefs. - Your default EC2 profile allows ssh to your running instances from the location you're running this script! - The most-used (and tested!) script (create_ec2_ami.sh) assumes that there are NetBSD 6.0 AMIs available on EC2 already, for bootstrapping. The public images in account 101367081206 (an account owned by me, Jeff Rizzo, a NetBSD developer) are available for use, and are searched in addition to the user's own AMIs. You can also specify a particular AMI to use for the bootstrapping. There is another script (create-new.sh) which is less well tested, and which may currently be broken, which allows you to use a Linux AMI to create a NetBSD one. You will need to create a 'bootstrap' image with an ext2 file system (see build_ec2_img.sh, specifically the "-b" flag). CREATING AN AMAZON MACHINE IMAGE (AMI) -------------------------------------- Once you've verified the Amazon EC2 command-line tools work (see above), the basic workflow is as follows: - Edit the set_ec2_env.sh script to have the location of your certs/keys. This file is read by several of the scripts. - Either build sets with build.sh, or identify where the release you want to base your AMI on lives. You can access built sets via http, ftp, or on the local fs. - Build an image for upload, using the build_ec2_img.sh script. This image will contain everything except the kernel (the kernel will be on a separate ext2fs partition, due to EC2 boot requirements), and will be resized to fit the volume created in the AMI creation script. An example of doing this: ./build_ec2_img.sh -d http://ftp.netbsd.org/pub/NetBSD/NetBSD-6.0_BETA/amd64/binary/sets ...this will create an image file in the current directory. - Create the AMI from the above image plus a kernel. You have to specify which region to create the AMI in: ./create_ec2_ami.sh -i ./NetBSD-AMI.img.xz -a x86_64 -d "AMI description goes here" -k http://ftp.netbsd.org/pub/NetBSD/NetBSD-6.0_BETA/amd64/binary/kernel/netbsd-XEN3_DOMU.gz -v "6.0_BETA-or-whatever-version-string-you-want" us-east-1 The process can take anywhere from 10-60 minutes, depending on which region you choose and how fast your connection is for uploading. After a successful run, be sure to test your AMI!	riz	12 years ago
1919	don't link with libthreshold any more don't link with libthreshold any more run simple tests in place without requiring lib installation	agc	12 years ago
1918	restore correct operation for odd-sized files restore correct operation for odd-sized files	agc	12 years ago
1917	+ don't allocate space if we're passed an in-memory array as... + don't allocate space if we're passed an in-memory array as either a source or a destination of split/combine + fix up the split_vec/combine_vec combinatino so that they both work a bit better now - still some work needed here, though + free the internal s4 structure as part of the combine/combine_vec operation, rather than doing it in ssss_end(). We know at that stage that we have allocated it, and the principle of freeing memory in the same place in the calling stack as it was allocated still applies + rip out parts of the initial comment which aren't really accurate any more - i'm not sure there's that much of the original code left	agc	12 years ago
1916	Update the ssss code: Update the ssss code: + add the ability to encode split shares as 16bit or 24-bit values, which greatly simplifies input and output, and allows constant-sized output for given inputs. This also gives us the ability to use larger primes, although space is wasted with these, and it's not immediately obvious what advantage would accrue. But it's there, just in case. + we advertise being able to do 255 (SSSS_MAX_SHARES) shares - so complete the inverse table for GF(16) arithmetic modulo P for 256 values + use SSSS_MAX_SHARES (255) internally, rather than the previous internal (and much smaller) limits + simplify the I/O in this library - it was a remnant from 1993 + remove private structs and definitions from the header file - they're not part of the public interface, and shouldn't be exposed + add ssss_split_vec() and ssss_combine_vec() which give input and oputput to split and combine via struct iovec + update the manual page	agc	12 years ago
1915	Typo Typo	skrll	12 years ago
1914	Numerous changes to ssss(1) and libssss(3): Numerous changes to ssss(1) and libssss(3): + don't use the threshold headers any more - just use our own header + "ssss" is the only algorithm supported by ssss(1) + add back (seamless) file I/O functionality, in the case that a memory mapping, or subsequent memory allocation, fails + don't store information in the ssss share header which could be used as part of an attack (specifically the threshold and total number of shares). + size the extra memory required on a much better estimate of the size needed, rather than a constant size + get rid of unneeded files, now that everything has been cleaned up	agc	12 years ago
1913	minor cosmetic changes to make this more readable - no funct... minor cosmetic changes to make this more readable - no functional change	agc	12 years ago
1912	Various fixes to improve stability and code readability Various fixes to improve stability and code readability + previously not enough space was allocated for the unusual occurrence that a 16-bit entity was greater than the largest 16bit prime (65521). Introduce an overhead allocation which works around this issue + the largest prime is not about to change - define it, and don't bother passing it as an argument to functions. + remove an extraneous "can't happen" call to exit() + fix some lint (as seen on amd64) + use better names for variables in the code + move occurrences of fprintf() to warn() - thanks, Christos! + use definitions instead of magic integer constants to make it obvious what's happening with end of file, and odd-sized files + hold on information on input parsing in a separate struct, rather than using arguments to functions + check input arguments to ssss(1), and improve error checking in general With this in place, ssss(1) will now combine shares accessible through httpdev(8) - which opens up a number of opportunities...	agc	12 years ago
1911	don't rely on an application (such as ssss(1)) being able to... don't rely on an application (such as ssss(1)) being able to mmap(2) the shares it presents via libssss - allocate space using calloc(3) if the mmap fails. this change allows us to use ssss(1) to combine shares from different websites, mounted using httpdev(8).	agc	12 years ago
1910	return the number of characters read/written in the small ta... return the number of characters read/written in the small target operation, not the size that was requested.	agc	12 years ago
1909	Document -n. Document -n.	wiz	12 years ago
1908	The default blocksize is 4096 bytes. The default blocksize is 4096 bytes.	wiz	12 years ago
1907	make sure we initialise the circa structures when decoding a... make sure we initialise the circa structures when decoding a file, and no header is present.	agc	12 years ago
1906	Recurse into 'window'. Recurse into 'window'.	jmmv	12 years ago
1905	This commit was generated by cvs2svn to compensate for chang... This commit was generated by cvs2svn to compensate for changes in r2280, which included commits to RCS files with non-trunk default branches.	jmmv	12 years ago
1904	Don't hardcode a blocksize of 512 bytes. Don't hardcode a blocksize of 512 bytes. Add a blocksize argument to httpdev (-b blocksize). Bump default blocksize to 4KB.	agc	12 years ago
1903	Add an argument to circa_file(3), and a command line flag (-... Add an argument to circa_file(3), and a command line flag (-n) to circa(1), to denote that we do/do not want the circa header placed in the output. This is useful in cases where the sectorsize is pre-defined, and so we do not need to place the value in a header in the output. Bump shlib major version for the change to the API for libcirca(3).	agc	12 years ago

Older »