~powersj/curtin/pycodestyle : changes

~powersj/curtin/pycodestyle » Changes from revision 451

From Revision 451 to 432

Rev	Summary	Authors	Date
451	pycodestyle cleanup of ambigious name pycodestyle cleanup of ambigious name	Josh Powers	7 years ago
450	helpers/common: Add grub install debugging output helpers/common: Add grub install debugging output At various times (bug 1642298) we have needed to see more information on what is being done during grub installation. Adding execution tracing exposes the exact command that's run for debugging. Also, add a dump of efibootmgr output before and after to aide in debugging UEFI grub installs. Capture this output and send it to log at debug level rather than letting it go to standard output / error.	Ryan Harper	7 years ago
449	flake8: remove unused variable. flake8: remove unused variable.	Scott Moser	7 years ago
448	curtin: add version module and display in output and logs curtin: add version module and display in output and logs It's difficult to determine what curtin version is running from the collected logs in a maas deployment which uses curtin by sending it to the target system. Resolve this by having curtin announce itself. - Introduce a new version module which does the following: - allows packaging to set the version by source code modification. packaging changed to replace a string @@PACKAGED_VERSION@@ with the version packaged. - if no packaged version is set, and .bzr dir exists, then run bzr revno to get information on that. - if no bzr, then use the static version. - Modify curtin.command.main to print the version string in argparse output - Update curtin logging setup to emit a version string as soon as logging is configured	Ryan Harper	7 years ago
447	vmtest: Add the ability to add extra config files to test ex... vmtest: Add the ability to add extra config files to test execution. This is what I used to test multipath-tools via curtin, but it can be much more helpful by generally allowing to add any external content	Christian Ehrhardt	7 years ago
446	content decoding in load_file, apply_net raise exception on ... content decoding in load_file, apply_net raise exception on errors This patch series fixes two issues. First, subcommands of the apply_net command were exiting non-zero but we failed to raise and exception which let curtin hide an error. We've modified apply_net to re-raise the exception when it occurs. Additionally update vmtest to look for stack-traces in the installation log and mark a test failed if it detects one; this should prevent future cases from re-occurring. The second error is when loading a file with and encoding, load_file did not handle this case. Merge in a version of load_file from cloud-init which already handles this case. Introduce new unittests to validate the function as it touches some block related code where we read partition data directly. Finally, the mock_open feature of unittest.mock only supports binary data in version 2.0.0 or newer, so skip this unittest on systems without a new-enough mock; note this does not affect the function of the code on the same release, only the unittest.	Ryan Harper	7 years ago
445	gpg: retry when recv'ing gpg keys fail gpg: retry when recv'ing gpg keys fail Retry gpg recv operations in case error is transient. Add unittests to validate curtin.gpg including retry behavior	Ryan Harper	7 years ago
444	Add clear_holders checks to disk and partition handlers Add clear_holders checks to disk and partition handlers In some cases disks may contain block device metadata outside of the current partition table. Due to Linux udev rules and behaviors; creating or modifying a disks partition table may trigger udev rules which may start up mdadm, bcache or lvm services which will obtain exclusive access to the disk preventing installation from completing. This branch addresses this issue with the following - block.wipe_volume now uses os.O_EXCL flag to obtain exclusive access to a device; this fails when something else (like mdadm) obtains a file handle on the device with O_EXCL flag - Upon failure to obtain exclusive access, we dump information about a devices block holders - During disk handling, after creating a partition table we check for holders and then clear them if present. - During partition handling, after creating a new partition we check for holders and then clear them if present.	Ryan Harper	7 years ago
443	vmtest: overhaul image sync vmtest: overhaul image sync Changes to support HWE and Centos images have exposed issues in the image sync process. Our 'make sync-images' was using a simplestreams filter that did not pickup any of the required HWE kernels which meant that during vmtest runs we would trigger new image syncs. Compounding this issue was the fact that the filter used was too-wide which picked up things like di-initrd,di-kernel files. Additionally for an empty IMAGE_DIR repository, there were two bugs; one it assumed that it could load the vmtest.json/vmtest-centos.json files. Second when the repo was empty, we did not return an iterable type. The following changes have been made to achieve the goals: - make sync-images should download everything we need to run a complete vmtest run without acquiring any new files from our image server. - Handle empty repositories properly - Do not attempt to sync with the streams on each test, but only if specific needed files are missing - Do not sync streams during jenkings-runner; the previous make sync-images should have pulled everything	Ryan Harper	7 years ago
442	vmtest: skip apt-proxy test if not set vmtest: skip apt-proxy test if not set Testing of apt-proxy requires one to be set in the host. If a proxy is not set then the test-case will fail. This was discovered on diglett host which usally has an apt-proxy but recently it was disabled which tripped this failure.	Ryan Harper	7 years ago
441	net: add new lines after rendered static routes. net: add new lines after rendered static routes. Static routes were being rendered in /etc/network/interfaces without a trailing newline after the 'pre-down' line. Add unittest and vmtest to validate static route configurations.	Scott Moser	7 years ago
440	vmtest: add 'webserv' helper vmtest: add 'webserv' helper Rather than running python -m SimpleHTTPServer for the web server add a program 'webserv' in tools/ dir and use that. The benefits of this approach are: a.) no longer need python2 (we were invoking with 'python') b.) webserv listens on both ipv4 and ipv6 addresses.	Scott Moser	7 years ago
439	vmtest: add CURTIN_VMTEST_CURTIN_EXE variable. vmtest: add CURTIN_VMTEST_CURTIN_EXE variable. This allows us to control which 'curtin' program is used. The program must support curtin's 'pack' interface, and needs to handle finding itself appropriately. Also here, we drop the PYTHONPATH setting in launch, as it is no longer needed as our bin/curtin wrapper handles that. The target use case is really for easily running vmtest with an installed version of curtin, allowing us to easily verify stable release updates.	Scott Moser	7 years ago
438	multipath: don't run update-grub; setup_grub will handle thi... multipath: don't run update-grub; setup_grub will handle this better. During an update-grub in the target system when using multipath the 'os-prober' tool used to discover other operating systems to add to the grub menu does not handle duplicate paths and mounts the same disk up twice causing unknown behavior which sometimes results in timeouts during installation. Dropping the direct call to update-grub and deferring to curtin's setup_grub method (which disables os-prober) works around the issue. Separate bug filed against os-prober. LP: #1656369	Ryan Harper	7 years ago
437	revert: Test Workaround: skip XenialTestNvme for a short tim... revert: Test Workaround: skip XenialTestNvme for a short time. Previous commit disabled XenialTestNvme due to broken kernel in xenial. That has been fixed in LP: #1651602, so re-enable the test.	Scott Moser	7 years ago
436	Test Workaround: skip XenialTestNvme for a short time. Test Workaround: skip XenialTestNvme for a short time. This skips test XenialTestNvme that is failing due to LP: #1651602. That fix should land in xenial -updates at some point in mid January. So, skip the test until Jan 21, and then raise exception to remind ourselves to revert the change after the 30th.	Scott Moser	7 years ago
435	pep8: fix pep8 errors found with 'make pep8' on zesty. pep8: fix pep8 errors found with 'make pep8' on zesty. This can also be seen by running pep8 in a tox environment, but as it is right now, our Makefile uses ./tools/run-pep8. That runs 'pep8', where our tox environment runs 'flake8'. We should really resolve that difference, but for now I am just quickly committing the fix.	Scott Moser	7 years ago
434	Workaround failures caused by gpg2 daemons left running in c... Workaround failures caused by gpg2 daemons left running in chroot. Kill dirmngr/gpg-agent after add-apt-repository as an interim workaround. The failure occurs only when gpg2 is used, so only on yakkety or newer. Add a Yakkety test to excercise this path.	Christian Ehrhardt	7 years ago
433	Install u-boot-tools when running on a system with u-boot. Install u-boot-tools when running on a system with u-boot. Previously lp:maas-images was adding u-boot-tools to all ARM images before they were published to images.maas.io. lp:maas-images now republishes the SquashFS images from cloud-images.ubuntu.com unmodified. The unmodified images do not contain u-boot-tools causing the kernel to fail to install on systems with u-boot. We now use flash-kernel to show which packages are needed for installation and install those as it indicates.	Scott Moser	7 years ago
432	block: fix partition kname for raid devices block: fix partition kname for raid devices mdadm RAID1 supports partitioning of the md device and utilizes the 'p' char to name partitions: md0p1, md0p2 etc. - Fix by adding 'md' to curtin/block:partition_kname - Update block unittest to test/check md names - Introduce a new vmtest to exercise partitioning RAID1 disks (test_mdadm_bcache.py:TestMirrorbootPartitions)	Ryan Harper	7 years ago

Older »