~powersj/curtin/fix-vmtest-log-exception : changes

~powersj/curtin/fix-vmtest-log-exception » Changes from revision 521

From Revision 521 to 502

Rev	Summary	Authors	Date
521	Adding exception handeling to curtin-log-print runs. Adding exception handeling to curtin-log-print runs.	Joshua Powers	6 years ago
520	Update network cloud-init related skiptest dates, SRU still ... Update network cloud-init related skiptest dates, SRU still pending Move skipdate to 2017-08-20 to allow more time for cloud-init SRU to land.	Ryan Harper	6 years ago
519	tests: Add CiTestCase common parent for all curtin tests. tests: Add CiTestCase common parent for all curtin tests. This branch makes sure all unittests subclass from CiTestCase which provides all unit tests with a tmp_dir and tmp_path creation. Drop unittest-specific tempfile usage in favor of self.tmp_dir and self.tmp_path. Adding this parent class gives us a single place to add test methods which have general application across unit tests, like log tracking and tmp directory cleanup.	Chad Smith	6 years ago
518	vmtests: Remove force flag for centos curthooks vmtests: Remove force flag for centos curthooks During development, and before our source images had updated curtin hooks we needed this flag to push test installations down the new code path. We've left it present for future development needs.	Ryan Harper	6 years ago
517	tools/jenkins-runner: improve tgtd cleanup logic tools/jenkins-runner: improve tgtd cleanup logic The previous tgtd cleanup logic was hard to follow at best. It was buggy in that it would always kill the TGT_PID process even if it didn't start it. both cleanup lines ended with: <condition> \|\| [ -n "${TGT_PID}" ] && kill -9 ${TGT_PID} which will always run the 'kill' if TGT_PID is non-empty. Also: * clean up the output of tools/jenkins-runner by redirecting the find-tgt.d to a file so you don't see its debug messages. * use modulus to get a port in the right range. * improve the picking of a port and then checking to see if it was used. I believe a race condition where tgt wasn't listening on the port when netstat would run lead to false positives for failures where a sleep would have sufficed. * remove the tgt socket file(s) if it was created.	Scott Moser	6 years ago
516	tests: Drop EOL Wily Vivid and Yakkety tests. tests: Drop EOL Wily Vivid and Yakkety tests. We no longer carry test classes for end-of-life/unsupported series.	Chad Smith	6 years ago
515	Disable yum plugins when installing packages, update ca-cert... Disable yum plugins when installing packages, update ca-certs for https The default yum configured plugins includes the 'fastmirrors' plugin which produces inconsistent results resulting in broken installs. Update the ca-certificates package to prevent https mirror download failures. Add some retries to the yum install commands to handle transient failures.	Ryan Harper	6 years ago
514	Rename centos_network_curthooks -> centos_apply_network_conf... Rename centos_network_curthooks -> centos_apply_network_config. Rename centos_network_curthooks and its upper-cased feature CENTOS_NETWORK_CURTHOOKS to centos_apply_network_config and CENTOS_APPLY_NETWORK_CONFIG respectively. Also drop the handling of 'cloudconfig' entries from that function. Handling of cloudconfig is up to the curthooks, and that is being added to maas-images built-in curthooks. The name centos_apply_network_config is left in curtin.cmd.curthooks. We add a unittest to ensures that entry point is never lost. Also add unit test to ensure 'write_files' is available from futil.	Scott Moser	6 years ago
513	tests: in centos_defaults use write_files for grub serial. tests: in centos_defaults use write_files for grub serial. This changes centos defaults to pass a file into the target with write_files and then use that program in late_commands to modify grub1 and grub2 config files so that we get console output to the serial console. Instead of running grub2-mkconfig it just updates both the defaults file and the rendered /boot/grub2/grub.cfg.	Scott Moser	6 years ago
512	write_files: write files after extract, change write_files s... write_files: write files after extract, change write_files signature. Instead of relying on curthooks to write files, write them during 'extract'. The reason for this is that curthooks can be over-ridden by the image that is being installed, but writing files is generic. The motivation for this was that I found centos images did not write files. Also present here is a cleanup of 'write_files' signature and a move from curtin.commands.curthooks to futil. It previously took a dictionary, and operated on only the 'write_files' entry inside it. Now we take that dictionary directly and change callers. The kwarg name 'target' was changed to 'base_dir' to more correctly identify what this is.	Scott Moser	6 years ago
511	pass network configuration through to target for ubuntu and ... pass network configuration through to target for ubuntu and centos curtin will now query the in-target cloud-init for the NETWORK_CONFIG_V2 feature flag. If present, curtin will copy in any custom networking config an defer rendering to cloud-init upon first boot. This function is also implemented for Centos/Enterprise Linux distros. Centos network curthooks has the following capabilities - Only runs when 'centos_network_curthooks: true' is passed in via curtin config. MAAS will modify centos built-in curthooks to use the centos_network_curthooks as needed. - If the target centos image does not contain a new enough cloud-init then the cloud-init-el-stable repo is configured and curtin installs epel-release, cloud-init and bridge-utils if needed Additional VMtests were added for Centos66 and Centos70 covering all of the network tests. Note that network_mtu is disabled as mixed v4 and ipv6 mtu settings are not supported in sysconfig. Other useful features in this branch: - tools/build-deb now lets users set the RELEASE value like: RELEASE=xenial ./tools/build-deb and the RELEASE value will be set in the changelog for the deb	Ryan Harper	6 years ago
510	tests: disable yakkety tests. tests: disable yakkety tests. Ubuntu 16.10 (Yakkety Yak) went EOL on 2017-07-20. Disable the tests for this release.	Scott Moser	6 years ago
509	tools/launch: automatically pass on proxy settings to curtin tools/launch: automatically pass on proxy settings to curtin This makes tools/launch automatically pass on proxy settings http_proxy, https_proxy, no_proxy from its environment into curtin via curtins' newly added 'proxy' config. It also puts the discovered IP address into 'no_proxy'. This means that by default, curtin will get the host environment's proxy settings but will not attempt to use the proxy for the host's IP address. This is done because it is likely that the configured proxy will not proxy traffic back to this host. As an implementation detail, it adds 'curtin_extra' arguments and integrates the previously added '--install-deps' flag into that.	Scott Moser	6 years ago
508	Add top level 'proxy' to config, deprecate top level http_pr... Add top level 'proxy' to config, deprecate top level http_proxy. This allows user to set http_proxy, https_proxy and no_proxy in their config. proxy: http_proxy: http://squid:3128/ https_proxy: http://squid:3128/ no_proxy: 10.0.0.2,127.0.0.1 Also migrates over the old 'http_proxy' top level config to proxy/http_proxy.	Scott Moser	6 years ago
507	tools/curtainer: fix to enable deb-src for -proposed. tools/curtainer: fix to enable deb-src for -proposed. This fixes a bug where: curtainer --proposed --source=out.d would not get source from -proposed, but rather from -updates or release pocket. Additionally, it puts a debug statement in to show the version that it did pull. Also here is removal of some trailing white space.	Scott Moser	6 years ago
506	Use unshare to put chroot commands in own pid namespace. Use unshare to put chroot commands in own pid namespace. Bug 1645680 showed problems when a process started detached and was left running. This starts processes in a chroot in their own pid namespace, so that the process is pid 1. Then, its children are killed when it exits. Also, backed out the work around changes made for 1645680.	Scott Moser	7 years ago
505	vmtest: let the socket level select a port rather than us pi... vmtest: let the socket level select a port rather than us picking one. Previously we had some code in tools/launch to pick a random port from a range of ports. That worked for the most part, but we recently had a failure in vmtest where we failed to select a port in 5 tries. Instead of selecting a port at random and then trying to bind to it, we can ask the socket layer for port 0, and it automatically assigns one. We then modify 'webserv' to output the port it binded and 'launch' to read it.	Scott Moser	7 years ago
504	Fix deb build by mocking call to udevadm settle Fix deb build by mocking call to udevadm settle The sbuild chroot environment does not have the 'udevadm' command. A recent unittest now invokes 'udevadm settle' which is not mocked and failed due to the missing command in the build environment. Fix by adding a mock to the call to udevadm.	Ryan Harper	7 years ago
503	Clear and re-order UEFI boot methods during UEFI grub instal... Clear and re-order UEFI boot methods during UEFI grub installation. Previously when installing Ubuntu using curtin it was default to pass '--no-nvram' to the grub-install. This branch reverts that has passing '--no-nvram' will prevent the system from booting if MAAS is down because Ubuntu not be a loader in the EFI system for the system to fallback on. When updating the nvram grub places Ubuntu before the currently booted method, which prevents the ability for the machine to boot from the network anymore. This branch reorders to boot order of the EFI system to place the currently booted method before all others, but Ubuntu is still second in the list so if MAAS is down the machine will still boot from its local disk correctly. Another issue is that older EFI loaders would be present in the EFI firmware even through curtin deletes and re-creates the entire EFI partition. This removes only those loaders before grub-install is performed to make sure that only the relevant loads for the current state of the system are present. Fixes: LP:#1680917, LP:#1686669	Blake Rouse	7 years ago
502	Resolve bcache race-conditions Resolve bcache race-conditions Creation and removal of bcache devices continues to be somewhat racy preventing reliable installation of bcache configurations. This patch resolves both removal and creation race conditions by checking additional attributes in sysfs and applying reasonable retries with bounded timeouts. In some cases, it is not sufficient to check only that /sys/class/block/bcacheN has been removed to ensure exclusive access to any of the underlying slave devices of bcacheN. In BcacheBasic vmtest, we bind a bcache device to a partition and separate block device and will sometimes not wait long enough for the kernel to release on or more of the slave devices. This patch resolves this by checking the bcache device and all of the slave devices to see if they're still bound by the bcache layer. If so, then we run the shutdown code on the bcacheN block device. We also introduce a udevamd_settle between wiping the cache set device and checking on the backing device. Even though we now check slave devices, there still may be other holders of the device (very likely udev related), thus we introduce a retry loop in clear_holders when we attempt to wipe a volume (which requires exclusive access). During testing, we also uncovered issues which are addressed - Allow strict parameter to be set from the top callers all the way down; this handles the case where a block device may be deleted during the resolution of a devname to a sysfs path.	Ryan Harper	7 years ago

Older »