~free.ekanayaka/landscape-client/amp-test-readability : changes

~free.ekanayaka/landscape-client/amp-test-readability » Changes from revision 674

From Revision 674 to 655

Rev	Summary	Authors	Date
674	Drop Words class Drop Words class	Free Ekanayaka	11 years ago
673	Merge retry-package-ops [f=1177419] [r=free.ekanayaka,jseutt... Merge retry-package-ops [f=1177419] [r=free.ekanayaka,jseutter] [a=Christopher Armstrong] Package installation failures, including failures to acquire the apt lock, are now retried a number of times with a delay in between.	Christopher Armstron...	11 years ago
672	Merge amp-cleanup-9 [f=1170669] [r=radix,tribaal] [a=Free Ek... Merge amp-cleanup-9 [f=1170669] [r=radix,tribaal] [a=Free Ekanayaka] Alright, one of the last episodes of the AMP cleanup saga. In this branch: - Drop landscape.lib.amp.RemoteObjectConnector, as most of its functionality has no been pushed down to MethodCallClientFactory which is a more standard Twisted primitive more importantly and is transport-agnostic, making landscape.lib.amp really generic. - Convert landscape.lib.tests.test_amp.RemoteObjectConnectorTest to MethodCallFunctionalTest. The RemoteObjectConnectorTest class was exercising the (now gone) RemoteObjectConnector in a functional-like way, that means spinning the real twisted reactor and listening/connecting to real Unix sockets. Even if from a strict unit-test point of view all the functionality in landscape.lib.amp is already tested, I felt it was nice to still have some functional-like coverage mainly to protect us against regressions or API changes in the Twisted machinery that we depend on. I've used @inlineCallbacks to make tests a bit more readable.	Free Ekanayaka	11 years ago
671	Rephrased comment. [trivial] Rephrased comment. [trivial]	Christopher Glass	11 years ago
670	Merge amp-cleanup-8 [f=1170669] [r=tealeg,ack] [a=Free Ekana... Merge amp-cleanup-8 [f=1170669] [r=tealeg,ack] [a=Free Ekanayaka] Some more cleanup around the AMP-based machinery: - landscape.amp.ComponentConnector: the disconnect method now uses Twisted's IConnector.disconnect(), without messing with the lower-level ITransport.loseConnection(). The objects implementing IConnector are the ones returned by reactor.connectUNIX/connectTCP/connectSSH and afaiu they're what is supposed to be used for terminating a connection. - rename RemoteComponentRegistry to ComponentRegistry, to match ComponentPublisher and ComponentConnector - landscape.configuration: the changes here are due because we know use IConnector.disconnect() vs ITransport.loseConnection(). Unfortunately the code in this module is factored and tested in a far less than ideal way, eventually we should take time to clean it up. - landscape.lib.amp.RemoteObject: update the constructor docstring and reduce a bit of logic duplication by adding a new _send_method_call helper method, which is now used by __getattr__ and _retry (this makes _retry flush the fake asynchronous connection used in tests, in the same way __getattr__ does). - landscape.lib.amp.MethodCallServerFactory: rename the "reactor" init parameter to "clock", since only the IReactorTime interface is needed (in this case instances are normally called "clock" and can be mocked with a twisted.internet.task.Clock instance in tests) - landscape.lib.tests.test_amp: Add a FakeConnector class implementing IConnector, so we can use it unit-tests. Also, I've converted a few deferred-returning (asynchronous) tests to synchronous equivalent. - landscape.tests.test_amp: Concerted asynchronous tests to synchronous equivalent.	Free Ekanayaka	11 years ago
669	Merge reboot-after-message [r=free.ekanayaka,ack] [f=1176121... Merge reboot-after-message [r=free.ekanayaka,ack] [f=1176121] Delay rebooting until after the package operation has been responded to (plus 5 seconds for safety). This avoids a race condition where the "shutdown" command would kill landscape-client before the message could be saved to disk by the broker.	Christopher Armstron...	11 years ago
668	Drop our copy of initgroups.c as it's only relevant for Pyth... Drop our copy of initgroups.c as it's only relevant for Python < 2.7 and it's fixed in Lucid's Twisted which uses 2.6 [trivial] [r=ack]	Free Ekanayaka	11 years ago
667	Merge amp-cleanup-7 [f=1170669] [r=tribaal,tealeg] [a=Free E... Merge amp-cleanup-7 [f=1170669] [r=tribaal,tealeg] [a=Free Ekanayaka] This branch is simplifies the setup methods of all our test helpers and test cases using AMP connections, by not making them return deferreds (and hence making twisted.trial.unittest.TestCase eventually spin the reactor, see http://twistedmatrix.com/documents/current/core/howto/trial.html#auto7).	Free Ekanayaka	11 years ago
666	Merge amp-cleanup-6 [f=1170669] [r=tribaal,radix,tealeg] [a=... Merge amp-cleanup-6 [f=1170669] [r=tribaal,radix,tealeg] [a=Free Ekanayaka] Another episode of the AMP-related cleanup: - Rename landscape.amp.RemoteComponentConnector to simply ComponentConnector (the fact that it's remote is implied by the fact that you connect to it, and the new name complements the existing ComponentPublisher nicely) - Drop the landscape.amp.ComponentProtocolClientFactory sub-class and use MethodCallClientFactory directly, tweaking the needed instance attributes with explicit assignements (this is a sort of composition-over-inheritance change) - Drop landscape.reactor.UnixReactorMixin. The landscape.reactor.FakeReactor class now really fakes the Unix socket transport, by re-using the machinery in landscape.lib.tests.test_amp, instead of delegating the behavior to the real twisted reactor (hence making it not fake at all) - Update tests that were using FakeReactor.listen_unix/connect_unix (directly or via helpers) to be synchronous. This means essentially using FakeReactor.advance to simulate time progresses and trigger scheduled calls. - Make landscape.lib.amp.RemoteObject aware of fake transports, so it can flush it transparently. This is a hacky workaround to the fact that twisted's AMP code doesn't like synchronous transports, and without it we'd need to explicitly flush the fake transport buffers explicitly all the times in a test we communicate with a remote component connected via a fake transport. - Drop landscape.service.LandscapeService.socket instance attribute, as LandscapeService-derived classes now all use the new landscape.amp.ComponentPublisher convenience introduced in former branches, that takes care of that.	Free Ekanayaka	11 years ago
665	Merge amp-cleanup-5 [f=1170669] [r=radix,tealeg] [a=Free Eka... Merge amp-cleanup-5 [f=1170669] [r=radix,tealeg] [a=Free Ekanayaka] This branch continues the AMP machinery cleanup started in former branches. In particular: - Drop the landscape.amp.ComponentProtocol sub-class and all its sub-classes, which is now not needed since the methods to be exposed are now passed to landscape.amp.ComponentPublisher - Rename ComponentProtocolFactory to ComponentProtocolClientFactory, since we now use 2 different factories for the client and server sides (they are asymmetric and implement different logic) - Use a uuid instead of a sequence number to track in-progress MethodCall. This fixes a latent bug that probably never manifested itself (i.e. monitor and manager connecting to the broker and independently sending the same sequence number for different MethodCalls) - Drop MethodCallProtocol which has become unused, along with some other temporary code not used anymore.	Free Ekanayaka	11 years ago
664	Merge amp-cleanup-3 [f=1170669] [r=radix,tealeg] [a=Free Eka... Merge amp-cleanup-3 [f=1170669] [r=radix,tealeg] [a=Free Ekanayaka] This is another branch in the a series that will make most client tests synchronous. In detail: - Add a landscape.amp.ComponentPublisher class, which is a convenience for Landscape client code around the generic low level landscape.lib.amp machinery. It basically supports publishing a Python object over a Unix socket using AMP. It is the server-side equivalent of RemoteComponentConnector (see next point). - Make landscape.amp.RemoteComponentConnector not inherit from landscape.lib.RemoteObjectConnector. The latter should be eventually dropped, as it's transport-specific (it hard-codes using Unix socket), while the machinery in landscape.lib.amp should be transport-agnostic, providing plain Twisted factories and protocols. It is a sort of replace-inheritance-with-composition change. - Create sub-classes of ComponentPublisher for each Landscape service that publishes an object, this replaces the low-level sub-classing of protocols that we add before. - Add a MethodCallServerFactory class, which is a little convenience around AMP, mainly usedfor instantiating an AMP protocol with the appropriate "locator" parameter, that will responde to L{MethodCall} commands send by clients.	Free Ekanayaka	11 years ago
663	Merge amp-cleanup-2 [f=1170669] [r=ack,tealeg] [a=Free Ekana... Merge amp-cleanup-2 [f=1170669] [r=ack,tealeg] [a=Free Ekanayaka] This branch is a step toward the goal of fully-synchronous tests in the client. In detail: - Drop the retry_on_reconnect parameter of the RemoteBroker constructor, and stick it to MethodCallClientFactory instead. The idea is to make the factory a sort of controller of RemoteBroker. Since user code is supposed to deal with factories directly, the can control the RemoteBroker's behavior by modifying the factory. - Move the MethodCallFactory class after the RemoteBroker class, since we now need to set the latter as 'remote' class attribute of the former. - Rename MethodCallFactory to MethodCallClientFactory, since this is really a protocol factory for clients, not for servers. - Implement a new interface for MethodCallClientFactory, which should be more convenient for user-code. This includes adding a getRemoteObject method and replacing the notify_success and notify_failure methods with a single notifyOnConnect (camel case for consistency with Twisted).	Free Ekanayaka	11 years ago
662	Merge amp-cleanup [f=1170669] [r=tealeg,ack] [a=Free Ekanaya... Merge amp-cleanup [f=1170669] [r=tealeg,ack] [a=Free Ekanayaka] This is almost a trivial change, in preparation of some follow-up cleanup work. It's mostly about cosmetics and little details in landscape.lib.amp: - Change the module docstring to reflect how this machinery's API will change (note that it hasn't been implemented yet tho). The new style is a bit more Twisted-y, and doesn't force the use to use unix sockets. - MethodCallSender.send_method_call now returns directly the result of the remote method invokation, rather then MethodCall.response low-level dict. Tests have been updated accordingly. - MethodCallSender.protocol is now public, and can be set by calling code.	Free Ekanayaka	11 years ago
661	Drop the legacy data upgrader, which was needed for very old... Drop the legacy data upgrader, which was needed for very old clients (earlier than dapper) [trivial] [r=tealeg]	Free Ekanayaka	11 years ago
660	Merge testable-amp [f=1169102] [r=ack,tealeg] [a=Free Ekanay... Merge testable-amp [f=1169102] [r=ack,tealeg] [a=Free Ekanayaka] This is a first step towards having synchronous tests for AMP-related code: - Use composition vs inheritance, so MethodCallServerProtocol has now become MethodCallReceiver and MethodCallClientProtocol has become MethodCallSender. They don't inherit from the base AMP protocol anymore, this will let us mock the AMP protocol itself later on. - Put in place temporary workarounds (marked with "XXX") for the few breakages that such refactoring causes. These will be cleaned up in a follow-up branch. - Changed most of the tests in l/lib/tests/test_amp.py to be synchronous and not use the real Twisted reactor. This is done by faking the transport, so we can simulate a connection rather than use real Unix sockets. These methods don't need to return deferreds anymore in order to perform their assertions, as they can trust that everything happens synchronously.	Free Ekanayaka	11 years ago
659	Drop out-of-date Twisted trial hacks (fixed since Twisted 2.... Drop out-of-date Twisted trial hacks (fixed since Twisted 2.5, in hardy) [trivial] [r=cglass]	Free Ekanayaka	11 years ago
658	Add successResultOf and failureResultOf to LandscapeTest, ta... Add successResultOf and failureResultOf to LandscapeTest, taken from Twisted 12.3.0 [trivial] [r=cglass]	Free Ekanayaka	11 years ago
657	Merge reactor-cleanup [f=1168315] [r=tealeg,ack] [a=Free Eka... Merge reactor-cleanup [f=1168315] [r=tealeg,ack] [a=Free Ekanayaka] This branch is actually a trivial change: - Improve docstrings everywhere, mentioning the surprising behavior around the use of listen_unix in FakeReactor. - Drop ThreadedCallsReactorMixin, which was meant to hold common functionality between TwistedReactor and FakeReactor, but had none. Its call_in_thread method has been moved to TwistedReactor while call_in_main to FakeReactor, where they are actually used. - Move FakeReactor after TwistedReactor in the reactor.py file. - Re-arrange the order of method definitions in both FakeReactor and TwistedReactor so they match. - Drop udp-related methods, which we don't use.	Free Ekanayaka	11 years ago
656	Better docstrings for broker test helpers [trivial] [r=teale... Better docstrings for broker test helpers [trivial] [r=tealeg]	Free Ekanayaka	11 years ago
655	Merge exchange-token [f=1161856] [r=tribaal,ack] [a=Free Eka... Merge exchange-token [f=1161856] [r=tribaal,ack] [a=Free Ekanayaka] This branch implements the client-side part of the changes needed to detect cloned computers. The idea is that at each exchange the server will give the client a token that the client will be required to give back at the following exchange to prove that it's still the same client. If two clients with the same secure ID try to perform an exchange using the same token, only the first one that completes the exchange will succeed, because the second will not be authenticated since the token has changed in the meantime (after the first exchange completed).	Free Ekanayaka	11 years ago

Older »