~time-drive-devel/time-drive/time-drive.sync : changes

~time-drive-devel/time-drive/time-drive.sync » Changes from revision 101

From Revision 79 to 60

Rev	Summary	Authors	Date
79	Modified timedrive-app.py and realtime methods to allow it t... Modified timedrive-app.py and realtime methods to allow it to run a check of files that have been modified locally as part of the program setup. The same routine can be repeated at a specified interval (likely to be set to every 5 minutes once all aspects of the program are working within tolerance).	Rob Oakes	11 years ago
78	Added initial support for real-time file sync using mdate an... Added initial support for real-time file sync using mdate and folder driven comparison, rather than file driven comparison. Todo: Create real-time methods that are able to call the new remote and local folder difference methods.	Rob Oakes	11 years ago
77	Fixed an issue with the update folder methods which were cau... Fixed an issue with the update folder methods which were causing the records for top-level folders to fail.	Rob Oakes	11 years ago
76	Initial implementation of real-time folder sync fow ownCloud... Initial implementation of real-time folder sync fow ownCloud 5. The algorithm makes use of folder eTags to see when a file has changed. If the toplevel folder eTag value remains unchanged from the last sync, then there have been no updates to the contents of the folder. ownCloud5: Modified the method used for retrieving the contents of an ownCloud directory so that they can include folder listings. syncAction: Updated the download file actions to also keep track of etag values.	Rob Oakes	11 years ago
75	Modified a bug in timedrive-app.processResults which was pre... Modified a bug in timedrive-app.processResults which was preventing the cueing of subsequent uploads/remote operations.	Rob Oakes	11 years ago
74	Continued work with exception and error handling. Started re... Continued work with exception and error handling. Started refactoring to move to a folder driven sync algorithm, rather than continuing to use the file drive sync algorithm used now.	Rob Oakes	11 years ago
73	Continued work on improving error and exception handling in ... Continued work on improving error and exception handling in background libraries. Created two new error classes AuthenticationError and ConnectionError, which can be used to manage different sorts of real-time failures.	Rob Oakes	11 years ago
72	Added preliminary support for deleting files to the real-tim... Added preliminary support for deleting files to the real-time sync client. Refactored common database operations into methods for easier condition checking. Todo: - Test offline support for deleting files - Debug dav.py methods for downloading remote file attributes - Make sure that the lower level sync methods follow the same API so that they are able to successfully detect when an operation has occurred and when it has not.	Rob Oakes	11 years ago
71	Initial version of upload queue and error library. Real-time... Initial version of upload queue and error library. Real-time sync client can detect when it is offline and has rudimentary support for determining why the remote server (owncloud only) may be rejecting authentication requests. To-do: - Implement the processing of downloads - Implement file renames, deletes - Implement a better method for uploading a file when it is locked by a program - Implement sync algorithm that is driven by folders rather than individual files	Rob Oakes	11 years ago
70	Started work on verifying that file uploads and downloads ha... Started work on verifying that file uploads and downloads have been completed successfully. Added errors module to sync to better detect certain types of failures.	Rob Oakes	11 years ago
69	Continued work on real-time sync module. Finished queueing o... Continued work on real-time sync module. Finished queueing of uploads. Started work on improving backend functions through addition of custom errors.	Rob Oakes	11 years ago
68	Continued work on real-time sync client. Solidified the queu... Continued work on real-time sync client. Solidified the queue method for processing uploads and downloads. Draft version of API. API, Local Files: Name of collection, action, and the local file path as tuple API, Remote Files: Name of collection, action, local file path as tuple	Rob Oakes	11 years ago
67	Started work on multithreading of real time sync application... Started work on multithreading of real time sync application. Added methods for managing the sync queue. Started moving real time checks to a separate background class for code reuse. To-do: 1. Determine the structure of the API, taking into account that there needs to be an easy way to cache the action should the transfer fail. Directly caching Qthreads seems like a waste of resources. 2. Start work on a way to save the transfer list to disk so that it can persist between network states.	Rob Oakes	11 years ago
66	Continued work on real time synchronization application. Fix... Continued work on real time synchronization application. Fixed a bug in how Unix timestamps were being calculated that caused file timestamps to be off by seven hours.	Rob Oakes	11 years ago
65	Enabled syncFolder to work recursively. Fixed a bug in backe... Enabled syncFolder to work recursively. Fixed a bug in backend which was preventing files from being delted correctly.	Rob Oakes	11 years ago
64	Worked on ownCloud5 backend. Added methods to create a remot... Worked on ownCloud5 backend. Added methods to create a remote file list without a manifest. Cleaned up the sync methods to remove references to the live server list. The server list has bee folded into the remote file list. Added methods for the remote file object to update its properties back to the server.	Rob Oakes	11 years ago
63	Moved funtions related to the creation of the manifest and g... Moved funtions related to the creation of the manifest and generation of the remote file list to the backend. This allows for each of the backends to hae a separate implementation and for the sync algorithm to work on only two points of data: the remote filelist and the local file list. Todo: - For the dav.py backend, create a routine which will reconcile the live server filelist with that in the manifest (should happen on a folder by folder basis, pushed by the local machine) - For the ownCloud backend, implement a routine which will pull down the remote list folder by folder. Create a fall-back procedure if information about checksums is required to reconcile a particular problem.	Rob Oakes	11 years ago
62	Started adding support for etags (a unique identifier) to th... Started adding support for etags (a unique identifier) to the various parts of Time-Drive: - New property to local and remote which tracks the identifier. (Destroys backward compatability with older databases.) - Added support to dav.py for parsing the WebDav XML for the property. - Started to move routines for generating a remote file list, downloading the manifest, and cleanup actions into the specific backends as the way that these actions are performed will vary greatly across different backends.	Rob Oakes	11 years ago
61	Added an additional tag to parseRemoteResource that is able ... Added an additional tag to parseRemoteResource that is able to detect etags (ownCloud 4.5+). This property can be used as a reliable indicator of whether the remote resource has changed or not (similar to the md5 checksum). For servers that support it, the etag identifier removes the need for a manifest, thereby makiny the sync a two-way comparison instead of a three-way comparison. eTags also exist for folders. This can be a very quick way to determine if there have been any folder level changes, thereby speeding up the synchronization operation considerably. Todo: - Implement the etag property into the sync routine for the ownCloud 5 provider.	Rob Oakes	11 years ago
60	Moved dav.py to a new folder called backends. Started work o... Moved dav.py to a new folder called backends. Started work on an owncloud specific backend. To-do: - Add methods to the backend which handle the creation of a manifest. - Restructure sync so that the manifest is created as part of a "cleanup" routine and that the sync operation can run with/without a manifest being present.	Rob Oakes	11 years ago