| | |
- Queue.Queue
-
- PriorityQueue
- __builtin__.object
-
- Coordinator
- Job
- TaskScheduler
- exceptions.Exception(exceptions.BaseException)
-
- JobError
- threading.Thread(threading._Verbose)
-
- BUThread
- FUThread
class Coordinator(__builtin__.object) |
| |
Responsible of the main control of the backup process, this class will
loop receiving in each iteration a possible new file to update and sending
it to the backups threads. Additionally it coordinates other threads such
as the TaskScheduler and the FileUpdater.
|
| |
Methods defined here:
- __init__(self)
- loop(self, callbacks=[])
- The main function of this class: will iterate for ever coordinating
the work of all Threads. Accept a list of callbacks functions
(executing them in FIFO style)
- process_events(self, events)
- Process every event received from inotify watcher. This are the steps
to perform (for every event):
1- validate the event (althoough watcher should send the right
events)
2- instantiate the corresponding Tasks and validate they still
exist
3- perform a globster inclusion to verify we need to backup the
value.
4- build the Job class and pass it to the FUQueue
- stop(self, stop_type='clean', sqlcon_=None)
- Ask the coordinator to stop every thread, all jobs and exit the
loop.
Parameters:
stop_type: accept two values: clean (all current jobs will
finish before stopping) and abort (jobs will be stop abruptly
in whatever state they are before stopping). This method won't
stop from logging the result of all already finished jobs
Data descriptors defined here:
- __dict__
- dictionary for instance variables (if defined)
- __weakref__
- list of weak references to the object (if defined)
|
class FUThread(threading.Thread) |
| |
Thread responsible for receiving the tasks scheduled and performing the
track files update (meaning updating to see if new files has been added in
the tracked paths and returning a list of all files that has changed since
last backup
|
| |
- Method resolution order:
- FUThread
- threading.Thread
- threading._Verbose
- __builtin__.object
Methods defined here:
- __init__(self, Tname, sleep_interval)
- run(self)
- Loop ad infinitum unless it gets a stop signal. Periodically check
TSQueue and find out if a new task has been queued. If so, update its
tracked files info to discover new files and changed files. Put them
on FUQueue and sleep before starting all over again
Methods inherited from threading.Thread:
- __repr__(self)
- getName(self)
- isAlive(self)
- isDaemon(self)
- join(self, timeout=None)
- setDaemon(self, daemonic)
- setName(self, name)
- start(self)
Data descriptors inherited from threading._Verbose:
- __dict__
- dictionary for instance variables (if defined)
- __weakref__
- list of weak references to the object (if defined)
|
class Job(__builtin__.object) |
| |
The object used to communicate files to update between queues.
This objects holds all the data we need to know to perform the task:
job_id: the job identification number
job_tas_id: the task identification number
job_from: the path to the file
jod_file: the name of the file
job_sto_id: the transport instance needed to transfer the backup'ed file
job_to: the directory where to put the file
job_priority: the value needed in the FUQueue Priority Queue to priorize or
not the job.
job_timestamp: the timestamp when this task was scheduled
jod_timestamp: the timestamp when a specific file from the task was scheduled
jod_waf_id: the watched file identification number
jod_backup_date: the date when the backup was performed.
jod_backup_name: the name assigned to the file in the backup repository
jod_log: a message with the final result (success, failure, etc)
|
| |
Methods defined here:
- BU_Data(self, values)
- Use this method to insert fresh BackUp info (see self.BU_attrs for
the attributes required)
- FU_Data(self, values)
- Use this method to insert fresh FileUpdater info (see
self.FU_attrs for the attributes required)
- TS_Data(self, values)
- Use this method to insert fresh TaskScheduler info (see
self.TS_attrs for the attributes required)
- __init__(self, id=None, data=None)
- Data must always be a list of dicts with the attribute:value as
explained in the class doc string. Only job_id will be ignored since it
will be assigned by the db. In case id is assigned, it will be used to
update the job info in the db (e.g. after a backup job was performed)
- job_distribution(self)
- This method reads the event, finds out the job fp and derivates the
job to the given activity
Data descriptors defined here:
- __dict__
- dictionary for instance variables (if defined)
- __weakref__
- list of weak references to the object (if defined)
Data and other attributes defined here:
- BU_attrs = set(['jod_backup_date', 'jod_backup_name', 'jod_file', 'jod_log', 'jod_path', 'jod_timestamp'])
- FU_attrs = set(['job_priority', 'job_sto_id', 'job_tas_id', 'job_timestamp', 'job_to', 'jod_file', ...])
- TS_attrs = set(['job_excluding', 'job_from', 'job_including', 'job_priority', 'job_scheduled_date', 'job_sto_id', ...])
- __fk__ = 'jod_job_id'
- __pk__ = 'job_id'
- __references__ = 'job_detail'
- __table__ = 'job'
- dummy = <class 'barkd.dummy'>
- job_id = None
|
class PriorityQueue(Queue.Queue) |
| |
Variant of Queue that retrieves open entries in priority order
(lowest first).
Entries are typically tuples of the form: (priority number, data)
This class can be found at: Python-2.6a3/Lib/Queue.py
|
| |
Data and other attributes defined here:
- maxsize = 0
Methods inherited from Queue.Queue:
- __init__(self, maxsize=0)
- empty(self)
- Return True if the queue is empty, False otherwise (not reliable!).
- full(self)
- Return True if the queue is full, False otherwise (not reliable!).
- get(self, block=True, timeout=None)
- Remove and return an item from the queue.
If optional args 'block' is true and 'timeout' is None (the default),
block if necessary until an item is available. If 'timeout' is
a positive number, it blocks at most 'timeout' seconds and raises
the Empty exception if no item was available within that time.
Otherwise ('block' is false), return an item if one is immediately
available, else raise the Empty exception ('timeout' is ignored
in that case).
- get_nowait(self)
- Remove and return an item from the queue without blocking.
Only get an item if one is immediately available. Otherwise
raise the Empty exception.
- join(self)
- Blocks until all items in the Queue have been gotten and processed.
The count of unfinished tasks goes up whenever an item is added to the
queue. The count goes down whenever a consumer thread calls task_done()
to indicate the item was retrieved and all work on it is complete.
When the count of unfinished tasks drops to zero, join() unblocks.
- put(self, item, block=True, timeout=None)
- Put an item into the queue.
If optional args 'block' is true and 'timeout' is None (the default),
block if necessary until a free slot is available. If 'timeout' is
a positive number, it blocks at most 'timeout' seconds and raises
the Full exception if no free slot was available within that time.
Otherwise ('block' is false), put an item on the queue if a free slot
is immediately available, else raise the Full exception ('timeout'
is ignored in that case).
- put_nowait(self, item)
- Put an item into the queue without blocking.
Only enqueue the item if a free slot is immediately available.
Otherwise raise the Full exception.
- qsize(self)
- Return the approximate size of the queue (not reliable!).
- task_done(self)
- Indicate that a formerly enqueued task is complete.
Used by Queue consumer threads. For each get() used to fetch a task,
a subsequent call to task_done() tells the queue that the processing
on the task is complete.
If a join() is currently blocking, it will resume when all items
have been processed (meaning that a task_done() call was received
for every item that had been put() into the queue).
Raises a ValueError if called more times than there were items
placed in the queue.
|
class TaskScheduler(__builtin__.object) |
| |
Manages the creation of new schedules for an existing task (only when
Temporal_Basis.teb_validate is True)
|
| |
Methods defined here:
- __init__(self)
- Will retrieve last init timestamp from coordinator_special_event or
insert it in case it does not exist
- calculate(self, how_many=1)
- Given a list of dicts populated with task information, will return
for each of them the next "how_many" scheduled dates (seconds since
epoch)
- next_scheduled_task(self)
- Given a populated instance, it will search for the next task(s)
-a list of tasks in case several of them share the same date-
- populate(self, how_many=None)
- Will query the database and bring all active tasks
If parameter how_many contains an integer it will calculate the next
"how_many" scheduled dates for every task
Static methods defined here:
- next_scheduled_date(cls, last_date=None, how_many=1)
- Given a date (datetime.now() in case None is defined)
return the next scheduled date(s) for a given task. If the
task's TemporalBasis does not need an scheduled data (e.g. On
Demand) it will return None.
Parameters:
cls: a task.Task instance
last_date: a datetime instance, it will be use to calculate the
next date after it
how_many: how many days after the compared date this method must
retrieve.
Data descriptors defined here:
- __dict__
- dictionary for instance variables (if defined)
- __weakref__
- list of weak references to the object (if defined)
Data and other attributes defined here:
- blacklisted = []
- dates = {}
- tasks = {}
| |