~jonathanmcdougall/baretorrent/trunk

This is about developing extensions for baretorrent.

1 Introduction
==============
Extensions are python scripts imported at runtime by baretorrent.
Scripts can register functions that will be called for specific events
(such as when a torrent is added) or call functions directly into
baretorrent.

There is currently no way to modify the user interface because I am not
able to bundle wxPython yet. Working on it.


2 Installing
============
There are two ways of making an extension available in baretorrent:
using the regular install from the Tools menu or by specifying paths in
a file. The former is used by end-users and copies the extension files
into the user's directory, which is fine for users, but probably not for
developers.

Developers can create a file "dev.txt" in the configuration directory
and have a list of paths pointing to directories that contain main.py.
The conf directory varies by platform:

. windows: c:\Users\name\AppData\Roaming\baretorrent\extensions
. linux: /home/name/.baretorrent/extensions
. osx: /Users/name/Library/Application Support/baretorrent/extensions

As an example, an extension might be in /home/user/dev/my-ext/ while
developing. Adding a file /home/user/.baretorrent/extensions/dev.txt
that contains this path on the first line will add the extension to
baretorrent as if it was installed normally.

Such extensions are called "dev extensions".

Dev extensions differ from regular ones in that they cannot be
uninstalled from the interface since they need to be removed from
dev.txt. They also get a "reload" button in the extension list in the
preferences that will do the equivalent of an uninstall and reinstall
without having to restart the program. Pressing F2 does the same thing
for all dev extensions.

3 Structure
===========
Two files are required in the directory: btx.txt and main.py. The former
contains the extension description and the latter the actual code.

The reason for separating the two is that the description is shown to
the user prior to the installation as a confirmation. If the description
was part of the script, it would need to be imported and executed even
though the user hasn't agreed yet. The description file is therefore a
simple text file. The scripts are never run unless the user installs the
extension.

3.1 btx.txt
-----------
This is a list of "key = value" lines. Lines starting with # are
ignored. For example, from the httpd extension:

guid = E5258701-DF85-4CA9-819F-E33F065C4E55
name = httpd
author = Jonathan McDougall
short_description = Handles web requests
long_description = Listens on a port for incoming requests ...
version = 0.2
email = jonathanmcdougall@gmail.com
website = https://sourceforge.net/projects/baretorrent/
requires = 0.4.3

The guid and name fields are mandatory. The following is a description
of the various fields:

. guid
  This is a sequence of hexadecimal digits separated into five groups
  without braces

. name
  The display name of the extension

. author
  The name of the author

. short_description
  A one-liner that describes the extension.

. long_description
  A longer description. This is displayed in a textbox with a vertical
  scrollbar.
  
. icon
  Path to a 64x64 icon relative to the root of the extension. If this is
  not specified, a file "icon.png" is used if it exists. This is
  displayed when this extension is selected in the options.

. small_icon
  Path to a 16x16 icon relative to the root of the extension. If this is
  not specified, a file "icon_16.png" is used if it exists. This is
  displayed in the extensions list next to names.
  
. version
  Version of the extension in the form of "major.minor.maintenance".
  Both minor and maintenance are optional. This field is important
  (although optional) because it determines whether an extension is a
  newer version or not and is displayed as such on the interface when
  the user installs it.

. email
  Email address of the author. Specifying this will make the author
  name clickable in the description.

. website
  URL of the extension's website. This will a clickable link in the
  description.

. requires
  The lowest version of baretorrent supported by this extension in the
  form of "major[.minor[.maintenance]]"; if not specified, no version
  check is done.

3.2 main.py
-----------
This is the script that is imported by baretorrent when the extension
is loaded. It can be empty, as long as it exists. See below for
predefined functions.


4 Script
========

4.1 Available modules
---------------------
The standard python modules are all available along with two more:
baretorrent and libtorrent. The latter makes the whole of libtorrent
available to scripting.

Baretorrent uses Boost.Python to interact with python scripts. It
exports several functions and classes for itself, along with libtorrent.
All of this is in the dynamic library (baretorrent.dll on windows, .so
on posix.)

Python on posix platforms can import a .so directly, so there is no need
for an additional layer. On windows, however, python needs modules to
have the pyd extension. Since both baretorrent and python need the dll
and windows needs a .dll extension, an additional small dll called
baretorrent.pyd exists that merely calls into baretorrent.dll.

The additional libtorrent.pyd/so file is so that libtorrent can be
imported through that name instead of through 'baretorrent'. Therefore:

                dependencies on posix platforms

                        baretorrent.so
                        /           \
                       /             \
              baretorrent            libtorrent.so


                    dependencies on windows

                        baretorrent.dll
                        /     |     \
                       /      |      \
              baretorrent.exe |      libtorrent.pyd
                              |
                       baretorrent.pyd
                        (windows only)

With

  import baretorrent

on posix, baretorrent.so will be imported. On windows, baretorrent.pyd
will be imported, which depends on baretorrent.dll, so the effect is the
same. With

  import libtorrent

on posix, libtorrent.so will be imported, which depends on
baretorrent.so. On windows, the same thing happens with libtorrent.pyd
and baretorrent.dll.


4.2 Hooks
---------

After importing main.py, baretorrent will call init() if it exists. Do
_not_ call into baretorrent or libtorrent in main.py when importing. If
anything from baretorrent or libtorrent is needed on startup, this
should be done in init(). For example,

  # main.py
  import baretorrent
  
  baretorrent.add_torrent("url")

will fail because baretorrent hasn't finished loading up the extension
yet, it has more bookkeeping to do after importing, after which it will
call init(). Doing this will disable your extension right away.

This will work:

  # main.py
  import baretorrent

  def init();
    baretorrent.add_torrent("url")

(although the user probably won't be happy.)

On shutdown, baretorrent will call cleanup().


4.3 Events
----------
Extensions can register for certain predefined events (such as
"torrent.added" or "option_changed") and have functions called when they
are fired. These callbacks should be kept short because they may be
called from baretorrent's user interface thread and therefore would
block the gui.

To register an event, baretorrent.register("name", "function") should be
called, probably from init(). baretorrent.unregister("name", "function")
can also be called.


4.4 Options
-----------
Extensions should try not to hardcode options if possible (or if it
makes sense.) Once I figure out something for modifying the user
interface, extensions will be able to create user-friendly dialogs for
options. In the meantime, the only way to modify options is from the
list of key/value in the preferences.

There are three different functions relating to options: set_opt(),
get_opt() and add_opt(). The latter is used to add new options to the
database with a default value. The option will only be added if it
doesn't exist yet. This should be called in install() instead of doing a
get_opt()/set_opt() check in init() since get_opt() will return an empty
string if the option doesn't exist, making it hard to give a default
value. add_opt() also allows giving a default value, which is used from
the advanced options so the user can reset the original value.


4.5 Behaving
------------
. make sure uninstall() restores baretorrent to what it was without
  your extension; repeatedly enabling/disabling the extension without
  restarting baretorrent should behave correctly and predictably

. return quickly from events or the user will be unhappy; be slow too
  many times and the user will be shows a warning that your extension
  sucks, along with a "disable" button

. this is a python script that is not in a sandbox and therefore has
  the same permissions as baretorrent itself does; be nice

. don't phone home, don't spy, don't show ads, don't track, don't
  report or god will strike you down; respect the user's privacy

  
4.5 Thread safety
-----------------
Don't call into baretorrent from two different threads at the same time.
It doesn't matter from which thread calls are made, as long as they can
never overlap.


5 References
============

In the following, "callback(...)" is the signature of the function that
is passed to baretorrent.register(). 

5.1 Events
----------
In the following, the 'torrent' parameter is a baretorrent.torrent
object.

. "torrent.added", callback(torrent)
  fired when a torrent is added, whether by the user or extensions
  (including the one that did the adding)

. "torrent.error", callback(torrent)
  fired when a torrent becomes errored; torrent.error() can be used
  to retrieve a boost::system::error_code

. "torrent.metadata", callback(torrent)
  fired when a torrent gets the metadata; this can be fired in various
  circumstances, such as when torrents are resumed on startup,
  immediately after "torrent.added" when a .torrent is added (because
  the metadata is part of the .torrent) or when metadata is received
  from a peer for a magnet link

. "torrent.complete", callback(torrent)
  fired when a torrent is completed and begins seeding

. "torrent.file_completed", callback(torrent, file)
  fired everytime a file is completed in the given torrent; 'file'
  is a libtorrent.file_entry.

. "torrent.updated", callback(torrent)
  fired regularly when anything about a torrent changes (progress,
  queue position, errored, etc.)

. "torrent.ready", callback(torrent)
  fired when a torrent is ready to download or seed (libtorrent has
  it, metadata is available, etc.)

. "option_changed", callback(name, value)
  fired when an option for this extension has been changed by the
  user; this may be called multiple times in succession if the user
  changes several options before clicking ok

. "icon_clicked", callback()
  fired when the extension has added an icon in the status bar (see
  5.2.10, add_icon()) and the user has clicked it
 

5.2 baretorrent API
-------------------
All of the following are in the baretorrent module. All the strings
are in utf-8.

5.2.1 Miscellaneous

. quit()
  quits baretorrent

. version()
  returns baretorrent's version as an object with the following
  properties:
    . major
      major version number as an integer, such as 0 for 0.2.3-alpha

    . minor
      minor version number as an integer, such as 2 for 0.2.3-alpha

    . maintenance
      maintenance version number as an integer, such as 3 for
      0.2.3-alpha

    . tag
      version tag as a string, such as "alpha" for 0.2.3-alpha; may be
      empty

    . build_date
      build date and time as a string with an unspecified format that
      depends on the compiler and platform; don't try parsing it
	  
    . string
      version as a string, such as "0.2.3-alpha"

. make_loc_version()
  returns a localized string that contains baretorrent's version and
  build date; this is the strin used in the about box
  
5.2.2 Logging
Although print() can be used, using baretorrent's logging is
recommended: it will prefix strings with the date and time along with
the extension's name; it may be logged into a file (or wherever else) if
the user wants it; and it may be used in the future to log strings
somewhere in the user interface.

There are three different logging levels: normal, verbose and very
verbose. The latter should be as chatty as possible, while the other
two should not be logging recurring events (such as network connections
or torrent updates, for example.) The level output in the console (or
perhaps also on the user interface) is set by the user.

Logging the string "Lorem ipsum" from an extension called "test" would
output something like this in the console:

[12/18 22:53:38] [extension test]: lorem ipsum

The exact format is unspecified, don't rely on it. Do not include a \n
at the end, this is added automatically.

. log(string)
  logs the given string

. log_error(string)
  logs the given error string; this behaves like log(string) but could
  for example make the text red in the console (the actual behaviour
  is unspecified)

  apart from logging the string, this has no effect on baretorrent and
  will not be shown to the user nor will it disable or fiddle with the
  extension

. log_v(string)
  logs a verbose string

. log_vv(string)
  logs a very verbose string

5.2.3 torrent
The torrent class (along with the manager) is the core of baretorrent.
It wraps a libtorrent.torrent_handle object (which is also exposed), but
baretorrent.torrent's functions should be used instead if possible.

. torrent.total_size()
  returns the total size in bytes of this torrent as an integer; see
  libtorrent.torrent_status.total_wanted

. torrent.handle
  returns the libtorrent.torrent_handle for this torrent
  
. torrent.downloaded()
  returns True if the torrent has been downloaded and is now seeding

. torrent.added()
  returns True if libtorrent has acknowledged this torrent; there may be
  a delay between when a torrent is added with manager.add_torrent()
  and when libtorrent acknowledges it

  this corresponds to the torrent.added event

. torrent.errored()
  returns True if this torrent is in an error state (such as if resuming
  failed or the drive is full)

. torrent.error()
  returns a libtorrent.error_code object

. torrent.start()
  starts this torrent; no-op if the torrent is already started

. torrent.pause()
  pauses this torrent; no-op if the torrent is already paused

. torrent.up()
  moves the torrent up one in the queue; no-op if the torrent is already
  on top

. torrent.down()
  moves the torrent down one in the queue; no-op if the torrent is
  already at the bottom

. torrent.top()
  moves the torrent to the top of the queue; no-op if the torrent is
  already at the top

. torrent.bottom()
  moves the torrent to the bottom of the queue; no-op if the torrent is
  already at the bottom

. torrent.name()
  returns the display name of the torrent; its content depends on the
  state of the torrent:
    1) for newly added torrents, this is constructed from the
       stem of the path (for files or urls), the dn field (for
       magnets) or the hash (btih) if dn wasn't in the url
    2) once libtorrent receives metadata for this torrent (either
       immediately for .torrent files, after downloading the
       .torrent from the network or after receiving metadata from
       peers for magnet links), this uses whatever libtorrent
       returns (see libtorrent.torrent_handle.name())
    3) for torrents that are resumed, name() was saved in the xml
       file and so is reused until libtorrent loads the metadata
       (which should be immediately anyways)

. torrent.hash()
  returns a hex string for the torrent's hash; its content depends on
  the state of the torrent:
    1) for newly added torrent, this comes from the content of
       the .torrent file or the btih field of a magnet link; in
       case of a .torrent that needs to be fetched, hash() will
       return an empty string until metadata has been received
    2) once libtorrent receives metadata for this torrent, this
       uses whatever libtorrent.torrent_handle.hash() returns
    3) for torrents that are resumed, hash() was saved in the xml
       file and so is reused until libtorrent loads the metadata
       (which should be immediately anyways)

. torrent.url()
  returns the url used to fetch the metadata, which is either a url to
  a .torrent or a magnet link; in case of a local file, this is always
  empty

. torrent.status()
  returns one of the torrent.statuses enum values

. torrent.sha1_hash()
  returns a libtorrent.big_number

. torrent.save_path()
  returns the path where the files in this torrent are being saved

. torrent.position()
  returns the queue position as an integer

. torrent.file_progress()
  returns the number of bytes downloaded for each file in the
  torrent as a list, such as [42, 55]; libtorrent.torrent_info.file_at()
  and libtorrent.file_entry.size can be used to get the size of the file

. torrent.file_priorities()
  returns the queue position for each file in the torrent as a list,
  such as [2, 0, 1]

. torrent.file_priorities(list)
  sets the queue position of all the files in the torrent; make sure the
  number of elements in the list is the same as the number of files in
  the torrent

. torrent.trackers()
  returns a list of libtorrent.announce_entry, one for each tracker used
  by the torrent

. torrent.trackers(list)
  sets the list of trackers used by the torrent; each element should be
  a libtorrent.announce_entry

. torrent.add_tracker(string)
  adds the given tracker url to the list
  
. torrent.peers()
  returns a list of libtorrent.peer_info that are connected to this
  torrent

. torrent.torrent_status()
  returns a libtorrent.torrent_status object

. torrent.eta()
  returns the number of seconds before this torrent is completed; this
  may be 0 if the torrent is seeding or if it its speed is too slow

. torrent.statuses
  this is an enum for the possible states of the torrent
   
    . torrent.statuses.paused
    the torrent is paused
	
	. torrent.statuses.stopped
	the torrent has been seeded to its max ratio
    
    . torrent.statuses.checking
    the torrent failed to resume properly and is now rechecking
    
    . torrent.statuses.resuming
    the torrent's fastresume file is currently being read
    
    . torrent.statuses.starting
    the torrent has been added but not acknowledged by libtorrent yet
    
    . torrent.statuses.queued
    the torrent is paused because other torrents currently have priority
    
    . torrent.statuses.metadata
    the torrent is currently waiting for metadata
    
    . torrent.statuses.stalled
    the torrent is downloading, but is not receiving anything from peers
    
    . torrent.statuses.downloading
    the torrent is downloading data
    
    . torrent.statuses.seeding
    the torrent is completed and may or may not be uploading data
    
	. torrent.statuses.moving
	the torrent is currently being moved
	
    . torrent.statuses.removing
    the torrent has been removed but libtorrent isn't done with it yet
    (such as when deleting the files, which may take a while)
    
    . torrent.statuses.bad_save_path
    error state where the save path for this torrent is not accessible
    anymore
    
    . torrent.statuses.resume_failed
    the fastresume data for this torrent couldn't be loaded and the
    torrent will need to be started manually; this state is carried
    through restarts
	
	. torrent.statuses.move_failed
	the torrent couldn't be moved	
    
    . torrent.statuses.generic_error
    any other error; use torrent.error()

. torrent.priorities
  this is an enum for the default priorities; although any integer may
  be used, these correspond to the context menu in the files panel
  
    . torrent.priorities.skip
      the file is not downloaded

    . torrent.priorities.normal
      default priority

    . torrent.priorities.high
      high priority

. status_to_loc(status)
  returns a localized string for the given status, such as "resuming"
  or "paused"

5.2.4 manager
The manager handles the list of torrents and wraps a libtorrent.session
object (which is also exposed.)

. manager.size()
  returns the number of torrents

. manager.empty()
  return True if there are no torrents

. manager.get(index)
  returns a torrent object at the given index; don't call this with an
  invalid index

. manager.add_torrent(string)
  adds a new torrent; the given string may be a url to a .torrent, a
  magnet link or the path to a local file

. manager.remove_torrent(torrent, keep_files)
  removes the torrent without warning the user; if keep_files is False,
  the downloaded files are also deleted

. manager.session()
  returns the libtorrent.session

. manager.download_rate()
  returns the total number of bytes per second downloaded for all
  torrents

. manager.updload_rate()
  returns the total number of bytes per second uploaded for all torrents

. get_manager()
  returns baretorrent's global manager object; there is only one

5.2.5 config

. config.windows()
  returns True if this platform is windows

. config.apple()
  returns True if this platform is OSX

. config.linux()
  returns True if this platform is any flavor of linux

. config.posix()
  returns True if this platform is posix (anything but windows)

. config.debug()
  returns True if baretorrent has been compiled in debug mode

. config.target_is_utf8()
  returns True if the native encoding for this platform is utf8

. config.target_is_utf16()
  returns True if the native encoding for this platform is utf16

. config.x86()
  returns True if this platform is 32-bit

. config.x64()
  returns True if this platform is 64-bit

5.2.6 utility

. urlencode(string)
  url-encodes the given string; from curl: "all characters that are not
  a-z, A-Z, 0-9, '-', '.', '_' or '~' are converted to their "URL
  escaped" version (%NN where NN is a two-digit hexadecimal number)",
  along with spaces being replaced by plus signs

. urldecode(string)
  reverses urlencode()

. make_loc_size(bytes)
  returns a localized human-readable string for the number of bytes,
  such as "42.3 KiB"

. make_loc_duration(seconds)
  returns a localized human-readable string for the given number of
  seconds, such as "4 hours" or "3 days"
  
. make_loc_duration_abbr(seconds)
  returns a localized human-readable string for the given number of
  seconds with abbreviated units, such as "4 h" or "3 d"
  
. make_loc_rate(bytes_per_second)
  returns a localized human-readable string for the given number of
  bytes per second, such as "42.3 KiB/s"

. make_loc_time(seconds_since_epoch)
  returns a localized human-readable string for the given time; its
  format is unspecified and depends on the user's locale

. make_loc_priority(priority)
  returns a localized string for the given priority, if one of the
  torrent.priorities enum

. directory_separator()
  returns a string for the directory separator on this platform (such
  as "/" or "\")

. natural_compare(string1, string2)
  compares the two given strings and returns -1, 0 or 1; this compares
  series of digits as numbers

. name_from_url(url)
  makes up a torrent name from the given url: for magnet links, this is
  either the dn or btih field; for other urls, this will use the last
  path component without the extension
  
  if nothing interesting is found, the given url is returned as-is

. set_opt(name, string)
  sets the option to the given value; this will create the option if it
  doesn't exist

. get_opt(name)
  returns the given option's value as a string

. add_opt(name, default_value)
  adds the given option with its default value if it doesn't already
  exist

. paths.conf()
  returns the directory in which the configuration files are stored,
  such as /home/user/.baretorrent

. paths.locales()
  returns the directory in which the locale files are stored

. paths.locale_suffix()
  extension of the locale files (such as ".txt")

. paths.incomplete_suffix()
  returns the suffix appended to incomplete files within a torrent

. paths.torrent_file_suffix()
  returns the suffix appended to torrent files when saved for the user;
  this should be used instead of assuming ".torrent"

. paths.resume_file_suffix()
  returns the suffix appended to fastresume files in the configuration
  directory

. paths.state_file_suffix()
  returns the suffix appended to state files in the configuration
  directory

. paths.session_filename()
  returns the filename of libtorrent's session settings file in the
  configuration directory

. paths.queue_filename()
  returns the filename of baretorrent's queue xml file in the
  configuration directory

5.2.7 locale

. loc(name, ...)
  returns a localized string for the given name; one or more arguments
  may be needed for certain strings
  
  for example, baretorrent.loc("torrent.add") would return "Add torrent"
  and baretorrent.loc("torrent.adding", "Test") would return "Adding
  Test"; calling the later without "Test" would fail

. loc_error(error_code)
  returns a localized string for the given libtorrent.error_code object

5.2.8 platform

. platform.open_directory(path)
  opens a platform-specific file browser (such as explorer, finder,
  etc.) pointing to the given path

. platform.download_directory()
  returns the default, platform-specific download directory for the
  current user as a string (such as /home/user/Downloads); this may not
  correspond to the "bt.path.complete_save_path" option if the user
  changed it

. platform.conf_directory()
  returns the default, platform-specific configuration directory for the
  current user as a string (such as /home/user/.baretorrent); this may
  not correspond to paths.conf() if the user specified --config on the
  command line

. platform.conf_directory_name()
  returns the default, platform-specific name of the configuration
  directory; this may be something like ".baretorrent" on posix
  systems (so it is hidden) or "baretorrent" on windows; this may not
  correspond to paths.conf() if the user specified --config on the
  command line

. platform.add_firewall_exception()
  adds baretorrent to the system's firewall; no-op if the system doesn't
  have a firewall or has no way of setting it in a generic way

. platform.compile_target()
  returns the target for which the program was compiled as a string,
  such as "Microsoft Visual C++ 1700 x64"; the actual format of the
  string is unspecified

. platform.name()
  returns the name of the platform on which baretorrent is currently
  running, such as "Microsoft Windows 7 Service Pack 1 (build 7601)
  64-bit"; the actual format of the string is unspecified

5.2.9 events

. register(event_name, function_name)
  will call the given function by name when the event is fired; see 5.1
  for the list of valid events

5.2.10 user interface

. add_icon(path, greyed)
  adds the given icon to the status bar; if an icon was already added
  for this extension, it is replaced

. remove_icon()
  removes the icon added to the status bar; this should be called on
  cleanup if add_icon() was called


5.3 libtorrent API
------------------
See http://www.rasterbar.com/products/libtorrent/python_binding.html#using-libtorrent-in-python


6 Packaging
===========
All the files, including btx.txt, must be archived using 7z and given
the btx extension. "Install extension" from the Tools menu will unpack
it, copy the files into the user's configuration directory and import
it.


EOF