Authors: | Stani (spe.stani.be#gmail.com) Nadia Alramli (nadiana.com) |
---|---|
Date: | 2010/2/19 |
License: |
This paper is collection of tips, suggestions and lessons we learned from our work on a cross-platform photo batch processor called ‘Phatch’. Python’s cross-platform libraries can perhaps solve 90% of the problems, but the remaining 10% can be tricky to fix. Our paper is aimed at these remaining bits. We are using examples from Phatch throughout the paper in the hope that some of them will save your time and sanity while developing and distributing Python applications. wxPython is used as an example for the GUI, but most of the tips are applicable to other toolkits as well. The main focus of the paper is applications, but most of it is also useful for modules. Cross-platform development is a broad subject that cannot be completely covered in one paper. This is why we try to link to external resources as much as possible. Be sure to check them out.
We presented this paper at PyCon 2010:
We suggest arranging your application directory structure first before writing code. This might save you time by minimizing the need to move modules around later. The structure has to be flexible to adapt to different ‘contexts’. The context is defined by:
It is important to keep in mind that the file locations of your application will be different depending on the ‘context’, For example:
Based on our experience, we recommend you to separate application files into different directories:
Data files: icons, images, ...
Docs: HTML, restructured text, ...
Locale: i18n translations for gettext.
Separate directory for each platform where you can put bin files, shortcuts, ...
A simplified directory structure of Phatch:
phatch
+--data
+--docs
+--locale
+--linux (bin file, '.desktop' file, man page, ..)
+--mac (bin file, Info.plist, ...)
+--windows (bin file, '.ico' icon, ...)
+--source
+--core
+--pyWx (graphical user interface)
+--cli (command line interface)
+--lib
+--pyWx
+--linux
+--windows
+--mac
It is important that all images and icons are in the data path and not in the source path. An alternative is to embed images as strings in Python code. wxPython provides the command line tool img2py for this purpose. Note that we use pyWx as the name of the wxPython code directory, and not:
The different contexts make the use of __file__ problematic or even impossible in the case of py2exe. It is expected in the future that distutils will improve handling of data files. We strongly suggest you write one module to handle path locations depending on the context instead of hard-coding locations or repeating path locating logic across your library. Imagine your application has a module named path:
import sys BIN = False # should be set to True by /usr/bin/app SETUP_PY = sys.argv[0].endswith('setup.py') def platform(): if sys.platform.startswith('win'): return 'windows' elif sys.platform.startswith('darwin'): return 'mac' return 'linux' def SETUP_PY or BIN: return not sys.argv[0].endswith('.py') def setup(): if hasattr(sys, 'frozen'): return 'frozen' elif is_package(): return 'package' return 'source' CONTEXT = (platform(), setup()) if CONTEXT == ('mac', 'frozen'): DATA = os.path.join(APP_ROOT, 'Contents','Resources') # py2app elif CONTEXT == ('windows', 'frozen'): DATA = os.path.join(APP_ROOT, 'data') # py2exe elif CONTEXT == ('linux', 'package'): DATA = os.path.join(sys.prefix, 'share', APP_NAME) # linux else: DATA = os.path.join(...) # when run from source
Afterwards you can import this path module to locate for example the data path:
>>> import path >>> path.DATA != path.SOURCE True >>> icon = os.path.join(path.DATA, 'images', 'icon.png')
As the context doesn’t change during the execution of a program, constant values can be used. For Phatch we used two context modules:
It is also important to add the following files in the root of your package:
These files should describe everything necessary for distribution. For example, list dependencies, a description of the installation process, ...
If you are packaging modules instead of an application, be sure to also read the Hitchhiker’s Guide to Packaging.
Most Python coders are familiar with python and pythonw. What is relatively unknown, is that they are implemented quite differently on each platform:
Windows:
- python: Start program in an outputting console to display stdout and stderr.
- pythonw: Start program without an outputting console.
Mac OS X: Since python 2.5 the programs python and pythonw link to the same file. Before python2.5 they were really different:
- python: Run a program without a GUI (Graphical User Interface).
- pythonw: Run a program with a GUI.
For example running a wxPython application with python is not possible on Mac OS X with python 2.3 or python 2.4 as wxPython programs have a graphical user interface. (Mac OS X Tiger still shipped with Python 2.3.5 by default. Mac OS X Leopard ships with Python 2.6.)
Linux: pythonw does not exist at all!
In order to force a GUI application to use pythonw on Mac OS X, you can restart your application. Example code from phatch/app.py:
import os import sys def reexec_with_pythonw(f): """'pythonw' needs to be called for any GUI app to run from the command line on Mac OS X.""" if sys.platform == 'darwin' and \ not sys.executable.endswith('/Python'): sys.stderr.write('re-executing using pythonw') os.execvp('pythonw', ['pythonw', f]+sys.argv[1:])
Some libraries such as PIL require the full path to font files. Finding fonts on a system in a cross-platform way is not trivial. For example wxPython provides a font dialog which returns font names, but not the font file names. The TTFquery project (http://ttfquery.sourceforge.net) provides a module named findsystem.py, which works very well for Windows. It also provides solutions for Linux and Mac OS X, but those are not optimal as they only search specific directories. For Linux and Mac OS X there are these alternatives:
Finding an executable on Linux and Mac OS X is easy by reading the output of the which command. Simplified example code from phatch/lib/system.py:
def find_exe(exe): return subprocess.Popen('which %s'%exe, stdout=subprocess.PIPE, shell=True).stdout.read() >>> find_exe('python') '/usr/bin/python'
Finding an executable on Windows is more complicated as there is no easy which command. Here are some possible strategies:
- Check the source of webbrowser module. It contains some useful code.
- Execute the commands ftype and assoc to find the appropriate application for a given file extension.
- Seach the current directory and the PATH while respecting the PATHEXT environment string.
- Scan the Windows registry for the corresponding file type and retrieve the associated application.
- A brute force method is to recursively search C:\Program Files for executables (.exe) one or two levels deep.
There is probably a better solution as we are not Windows experts.
Every platform has the concept of a ‘Desktop’. Interacting with the desktop enhances the native feel of your application. This is how the desktop directory can be retrieved in a cross-platform way:
Mac OS X: The Desktop directory is always in the user home directory.
Windows: The Desktop directory can be retrieved from the Windows registry.
Linux: Special directory paths (Desktop, Documents, ...) are described in the ~/.config/user-dirs.dirs file if the language is not English. Here is an example file for Dutch:
# This file is written by xdg-user-dirs-update # If you want to change or add directories, just edit the line you're # interested in. All local changes will be retained on the next run # Format is XDG_xxx_DIR="$HOME/yyy", where yyy is a shell-escaped # homedir-relative path, or XDG_xxx_DIR="/yyy", where /yyy is an # absolute path. No other format is supported. # XDG_DESKTOP_DIR="$HOME/Bureaublad" XDG_DOWNLOAD_DIR="$HOME/Downloads" XDG_TEMPLATES_DIR="$HOME/Sjablonen" XDG_PUBLICSHARE_DIR="$HOME/Openbaar" XDG_DOCUMENTS_DIR="$HOME/Documenten" XDG_MUSIC_DIR="$HOME/Muziek" XDG_PICTURES_DIR="$HOME/Afbeeldingen" XDG_VIDEOS_DIR="$HOME/Video's"
This Python function will retrieve the Desktop directory on all platforms. Example code from phatch/lib/desktop.py:
if sys.platform.startswith('win'): # Windows from win32com.shell import shell, shellcon DESKTOP_FOLDER = shell.SHGetFolderPath(0, shellcon.CSIDL_DESKTOP, None, 0) elif sys.platform.startswith('darwin'): # Mac OS X DESKTOP_FOLDER = os.path.expanduser('~/Desktop') else: # Linux DESKTOP_FOLDER = os.path.expanduser('~/Desktop') user_dirs = os.path.expanduser('~/.config/user-dirs.dirs') if os.path.exists(user_dirs): match = re.search('XDG_DESKTOP_DIR="(.*?)"', open(user_dirs).read()) if match: DESKTOP_FOLDER = os.path.expanduser( match.group(1).replace('$HOME', '~'))
Temporary files are handled quite differently by platforms as described in a blog post from Logilab. Using the mkstemp function of the standard tempfile module requires some caution, as you are left with the responsibility to handle some low level file garbage collection.
The mkstemp function in the tempfile module returns a tuple of 2 values:
- An OS-level handle to an open file (as it would be returned by os.open()).
- The absolute pathname of that file.
Often only the absolute pathname is used. This seems to be working fine, but there is a bug lurking there. This bug will show up on Linux if you call this function many times in a long running process. On Windows, it will show up on the first call. The problem is that the file descriptor was not closed. The first element of the tuple returned by mkstemp is typically an integer used to refer to a file by the OS. In Python, Leaving a file open is usually not a big deal because the garbage collector will ultimately close the file for you, but here we are not dealing with file objects, but with OS-level handles. The interpreter sees an integer and has no way of knowing that the integer is connected to a file.
The issue on Windows is that you can’t remove any temporary file without closing the file descriptor, while on Linux you can. So on Windows this bug (removing before closing file descriptor) will show up from the first call of os.remove(temporary_file), while on Linux it only shows up when you exhaust the available file descriptors. Sometimes your application won’t arrive at that point on Linux when it only uses a limited amount of temporary files. In that case your application seems to work fine on Linux, while it doesn’t work on Windows. In reality the code is wrong both for Linux and Windows. That’s another reason why you need to make sure you test your application on every platform.
In Python, this translates to the following code:
from tempfile import mkstemp fd, temp_path = mkstemp() subprocess.call('command --output %s' % temp_path) file = open(temp_path, 'r') data = file.read() file.close() os.close(fd) # do not forget to close the file descriptor os.remove(temp_path)
For Phatch we wrapped this in class Tempfile in phatch/lib/system.py:
>>> t = TempFile('.png') >>> t.path.endswith('.png') True >>> t.close()
Unfortunately each platform has its own way to store thumbnails of files:
Each directory stores its thumbnails in a hidden thumbs.db file. Unfortunately this file is not straightforward to access with Python in Windows. The vinetto project solves this, but unfortunately it must be run in Linux or Cygwin. (http://vinetto.sourceforge.net/)
How thumbnails are cached depends on the file system according to this page:
HFS:
The HFS filesystem, for reasons of compatibility with ancient Apple operating systems, stores files in two “forks”: a data fork, which is what you normally think of as a file, and a resource fork, which is an indexed set of typed data objects. For those of us accustomed to the concept of single-forked files, the resource fork is sort of mysterious. It is part of the file, but not part of the byte stream you would receive if you read the file. In the old days the resource fork of an application would store things like strings, user interface elements, icons and bitmaps, and other blocks of largely immutable data. These days it’s hardly used at all except by Finder.
You could reasonably conceive a file’s resource fork as being filesystem metadata, but from a practical perspective that’s not actually the case. HFS has metadata support, but it’s distinct from the concept of forked files. Other filesystems (like NTFS) supports multiple streams of data addressed by the same filename, but such capabilities are almost entirely unused.
In any case, the technical answer to “where is the file’s resource fork?” is “in the file’s resource fork.” Where is it stored? In the file’s resource fork.
FAT:
FAT-16 or FAT-32, which has no support for resource forks. These are put in dot-files (files with the same name, but with a . in front of them), which the Finder ignores. You can see these from the Terminal. Windows ignores them too because they usually have the -h (hidden) attribute.
Mac OS X does this on all filesystems that don’t support resource forks.
Thumbnails are cached in the ~/.thumbnails directory according to the Freedesktop specification (http://jens.triq.net/thumbnail-spec/). This standard is valid for all desktop environments such as Gnome and KDE. Most open source applications which support thumbnails on multiple platforms (such as Gimp and Blender) use the Freedesktop standard for all platforms, as it is straightforward to implement. For Phatch we developed a library in pure python, which enables to use the Freedesktop standard on all platforms for thumbnails: phatch/lib/thumbnail.py.
The most widely used free software X desktops, GNOME, KDE and Xfce, are working closely with the Freedesktop.org project. This organization seeks to ensure that differences in development frameworks are not user-visible. Freedesktop.org was formerly known as the X Desktop Group, and the acronym “XDG” remains common in their work:
PyXDG is a python library to access Freedesktop.org standards. Currently supported are:
- Base Directory Specification.
- Menu Specification.
- Desktop Entry Specification.
- Icon Theme Specification.
- Recent File Specification.
- Shared-MIME-Database Specification.
These are three pyxdg examples:
Before the application data was stored in a hidden ~/.application directory. This method is now deprecated and three different directories have to be used for configuration, data and cache files:
>>> import xdg.BaseDirectory as bd >>> bd.xdg_config_home # preferences/settings/... '/home/user/.config' >>> bd.xdg_data_home # user plugins/images/... '/home/user/.local/share' >>> bd.xdg_cache_home # user cached items '/home/user/.cache'You can also view and edit the list of the recently-opened files:
>>> import xdg.RecentFiles as rc >>> all = rc.RecentFiles() >>> all.parse() >>> last = all.getFiles()[0] >>> last.URI u'file:///home/user/recent_file.txt'The Python standard library has a module called “mimetypes”, but it works using lists of mapped file extensions. The xdg.Mime module allows to determine file mime type very accurately using magic number tests. The advantage is that if a file has wrong or no extension, the test will return the correct MIME:
>>> import xdg.Mime >>> xdg.Mime.get_type('icon', name_pri=0) 'image/png'For more information visit: http://www.Freedesktop.org/wiki/Software/pyxdg
xdg-utils http://portland.Freedesktop.org/wiki/
Xdg-utils consists of the following installation related tools:
- xdg-desktop-menu, install desktop menu items.
- xdg-desktop-icon, install icons to the desktop.
- xdg-icon-resource, install icon resources.
- xdg-mime, query information about file type handling and add descriptions for new file types.
It also contains the following runtime integration tools:
- xdg-open, open a file or URL in the user’s preferred application.
- xdg-email, send mail using the user’s preferred e-mail composer.
- xdg-screensaver, control the screensaver.
Be sure your application can handle filenames that contain:
Non-ASCII characters: A rule of thumb is to use Unicode for paths you pass to functions like os.listdir. That will cause Python to call the Unicode version of the function, and you get the true Unicode filenames. For examples and more details refer to this article.
Spaces: To be safe it is better to preprocess filenames and add quotes only if necessary. Example code from phatch/lib/system.py:
import re RE_NEED_QUOTES = re.compile('^[^\'"].+?\s.+?[^\'"]$') def fix_quotes(text): """Fix quotes for a command line parameter. Only surround by quotes if a space is present in the filename. """ if not RE_NEED_QUOTES.match(text): return text if not ('"' in text): return '"%s"'%text elif not ("'" in text): return "'%s'"%text else: return '"%s"'%text.replace('"', r'\"')
Demonstration:
>>> fix_quotes('blender') 'blender' >>> fix_quotes('/my programs/blender') '"/my programs/blender"'
Filenames on Linux can be formatted differently:
There are two common formats:
- Normal filename notation: /home/user/foo.png.
- URI notation: file:///home/user/foo.png.
Not all Linux file browsers use the same format, when a file location is copied to the clipboard or when a file is dragged & dropped:
- Nautilus: /home/user/foo.png
- Thunar: file:///home/user/foo.png
This can be solved easily with the following function:
import urllib def fix_path(path): if path.startswith('file://'): return urllib.unquote(path[7:]) return path >>> fix_path('file://hello%20world') 'hello world'
There is a handful of useful methods and modules to use when developing cross-platform applications. The os.path module offers path manipulation functions where you don’t have to worry about platform differences. The platform module gives you access to underlying platform’s identifying data. It is very useful when you need to write a platform specific piece of code without worrying about other platforms.
os
os.path.join(path1 [, path2 [, ...]]): Join one or more path components intelligently.
os.path.split(path), os.path.splitdrive(path) and os.path.splitext(path): Convenient path splitting functions.
os.linesep: The string used to separate (or, rather, terminate) lines on the current platform.
os.pardir: The constant string used by the operating system to refer to the parent directory.
os.environ: This can be used to determine which desktop environment is running in Linux:
'GNOME_DESKTOP_SESSION_ID' in os.environ 'KDE_FULL_SESSION' in os.environ
sys
platform
If you use shell=True with the subprocess module, pass the command line as a string, not as an argument list. On Linux it will fail, when the arguments contain spaces or only a command line option:
.. sourcecode:: python
>>> from subprocess import Popen
>>> Popen(['gcc', '--version'], shell=True)
gcc: no input files
On Windows an argument list works correctly:
>>> from subprocess import Popen >>> Popen(['gcc', '--version'], shell=True) gcc (GCC) 3.4.5 (mingw-vista special r3) ...
The os module does not provide a cross-platform way to start a file with the associated application. Luckily there is any easy way to do it. An example from phatch/lib/system.py:
def start(path): if hasattr(os, 'startfile'): # Windows os.startfile(path) else: if sys.platform.startswith('darwin'): # Mac OS X command = 'open' else: # Linux command = 'xdg-open' subprocess.call([command, path])
Phatch needed to create shortcuts dynamically. For example a desktop shortcut on which images could be dropped.
Shortcuts in Windows are defined by .lnk files. These files can not be created with file write. However you can create them using the pywin32 API.
Phatch uses the following code to generate Windows shortcuts dynamically in phatch/lib/windows/shortcut.py:
import win32com.client def create(save_as, path,arguments = "", working_dir = "", description = "", icon_path = None, icon_index = 0): # initialize shortcut shell = win32com.client.Dispatch("WScript.Shell") shortcut = shell.CreateShortCut(save_as) # set shortcut parameters shortcut.Targetpath = path shortcut.Arguments = arguments shortcut.WorkingDirectory = working_dir shortcut.Description = description if icon_path: shortcut.IconLocation = icon_path # save shortcut shortcut.save()
Shortcuts in Linux are defined in .desktop files. These files provide information about your application such as the name, icon, and description. You can find the specifications here. They are used for application launchers and to create menus. The .desktop files should be made executable, unless they are placed in /usr/share/applications (see Start Menu topic).
Phatch uses the following code to generate a Linux .desktop shortcut dynamically in phatch/lib/linux/desktop.py:
import os DROPLET = """\ #!/usr/bin/env xdg-open [Desktop Entry] Version=1.0 Type=Application Name=%(name)s Terminal=false Exec=%(command)s Icon=%(icon)s""" def create_droplet(name, command, folder='~/Desktop', icon='gnome-panel-launcher'): filename = os.path.expanduser(os.path.join(folder, name + '.desktop')) data = {'name': name, 'icon': icon, 'command': command} droplet = open(filename, 'w') droplet.write(DROPLET % data) droplet.close() os.chmod(filename, 0755) # make it executable
Here is an example output:
#!/usr/bin/env xdg-open
[Desktop Entry]
Version=1.0
Type=Application
Name=Image Inspector
Terminal=false
Exec=phatch -n %U
Icon=phatch
Note the %U beside the exec command name. This is a special field code that will be expanded by the file manager or program launcher when encountered in the command line. Field codes consist of the percentage character (“%”) followed by an alpha character. In this case %U will be expanded into a list of URLs which is very important if you want the shortcut to act as a multi-file droplet. The .desktop specification website has the full list of special field codes and description.
You can enhance the interaction with your program by registering special actions with file types. These actions will appear in the context menu of the file browser. If appropriate, implement the Open action as it will allow the user to double click a file to open your application. This is easily achieved by registering file types with your application. Other actions can be added in different ways depending on the platform or file browser. Do not exaggerate and clutter the user experience.
The Open action is defined by the MimeType fields of /usr/share/applications/*.desktop files. This works for any application, not just file browsers. If you develop a JPEG application, it will be automatically integrated in photo browsers, such as F-Spot. This is a simplified example extracted from the Phatch’s desktop file:
#!/usr/bin/env xdg-open
[Desktop Entry]
...
MimeType=application/x-phatch;image/gif;image/jpeg;image/jpg;image/png;image/tiff;inode/directory;
Defining this field will associate your application with the MIME types of your choice. Note that inode/directory is the MIME type for directories.
For additional actions in the context menu, you need to write configurations specific to the file browser:
Nautilus:
- If the package python-nautilus is installed, actions can be defined by a Python file in ~/.nautilus/python-extensions. This requires the dependency python-nautilus.
- For Phatch we wrote a library to create Nautilus actions dynamically through Python: phatch/lib/linux/nautilusExtension.py
Thunar file manager:
- Actions can be defined in the XML config file ~/.config/Thunar/uca.xml.
- For Phatch we wrote a library to create Thunar actions dynamically through Python: phatch/linux/thunar.py.
File type association is typically done by including a property list file with a .plist (e.g. Info.plist) extension in your application bundle. To define file types supported by application you need to fill in the CFBundleDocumentTypes property, for example:
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC -//Apple Computer//DTD PLIST 1.0//EN http://www.apple.com/DTDs/PropertyList-1.0.dtd > <plist version="1.0"> ... <key>CFBundleDocumentTypes</key> <array> <dict> <key>CFBundleTypeExtensions</key> <array> <string>html</string> <string>htm</string> </array> <key>CFBundleTypeIconFile</key> <string>icon.icns</string> <key>CFBundleTypeName</key> <string>HTML Document</string> <key>CFBundleTypeRole</key> <string>Viewer</string> </dict> </array> </plist>
You can view/edit .plist files using the “Property List Editor” application available as part of Apple Developer Tools.
The plist file should be located in the Contents directory of the application bundle, for example:
MyApp.app/Contents/Info.plist
You can register file types and add actions to the context menu by writing to the Windows registry. This is typically done by an installer program such as Inno Setup. You can also do it at run time using Python. We created a library to do this for Phatch, you can find it in phatch/lib/windows/register.py. Here is a simplified example:
def register(label, action, filetype='Python.File', suffix='"%1"'): try: k = '%s\\shell\\%s' % (filetype, label) key = _winreg.CreateKey(_winreg.HKEY_CLASSES_ROOT, k) except: pass try: command = '%s %s'%(action, suffix) _winreg.SetValue(key,"command", _winreg.REG_SZ,command) return key except: return False
Alternatively you can use assoc and ftype. They write to the registry on your behalf:
For example:
C:\temp>assoc .pdf
.pdf=AcroExch.Document
C:\temp>ftype AcroExch.Document
AcroExch.Document="C:\Program Files\Adobe\Reader 8.0\Reader\AcroRd32.exe" "%1"
Creating cross-platform applications is not just a technical issue, but also a design issue. Each serious platform has Human Interface Guidelines (HIG) that define design rules. Unfortunately they are all different. They are partly based on usability studies, but also contain arbitrary conventions.
Each platform has its own differences and variations when it comes to user experience.
There are three types of a document interfaces:
- Single Document Interface (SDI), which gives each document its own frame.
- Tabbed Document Interface (TDI), which puts all documents within one frame with tabs.
- Multiple Document Interface (MDI), which places all documents as child frames within one bigger parent frame. Child windows can be tiled or arranged as a cascade.
The use varies between platforms and applications:
- Mac OS X: SDI, TDI (mostly for web browsers or certain text editors)
- Linux: TDI, SDI
- Windows: MDI, TDI, SDI
It would take quite some effort to satisfy everyone and support all three interfaces. In case you select one, choose a single or tabbed document interface depending on the type of application, as the Multiple Document Interface is only supported by Windows. A good compromise is TDI where tabs can be detached to a new frame like in Firefox or Google Chrome.
Take into account that user interface widgets might behave differently on each platform. For example the native notebook widget in wxPython:
If you target Linux be sure to test your application both in a Gnome and KDE as the difference in rendering might cause some layout bugs.
Mac OS X and Windows have a fixed color palette, however Linux users feel strongly about their color palettes. For example, most users change Ubuntu’s default brown palette.
Use system colors whenever possible. Try to avoid:
- Introducing your own color palette.
- Using a system color from one platform (for example: the background color), as a default color on another platform.
Cross-platform GUI kits normally give you API access to the system colors. For example in wxPython:
>>> import wx >>> [color for color in dir(wx) if color.startswith('SYS_COLOUR')] ['SYS_COLOUR_3DDKSHADOW', 'SYS_COLOUR_3DFACE', 'SYS_COLOUR_3DHIGHLIGHT', 'SYS_COLOUR_3DHILIGHT', 'SYS_COLOUR_3DLIGHT', 'SYS_COLOUR_3DSHADOW', 'SYS_COLOUR_ACTIVEBORDER', ....]
For Phatch we developed code in wxPython to draw a nice gradient based on Linux system colors in phatch/lib/pyWx/vlist.py:
def GradientColour(color): rgb = r, g, b = color.Red(), color.Green(), color.Blue() # wx2.6 m = max(rgb) rgb_without_max = [x for x in rgb if x != m] if not rgb_without_max: return wx.Colour(128, 128, 128) n = max(rgb_without_max) keyw = {} for c in ('Red', 'Green', 'Blue'): x = getattr(color, c)() if x == m: keyw[c.lower()] = x elif x == n: keyw[c.lower()] = x/2 else: keyw[c.lower()] = x/8 return wx.Colour(**keyw)
This is used as follows for the add actions dialog:
hilight = wx.SystemSettings_GetColour(wx.SYS_COLOUR_MENUHILIGHT) color_from = GradientColour(hilight) color_to = hilight
If you target Linux, you need to be familiar with the notion of ‘stock icons’. Stock icons are provided by the user theme. They ensure consistent artwork throughout different applications. Use stock icons wherever possible and try to avoid custom icons.
Most frequently used stock icons are accessible through the wxPython API:
>>> import wx >>> [icon for icon in dir(wx) if icon.startswith('ART_')] ['ART_ADD_BOOKMARK', 'ART_BUTTON', 'ART_CDROM', 'ART_CMN_DIALOG', 'ART_COPY', 'ART_CROSS_MARK', 'ART_CUT', 'ART_DELETE', 'ART_DEL_BOOKMARK', 'ART_ERROR', 'ART_EXECUTABLE_FILE', 'ART_FILE_OPEN', 'ART_FILE_SAVE', ...] >>> file_open_bitmap = wx.ArtProvider_GetBitmap(wx.ART_FILE_OPEN, wx.ART_OTHER, (48, 48))
This is the equivalent code for the GTK API:
>>> import gtk >>> [i for i in dir(gtk) if i.startswith('STOCK_')] ['STOCK_ABOUT', 'STOCK_ADD', 'STOCK_APPLY', 'STOCK_BOLD', 'STOCK_CANCEL', 'STOCK_CAPS_LOCK_WARNING', 'STOCK_CDROM', 'STOCK_CLEAR', 'STOCK_CLOSE', 'STOCK_COLOR_PICKER', 'STOCK_CONNECT', 'STOCK_CONVERT', ...]
Use pyXDG to access other icons:
>>> import xdg.IconTheme >>> xdg.IconTheme.getIconPath('firefox') '/usr/share/pixmaps/firefox.png' >>> xdg.IconTheme.getIconPath('fileopen', size=48, theme='gnome') '/usr/share/icons/gnome/scalable/actions/fileopen.svg'
Choose your own set of icons for Windows and Mac OS X as they don’t provide (freely-licensed) stock icons.
Use the standardized icon naming specification of Linux to make theme swapping easy.
If you use icons from the web, make sure to check license compatibility.
Try to avoid mixing licenses unless you know what you are doing.
Debian/Ubuntu only accepts version 3 of Creative Commons Share Alike to combine with GPL or LGPL. The simplest solution is often to contact the author and ask:
- To upgrade his CC-SA license to version 3.
- To dual license the icons.
Linux: gnome-look.org, kde-look.org, …
- Tango is a ‘neutral theme’ and is designed to work also well on Windows and Mac OS X.
- Humanity: orange-based, default theme of Ubuntu.
Open Clipart Library: openclipart.org:
- SVG icons are scalable and resolution independent. These can be rendered off-line to the appropriate format for your platform.
- The License is ‘public domain’ which also allows commercial use.
Other:
There are occasions in which your application wants to notify the user about something: software updates are available, a new instant message have been received, the 300 page print job has finally finished, etc.
To keep notifications easy to port cross-platform, don’t make them interactive. For example Ubuntu does not support notifications that require user interaction.
These are the most important libraries:
- Linux: pynotify.
- Mac OS X: Growl, which is not standard, is usually installed.
- Windows: a good wxPython solution is ToasterBox of Andrea Gavana, which mimics the look of Firefox or Thunderbird notifications.
For Phatch we developed a library that unifies these three systems in one API: phatch/lib/notify.py.
Model–View–Controller (MVC) is an architectural pattern which is often recommended for GUI programming. If this pattern seems overkill for your application, try at least to separate the GUI code (View) clearly from the other code. Let the two code parts interact with each other through events.
First, it is good to develop a simple command line application with the most basic functionality of your application. Afterwards build the GUI part on top.
PyPi lists many event frameworks e.g. pubsub
Hello world example:
''' One listener is subscribed to a topic called 'rootTopic'. One 'rootTopic' message gets sent. ''' from pubsub import pub # create a listener def listener1(arg1, arg2=None): print 'Function listener1 received:\n arg1="%s"\n arg2="%s"' % (arg1, arg2) # register listener pub.subscribe(listener1, 'rootTopic') # create a function that sends a message def doSomethingAndSendResult(): print 'lahdida... have result, publishing it via pubsub' pub.sendMessage('rootTopic', arg1=123, arg2=dict(a='abc', b='def')) # define the main part of application if __name__ == '__main__': doSomethingAndSendResult()This will produce the following output:
lahdida... have result, publishing it via pubsub Function listener1 received: arg1="123" arg2="{'a': 'abc', 'b': 'def'}"
wxPython is a GUI toolkit for the Python programming language. It is implemented as a Python extension module that wraps the popular wxWidgets cross-platform GUI library. Currently-supported platforms are 32-bit Microsoft Windows, most Unix or Unix-like systems (wrapping GTK), and Mac OS X. Other examples of cross-platform GUI toolkits are pyQt and Tkinter.
The best resources to learn wxPython are:
Supporting interaction with clipboard greatly enhances the integration with the desktop and is easy to implement:
Example code to read the clipboard:
if not wx.TheClipboard.IsOpened(): # may crash, otherwise clip_data = wx.TextDataObject() wx.TheClipboard.Open() success = wx.TheClipboard.GetData(clip_data) wx.TheClipboard.Close() if success: self.text.SetValue(clip_data.GetText()) else: self.text.SetValue("There is no data in the clipboard in the required format")
Example code to write to the clipboard:
clip_data = wx.TextDataObject() clip_data.SetText("Hi folks!") wx.TheClipboard.Open() wx.TheClipboard.SetData(clip_data) wx.TheClipboard.Close()
More info at http://wiki.wxpython.org/ClipBoard
To display a context menu, use a wx.ContextMenuEvent, not mouse down or up events. An Apple mouse has only one button. Right-click and Ctrl-Left-click are both ContextMenuEvents on Mac OS X. As gesture user interface develops, other gestures may generate a ContextMenuEvent.
Make sure that widgets which can have multiple subitems such as a listbox or a drop down control, are not empty when the GUI frame is initialized. Instead prepopulate them with some dummy value(s) such as Select which can be replaced later. Otherwise your application might crash on Mac OS X.
wxPython defines -1 as the default initial value for indices, sizes, position, ... Avoid explicitly writing the default value of -1 in your code.
- It can crash applications on Mac OS X.
- wx.DefaultSize (-1, -1) which is always transformed to (0, 0) on Mac OS X. Use reasonable minimal values instead, such as wx.Size(16,16).
Turn on anti-aliasing in the STC control:
SetUseAntiAliasing(True)
wxPython provides these overwritable wx.App methods to deal with Apple events:
- MacNewFile () Called in response of an “open-application” Apple event.
- MacOpenFile (filename) Called in response of an “open-document” Apple event.
- MacOpenURL (url) Called in response of a “get-url” Apple event.
- MacPrintFile (fileName) Called in response of a “print-document” Apple event.
- MacReopenApp () Called in response of a “reopen-application” Apple event.
As more and more applications are available on the cloud, it might be worth considering building your application as a web application instead of a desktop application.
Mozilla Prism and Google Chrome allow you to distribute a web page as a desktop application.
Cloud applications lack certain aspects of necessary interaction with the operating system:
- Dragging & dropping multiple files from a local computer to a web page is not yet supported.
- For some issues there are some workarounds, which usually requires Flash. For example to upload multiple files (http://www.uploadify.com/).
- HTML5 will address many of these limitations.
- One possible strategy for now is to use a GUI toolkit only for the frame and embed a Gecko or Webkit control to display a web page. The wxPython, GTK or pyQt frame can handle all the missing functionality such as drag and drop. Gwibber, an open source microblogging client for GNOME, is constructed this way (https://launchpad.net/gwibber).
- Pyamas: Like the Google Web Toolkit (Java), pyjamas compiles an application (and its libraries, UI widgets and DOM classes) from Python to Javascript. Pyjamas handles the packaging of the generated JavaScript and takes care of browser interoperability issues. This leaves you free to focus on application development. (http://pyjs.org/)
- Raphaël is a small JavaScript library that will simplify your work with vector graphics on the web. For example: if you want to create your own chart or an image widget which can crop and rotate, this can be simply and easily developed with this library. Raphaël could be considered the web alternative to Cairo, which is used by GTK to draw custom controls.(http://raphaeljs.com)
- At some point it will require knowledge of JavaScript and server architecture. If you are not familiar with these, maybe it is not worth the hassle for developing desktop applications.
- The problem of platform compatibility between Linux, Windows and Mac OS X, is deferred to the compatibility problems between Firefox, Google Chrome, Safari, ... and (especially older versions of) Internet Explorer.
We will focus on distributing your application to end users who are probably not familiar with Python. Therefore we will limit ourselves to:
If you want to distribute python tools or modules, you should have a look at PyPi, but this is out of the scope of our study. Freezing for Linux is also possible, but this is not the common approach.
We will not fully cover setup.py, py2app or py2exe. Instead we will give some less obvious and hopefully useful tips. For more information about them please refer to the official documentation.
py2exe is a distutils extension to convert Python scripts into executable Windows programs. The generated executable can run without a Python installation.
In setup.py write the following:
from distutils.core import setup import py2exe # loading py2exe so that it can add its command. setup( windows=[ { 'script': 'MyApp.py', 'icon_resources': [(1, 'resources/icon.ico')], } ] )
Then simply run the script like this:
python setup.py py2exe
Of course in real applications things are not that easy...
For more details visit http://www.py2exe.org/old/.
__file__ is not available in the frozen main script, That’s one of the reasons we suggest using sys.executable to get your application executable path:
if hasattr(sys, 'frozen'): return os.path.dirname(unicode(sys.executable, sys.getfilesystemencoding()))
Note that sys.executable uses the file system encoding.
The full instructions about providing the Microsoft Visual C runtime DLL are in the py2exe tutorial
There are many options to build Windows installers, including:
py2app is a Python setuptools command that allows you to make standalone Mac OS X application bundles from Python scripts. It is similar in purpose and design to py2exe.
In setup.py write the following:
from setuptools import setup setup( setup_requires=["py2app"], app=["MyApp.py"], options={ 'py2app': { 'iconfile': 'resources/icon.icns', } } )
Then simply run the script:
python setup.py py2app
Of course, things can be a little more complicated in a real application.
For more details check the official py2app documentation.
Droplets are scripts that can handle files dropped into its icon. To make your application a drop target for files, you can do the following:
Override wx.App.MacOpenFile(self, filename) with code for loading the file. For example:
class MyApp(wx.App): def OnInit(self): #do init stuff here def MacOpenFile(self, filename): print filename #code to load filename goes here.
Set the argv_emulation option in py2app to True. This will put the names of the dropped files into sys.argv. In fact you should set this option to True if your application accepts any kind of command line arguments.
Associate file types with your application as explained previously.
For more details and tips check: wxPython Optimizing for Mac OS X
py2app allows you to make some tweaks to your Info.plist file to change how your application behaves and interacts with Mac OS X. A property list file will be created from the dict given in the plist option and py2app will fill in any missing keys as necessary. File type association explained in a previous section can be done like this with py2app:
# Associating html and htm files with your application Plist = { 'CFBundleDocumentTypes': [{ 'CFBundleTypeExtensions': ['html','htm'], 'CFBundleTypeName': 'HTML Document', 'CFBundleTypeRole': 'Viewer', 'CFBundleTypeIconFile': 'Icon.icns', }] } setup( app=['MyApp.py'], options={ 'py2app': {'plist': Plist} }, )
Here are some commonly customized property list keys relevant to py2app applications:
CFBundleGetInfoString: | |
---|---|
The text shown by Finder’s Get Info panel. | |
CFBundleIdentifier: | |
The identifier string for your application (in reverse-domain syntax), e.g. "org.pythonmac.py2app". | |
CFBundleURLTypes: | |
An array of dictionaries describing URL schemes supported by the bundle. | |
LSBackgroundOnly: | |
If True, the bundle will be a faceless background application. | |
LSUIElement: | If True, the bundle will be an agent application. It will not appear in the Dock or Force Quit window, but still can come to the foreground and present a UI. |
NSServices: | An array of dictionaries specifying the services provided by the application. |
Apple Disk Image (.dmg) is usually the simplest way to distribute Mac OS X applications. There is a command line utility hdiutil to create the image:
hdiutil create MyApp.dmg -srcfolder /path/to/app/
For a fancier installer, create an installation package *.pkg. There is a PackageMaker application as part of Apple Developer Tools.
GUI2EXE is a Graphical User Interface frontend to all the “executable builders” available for the Python programming language. It can be used to build standalone Windows executables, Linux applications and Mac OS X application bundles and plugins starting from Python scripts.
At the moment the supported executable builders are:
GUI2EXE has some extra advanced features for py2exe only, such as:
The website has an extensive list of features. The author of GUI2EXE, Andrea Gavana, is very responsive on the wxpython-users mailing list in case you run into any issues.
No matter which distribution you are targeting, we suggest you contact the distribution maintainers to do the packaging. But first make their job easier by following some good practices during development:
Clearly state the license of the code and make sure that there is a copy of the license in the tarball.
Document the building process as much as possible:
Separate data files from source files. This is important for Windows and Mac applications as well.
Avoid using __file__ to locate files:
Don’t ship pre-compiled files (.so, .pdf, etc.) in the tarball without sources (.rest, .tex, ...). For example, if you are using Sphinx to generate documentation, ship conf.py and the .rst or .txt source files, not the generated HTML. A post installation script can do the HTML generation.
Do not copy other modules in the source tree of your application. Providing security support for such modules is a nightmare. If you’re missing something in foo module, send its upstream author a patch instead.
Never depend on an unreleased version of any library.
When you fix a bug, don’t re-release a modified tarball with the same version. Always increment the version.
Before releasing a version of your application, it is recommended to check your source code automatically for example with pyflakes:
$ pyflakes phatch/ | grep -v "undefined name '_'" | grep -v 'but unused'| grep -v redefinition
Getting your package in the official Debian repository has a big advantage as many Linux distributions derive from it (Ubuntu, Xandros, Mepis, ...). So if you want to package your application for Ubuntu, it means you want to package it for Debian. Debian always has at least three distributions in active maintenance: stable, testing and unstable. Ubuntu is usually based on Debian unstable, future Long Term Support (LTS) versions are based on Debian testing. A Debian source package has the .dsc extension, while a binary package has the .deb extension. (Technically a .dsc file glues all files together: one or multiple tarballs with upstream sources and with Debian modifications, which have mostly the .diff.gz or .debian.tar.gz extension. All of them together are a “source package”.)
Linux uses a river metaform in which ‘upstream’ refers to the original software and its developers, and ‘downstream’ refers to packaging an application for distribution. (According to the metaform of a ‘stream’, Debian is also considered upstream for Ubuntu.) The Python Applications Packaging Team (PAPT) is the Debian packaging team of Python programs that don’t provide public modules. (If you want to distribute a python module, contact the Debian Python Module Packaging Team.) PAPT prefers upstream authors to work with them if you wish to have your code in Debian. This is more desirable than creating unofficial packages which are never uploaded to Debian.
If you don’t know how to make Debian packages, the first thing to do is report a Request For Packaging (RFP) bug by sending a mail to submit@bugs.debian.org which follows the following template:
Subject of email:
RFP: phatch -- simple to use Photo Batch Processor
Body of email:
Package: wnpp
Severity: wishlist
* Package name : phatch
Version : 0.2.1
Upstream Author : Stani M <spe.stani.be@gmail.com>
* URL : http://photobatch.stani.be
* License : GPL
Programming Lang: Python
Description : simple to use Photo Batch Processor
Phatch handles all popular image formats and can duplicate (sub)directory
hierarchies. It can batch resize, rotate, apply perspective, shadows, rounded
corners, ... and more in minutes instead of hours or days if you do it
manually. Phatch allows you to use EXIF and IPTC tags for renaming and data
stamping.
Submitting a RFP gives no guarantee that your application gets packaged. In order to speed up the process, do everything you can to prepare the package. Look for packages with similar requirements in the PAPT packages sources and use these as a base for your own package. Also read and understand the Debian Python policy, for example:
After you’ve prepared the package, contact the PAPT team and there will be a high chance they will finish the package for you and upload it to the repositories. You might want to join PAPT to continue assisting with future releases of your application.
If you know how to make Debian packages, you can report an Intend To Package (ITP) bug. The template for the bug looks exactly the same as for RFP bugs except “RFP” is replaced with “ITP” in the Subject header. Others might offer help or ask about the status of work be replying to NNNNNN@bugs.debian.org (so just like in any other Debian bug). ITP is not a must, but it’s polite to tell others that you’re working on something so that they’ll not waste their time by duplicating the work. ITP bugs are also sent to debian-devel mailing list by default so other Debian Developers might let you know that you’re wasting your time because there are some copyright issues or any other problems with the software. When you are ready, ask PAPT on debian-python mailing list or #debian-python IRC channel (OFTC network) to check the source .dsc package and upload it to the official repositories. Mention that you are the upstream author.
Debian packaging is described by files in a debian directory. For example the debian/control contains all the dependencies and a description. The following tools helps you to create a template for the debian directory:
The following are Python specific tools:
These are some examples from the python-stdeb documentation:
$ pypi-install mypackage # install from PyPi
$ python setup.py --command-packages=stdeb.command bdist_deb # make a deb binary package
$ py2dsc source.tar.gz # make a dsc source package
$ # build a deb binary from the dsc source
$ cd deb_dist/reindent-0.1.0/
$ dpkg-buildpackage -rfakeroot -uc -us
During the package process you will run into more abbreviations such as PAPT and RFP. You can look them up in this glossary or install the wtf tool:
$ sudo apt-get install bsdgames
$ wtf apt
apt: apt (8) - Advanced Package Tool
It is recommended to submit your application to the Debian repositories through PAPT for Ubuntu distribution. Depending on the release schedule it will be synced from Debian to Ubuntu automatically or after a sync request. Ubuntu has a release cycle of 6 months. The last two months there is a Feature Freeze, during which no new packages or updates are allowed.
In case you want to test your application directly with an audience of Ubuntu users without any bureaucracy, Ubuntu’s Personal Package Archives (PPA) are the right solution. A PPA is a system to distribute software and updates directly to Ubuntu users. It uses Launchpad, which is a collaboration website like Sourceforge or Google Code. Create your source package, upload it and Launchpad will build binaries and then host them in your own apt repository:
We use PPA’s for Phatch for experimental development versions. Once ‘stable’ enough we push it into Debian Unstable. To upload to Debian Unstable you need permission of somebody else, while a PPA is under your own control (you can also remove packages). To upload your package use the command dput:
$ dput ppa:stani/ppa <source.changes>
Starting to Karmic/9.10, end user can add the PPA and its key as simple as:
$ sudo add-apt-repository ppa:stani
The Ubuntu community is trying to build an ecosystem for the opportunistic developers around Launchpad and Bazaar with the following tools:
For more information attend the Ubuntu Opportunistic Developer Week.
The RPM format is used by many popular Linux distributions, including Red Hat, Fedora, SUSE, CentOS and Mandriva. distutils has a good basic support for the RPM format using the bdist_rpm command:
python setup.py bdist_rpm
However, since each distribution has its own path specifications and packaging policies, the resulting RPM will most likely not be accepted into the distribution repository. Therefore we suggest you carefully read about the packaging process first and contact the list of packagers if you intend to distribute your application to end users.
An RPM build for Fedora can be used on SUSE. (SUSE on Fedora is also possible). However, if your binary is linked to a library that is not available on the other distribution, that binary will not execute. Also, some path locations may be different.
Fedora has a Special Interest Group (SIG) for Python and also a Python packaging policy, on which the Python packaging policy of Mandriva is based.
Red Hat, Fedora and CentOS use Koji as the platform to build RPM’s and do packaging tests. The Koji wiki explains the process. OpenSUSE has it is own build service. It claims to be the only service that allows developers to package software for all major Linux distributions.`
Include an .xml file that describes your MIME Type and copy it to /usr/share/mime/packages. For example phatch.xml:
<?xml version="1.0" encoding="UTF-8"?> <mime-info xmlns="http://www.freedesktop.org/standards/shared-mime-info"> <mime-type type="application/x-phatch"> <sub-class-of type="text/plain"/> <comment>Phatch action list</comment> <comment xml:lang="nl">Phatch actielijst</comment> <glob pattern="*.phatch"/> </mime-type> </mime-info>