~msapiro/mailman/topics

Mailman - The GNU Mailing List Management System

Copyright (C) 1998-2004 by the Free Software Foundation, Inc.

51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA

UPGRADING FROM PREVIOUS VERSIONS

    For the most part, upgrading Mailman entails installing the latest version

    over the existing version.  Usually, you can unpack the new release, run

    'configure' with the same options you used in your previous install, and

    then do a 'make install'.  However, there are some changes that may need

    to be taken care of manually.

    What you need to do depends on the version you are using and the version

    you are upgrading to.  In all cases, you should first turn off your mail

    and web access to your Mailman installation.  You're essentially upgrading

    a database, and it's usually a good idea to make sure the database cannot

    be modified in the middle of the upgrade.

    My recommendations are:

    - Turn off your incoming mail daemon.  Most remote smtp servers will

      simply queue up messages destined for your domain if port 25 is shut

      off.

    - Temporarily disable web access to Mailman.  You can do this by either

      turning off your web server temporarily, or by setting up a temporary

      redirect to a "service unavailable" page for the Mailman URLs.  Refer to

      your web server documentation for details.

UPGRADING FROM 2.1.4 to 2.1.5

    In Mailman 2.1.5, some significant changes have been made to the file

    formats for qfiles and the pendings database.  All care has been taken to

    make sure the upgrades happen automatically and smoothly, but you should

    double check and, for the ultra-paranoid, make backups of your Mailman

    site before you upgrade.  BE SURE TO TURN OFF MAILMAN AS DESCRIBED ABOVE

    BEFORE YOU UPGRADE.

    Specifically, in MM2.1.4 every message in the queues was represented by

    two files, a .msg or .pck file containing the email message, and a .db

    file containing metadata about the message.  In MM2.1.5 this has been made

    more efficient by using only one file (with a .pck extension) for both the

    message and metadata.  This should make MM2.1.5 half as hostile to the

    file system.

    The bin/upgrade script, which is run automatically when you upgrade,

    should convert all the old style qfiles to the new style qfiles.  Note

    that this could take a long time if you have a lot of files in your qfiles

    subdirectories.  Pay particular attention to files you might have in

    qfiles/shunt; these will get upgraded too, although files in qfiles/bad

    will not.

    In MM2.1.4, the database file containing pending actions (i.e

    subscriptions, unsubscriptions, message holds, etc.) was shared globally

    among all mailing lists.  In MM2.1.5, each list now has its own pending

    database file.  All care has been taken to properly split pending actions

    from the global to the list-specific files, but it's possible there are

    bugs here.  Best practice is to clear all pending actions before you

    upgrade, although this is not always possible.

UPGRADING FROM 2.0.x to 2.1

    When you upgrade from Mailman 2.0.x to Mailman 2.1, you should double

    check that your moderation and privacy options are still set the way you

    want them.  The Mailman options dealing with moderation and privacy have

    changed significantly, to make them easier to understand and control.

    Ever effort was taken to translate the old configuration variables to the

    new configuration variables, but because the old semantics were so

    complex, it is possible your settings may not have been correctly

    translated.

    Check especially the values for (in Privacy -> Sender Filters)

    default_member_moderation, generic_nonmember_action, and

    accept_these_nonmembers.  Also check the moderation flag on member

    accounts in the Membership Management screen.

    In Mailman 2.1, the qrunner subsystem has been completely

    rewritten.  You no longer start qrunner from cron!  Instead, there

    is a bin/mailmanctl script which is used to start, stop, and

    restart mail delivery.  This script is appropriate to use as a

    Unix init script.  Be sure to update your crontab with the new

    cron/crontab.in file.

    NOTE: It is very important that if you are upgrading from a

    pre-MM2.1alpha2 system to a post-MM2.1alpha2 system that you let

    the old qrunner process clear any and all messages sitting in the

    qfiles/ directory *BEFORE* you upgrade.  Otherwise after the

    upgrade, those messages will not get delivered, and there is no

    easy way to upgrade those pending messages.

    NOTE: When upgrading to Mailman 2.1, you will need to regenerate

    your aliases files.  There have been many changes to the alias

    names, the programs they map to, and the name of the wrapper

    script.  See the Mailman Installation Manual for details of making

    Mailman work with your mail server.

    To regenerate your aliases, use the bin/genaliases script.

    IMPORTANT: The encryption algorithm for list admin passwords has

    changed which means that after upgrading from 2.0.x to 2.1.x, list

    passwords will have to be reset.  There is a new bin/change_pw tool

    to help with this.

    Mailman 2.1 introduces multilingual (a.k.a. internationalization

    or i18n) support.  Previously only one language per list was

    supported, and it was assumed that this language would be English.

    The upgrade script for Mailman 2.1 creates a subdirectory `en'

    inside each lists/<listname> directory.  It then copies all the

    .txt and .html files from lists/<listname> into

    lists/<listname>/en.

    If you have modified those templates to contain non-English text,

    you will have to manually rename the en subdirectories to the

    language code for the language of your templates.  Mailman's

    upgrade script should handle cleaning up any templates which are

    duplicates of the defaults, but you'll want to double check this

    manually.

    If you are running a MM2.0.x system with non-standard patches

    applied, you might have some other problems with your upgrade.

    Here are some instances we know about:

    - If you've applied patch #413752 (coerce to plaintext), then your

      upgrade will not go smoothly.  Take a look at patch #651406 for

      an unofficial solution.

      http://sf.net/tracker/?group_id=103&atid=300103&func=detail&aid=413752

      http://sf.net/tracker/?group_id=103&atid=300103&func=detail&aid=651406

UPGRADING INDIVIDUAL LISTS

    If you're nervous about upgrading all of your lists to 2.1 in one

    go, you can move them and upgrade them one at a time.  Start by

    doing a clean Mailman 2.1 installation in an empty directory --

    call it $MM21.  (I'll assume your Mailman 2.0 installation is in

    $MM20.)

    Doing this means you'll have co-habiting Mailman 2.0 and 2.1

    installations for a while, until you have moved all of your lists

    to Mailman 2.1.  Depending on your MTA and web server, this could

    be transparent and painless, or it could be an ongoing headache.

    If you use Apache with mod_rewrite, then it's fairly

    straightforward to set things up so that both Mailman 2.0 and 2.1

    inhabit the /mailman and /pipermail URL-space of your server; this

    makes the transition almost transparent to list admins and

    subscribers.  See below for details.

    Now, for each list that you want to move, you'll have to

      * Shut down your MTA.

        If you have a lot of outgoing list traffic, you might need to

        leave your MTA up but only let it accept connections from

        127.0.0.1 (localhost), so Mailman 2.0 can flush its queue.

        How to do this is MTA-dependent; for Exim, you can set

        "local_interfaces = 127.0.0.1" and "kill -HUP" the Exim

        daemon.

      * Shut down your web server.  For a more professional look, or

        if you want to allow people to keep accessing the rest of your

        web site, you could make your web server respond to all

        /mailman/ URLs with a "temporarily unavailable" message.

        How to do this is web server-dependent; with Apache and

        mod_rewrite, this does the trick:

          RewriteRule ^/mailman/.* /var/www/unavailable.html [L]

        (Of course, you'll have to supply your own

        /var/www/unavailable.html.)

      * Force Mailman 2.0 to process its queue:

          python -S $MM20/cron/qrunner

        (This is only necessary if there are any files in $MM20/qfiles;

        if you need to do this, make sure you left your MTA listening to

        127.0.0.1.)

      * Move the list:

          cd $MM20

          mv -i lists/foo-list $MM21/lists

          mv -i archives/private/foo-list $MM21/archives/private

          mv -i archives/private/foo-list.mbox $MM21/archives/private

          rm archives/public/foo-list

          rm archives/public/foo-list.mbox

          cd $MM21

          bin/withlist -l -r fix_url mylist

        (The fix_url step will not be necessary if your Mailman 2.0

        and 2.1 installations will be sharing the same URL-space.)

      * Edit your web server config so the list's URLs continue to

        work.  There are two possible approaches here; the simpler way

        is to setup a new slice of URL-space that will be used by your

        Mailman 2.1 installation, eg. /mailman-21:

        With Apache and mod_rewrite:

          RewriteRule /mailman/(.*)/(foo-list.*) /mailman-21/$1/$2 [R=temp]

        (The [R=temp] assumes that "/mailman-21/" is a temporary URL,

        and you'll move all your lists to "/mailman/" when the

        transition to Mailman 2.1 is complete.)

        If you don't want to expose ugly temporary URLs like

        "/mailman-21" to the world, it's only slightly more work to make

        Mailman 2.0 and 2.1 share the same slices of URL-space.  Here's

        how to do it with Apache and mod_rewrite:

          RewriteRule ^/mailman/(.*)/(foo-list.*) \

                      $MM21/cgi-bin/$1/$2 \

                      [T=application/x-httpd-cgi]

        Not only is this more aesthetically pleasing, it's faster -- no

        redirects.

        In either case, you'll want to rewrite the list's archive URLs

        to Mailman 2.1's archive:

          RewriteRule ^/pipermail/(foo-list.*) $MM21/archives/public/$1

      * Restart your web server (or disable the "temporarily

        unavailable" stuff).

      * Restart your MTA (or make it listen to more than just

        127.0.0.1).

UPGRADING FROM 2.0 to 2.0.x (where x >= 1)

    Nothing much more than running "make install" (after upgrading)

    should be necessary.

UPGRADING FROM 2.0 beta to 2.0 final

    You MUST re-run configure; running config.status is not sufficient

    due to some recent changes in the autoconf scripts.  You can do a

    head of config.status if you don't remember the options you

    originally ran configure with.

    The cron jobs for Mailman 2.0 final have changed considerably,

    including the frequency with which they run.  You should reload

    misc/crontab.in for the `mailman' user to get the right settings.

    See the INSTALL file for details.

    FAILURE TO DO THIS WILL RESULT IN A LESS THAN OPTIMALLY FUNCTIONAL

    MAILMAN INSTALLATION.

UPGRADING FROM 1.x to 2.x

    In addition to the instructions above, I highly recommend that you

    make sure your Mailman queue is cleared /before/ upgrading.

    Mailman version 1.x had a cron script called run_queue which was

    part of its bulk mailer.  With Mailman 2.x there is no default

    bulk mailer (it lets the MTA handle this), and it is currently

    unknown what the effects of upgrading are on the run_queue script,

    but I'll bet it's not good. :)

    The way to make sure that your Mailman queue is empty is to look

    in your $prefix/data directory.  If you see any files that start

    with "mm_q." you've still got messages waiting on the queue.  You

    can run $prefix/cron/run_queue by hand until the queue is cleared.

    Multiple invocations of this script won't help though; they lock

    each other out.  Also, be warned that clearing the queue can take

    a while and may cause a large load on your system (two reasons why

    all this stuff has been redesigned in 2.x :).

    You do not need to run "make update" if you are upgrading from

    version 1.0 or 1.1 to version 2.0, since this is now run

    automatically when you do a "make install".  However you should

    modify your crontab entries to execute cron/qrunner instead of

    cron/run_queue.  You can also safely remove the file

    $prefix/cron/run_queue.

    If you are upgrading from a pre-1.0 beta, you need to follow the

    instructions below.

UPGRADING FROM PRE-1.0 to 2.x

    You need to do a few extra things to make sure that the file

    system layout for the early 1.0 betas is upgraded to the 1.x

    configuration.  There are two ways to do this.

    First, from the source directory, after you've done a "make

    install" you can run "make update".  "make update" creates a file

    named "update.log" in the top level of the source distribution.

    If the script that updates the Mailman filesystem encounters

    something that is not resolvable, it will log info about this to

    "update.log".  This is worth checking after the upgrade completes.

    You can also just change to the installation directory (i.e. $prefix)

    and run bin/update.  This is the same as above except that the

    update.log file is not generated.

    Check your crontab entry.  Remove any runs of obsolete scripts, in

    particular cron/upvolumes_yearly, cron/upvolumes_monthly, or

    cron/archive.

WHAT "MAKE UPDATE" DOES

    Below is an annotated listing of the things that "make update"

    does.  Hopefully, this will help resolve any problems you are

    having.

    Note that it can't hurt to run "make update" each time you

    upgrade, but if you're running version 1.0 or newer, it won't help

    much either!

    - To upgrade to 1.0b10, you will need to copy

      templates/options.html to lists/<listname>/options.html for each

      mailing list you have.  However, if you have edited the

      options.html file, say from the Web interface, you will have to

      merge these changes in manually.

    - The upgrade to 1.0b7 included the removal of

      Mailman/smtplib.py{,c} since Mailman now uses the default Python

      1.5.2 version of smtplib.

    - Archiving files are moved around as part of integrating

      Pipermail into Mailman, as of 1.0b6.  In particular,

      1) if a list has only a private mbox archive

      $prefix/archives/private/<listname> is moved to

      $prefix/archives/private/<listname>.mbox/<listname>

      2) if a list has only a public mbox archive

      $prefix/archives/public/<listname> is moved to

      $prefix/archives/private/<listname>.mbox/<listname>

      and a symlink is made that points

      $prefix/archives/public/<listname>.mbox to

      $prefix/archives/private/<listname>.mbox/<listname>

      3) if a list has both private and public mbox archives,

      make update picks one of the above 2 configurations based on

      whether or not the list currently is archived publicly.  It then

      renames the other mbox to mbox.preb6.

      4) if a list used recent CVS sources, where archives were placed in

      $prefix/public_html/archives, then these are moved to

      $prefix/archives/private/<listname> and a symlink is made from

      $prefix/archives/public/<listname> to that spot if the list's

      archives are public.  Also, a permissions-related security

      problem is removed.

      To integrate mbox archives of old lists, log in as user `mailman'

      and run $prefix/bin/arch <listname> <path-to-mbox-archive>.

      Also, by default, beta6 does both mbox and html based archiving,

      but you can configure Mailman to do one, both, or neither.

      Please see $prefix/Mailman/Defaults.py for details.

      There was a short period of time when the CVS sources archiving

      code was not organized into its own package.  The pickled

      articles in the archives that were placed into archives during

      this period stored the path to the module HyperArch, but that

      module has moved.  You can quick fix this by running

      ln -s $prefix/Mailman/Archiver/HyperArch.py \

              $prefix/Mailman/HyperArch.py

    - If upgrading from version 1.0b4 or earlier, "make update" moves

      list-specific templates.  For each list,

      $prefix/templates/<listname>/* is moved to $prefix/lists/<listname>.

      Please reference the generic templates in $prefix/templates to see

      if any variables have changed (There shouldn't be many, only

      options.html was updated from b5 to b6).

      For really old versions of Mailman, you may not even have

      <listname> subdirectories in $prefix/templates!  In this case

      you will need to manually copy some files into your new list

      directories.  Here's an example shell command that will do the

      trick:

      cp templates/{archives,handle_opts,listinfo,roster,subscribe}.html lists/<listname>

    - Some modules that existed in previous versions, but that have

      been replaced with newer (differently named) modules, are

      removed.

Local Variables:

mode: indented-text

indent-tabs-mode: nil

End: