~oddbloke/pqm/simpletal

<?xml version="1.0" encoding="UTF-8"?>
<!-- arch-tag: manual for the tla patch queue manager
-->

<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" [
<!ENTITY tla "<application>tla</application>" >
<!ENTITY pqm "<application>pqm</application>" >
<!ENTITY exampledir "<filename>/usr/src/pqm/</filename>" >
<!ENTITY examplerepository "example-dev@example.com--2003" >
] >
<article>
  <title>Introduction to the Patch Queue Manager</title>

  <articleinfo>
    <date>2003-10-16</date>
    <author>
      <firstname>Colin</firstname>
      <surname>Walters</surname>
      <email>walters@verbum.org</email>
    </author>
    <legalnotice>
      <para>Copyright 2003 Colin Walters</para>
      <para>Portions Copyright (c) 2004 Robert Collins</para>
      <para>Portions Copyright (c) 2003, 2005 Walter Landry</para>
      <para>Portions Copyright (c) 2005 Canonical Limited</para>
      <para>Permission is granted to copy, distribute and/or modify this document under the terms of the GPL.</para>
    </legalnotice>
    <keywordset>
      <keyword>arch</keyword>
      <keyword>arx</keyword>
      <keyword>bazaar</keyword>
      <keyword>bzr</keyword>
      <keyword>tla</keyword>
      <keyword>pqm</keyword>
      <keyword>patch</keyword>
      <keyword>changeset</keyword>
      <keyword>repository</keyword>
    </keywordset>
  </articleinfo>

  <section>
    <title>Getting &pqm;</title>
    <para>
      &pqm; is maintained in <command>bzr</command>.
    </para>
    <variablelist>
      <title>'official' &pqm; repository location</title>
      <varlistentry>
	<term>Location</term>
	<listitem><para><ulink url="http://people.ubuntu.com/~robertc/pqm/trunk">http://people.ubuntu.com/~robertc/pqm/trunk</ulink></para></listitem>
      </varlistentry>
    </variablelist>
  </section>
  <section>
    <title>What problem does it solve?</title>
    <para>
      The idea is simple.  You have a project with a number of
      developers.  With a revision control system like CVS, it's
      obvious that the project code will be kept in a single
      repository, which all the developers use.  You really don't have
      much of a choice.
    </para>
    <para>
      But a decentralized version control system (DVCS) allows your
      developers to commit while disconnected (say while they are
      travelling with a laptop), easily create their own temporary
      branches without affecting the main repository, and more.  To
      accomplish these things, each developer needs to have their own
      repository.
    </para>
    <para>
      This then raises a question - where <emphasis>is</emphasis> the
      project?  One solution is to pick a specific developer to
      perform the task of merging in the other developer's code.  That
      developer's repository becomes the canonical one for the project.
    </para>
    <para>
      However, there is a better way.  The main idea of the
      patch queue manager is to have a special repository which is
      managed entirely by the patch queue software.
    </para>
  </section>
  <section>
    <title>How it works</title>
    <para>
      You create a special repository to be managed by the patch
      queue manager.  Then, every developer branches off of it.  When
      a developer has reached a milestone and wants to sync up the
      main tree with their repository, they submit a merge request.
    </para>
    <para>
      Essentially, this just leverages merge
      functionality in the DVCS to perform the actual merge.
    </para>

    <section>
      <title>Handling conflicts</title>
      <para>
	One question that arises at this point - what happens if
	there's a conflict?  If two people submit merge requests, the
	second could conflict with the first.  Obviously the patch
	queue manager is not an AI; it can't automatically resolve all
	conflicts.  So the simple solution is to just to inform the
	submitter of the second patch of this fact, and reject the
	merge request.  They can then sync up their tree with the main
	branch, and resubmit the merge request.
      </para>
    </section>

    <section>
      <title>Running tests</title>
      <para>
	However, textual conflicts aren't the only kind of conflict;
	you could also have _semantic_ conflicts, where the patches
	don't touch the exact same portion of code, but taken together
	they break it.  One way to help solve this problem is to run a
	testsuite before every commit to the main repository.  In
	addition to any functionality built into the DVCS, the patch
	queue manager lets you do this with a precommit hook.  You can
	actually do pretty much anything you want inside this hook.
      </para>
    </section>

    <section>
      <title>Security</title>
      <para>
	You have two choices for submitting merge requests; one option
	is to send them via email. &pqm; supports GPG-signed
	messages for security.
      </para>
      <para>
	If it can be arranged for developers to all have write access
	to the same filesystem (via NFS/SFS, over sftp, etc), then you
	can have them simply drop merge requests into a special queue
	directory.
      </para>
    </section>
  </section>
  <section>
    <title>Setup</title>
    <section>
      <title>Requirements</title>
      <orderedlist>
        <listitem>
          <para>Python 2.4</para>
        </listitem>
        <listitem>
          <para>config-manager (used for url mapping and nested tree support).</para>
        </listitem>
        <listitem>
          <para>Pybaz (optional, for baz support).</para>
        </listitem>
        <listitem>
          <para>bzr (optional, for bzr support).</para>
        </listitem>
        <listitem>
          <para>GNUPG (optional, but <emphasis>highly</emphasis> recommended)</para>
        </listitem>
      </orderedlist>
    </section>
    <section>
      <title>Compilation and installation</title>
      <orderedlist>
	<listitem>
	  <para>
	    Use the normal <literal>configure</literal>,
	    <literal>make</literal>, <literal>make install</literal>
	    commands to install the software.  If you downloaded
	    &pqm; directly from the repository, you may have to
	    run <literal>autoreconf -i</literal>.
	  </para>
	</listitem>
	<listitem>
	  <para>
            Next, you need to create a ~/.pqm.conf configuration
            file. This lists the location of the queue, the path to
            the GNUPG keyring, whether or not to verify signatures,
            etc. In addition, the pqm.conf file also specifies
            which repositories are valid to merge into. This is not
            checked for all commands.  In the future, it will also
            allow specifying a keyring per-repository, and some other
            nice things.

            Please see the example file as a starting point.  Note
            that if you want to use an implementation besides
            baz, you must set the arch_path argument in .pqm.conf.

	    TODO: document all the options here in the manual.

	    The path for pqm.conf can be overridden via the environment
	    variable PQM_CONFIG.
	  </para>
	</listitem>
	<listitem>
	  <para>
            Set up pqm to process requests.  If you are using a
            shared filesystem, then you can just drop requests into a
            the queue directory.  If necessary, set up pqm to
            process requests through email.  You will likely want to
            use procmail. If you have an account dedicated to
            processing merge requests (a good idea), you could use
            this sample .procmailrc entry:

    <programlisting>
:0:
| pqm --read
    </programlisting>
            If you don't have a dedicated account, you will have to
            pick a specific <literal>Subject:</literal> or other
            header to use to differentiate merge requests from your
            regular email.  This will scan and verify any emails that
            come in.  You will need to populate the patch queue's GPG
            keyring.

            To actually perform these actions, you need to run

    <programlisting>
$ pqm --run
    </programlisting>
      

            That simply iterates through the pending merge requests in
            order (by mtime), and processes them. One way to run it is
            via a cron job.

	  </para>
	</listitem>
	<listitem>
	  <para>
            Create any precommit or postcommit hooks.
          </para>
	</listitem>
      </orderedlist>
    <para>
            Almost everything else can be done using with pqm actions.
    </para>
    </section>
  </section>
  <section>
    <title>Using &pqm;</title>
    <section>
      <title>Actions</title>
      
      <para>
            There are currently eight different actions that pqm
            can do.  The first five commands
      </para>
      <itemizedlist>
        <listitem>
          <simpara>
    make-repo REPOSITORY-NAME REPOSITORY-LOCATION
          </simpara>
        </listitem>
        <listitem>
          <simpara>
repo-cache-revision REPOSITORY/REVISION
          </simpara>
        </listitem>
        <listitem>
          <simpara>
repo-uncache-revision REPOSITORY/REVISION
          </simpara>
        </listitem>
        <listitem>
          <simpara>
tag FROM-REPOSITORY/FROM-REVISION TO-REPOSITORY/TO-REVISION
          </simpara>
        </listitem>
      </itemizedlist>

      <para>
      function as you would expect.  Note that everything must be
      fully specified.  The patch queue manager does not, in general,
      have a default repository.
      </para>
      <para>
        For merging, use 
      </para>
      <itemizedlist>
        <listitem>
          <simpara>
            star-merge FROM-REPOSITORY/FROM-REVISION
          </simpara>
        </listitem>
      </itemizedlist>

      <para>
         and to create a completely new line of development
      </para>
      <itemizedlist>
        <listitem>
          <simpara>
            create-version REPOSITORY/VERSION
          </simpara>
        </listitem>
      </itemizedlist>

      <para>
        Again, the commit message will be the
        <literal>Subject:</literal> line of your merge request.
      </para>

      <para>
        So the first command you will use is probably
        <literal>make-repo</literal>.
        If you are just starting development, you will then use
        <literal>create-version</literal>.  If you have already
        started development, you will use
        <literal>create-branch</literal>.  The developers then branch
        off of the patch queue manager's branch.  When the developers
        are ready, they will merge back with
        <literal>star-merge</literal>.  You will use
        <literal>repo-cache-revision</literal> and
        <literal>repo-uncache-revision</literal> to speed up get's
        and free up space. Finally, you will use
        <literal>tag</literal> to mark a release.
      </para>

    </section>

    <section>
      <title>Defining the upstream repository</title>
      <para>
        The commands in this section will be easier if we record in a
        little file which repository is the "upstream". It is recommended
        you do something like this:
      </para>
<screen>
$ echo 'example-dev@example.com/example-project' > _arx/+upstream
</screen>
    </section>

    <section>
      <title>Submitting a merge request via a shared filesystem</title>
      <para>If you can provide all of the developers with write access
      to the same filesystem, then you can simply have them place
      merge request files in the queue directory, using a small shell
      script such as this:
    </para>
    <programlisting>
#!/bin/sh (echo "From: $(arx param id)"; echo "Subject: $1"; echo; echo
star-merge "$(arx tree-branch)" "$(cat _arx/+upstream)") >
&exampledir;patch.$(date +%s)
    </programlisting>
    <para>
      Save that as <filename>pqm-submit-merge</filename>.  Then you
      would use this script by saying:
    </para>
<screen>
$ pqm-submit-merge 'fix lots of bugs'
</screen>
    <para>
      In the future, we would like to have better support integrated
      in the various version control systems for this kind of thing.
    </para>
    </section>
    <section>
      <title>Submitting a merge request via GNUPG-signed email</title>
      <para>
	If your site is widely distributed, or giving shell accounts
	to all developers doesn't work well, you can also use
	GNUPG-signed email.  Here's how you could modify the script
	above:
      </para>
<programlisting>
#!/bin/sh
echo star-merge "$(arx tree-branch)" "$(cat _arx/+upstream)" | gpg --clearsign | mail -s "$1" "$2"
</programlisting>
    <para>
      You might use the above script by saying:
    </para>
<screen>
$ pqm-submit-merge 'fix bugs' pqm@example.com 	
</screen>
    </section>
    <section>
      <title>Controlling the queue</title>
      <para>
      Typically you setup PQM to run from cron every minute so that
      it processes new requests as they come in. Each time it runs it
      will look in the queuedir for patches, and then process each of 
      them in turn. If for any reason you want to stop this process, 
      simply add a file called stop.patch to the queuedir. PQM will 
      then stop processing after the current merge request (or if 
      there are none, it will not process any incoming requests). To
      allow PQM to restart processing, simply remote the stop.patch
      file.
      </para>
    </section>
  </section>
</article>