NAStore External Reference Specification

NAStore
Virtual Volume Manager
External Reference Specification

Bill Ross

The Virtual Volume Manager (VVM) provides a disk cache layer between clients and removable volumes (e.g. tape) in the form of "virtual volumes" (VV's). Its main components are:

In General

Hosts

Clients, console and server processes may all run on different hosts (as defined in the Configuration), sharing filesystem access to the disk cache. For now, only root clients are supported.

Virtual Volumes

Virtual volumes (VV's) are files in the disk cache which are created by the VVM in response to client requests. Clients open, read and write them using routines in the Client Library. Each VV has a unique name generated by the VVM (such as X.1aZe), and is described by two attributes specified by the client: Id (an arbitrary string) and Class (described below in the Configuration section). When a VV is 'finished', it is written to removable volume(s) according to the policy configured for its Class.

All writes to VV's must be appends, and a 'finished' VV cannot be appended to. A VV can become 'finished' either when the client declares it to be so, when it reaches the maximum size for its Class, or if it has not been written for a given time. Once a 'finished' VV has been written to removable volume(s) (typically tape), the disk version may be deleted in order to free space; in this case, it will be restored to disk upon a client open (for read) request. The class configuration allows deletion of a VV immediately after it has been saved.

More details of VV use are described in the Client Library section, below.

Cache Management

The main VVM server (the VVM Daemon, or vvmd) monitors the amount of space used in the disk cache, and begins deleting the on-disk copies of saved VV's based on access time according to the target percentage of free space in the cache filesystem (currently hard-coded to 10%).

Virtual Volume Manager Daemon Processes

The VVM's main server is the vvmd; it handles VV access requests from the Client Library, maintains a database on VV's and on physical volumes allocated to the VVM, and communicates with the volman daemon (vold) to mount and dismount physical volumes. It also communicates with Virtual Volume Mover Daemons, or vvmvd's, which copy VV's back and forth between the disk cache and physical volumes. A vvmvd runs on each host that can mount tapes under
volman.

The VVM Daemon processes are described in more detail in the VVM Internal Design Specification.


Configuration

The Configuration has two basic components, compile-time and run-time.

Virtual Volume Classes

VV classes can be used to allocate resources according to amounts of data involved, desired access times, and possibly budgets of different users. The client chooses a class when requesting a VV for writing.

The VV classes are defined in a hard-coded table because they are not expected to change frequently; in fact, once any VV's have been written to physical volumes, it is better not to change their existing class definitions, and rather add new classes to match current needs. This is because the classes become in effect frozen in the VVMD database, both in the definitions of VV's and in the uses of scratch volumes allocated from volman.

The key part of the class definition is the prefix string which is used for the names of VV's, such as "A" in "A.1d4F". These identify the classes readily for humans and provide a name space for each class.

The Class definitions are in the the ClassInfoTab[] array in src/mss/vvm/lib/vv_class.c according to the structure given in (tt>vv_class.h in the same directory:


    typedef struct VsnT {
        char    *vs_rc;           /* volman Repository Controller            */
        char    *vs_voltype;      /* volume type (medium, capacity)          */
    } VsnT_t;

    typedef struct ClassInfo {
        char    *cs_name;        /* human-readable name                     */
        char    *cs_prefix;      /* prefix for VV names; 
                                 /* maximum len = VV_CLASSLEN               */
        int     cs_flags;        /* attributes                              */
        OFF_T   cs_vvsize;       /* maximum size allowed                    */
        OFF_T   cs_finishsize;   /* size at which it's ok to finish         */
        int     cs_nvsns;        /* number of vsn types in class (!>MAXVSN) */
        VsnT_t  cs_vsns[MAXVSN];/* vsn type definitions ('slots')           */
    } Class_t;

Note that the order of the VsnT_t's determines the preferred order for mounting VSN's to restore a VV, so faster, more accessible media should be listed first.

Here is an example ClassInfoTab with MAXVSN=2, showing four classes:

  static Class_t  ClassInfoTab[] = {
    /* name/prefix  flags         vvsize finishsz nvsns   rc1    type1      rc2  type2 */
    "class1","X",            0,    99999,  90000,   2,   "ACS1","SQR800", "ACS2","SQR800",
    "class2","Y",            0,   999999, 900000,   2,   "ACS1","SQR800", "ACS2","D3",
    "class3","Z",            0,  9999999,9000000,   2,   "ACS1","D3",     "ACS2","D3",
    "class4","bk", CS_FREEDISK,  9999999,9000000,   1,   "ACS1","D3",        0,  0,
    0,
  };
Here, the VV's of "class1" will have names beginning with "X", will be limited in size to 99999 bytes, are more likely to be finished by vvmd once the size exceeds 90000 bytes, and each one will be saved to two tapes, both the same type, "SQR800" (according to volman configuration), but in different repositories. The other classes of VV are bigger in size and thus a slower, higher-capacity, more cost-effective medium (D3) is used for the secondary copy in class2 and for both copies in class3. Class4 has only one copy (nvsns=1; note that 0's are used as placeholders for the secondary copy information) and is automatically freed from disk when it is saved; this configuration is designed to be used for workstation backups.

Run-time configuration file rc.vvm

The rc.vvm file, the name of which is specified in src/mss/vvm/Mkconfig.vvm, is read at VVMD startup; modifications to it have no effect on a running VVMD. rc.vvm contains the more dynamically configured parameters. These are:

An example rc.vvm:
      #
      #  some hosts are given for a distributed setup
      #
      BEGIN_HOSTS
         alpha beta gamma
      END_HOSTS
      #
      #  two classes have an extra set of 'hot' tapes
      #    mounted
      #
      BEGIN_PANELS
         class1  2
         class2  2
      END_PANELS
      END

Console commands

Basic console commands are vvstat (get status), vvconl (list VV info) and vvcons (shut down vvmd):
Other informational commands are vvclasses (list defined storage classes) and vvtpl (list information about VSNs owned by the VVM system): Finally, there is a collection of administrative tools for checking and repairing correspondences between disk cache, database and VSNs, and for reclaiming VSNs used by VVM which may be relatively full of deleted VV's:

Virtual Volume Manager Client Library

The Client Library contains the client-side routines used to access VV's. It is loaded via libvvm.a. These routines communicate with the Virtual Volume Manager Daemon (VVMD) process and perform normal file operations on the on-disk VV's. Virtual Volume equivalents of open(), lseek(), read(), write() and close() are provided. The vv_openwrite() and vv_openread() routines return handles for virtual volumes, which are used by the other routines. In addition to these analogs to standard filesystem routines, there is a vv_finish() routine which permanently closes the virtual volume for writing, at which point it is saved to a physical volume such as tape. (There is also a 'finish' argument to vv_close().) These routines communicate with the VVMD as necessary to get approval and to cause the VVMD's database to be updated.

See the Virtual Volume Manager Internal Design Specification for detailed coverage of the components.

Author (mail): Bill Ross


 NAS HOME PAGE  Storage Systems home page WebWork: Harry Waddell
NASA Official: John Lekashman