MSSL Basic User Note #1: V0.3 05/8/99 atp


How to use magtapes on UNIX

Contents

  1. Device names: Their lore and meaning.
  2. Manipulating tapes: The mt command.
  3. Tape Archiving: The tar, dump and restore commands.
  4. Foreign Tapes: VMS BACKUP savesets and ANSI labeled tapes.
  5. Misc: "None of the above" tapes, Tape to Tape copies.
  6. Where to go for more help.

1) Device names: Their lore and meaning

Under UNIX, magnetic tape drives are given individual numbers, starting from zero. Tape drives are accessed by means of a device special file which is located in the /dev directory. These device special files appear to the user as normal files, which can be opened, read from and written to. All operations that you would do on a normal file can be done on a special file. The only difference is that the data in the file resides on the tape loaded in the drive, not on the filesystem. In addition, because there are different ways of physically writing data to a tape (e.g. low/high density), there are several different files that correspond to the same tape drive.

Each variant of UNIX has its own names for the tape drives. In this document I will cover Digital UNIX (aka Tru64 UNIX  aka OSF/1) and Linux.

Linux

 Tape devices under linux have different names depending on the type of tape involved. The most common sort of tape (SCSI Tape) will have a device name of  'st'.
So for example the first scsi tape will have the following device special files

/dev/st0    - normal rewinding scsi device
/dev/nst0   - non-rewinding scsi tape device

Floppy tape drives (e.g. iomega ditto, Travan tapes and other QIC  Format tape drives) will be called ;

/dev/rft0   - normal rewinding floppy tape device.
/dev/nrft0  - non-rewinding floppy tape device.

One major difference between Linux and Digital UNIX devices is that the density code is not used on linux. If you want to write a tape with a density other than the default (usually highest) density then you may need to use the mt command with the densities or setdensities arguments to set the density of the device.  Read operations usually do not need the density setting explicitly, as this will be automatically picked up.

Digital UNIX

 Device names on Digital UNIX are more complex, As an example let us take the DAT tape on msslae. (mt0)

msslae~> file /dev/rmt0h
/dev/rmt0h:     character special (9/21506) SCSI #1 TLZ06 tape #104 (SCSI ID #5)
                errors = 0/900 offline 
msslae~>  ls /dev/*mt0?
/dev/nrmt0a /dev/nrmt0l /dev/rmt0a  /dev/rmt0l  
/dev/nrmt0h /dev/nrmt0m /dev/rmt0h  /dev/rmt0m  
msslae~>
In the listing there are eight special files which correspond to this tape drive. Each name specifies a different way of treating the tape to be used. From the example above, for magnetic tape 0 (mt0), there are four different densities.


/dev/nrmt0h /dev/rmt0h == high density
/dev/nrmt0m /dev/rmt0m == medium density
/dev/nrmt0l /dev/rmt0l == low density
/dev/nrmt0a /dev/rmt0a == another density (QIC format drives only)

In fact the density designation depends very much on the drive hardware. For the case of mssly1's DAT drive, which is a TLZ06, "man tz" gives the following snippet of information
 
 

          TLZ06: Single-density drive, compaction support.
              rmt?a 61000 BPI
              rmt?l 61000 BPI
              rmt?m 61000 BPI,compaction turned on.
              rmt?h 61000 BPI, compaction turned on.
So, in fact, the only difference is that rmt0h and rmt0m have hardware compression turned on, and rmt0a and rmt0l have it turned off. This means that for this device there are only two effective densities.
For the remainder of this document I will use the high density drive letter, (rmt0h) only, as this is the one most people use in practice, with the understanding that the other drive densities can be interchanged as needed.

 Going back to the list of files, there are two further divisions to be made. Those files starting "r" and those starting with "nr". These two sorts refer to the tape handling after all the operations are complete, and the drive is closed. Drives accessed using the "r" filename like /dev/rmt0h will rewind the tape on being closed. Those files starting "nr" will not rewind the tape on closing.

 This may seem, on first sight, a funny way of going about things, but it is logical, when you consider the differences between a tape and an ordinary file. If you open and close a file, then open it again you are placed (by default) at the start of the file. For a tape drive this corresponds to the start of the tape, so the normal behaviour is to be like a standard file, and rewind the tape on closure, so that the next time the tape drive is opened, it is positioned at the start of the tape.

 This may not be the ideal in many cases however, particularly if you want to append to a tape. In the case of a normal file, opening, and moving to the end of the file, ready to append, is an inexpensive, quick operation. For a tape drive, this sequence can take a significant amount of time however, and so the tape drive can be told (by using the nrmt0h device, rather than the default rmt0h device) to leave the tape where it is once the current operation has been completed.

A frequent problem occurs when users migrating from VMS forget the difference between the rmt0h and nrmt0h devices, and wonder why they cannot position the drive correctly. This is because the default behaviour is different between the two systems. This typically occurs when positioning a tape using the "mt" command. For example the command

mssly1~> mt -f /dev/rmt0h fsf 2
to supposedly skip forward by 2 files will, in fact, achieve nothing at all, as the tape will be rewound to its start immediately following the completion of the "mt" command. The correct form of this command would be instead;
mssly1~> mt -f /dev/nrmt0h fsf 2

Summary

2) Manipulating tapes: the "mt" command

The mt command is the magnetic tape handling program for UNIX. This command has the following syntax;
  mt [-f tape_device] command [count]
Where tape_device is one of the device special files discussed in the first section, normally a "no rewind" device like /dev/nrmt0h. The commands are detailed in the man page for "mt", but a few important ones will be listed here.

Command Description

rewind Rewinds a tape. status Prints out information about a tape. offline Unloads a tape. fsf Forward Skip File. bsf Back Skip File. Some of these commands can take an argument, e.g.
mssly1~> mt -f /dev/nrmt0h fsf 2
Will skip forward two files on a DAT on mssly1.

The drive name is optional, if you are accessing the first tape drive (mt0). For all other drives the drive name must be explicitly specified on the command line. This is good practice even if you are using the default drive.
 
 

.

3) Tape Archiving: The "tar", "dump" and "restore" commands.

There are many programs that handle tapes. The most common general purpose ones are tar and dump.

 tar is the tape archiver program, and provides a quick and easy way of transporting data between UNIX systems, and making user backups. It is stressed that tar is not in the UNIX equivalent of the VMS BACKUP command, as it was not designed to do the same job. VMS BACKUP was designed as a tool for system managers to manage system backups with. In this respect the UNIX analogue is the dump command. The dump and restore commands are used to backup entire disks to tape, normally as part of a system backup schedule. Both these commands provide high data integrity, an incremental facility, and a network protocol for backing up remote workstations. As a consequence dump and restore are quite tightly bound to a particular filesystem and operating system type. For these reasons the reader is referred to the man pages for dump(8) and restore(8) for further information.

 The tar program is simpler, and provides a readily transportable format for data interchange, and can be used whenever guaranteed data integrity is not an absolute requiement. As the tar program can write to a tape or a file (The two are essentially the same on UNIX) it also provides a handy means of packaging files for distribution. It is normally invoked with one major flag, and several optional flags. Each major flag defines the purpose of the program. Under OSF/1 the following flags are supported.


   Flag                 Meaning
   x                  Extract files from tape.
   c                  Create a new archive.
   t                  List the table of contents for an archive.
   u                  Updates an archive.

Files stored in a tar file or tape are stored with a pathname attached. If files are stored with absolute pathnames (with a leading '/') made on a remote system with a different directory structure, then they will not extract on the local system, as the directory structure is different, or you may not have write permission to the directory path stored in the tarfile. Likewise when you create tarfiles, it is a bad idea to create archives with absolute pathnames, for similar reasons. If you do encounter one of these tapes then use the 's' flag to strip off the leading '/'.

A selection of typical commands;

mssly1~> tar cvf /dev/rmt0h  /ydb/evn/*
mssly1~> tar xvf /dev/rmt0h  
mssly1~> tar cf  /scratch/fred.tar  /home/atp/idl /data0/atp/progs/*.pro
mssly1~> tar tfv /dev/nrmt1h
tar cvf /dev/rmt0h /ydb/evn/*
This command creates an archive on the named DAT tape (rmt0), with the contents of the /ydb/evn directory. The v flag gives a verbose listing of the process.
tar xvf /dev/rmt0h
This command extracts all the files from the tape archive on /dev/rmt0h, and gives a listing of the files as they are extracted. The files are extracted into the direcories stored on the tape.
tar cf /scratch/fred.tar /home/atp/idl /data0/atp/progs/*.pro
This command takes the contents of the contents of the directory /home/atp/idl and its subdirectories plus all the files matching /data0/atp/progs/*.pro and makes the archive silently into the file /scratch/fred.tar.
tar tfv /dev/nrmt1h
This command verbosely lists the table of contents of the tar archive on the EXABYTE tape drive on mssly1 (drive mt1).
The tar command can be made to read from standard input, and write to standard output. This means that it can be put in a pipeline of commands. One extremely useful example is this command;
mssly1~/idl> tar cf - *.pro | (cd /data0/atp/progs ; tar xfv - )
This may look horrendous, but works like this;

 The first tar program (tar cf - *.pro) makes an archive of all the .pro files in the current directory, and writes the result into a pipe. Waiting at the other end of the pipe, there is the second tar program (tar xfv - ) which has changed its current directory to /data0/atp/progs and which expands the archive written by the first tar program into the new directory. This particular command comes in very handy when you want to move an entire directory tree from one place to another, but dont want to change any of the dates or attributes of the files.

4) Foreign Tapes

There are three commonly encountered types of "foreign tape". A foreign tape is defined here as not having been written a compatible unix operating system,
  1. Raw Data Tape
  2. Ansi Labeled Data tape
  3. VMS BACKUP saveset
All of these types can be handled by Digital UNIX. Linux  does not presently have the tools to read ANSI or VMS BACKUP format tapes.
 

Raw Data Tape

This is handled in the same way as on any other system. Reading the data off such a tape is normally accomplished by using an user supplied program. Normal unix tape handling commands like mt can be used to position and manipulate the tape. If you just want to get the contents of the tape onto disk, then the dd command can be used to dump a raw image of the tape into a file, or other device.

 e.g.

   mssly1~> dd if=/dev/rmt1h of=/tmp/magtape
   2+0 records in
   2+0 records out
   mssly1~> ls -l /tmp/magtape 
   -rw-r--r--   1 atp      10          1024 Jan 20 17:37 /tmp/magtape
   mssly1~> 

Ansi Labeled Data tape

Ansi Labeled tapes are tapes written to conform with "ANSI X3.27-1978" (ansi 3) or "ANSI X3L5/83-28T" (ansi 4). These are standards for information exchange, and should work if all else fails. The program for reading and writing these tapes under OSF/1 is called ltf. It is similar to tar in many respects, and has a similar calling sequence. see ltf(1) and ltf(4) for more details

 e.g. To list the contents of a tape.

   mssly1~> ltf -tvf /dev/rmt0h
   ltf:  Volume ID is:  ATP_1     Volume is:  ANSI Version 3
   ltf:  Owner  ID is:                
   ltf:  Implementation ID is:   DECVMSBACKUP

   t(1,1) ---------   -/-   Sep 23  199410514(8192)F fred1.bck
   t(2,1) ---------   -/-   Sep 23  199410514(8192)F fred2.bck
   mssly1~> 

VMS BACKUP

VMS BACKUP savesets can be read and extracted on OSF/1 but not created. There are several compromises that have to be made, due to the differences between the two systems. For example there is no support for the multiple file types of VMS, nor for any of the extended file attributes.

 The program that is used to read VMS backup tapes is called vmsbackup. Again this follows the same pattern as the tar program, with major flags indicating the type of operation, and minor flags modifying some aspects of the selected process.

 e.g. To list the contents of a VMS BACKUP tape:

   mssly1~> vmsbackup -tvf /dev/rmt0h
   Volume: SOLAR_BACKUP 
   Saveset name: FRED.BCK   number: 1
   [ATP]16.DIR;1                                          7680     16 
   [ATP.16]16DEC_BFT.DATA;1                              33294     66 
   [ATP.16]2BCS_LC.EPS;1                                 12706     25 
   [ATP.16]6_CAXIX_PLOTS_LONG.EPS;1                     115494    226 
      :                                                    :       :
      :                                                    :       :
   [ATP.X]XV_211_TVTWM.EXE;1                            398336    779 
   [ATP.X]XV_MOTIF.EXE;1                                683520   1336 
   [ATP.X]XWATER.EXE;28                                  10752     22 
   [ATP]XTERM.HLB;1                                      44544     88 
   End of saveset: FRED.BCK

   End of tape
   mssly1~> 
For more information please see the vmsbackup man page. Also please note that the VMS BACKUP format is proprietary to Digital Equipment Corp. The vmsbackup command is not supplied by DEC, but is public domain software, and therefore not a supported program. It can deal with almost all common types of BACKUP tape, but occasionally it will encounter a format that it cannot decipher, and will be unable to read the data on that tape.
 
 

5) "None of the Above" tapes and misc

Mysterious Tapes

There are many different tape formats which I have not mentioned here, of which the most commonly encountered will be pax, cpio and ustar archives. If the tape is not one of the ones listed above, information about it can be listed by using the tcopy program.

 e.g.

    mssly1~> tcopy /dev/rmt1h
    file 0: block size 8192: record 0 
    file 0: block size 4291: record 1
    file 0: eof after 2 records: 12483 bytes
    file 1: block size 8192: record 0
    file 1: block size 4702: record 1
    file 1: eof after 2 records: 12894 bytes
    file 2: block size 5120: 23781 records
    file 2: eof after 23781 records: 121758720 bytes
    file 3: block size 5120: 1813 records
    file 3: eof after 1813 records: 9282560 bytes
    file 4: block size 5120: 
              :
              :
    mssly1~>
This information about blocksizes allows me to recognise that this tape is a Yohkoh data tape. It would also allow me to write a program to read the tape, with the correct blocksizes.
 
 

Tape to Tape copies

The simplest and easiest way to copy a tape is to use the tcopy command. This will duplicate tapes, independent of blocksize considerations. For example, to duplicate the contents of /dev/rmt0h to /dev/rmt1h:
     mssly1~>tcopy /dev/rmt0h /dev/rmt1h
     mssly1~>

6) Further Information

The online manual pages hold all of the information extracted for this document, plus all the details about the options not listed here. If you are at a loss to start, it is normally a good idea to use the keyword search facility of man. For example
 
 
    mssly1~> man -k tape
    vmsbackup (1)           - read a VMS backup tape
    gentapes (1)            - produce distribution magnetic tape media
    kitcap (4)              - Kit descriptor database for gentapes and 
                              gendisk utilities.
    ltf (1)                 - labeled tape facility
    ltf (4)                 - labeled tape facility
    mt (1)                  - Magnetic tape manipulating program
    mtio (7)                - magnetic tape interface 
    restore, rrestore (8)   - Restores files from tapes written with the 
                              dump or rdump command, respectively
    rmt (8)                 - Allows remote access to magnetic tape devices
    tapex (8)               - Tape exerciser program
    tar (1)                 - Manipulates tape archives
    tar (4)                 - Tape archive file format
    tcopy (1)               - Copies magnetic tapes
    tms (7)                 - TMSCP magnetic tape interface
    topen, tclose, tread, twrite, trewin,     
    tskipf, tstate (3f)     - FORTRAN tape I/O interface  
    tz (7)                  - SCSI magnetic tape interface
This lists the names of the manual pages (and the sections in brackets) of all the pages which match the keyword "tapes". To read the manual page for the tcopy program mentioned above the command man tcopy would be used.
 
 
atp@mssly1.mssl.ucl.ac.uk vn 0.1 20/1/95
atp vn 0.2 23/6/95
atp vn 0.3 5/8/99

http://www/www_computing/buns/tapes.html