ITEM: BC3392L

Information on tapes and tape drives



Response:

 1.200  Some info about tape backups
 1.201  How do I do remote backup?
 1.202  How do I backup a multi-disk volume group?
 1.203  How do I put multiple backups on a single 8mm tape?
 1.204  How can I make an exact duplicate of a tape over the network?
 1.205  What is tape block size of 0?
 1.206  Resetting a hung tape drive...
 1.207  How do I read a mksysb tape with tar?


1.200: Some info about tape backups

The following supplements the information on rmt devices in
InfoExplorer. It is based on my own personal experience with IBM tape
drives running on AIX 3.1. No warranty is expressed or implied.

CONFIGURING THROUGH SMIT:
    BLOCK size (0=variable length)              (ALL)
        Sets the tape block size.  When reading, the block size must be
        set to the block size set when the tape was written.  When
        using some commands, tapes written with ANY block size can be
        read if the block size is set to 0 (variable length) (see
        "BLOCK SIZES" below).

    Use DEVICE BUFFERS during writes            (ALL)
        Set to yes, the device will buffer data internally on writes.
        This greatly improves performance, but under certain cases may
        be undesirable since the data is not written to tape before
        returning a good indication.

    Use EXTENDED file marks                     (8mm only)
        Extended file marks take up much more space than short (or
        non-extended) file marks.  But extended file marks can be
        overwritten, allowing data not at the beginning of tape to be
        overwritten (see "FILE MARKS" below).

    RETENSION on tape change or reset           (1/4" only)
        If set to "no" then the tape will not be retentioned
        automatically when the tape is inserted.  Note that this will
        take effect only after the device is used.

FILE MARKS:
     Tape devices support multiple tape files.  Tape files are the
     result of a backup/cpio/tar/dd type command, where the device is
     opened, written to, and closed.  Because tapes allow large
     quantities of data to be written on a single tape, several backups
     (that is, tape files), may be combined on one physical tape.
     Between each tape file is a "tape file mark" or simply "file
     mark".  These file marks are used by the device driver to indicate
     where one tape file ends and another begins.

                              B       E
                   \<-------   O       O   ------->
                              T       T
              __ ____________________________ _______________
     physical | \\            |       |      \\             |physical
     beginning|  \\           | tape  |       \\            | end
       of     |   \\          | file  |        \\           |  of
      tape    |    \\         | mark  |         \\          | tape
              |_____\\________|_______|__________\\_________|
      Note that there is a distinction between the beginning of tape
     (BOT) side of a file mark and the end of tape (EOT) side of a file
     mark.  If the head is on the BOT side of a file- mark, "tctl fsf
     1" command will move only to the EOT side of the same file mark.

     With the 1/4" tape drive, writing can only take place
     sequentially, or after blank tape has been detected.  You cannot
     write over data on the tape (except at BOT).  If you wish to add
     data to a tape which has been written and then rewound you should
     space forward file mark until an error occurs.  Only then can
     you start writing again.

     With an 8mm tape drive, writing can only take place before blank
     tape, an EXTENDED file mark, or at BOT.  Thus if several backups
     have been made on one tape and you wish to overwrite one of the
     backups, position the tape to the place you wish to start writing
     and issue the following commands:
        tctl bsf 1
        tctl eof 1
     The first command skips back to the BOT side of the same file
     mark.  The second command rewrites the file mark (writing is
     allowed before extended file marks).  The erase head will erase
     data ahead of the write head, so that after writing the file mark
     the head will be positioned before blank tape.  Only after this
     may you start writing over data in the middle of the tape.  (All
     data beyond where you are currently writing will be lost).  Note
     that you cannot write over short file marks.  In order for this to
     work, the tape must have been written with extended file marks
     (use smit to change this).

     With the 9-track drive writing can take place anywhere on the
     tape although overwriting single blocks of data is not supported.

     On the 8mm drive extended filemarks use 2.2 megabytes of tape and
     can take up to 8.5 seconds to write.  Short filemarks use 184K
     and take up to 1.5 seconds to write.

BLOCK SIZES:
     When data is written to tape it is written in blocks.  The blocks
     on a tape are separated by inter-record gaps.  It is important to
     understand the structure of the written tape in order to
     understand the problems which can occur with changing block
     sizes.

     In fixed block size mode all blocks on the tape are the same
     size.  They are the size of the block size set in the device
     configuration.  All read()s and write()s to the tape drive must be
     a multiple of the fixed block size.

     In fixed block mode a read() will return as many blocks as needed
     to satisfy the read() request.  If a file mark is encountered
     while reading the tape only the data up until the file mark will
     be returned.

     It is not possible for the tape drive to read a tape whose block
     size is not the same as the block size in the device
     configuration.  (Unless the device configuration is in variable
     size blocks.)

     In variable block size (0) mode, the blocks written on the tape
     are the size of the read() and write() requests to the device
     driver.  In this case, the actual block sizes on the tape can be
     changed using the options to the backup commands (tar -C, cpio -C,
     backup -C).

     In variable mode, read() requests greater than size of the block
     on the tape will return only the data from the next block on the
     tape.  It is this feature that allows tapes written in any block
     size (fixed or variable) to read with the dd command (the output
     from the dd command may be piped to restore, tar, or cpio for
     example.)  Note that backup, tar, and cpio cannot read all tapes
     by using a large block size because they assume there is an error
     if they get a short read().
                dd ibs=128k obs=16k if=/dev/rmt0 | ...

     The tape head is always positioned at an inter-record gap, file
     mark, or blank tape after reading or writing.

     With the 8mm tape drive, using a fixed block size which is not a
     multiple of 1K is inefficient.  The 8mm tape drive always writes
     internally in 1K blocks.  It simulates the effect of variable
     block sizes, but, for example, using a fixed block size of 512
     bytes (or using variable block size and write()ing 512 bytes at a
     time) wastes one half of the tape capacity and gives only one half
     the maximum transfer rate.

     To figure out a tape's actual block size try:

     1). Set the tape to variable block size.
     2). "dd if=\ of=/tmp/dummy bs=128k count=1"
     3). "ls -l /tmp/dummy"
     4). The number of bytes in "/tmp/dummy" is the physical block size.

EXCHANGING DATA WITH NON-UNIX AND OTHER VENDORS MACHINES:
     Many tape drives support both variable and fixed block sizes.

     Variable block mode writes block sizes the size of the write
     command issued (tar and backup specify this with the -b option). 
     In fixed mode, block sizes are fixed and all writes must be a
     multiple of the fixed block size.

     Unix often internally chops larger reads and writes up into
     manageable pieces (often 65535, 65534, or 65532 bytes) before
     doing the actual reads and writes.  This means reads and writes of
     64K bytes are often broken up into a 65535 byte record and a 1
     byte record (In fixed mode the write will fail).  Block sizes >=
     64K (-C128 and greater) should be avoided for this reason.  AIX
     does not break up read and write requests, but be aware of the
     situation on other machines.

     If the tape is written in an unknown block size then set the
     device configuration in smit to use variable size blocks, use the
     "dd" command with a large input block size, and pipe it to the
     restore command.  For example:
          chdev -l rmt0 -a block_size=0
          dd if=/dev/rmt0 ibs=128k obs=16k | tar -tvf-


1.201: How do I do remote backup?

There seems to be several ways of doing this. I found the following works:

  tar -b1 -cf - . | rsh REMOTEHOST "dd ibs=512 obs=1024 of=/dev/TAPEDEVICE"


mksysb will not back up to remote tape devices. The following script
will do remote backups.

\#!/bin/ksh
\# @(\#) Create a backup tape of the private user data.
\#=================================================================\#
\#   Script :  usave.sh                                            \#
\#   Author :  F. Kraemer                                          \#
\#   Date   :  92/02/19                                            \#
\#   Update :  92/10/29                                            \#
\#   Info   :  the ultimative backup script                        \#
\#   Example:  usave.sh /dev/rmt0      -  save to local tape       \#
\#             usave.sh /save/save.me  -  save to local file       \#
\#             usave.sh /tmp/pipe      -  save to remote tape      \#
\#-----------------------------------------------------------------\#
PS4="(+) "
\#set -x
PROG=$(basename $0)
HOST=$(hostname)
TODAY=$(date +%H:%M:%S)
\#
\# cleanup
\#
cleanup ()
{ec=$1
error=$2
case "$ec"
in
   "$USAGE_EC")    \# usage error
     error="Usage:\\t$PROG DeviceName\\n" 1>&2
    ;;
   "$NOTAP_EC")    \# Tape error
     error="error:\\t$PROG: $DEVICE is not available on the system.\\n" 1>&2
    ;;
   "$LISTE_EC")    \# list error
     error="error:\\t$PROG: could not create tar list for $LOGNAME.\\n" 1>&2
    ;;
   "$NOTAR_EC")    \# tar command error
     error="error:\\t$PROG: tar command failed.\\n" 1>&2
    ;;
   "$PIPEP_EC")    \# pipe error
     error="error:\\t$PROG: mknod command failed.\\n" 1>&2
    ;;
   "$NORSH_EC")    \# rsh error
     error="error:\\t$PROG: rsh - Remote Shell command failed.\\n" 1>&2
    ;;
   "$RHOST_EC")    \# remote host error
     error="error:\\t$PROG: Remote Host unknown.\\n" 1>&2
    ;;
   *)
   ;;
esac
case "$DEVICE"
in
    \#
    \# Fix the block size if $DEVICE is a tape device
    \#
    /dev/rmt[0-9]*)
        echo "\\n\\t$PROG: Rewinding tape to begin.........(please wait)\\n"
        tctl -f $DEVICE rewind 2>/dev/null
        ;;
    *) ;;
esac
rm -f ${LIST} ${PIPE} 2>/dev/null
[ -n "$error" ] && echo "\\n${error}\\n"
trap '' 0 1 2 15
exit "$ec"
}\#
\# Variables
\#
USAGE_EC=1                         \# exit code for usage error
NOMNT_EC=2                         \# exit code wrong device name
NOTAP_EC=3                         \# exit code no tape available
LISTE_EC=4                         \# exit code backup list error
NOTAR_EC=5                         \# exit code for wrong tar
TRAPP_EC=6                         \# exit code for trap
PIPEP_EC=7                         \# exit code for pipe
RHOST_EC=8                         \# exit code for bad ping
NORSH_EC=9                         \# exit code for bad rsh
DEVICE="$1"                        \# device to tar into
LIST="/tmp/.tar.$LOGNAME.$$"       \#
REMOTEH=""                         \# Remote host for backup
REMOTET=""                         \# Remote tape for backup
tapedev=                           \#
PIPE="/tmp/pipe"                   \# Pipe for remote backup
\#
\# main()
\#
tput clear
echo "\\n\\t$PROG started from $LOGNAME@$HOST on $TERM at $TODAY.\\n"
rm -f $LIST 2>/dev/null
\#
\# Trap on exit/interrupt/break to clean up
\#
trap "cleanup $TRAPP_EC \\"Abnormal program termination. $PROG"\\"  0 1 2 15
\#
\# Check command options
\#
[ "$\#" -ne 1 ]  &&  cleanup "$USAGE_EC" ""
\#
\# Check device name
\#
[ `expr "$DEVICE" : "[/]"` -eq 0 ] && cleanup "$NOMNT_EC" \\
        "$PROG: Backup device or file name must start with a '/'."
\#
\# Check tape device
\#
case "$DEVICE"
in
    \#
    \# Fix the block size if $DEVICE is a tape device
    \#
    /dev/rmt[0-9]*)
        \#
        echo "\\n\\t$PROG: Verify backup media ($DEVICE)............\\n"
        \#
        \# see if a low or high density tape device was specified
        \# (eg rmt0.1)
        density="`expr $DEVICE : \\
                "/dev/rmt[0-9]*\\.\\([0-9]*\\)"`"
        \#
        \# strip /dev/ from device name and
        \# get the base name (eg translate:
        \# /dev/rmt0.2 to rmt0)
        \#
        tapedev="`expr $DEVICE : \\
                "/dev/\\(rmt[0-9]*\\)[\\.]*[0-9]*"`"
        \#
        \# Check if the tape is defined in the system.
        lsdev -C -c tape -S Available -F "name" | grep $tapedev >/dev/null 2>&1
        rc=$?
        [ "$rc" -ne 0 ] && cleanup "$NOTAP_EC" ""
        \#
        \# Restore old tape name.
        \#
        [ "${density:-1}" -lt 4 ] && density=1 || density=5
        DEVICE="/dev/${tapedev}.${density}"
        echo "\\n\\t$PROG: Insert a tape in ($DEVICE)........(press enter)\\n"
        read TEMP
        echo "\\n\\t$PROG: Rewinding tape to begin...........(please wait)\\n"
        tctl -f $DEVICE rewind 2>/dev/null
        ;;
    \#
    \# Backup is done on remote host. The remote shell facility
    \# must be set up and running.
    \#
    ${PIPE}*)
        \#
        echo "\\n\\t$PROG: Assuming remote backup via network.\\n"
        echo "\\t$PROG: Enter name of Remote Host   ===> \\c"
        read REMOTEH
        echo "\\n\\t$PROG: Pinging Remote Host to test connection.\\n"
        ping ${REMOTEH} 1 1 >/dev/null 2>&1
        rc=$?                                    \# give up unknown host
        [ "$rc" -ne 0 ] && cleanup "$RHOST_EC" ""
        JUNK=$(rsh ${REMOTEH} "/usr/sbin/lsdev -C -c tape -S Available")
        rc=$?                                    \# give up rsh failed
        [ "$rc" -ne 0 ] && cleanup "$NORSH_EC" ""
        echo "\\t$PROG: Available Tapes on ${REMOTEH} are :\\n\\n\\t\\t${JUNK}\\n"
        echo "\\t$PROG: Enter name of Remote Tape (e.g. /dev/rmt0) ===> \\c"
        read REMOTET
        echo "\\n\\t$PROG: Insert tape on ${REMOTEH} in ${REMOTET}..(press enter)"
        read TEMP
        echo "\\t$PROG: Rewinding Remote Tape ${REMOTET} on ${REMOTEH}.\\n"
        rsh ${REMOTEH} "tctl -f ${REMOTET} rewind"
        rc=$?                                    \# give up rsh failed
        [ "$rc" -ne 0 ] && cleanup "$NOTAP_EC" ""
        rm -f ${PIPE} 2>/dev/null
        mknod ${PIPE} p
        rc=$?                                    \# give up mknod failed
        [ "$rc" -ne 0 ] && cleanup "$PIPEP_EC" ""
        cat ${DEVICE} | rsh ${REMOTEH} "dd of=${REMOTET} obs=100b 2>/dev/null" &
        ;;
    *)  ;;
esac
\#
\# Prepare the list
\#
echo "\\n\\t$PROG: Create list of files to be saved...."
find $HOME -print > $LIST
rc=$?
[ "$rc" -ne 0 ] &&  cleanup "$LISTE_EC" ""
\#
\# tar the files
\#
echo "\\n\\t$PROG: Changing current directory to (/)...."
cd / > /dev/null 2>&1
echo "\\n\\t$PROG: Running tar format backup from user ($LOGNAME)...."
tar -cvf "$DEVICE" -L "$LIST"
rc="$?"
[ "$rc" -ne 0 ]  && cleanup "$NOTAR_EC" ""
\#
\# Backup completed
\#
TODAY=$(date +%H:%M:%S)
echo "\\n\\t$PROG ended at $TODAY............................\\n\\n"
cleanup 0


1.202: How do I backup a multi-disk volume group?

  1. If you have a set of three or more disks in a volume group
     (typically 3 for 5xx machines with three internal drives;
     with only two, the procedures outlined here have to be modified
     to ignore the fact that you don't have a quorum in the volume group)

  2. If one drive has failed (usually only one fails at a time :-) )

It is possible to go through a service boot (the volume group is called
rootvg and one of the 2 good disks on it is called hdisk0):

  importvg -y rootvg hdisk0
  varyonvg -f -n -m1 rootvg

These commands will work, but give error messages. If you wish to mount
a user filesystem, say /u on logical volume /dev/lv00, then

  mount -f /dev/lv00 /v

will work only if jfslog, the journaled file system log device, is not
on the damaged disk. If it is, you must (and can in any case) mount the
filesystem read-only:

  mount -f -r /dev/lv00 /v

This crucial and rather obvious point baffled several level 3 support
personnel at Austin as well as myself for almost a week. Once the file
system(s) of interest are available, they can be saved to tape for
restoration later. Of course, one can expect only about two thirds of a
filesystem to be recoverable if it spans all 3 physical disks. One
other point to remember is that the standard boot procedure from floppy
includes the restore command but does not include the backup command.

*****************************************************************************
* If you do not have other RS6000 machines at your site it is imperative    * 
* that you either build a bootable tape which includes either restore or    * 
* tar or cpio (a bootable floppy set will not have enough space) or at the  *
* very least copy onto a spare floppy backup, cpio, or tar.  The floppy     *
* should be created with backup -ivq so that its contents can be read into  *
* the memory resident system after booting.                                 *
*****************************************************************************

All is not lost if tar, cpio or backup are available on an undamaged
disk that can be mounted. Since tar and cpio are in /bin, they may both
very well be unavailable.

It is a very good idea for those who have tape devices to build a
bootable tape with their desired extra commands in it. Follow the
instructions from IBM but add your desired commands to the following
three files:

    /usr/lpp/bosinst/tape2
    /usr/lpp/bosinst/diskette/boot2
    /usr/lpp/bosinst/diskette/inslist

If you have anything other than a minimum memory configuration, you
should be able to add many commands.


1.203: How do I put multiple backups on a single 8mm tape?

There are two possible solutions to this, both of which use /dev/rmt0.1
which is non-rewinding.

SOLUTION \#1
-----------

To put multiple backups on a single tape, use /dev/rmt0.1, which is a
no-rewind device, using either rdump or backup (both by name & inode
work). Using rdump or backup "byinode" both generate the message that
the tape is rewinding but actually do not. This is an example that
works on my system:

\# rsh remote1 -l root /etc/rdump host:/dev/rmt0.1 -Level -u /u
\# rsh remote2 -l root /etc/rdump host:/dev/rmt0.1 -Level -u /u
\# tctl -f /dev/rmt0.1 rewind       \# rewinds the tape

where I am implementing the command from host.
To restore a table of contents of the first I would use 

\# restore -f /dev/rmt0.1 -s1 -tv

where the -s1 flag tells restore to go to the first record on the tape. 
Type the exact command again to get the second record. The -s(Number)
means go to Number record from this spot. It works pretty well.

SOLUTION \#2
-----------

------------------- Dump.sh --------------------
CONTENTSFILE=`date |dd conv=lcase |sed -e 's/19//' |awk '{print $6 $2 $3}'`
set -x
LEVEL=$1
shift

backup -c -b 56 -$LEVEL -uf /dev/rmt0.1 /
backup -c -b 56 -$LEVEL -uf /dev/rmt0.1 /usr
backup -c -b 56 -$LEVEL -uf /dev/rmt0.1 /u
tctl -f /dev/rmt0 rewind

touch /usr/local/dumps/Contents.$CONTENTSFILE
echo "Dumping /" >>/usr/local/dumps/Contents.$CONTENTSFILE
restore -t -s 1 -f /dev/rmt0.1 >>/usr/local/dumps/Contents.$CONTENTSFILE
echo "Dumping /usr" >>/usr/local/dumps/Contents.$CONTENTSFILE
restore -t -q -s 1 -f /dev/rmt0.1 >>/usr/local/dumps/Contents.$CONTENTSFILE
echo "Dumping /u" >>/usr/local/dumps/Contents.$CONTENTSFILE
restore -t -q -s 1 -f /dev/rmt0.1 >>/usr/local/dumps/Contents.$CONTENTSFILE
tctl -f /dev/rmt0 rewind

I process the table-of-contents first by a little program that does
common prefix encoding, and then compress.

This gives a table of contents file I can keep on-line until the tape
is reused.


1.204: How can I make an exact duplicate of a tape over the network?

The challenge here is not to have to create a temporary file (disk space
limitation) and work across heterogeneous networks.

This script might work:

LOCAL=/dev/tape_dev
REMOTE=/dev/tape_dev
dd if=$LOCAL ibs=64k obs=512 | rsh remote_host dd ibs=512 obs=64k of=$REMOTE

From: pack@acd.ucar.edu (Daniel Packman)

Daniel provides the following perl script to convert from the known
world's function codes to AIX for compatibility.

\#!/bin/perl
\# Wrapper to convert input rmt requests to
\# AIX 3.2 ioctl numbers.  We pass on all commands we don't understand
\# I0 MTWEOF -> I10  STWEOF write and end-of-file record
\# I1 MTFSF  -> I11  STFSF  forward space file
\# I2 MTBSF  -> I12  STRSF  reverse space file
\# I3 MTFSR  -> I13  STFSR  forward space record
\# I4 MTBSR  -> I14  STRSR  reverse space record
\# I5 MTREW  -> I6   STREW  rewind
\# I6 MTOFFL -> I5   STOFFL rewind and unload tape
\# I7 MTNOP  -> I0   (no-op? should ignore following count)
\# I8 MTRETEN-> I8   STRETEN retension tape, leave at load point
\# I9 MTERASE-> I7   STERASE erase tape, leave at load point
\#I10 MTEOM (position to end of media ... no ibm equivalent?)
\#I11 MTNBSF  (backward space file to BOF ... no ibm equivalent?)
@iocs = (10,11,12,13,14,6,5,0,8,7);
open(RMT,"|/usr/sbin/rmt") || die "Can't open pipe to rmt\\n";
select(RMT);
$| = 1;
while (\) {
  s/(\^I)(\\d$)/I$iocs[$2]/;
  exit 0 if $_ =~ /\^[Qq]/;
  print RMT $_ ; }
exit 0;


1.205: What is tape block size of 0?

Tape devices are generally split into two categories: fixed block and
variable block.  1/4" tape is the fixed block, and 8mm is variable.

On a fixed block size device, the kernel always sends data to the device
in suitable block size lumps, and varying the size passed to write(2)
(e.g., via the bs option to dd) gives the kernel more data to stream. 
On a variable block size device, the kernel writes to the device
whatever passed to it. On an 8mm, it had better be a multiple of 1024
to get efficient tape usage.

AIX has the World's Only Variable Block Size 1/4" tape drive. If you
use SMIT to set the block size to a nonzero value, AIX treats the device
as fixed block size, whether it is or not. By default, 8mm drives are
set to the same size as 1/4", 512 bytes. This is wasteful, but
otherwise mksysb and installp would fail.

If you set the block size to 0, the device is treated as variable block
size, and the size passed to write becomes the physical block size. 
Then if you use a sensible block size to dd, all should be wonderful.


1.206: Resetting a hung tape drive

A process accesses the tape drive. The process stops, exits, or whatever,
but still hold on to the drive. When this happens, the process cannot be
killed by any signal and the tape drive cannot be used by any other
process until the machine is rebooted.

The following should help:

RESET:

AIX, like most UNIX systems has no reset function for tape drives. You
can however send a Bus Device Reset (a standard SCSI message) to the
tape drive using the following piece of code. If the tape drive does
not respond to the BDR, then a SCSI Bus Reset will be sent (and this
will reset every device on the SCSI Bus). SCSI Bus resets are rather
extreme so you should refrain from using this program unnecessarily. 
But there are times (like after you've inserted a jammed/old/bad tape in
an 8mm drive), when there's no other way to reset the device other than
to shutdown and reboot (obviously you can power down and up an external
drive to reset it - and this would be the better choice).

This is actually documented in info, but can be hard to find and
there's no complete program.

/* taperst: resets the tape drive by sending a BDR to the drive. */
\#include \
\#include \
\#include \
\#include \

int main(int argc, char **argv)
{         /* This can be run only by root */

         if (argc != 2) {
             fprintf(stderr, "Usage: %s /dev/rmt\#\\n", argv[0]);
             return 1;
         }

         if (openx(argv[1], O_RDONLY, 0, SC_FORCED_OPEN) \< 0) {
             perror(argv[0]);
             return 2;
         }
         return 0;
}1.207: How do I read a mksysb tape with tar?

To recover specific files from a backup made with mksysb, try
$ tctl fsf 3
$ tar xvf/dev/rmt0.1 ./your/file/name



Support Line: Information on tapes and tape drives ITEM: BC3392L
Dated: November 1995 Category: N/A
This HTML file was generated 99/06/24~13:30:23
Comments or suggestions? Contact us