ITEM: F5728L
Problem copying sparse files to a new filesystem
Question:
I have a system where I am trying to copy a filesystem from one volume
group to another volume group. I am using the "cp" command to
accomplish this. The source filesystem is called "/uo2" and is 800meg
in size. The actual amount used from the "df" command is 600meg. I am
trying to copy this to another filesystem that is 655 MB in size. The
copy only gets about 2/3 of the way and stops with the filesystem
becoming full. What gives?
Response:
The customer has Oracle database files in his system. It turns out
that the database files were created as sparse datafile. This means
that though they might have been allocated by Oracle at 100meg, since
Oracle uses null bytes to allocate it, the null blocks are NOT
actually allocated physical space. The problem the customer is
experiencing is related to problem C6019. Apparently, "cp" is
expanding these files when they hit the destination filesystem and
causing the filesystem to fill up before completion of the copy.
Response:
To understand this problem I wrote the following c program:
\# mknullfile.c - create a file full of null characters
\#include \
\#include \
\#define FILE_NAME "null.file"
main()
{ int i;
FILE *out_file;
out_file=fopen(FILE_NAME,"w");
if (out_file == NULL ) {
(void) fprintf (stderr,"can not open output file\\n");
}
for (i=1;i\<=102400;i++) {
fputc('\\0',out_file);
}
(void) fclose(out_file);
}
This creates a file full of nulls that is 102400 bytes called null.file.
\# fileplace -v null.file
File: null.file Size: 102400 bytes Vol: /dev/hd1 (4096 byte blks)
Inode: 34869 Mode: -rw-r--r-- Owner: dbraden Group: system
Logical blocks
--------------
34949-34956 8 blks, 32 KB, 32.0%
34958-34974 17 blks, 68 KB, 68.0%
25 blocks over space of 26: space efficiency = 96.2%
2 fragments out of 25 possible: sequentiality = 95.8%
\# find . -name null.file -print | backup -ivf/tmp/null.bu
then cd to another directory and
\# restore -xvf/tmp/null.bu
\# fileplace -v null.file
File: null.file Size: 102400 bytes Vol: /dev/hd1 (4096 byte blks)
Inode: 88110 Mode: -rw-r--r-- Owner: dbraden Group: system
Logical blocks
--------------
unallocated 24 blks, 96 KB, 96.0%
88131 1 blk, 4 KB, 4.0%
1 blocks over space of 1: space efficiency = 100.0%
1 fragment out of 1 possible: sequentiality = 100.0%
Thus we have a sparse file. ls -l shows the same size for both files
\# cp null.file null.file2
\# fileplace -v null.file2
File: null.file2 Size: 102400 bytes Vol: /dev/hd1 (4096 byte blks)
Inode: 88154 Mode: -rw-r--r-- Owner: dbraden Group: system
Logical blocks
--------------
88152-88159 8 blks, 32 KB, 32.0%
88161-88177 17 blks, 68 KB, 68.0%
25 blocks over space of 26: space efficiency = 96.2%
2 fragments out of 25 possible: sequentiality = 95.8%
Now the file is not sparse. Thus the customer's problem of running
out of space when he does a copy.
backup & restore will preserve sparseness
tar will not preserve sparseness - it will take more blocks after
extraction
pax will aggressively preserve sparseness - a non sparse fill will
become sparse after being backuped and restored with pax
du incorrectly shows the number of blocks for a sparse file. In the
previous example "du -a null.file" will show either 8 or 200 blocks
for these files if they are sparse/not sparse respectively. df also
incorrectly states the amount of space taken by sparse files!!!
With the sparse.file in /home
\# df /home
Filesystem Total KB free %used iused %iused Mounted on
/dev/hd1 794624 120432 84% 18436 9% /home
\# del sparse.file (which is 102400 bytes)
\# df /home
Filesystem Total KB free %used iused %iused Mounted on
/dev/hd1 794624 120440 84% 18435 9% /home
It freed up only 8K!!! This is a potential problem in reducing
filesystems. Use either pax or backup/restore (not tar) when
reducing filesystems.
Customer is on AIX 3.2.3. He would like us to report a defect because
he cannot use cp; however, the man page for du states that "unallocated
blocks are not accounted for in the reported block counts." The df
command states that "Under certain exceptional conditions, these counts
may be in error. For example, if a file system is being actively
modified ... "
I also found a document on sparse files:
/public/support/aix/backup/sparse_files
which also documents the effects of pax and dd on sparse files.
A way to determine which, if any, files are sparse follows:
find . -print | xargs -n1 fileplace /tmp/fileplace.output
Then vi /tmp/fileplace.output and search for "unallocated".
Response:
I called the customer and explained why they reporting a defect would
be to no avail. Now that they understand the problem, they can live
with it though they are somewhat disappointed they have to deal with
it.
HR
CENTER
FONT size=2Support Line: Problem copying sparse files to a new filesystem ITEM: F5728L
BRDated: January 1994 Category: N/A
BRThis HTML file was generated 99/06/24~13:30:51
BRComments or suggestions?
A href="../../../../feedback.htm"BContact us/B/ABR
/FONT
/CENTER
/BODY
/HTML