PSF/AIX: MVS Download file receipt and spooling
ITEM: RTA000107669
Q:
ABSTRACT: PSF MVS Download file receipt and spooling
SEARCH ARG: psf download spooling
TOPIC THREAD: PRINT
PSF/AIX
..
In the MVS Download printing scenario, when a print file is
transferred to the AIX Server our documentation states it is
stored in an AIX filesystem. Subsequent to the completion of
the transfer, after the print file is sitting in the filesystem,
a SHELL script can execute to use the PSFIN command to submit
the print file to an AIX print queue for printing.
Can you clarify what is involved relative to the actual print
file on disk when PSFIN submits it to the queue? Is a pointer
added to the AIX print queue which points to the print file sitting
in the filesystem so that no double spooling takes place on the
AIX system? I can't answer this from the download manual or PSF AIX
admin or submitter manuals. Thanks.
A:
What psfin does depends on how the input is received (tape, file or
stdin) and the type of input datastream. In the section called "An
Overview of PSF for AIX Input Manager" (page 237 in my copy of Print
Admin, S544-3817-03), there is an outline of what occurs and the
corresponding Figure 51 on the following page. (All page references
in this item are to PSF/AIX Print Administration, S544-3817-03.)
You've said your customer is using MVS Download. In the most recent
version of the sample shell script provided (/usr/lpp/psf/bin/mvsprs.sh)
the input is passed to the psfin command as stdin. Thus, according to
the outline on page 237, no modification of the data is done.
Note that the sample shell script assumes the incoming data is line
data. If the data is another format, the sample script must be
modified accordingly.
If the incoming data is AFPDS, then no transform is invoked and psfin
invokes segmenter code to create job segment files and a job segment
list in (by default) /var/psf/segments and /var/psf/seglist respectively,
and enq's the segment list to the PSF/AIX queue for printing. The
enq command does not then make another copy of those segments; it points
to them through the seglist for that job.
On the other hand, if the incoming data is a type of data that must be
transformed (for example, by acif/line2afp), then psfin invokes the
appropriate transform according to the value in the JobScript. Some
transforms do create temporary copies of the incoming file in work areas
as part of the transform process. See page 246, "Processing Large
File with Input Manager" for information on the specific datastreams.
For example, if the incoming data is line data, no temporary files
are created by acif/line2afp. The output from the transform is then
segmented (according to the segment size you have specified in the
configuration, approximately 100KB by default) and can begin to print
while the remainder of the job finishes transforming. Again, the
enq command does not make another copy of those segments; it points
to them through the seglist for that job.
So if what you're really trying to determine when you ask about double
spooling is how much disk space you need in order to print a file of a
certain size, it will depend on the transform that is used and if
that transform makes a temporary copy in its workarea.
Q:
I have reviewed the chapter you referenced on page 237 and need
to confirm my understanding. My question is whether the print
file sent down from MVS Download is going to be written to disk
twice. Based on the answer above and reviewing the chapter
please confirm that I am accurate.
For example, assume I send a 5MB AFPDS print file from a host
to AIX Server using MVS Download. The MVS Download Server receives
that file on disk into a filesystem. At this point I have used
5MB of disk. The Download Server process invokes the shell
script, which feeds the file to the Segmenter via stdin. The
segmenter reads the 5MB file in and begins creating both a Segment
List File and Segment Data Files. By default, each Segment Data
File will be 100KB. I assume the first Segment Data File contains
the first 100KB of the downloaded file, the second Segment Data
File contains the second 100KB of the downloaded file and so on...
Therefore, when the segmenter has finished this process, I have
used 10MB + the size of the Segment List File (which I assume
are small) in disk space and created 50 Segment Data Files in my
work area. The Segmenter has effectively "copied or double written"
the original file on the AIX System. Is this an accurate
representation of what is going to happen?
---------------------------------------------------------------------
I understand that the Segment Data Files will not be read/written
again when they are enqueued to the AIX Spool, which would have
resulted in a triple write and double read.
1st Write by MVS Download, 1st Read by Segmenter, 2nd Write by
Segmenter, 2nd Read by Enq and 3rd Write by Enq.
---------------------------------------------------------------------
My net understanding is:
1. MVS Download Server will write the file once to a filesystem
as it is received from the host. After complete receipt step 2.
2. PSF AIX Segmenter will read the file from the filesystem and
write the file a second time as it creates Segment Data Files,
which are effectively 100KB copies of the original downloaded
file when printing AFPDS.
3. The Segment Data Files are enqueued to AIX via the Segment
List File and are read directly by PSF/AIX as they are
sent to the physical printer.
--------------------------------------------------------------------
Thanks for the further clarification.
A:
I checked with the developer and am including his complete process. He
reiterates a lot of what you and I have said, but I wanted to include
it in its entirety. I hope this helps.
----------------------------------------------------------------------
What I see in the attached note is pretty accurate. We do end up
"double-spooling" the MVS-Download files (plus or minus some space for
for files which are transformed (non-AFPDS files)). I'll try and
describe the MVS-Download process for an AFPDS file since it's easiest,
and the need to transform the data doesn't really affect the questions
asked in these notes.
The MVS-Download script receives the file into an AIX file, creates a
temporary job script (based on job attributes), then invokes psfin using
this job script, which will basically chop the incoming AFPDS into 100K
(the default segment size) segments. psfin creates a segment list file,
and once the first physical segment has been successfully created, it
then submits (enq) a job to the spool pointing to the segment list file.
This submission identifies that the segment list file is not to be
printed itself, but what it references is to be printed, and the print
process spools this little segment list file which contains mainly fully
qualified path names to the actual segment files that make up the print
job. At this point, there'd be 2X the size of the original file
(depending on how you've configured your workarea management
parameters--see below on cleanup).
Once psfin returns with a good (RC=0) return code, the download script
will delete the file that it had created, as well as any other temporary
files, and it is done with the job. The print request is then handled
(for reprint, restart, interrupt, etc.) like any other segmented job on
the system. At this point, you're back to having only one copy
(segmented) of the file, but for a period of time (how long depends on
the size of the file), you had double space requirements. There are some
error recovery/avoidance benefits to dumping the incoming data to a file
rather than directly spooling the data to psfin, and since not losing
jobs was a prime concern, this was the selected method.
We should also cover the question of cleanup of the segments. The
segments will remain in the system until they are manually deleted, or
the automated (preferred) process cleans them up. Once the job has
completed printing (or been taken off of the queue), the segment list
has a print status set for it. Under normal conditions (print completed
normally), the status is 'retained'. These jobs can be reprinted as
long as they remain on the system. If the job was cancelled (with errors
or by the user), the segment list is placed in 'cancelled' status,
and under normal circumstances cannot be reprinted without resubmitting
the job from the source location. Jobs for which there was a printer
problem, or printing was halted by the customer, segment lists are
placed in 'queued' state. These will print (from the beginning) when
the printer is available.
Cleanup of segments goes like this: No segments (or segment lists)
are deleted until the upper threshold (default of 90% of /var
filesystem) is reached. Once the threshold is reached, the automatic
cleanup process is invoked by psfin (actually the segmenter program)
when space is needed for a segmenting job.
The cleanup process does:
- Jobs which are in 'queued' state but reference spool ids which do
not show up on the AIX spool (have been deleted manually by the
customer using the enq -x command), are marked as 'cancelled'.
- All 'cancelled' jobs are deleted
- If there still is a need for more space, we'll check to see if the
number of retained bytes (total number of bytes associated with
all jobs in 'retained' state) is above the retain threshold. If
it is, we delete jobs, starting with the oldest, until we're back
below the retain threshold.
- If we still have no space, we'll check jobs in printing state, and
if segments have been 'stacked', we'll delete segments which have
been printed. This is needed for times when you're printing that
1GB file using a 500 MB filesystem.
- If we still have no space available, psfin will just keep polling
until something changes (a new filesystem is added to segment path,
the percentages are modified, segments are stacked, jobs cancelled,
etc.). At no time do we delete segments which are associated with
'queued' jobs.
------------------------------------------------------------------------
I hope this helps.
(Reviewed 12/1999)
S e a r c h - k e y w o r d s:
psf/6000 psf/aix psf aix mvs download psfin shell mvsprs.sh spool
segment segmenter Input Manager workarea performance disk double
infoprint ipmgr ipm os/390
WWQA: ITEM: RTA000107669 ITEM: RTA000107669
Dated: 12/1999 Category: XPSF6000
This HTML file was generated 2000/11/30~13:34:07
Comments or suggestions?
Contact us