A brief technical description of hardware/software interaction follows,
ending in a set of questions and issues.
Introduction
=============
Customer is developing rostering applications using RS6000 architecture
Customer is running scheduling and graphics applications which require
fast access to large quantities of data. The speed of access required
is too slow for current implementation of relational database. In
order to obtain the data quickly enough, files containing a networked
model of the data are being used.
As we are using client server architecture and have requirements for
sharing data, we use memory mapping (mmap()) and shared memory(shmem())
to gain access to the files
The set of memory mapped files will be in the range of 100-125
files per application.
Concerns
========
This architecture has previously been implemented using SUN Microsystem
architecture. The current project on Rs6000 is more complex and more
data are involved.
It appears(from reading Inforexplorer) that the Virtual Memory Manager
controls mmap() calls. This would appear to differ from SUN Microsyste
architecture which handles mmap() calls through the buffer cache.
For the machines where scheduling applications take place it is assumed
that the main memory(RAM) will be large enough to contain all data as
well as the application(this is because any paging causes major
degradation performance for these applications)
However, the machines on which graphics applications take place are
assumed to have more data than there is main memory. It is
nevertheless important that the application is not paged in or out.
Model
======
ROSTERING MODULE
---------------------------
| --------- |
||Server | ------ | Rostering Server
||Process | |mmap | |
|| |----->|files | |
|| mmapp()| ------ |
|| | | |
| --------- | |
| --------- |
------------|--------------
|NFS
|
------------|--------------
| --------- | | Graphics Workstations
||Graphics | | ------ |
||Appl | | |shmem | |
|| | | |() | |
|| mmap()---- ------ |
|| | |
| --------- |
---------------------------
SCHEDULING MODULE
---------------------------
| --------- |
||Server | |
||Process | |
|| | |
|| | |
| --------- |
| |
| ---------- ---------- |
||Scheduling| |shmem() or|| Rostering Server
||Process | |malloc() ||
|| | |data{same ||
| --------- |that were ||
| |mmapped ||
| |above ||
| -----------
---------------------------
Issues and Questions
====================
1. Since there are 100-125 files per application, the
server machine will have up to 500 files open at one time.
Is that possible ??
Are there performance implications??
is file size a factor??
2. What happens to main memory when files are memory mapped??
Does the Virtual Memory Manager only store one page a time
in main memory or does it attempt to fill the entire memory
(since the size of the memory map files on the workstations will
be larger than their main memory)??
Is there a limit to the number of files that can be memory mapped(
per machine, per user or per process)?
Is there a limit to the size of files to be memory mapped??
Is there a limit to the number of machines/processes which can
mmap() a given file??
Can the size of virtual memory be configured??
3. If the number of open files is a problem, we can collect a
group of files in a single, larger file. However if the
virtual memory manager only takes a page from a file at a time, then
will that adversely affect performance if all of the sub-files in
a file need to be accessed simultaneously.
4. If the VMM allocates all main memory for memory mapped files,
is there a way to ensure that a process is never swapped out
of main memory??
That is, is file swapping preferred over process swapping??
5. When using the msync() function with the MS_SYNC flag set,
does the function wait until the file has actually been written to
or only until the NFS daemons have been scheduled??
6. If the VMM controls memory mapped files and msync() is called
with the MS_INVALIDATE flag, what is the function actually
invalidating??
hi,
Can U tell me if you an answer for item 5 and 6?
ANSWER
R: Let me try and answer your questions as follows:
1) AIX will allow 2,000 files per process and 2,000,000 files max open
on the system at once. So, a process can open 500 files.
"Are there performance considerations" yes, the more resources
in use on the system the more performance will be decreased. But
this should not be a major problem. Having one file of 5MB or
10 at 500K each should not make much of a difference in performance.
2) AIX uses a demand based paging algorithm. This means that pages
are not removed or brough into memory until needed. The Virtual
Memory Manager VMM will bring into main memory as much of a file
as needed when it is needed. It will aslo use a page ahead
algorithm when sequential reads a requested.
3) From development no real limit has been set for mmap
in terms of the number of file mmap'ed. The real limits is
a per process data space of 2.5GB. Also, virtual memory or
page space can be a limit in terms of size of mmap'ed files.
4) Yes, it is possible to "pin" a process. Or the schedtune
command can be used to balance the ratio of memory used for
processes and memory used for data storage.
The last two questions will require a little more research. If
you would like to have more information on the previous items reopen
this item other wise, the answer to items 5 and 6 will be appended
when available.
R: The following is the response I got back from development.
This is the answer regarding the MS_INVALIDATE flag
Invalidate. For working storage segments all pages
in memory are released, all paging space blocks are
freed and the page states are set to logically zero.
For files, all pages in memory are released except
when the disk block is uninitialized in which case
when the disk block is uninitialized in which case
the page is zeroed and kept. For deferred-update
files, any extension memory disk block is freed.
--
I am checking on the MS_SYNC case. It stated in the code that
If synchronous, wait for I/O on SCB to complete
(except for working storage segments for which
waiting on an I/O level isn't supported).
--
Regarding the question about msync() and NFS. This is the answer:
The NFS client, after receiving the page outs driven by the VMM,
will ensure that they are written to the NFS server before respondin
to the VMM that they have been completed successfully.
VMM does pageout for these pages and waits with vms_iowait for
the I/O to complete. What this means is that the vms_iowait()
will effectively wait for all data to be written to the NFS server.
Therefore the short answer is: msync() should behave as an fsync()
for an NFS file system.
S e a r c h - k e y w o r d s:
MSYNC() AND MMAP() USAGE
WWQA: ITEM: RTA000085993 ITEM: RTA000085993
Dated: 03/1995 Category: ITSAI6000GE
This HTML file was generated 99/06/24~12:43:28
Comments or suggestions?
Contact us