ITEM: RTA000022311

Q: PLATFORM: RISCSYSTEM                                                         
** This item contains information that is intended to be helpful,  **           
** accurate and act as a guide of HOW TO accomplish a given system **           
** task.  The provided explanations, techniques and procedures     **           
** have been reviewed for technical accuracy and applicability,    **           
** but have not been tested in every possible environment or       **           
** situation. Normal precautions should be taken in adopting these **           
** same techniques and procedures, because as product and system   **           
** interfaces change, so would the usage of this information       **           
** change.                                                         **           
The Risc System 6000 hardware divides the currently active 32 bit               
virtual address space into 16 independent segments, each addressed              
by a segment register. In AIX V3, segment 2 (virtual address                    
0x20000000) is referred to as the process private segment. This                 
segment contains most of the per process information including user             
data, user stack, kernel stack, and user block.                                 
As a result of placing user data and the user stack within a single             
segment, the system limits the maximum amount of stack and data to              
slightly less than 256 megabytes (the kernel stack and u block are              
relatively small and of fixed size). While this is sufficient for               
the majority of applications, certain applications require extremely            
large data areas.                                                               
In AIX V3.1, it is possible to provide a large data model by supplying          
shared memory segments in which the extremely large data areas might            
reside.  Please refer to ASKQ items 3VQBM, 5VRBV, and 4VRBV. While              
the techniques described in these items continue to work in AIX V3.2,           
there has been a change in the AIX address space model that may                 
address many of the program requirements addressed by the cited ASKQ            
In AIX V3.2, a new address space model is available to enable large             
data applications. Programs that do not need this large address                 
space will continue to be described by the smaller, more efficient              
model. Programs that need the larger model indicate it to the system            
by setting the o_maxdata field in the a.out header in the executable           
to indicate the actual amount of data needed.                                   
With the large address space model, the program's data is laid out              
beginning in segment 3. The program consumes as many segments as are            
needed to hold the amount of data indicated by o_maxdata up to a                
maximum of 8. This allows the process to have up to 2 gigabytes of              
data, if needed.                                                                
Other aspects of the process address space are unchanged. The user              
stack, kernel stack, and u block continue to reside in segment 2.               
Also, the data resulting from being linked with any shared libraries            
is also placed in segment 2. Only program data is placed in segments            
3 and above. There are several consequences of this organization that           
are worth noting.                                                               
First, the user stack is still bounded by the size of segment 2                 
though it can be relocated into a shared memory segment if desired.             
Second, fewer segments are available for mapped files. Third, the               
total size of the executable must be less than 256 megabytes so as              
to fit into segment 1.  A consequence of this restriction is that a             
program cannot have very large initialized data or text.                        
The large data model is enabled by a -b option to the linker (/bin              
/ld). For example, to link a program that will have the maximum 8               
segments reserved to it, the following command line option might                
be used:                                                                        
cc -o bigdata bigdata.o -bmaxdata:0x80000000                                    
Note that the -bmaxdata option can also be used with a compiler                
command for convenience. The number 0x80000000 is the number of                 
bytes, in hexadecimal, that can be represented by 8 256 megabyte                
segments. Larger numbers can be used, but they are ignored since a              
maximum of 8 segment can be reserved. The value may also be specified           
in decimal.                                                                     
The large address space model is used if any non-zero value is given            
for maxdata. The -bmaxdata option should be used only if the                    
program actually needs large data spaces since the large address                
space model is slightly less efficient since additional segments must           
be managed.  When a program, such as the shell, attempts to execute             
a program with large data, the system recognizes the requirement and            
attempts to modify the soft limit on data size to accommodate the               
process. This will fail and the process will be killed if the process           
does not have privilege or if the new soft limit is above the hard             
limit for the process. This is not usually a problem as the login               
process normally sets the hard limit to infinity.  However, if the              
calling process has modified its hard limit (via the unlimit command            
in the shell or the limit command in the C-shell) then this can                 
cause a problem.                                                                
After placing the program's initialized and uninitialized data in               
segment 3 and beyond, the break value is computed. The break value              
defines the end of the processes' static data and the beginning of              
its dynamically allocatable data. The process is free to move the               
break value further up (via malloc, brk, or sbrk) to the end of the             
segment identified by the maxdata field. For example, if maxdata is             
0x80000000, then the maximum break value would be up to the end of              
segment 10 or 0xafffffff. The brk system call will extend the break             
across segment boundaries but not beyond the maxdata point.                    
The majority of system calls are unaffected by large data programs.             
The semantics of the fork system call are unchanged. Large data                 
programs may exec other programs, large or small. Large data programs           
can load and unload other modules.                                              
The sbrk, brk and setrlimit system calls interact in how they allow             
the break value to be set and how they allow limits to be set. The              
data region of a process must not overlap with the stack region of the          
As a result, the brk and sbrk system calls only allow the break to              
be moved up to the soft stack limit for small data programs or up               
to the maxdata point for large data programs. Likewise, the setrlimit           
system does not allow the soft stack limit to be set low enough to             
overlap with the other data in segment 2.                                       
The setrlimit system call allows the soft data limit to be set to               
any value up to the hard limit. However, this may not allow the                 
process to be able to increase its size up to that value due to the             
inherent limitation of the address space model being used by the                
Running programs with large data spaces may require massive amounts             
of paging space. Indeed, if a program with a 2 gigabyte address space           
actually tries to access every page in its address space, the system            
will require 2 gigabytes of paging space to deal with the program.              
Naturally, the AIX page space monitor will terminate processes if               
paging space runs low.  Programs with large data spaces will be the             
first to go since they would typically be large consumers of paging            
Debugging programs with large data is no different from debugging               
other programs. Dbx is able to debug these programs either actively             
or from a core dump.  A full core dump is ill advised since programs            
with large data areas would produce massive core dumps and consume              
large amounts of file system space.                                             
Provided in this article is big32.f, a demonstration program                    
originally provided as an example in the previously cited ASKQ items            
and the compiler command line demonstrating the use of the -bmaxdata            
------------------- file big32.f ---------------------------------              
 integer*4 myarray,total_size(2)                                                
 common /glenn/ myarray(67139936*2)                                             
 total_size(1) = 2**28                                                          
 total_size(2) = 2**28                                                          
c Let's look at the empty array ...                                             
 write(*,100) 1,myarray(1)                                                      
 write(*,100) total_size(1)/4,                                                  
 write(*,100) total_size(1)/4+1,                                                
 write(*,100) ((total_size(1)+total_size(2))/4),                               
c Change the values in the array ...                                            
  call try(myarray,total_size)                                                  
c Now let's look at it ...                                                      
 write(*,110) 1,myarray(1)                                                      
 write(*,110) total_size(1)/4,                                                  
 write(*,110) total_size(1)/4+1,                                                
 write(*,110) ((total_size(1)+total_size(2))/4),                                
100 format('Before update array value at location',i10,' is:',i10)              
110 format(' After update array value at location',i10,' is:',i10)              
120 format(/,'In main program, common block GLENN_ contains MYARRA              
Y_') endc subroutine try(idummy,total_size)                                     
 integer*4 idummy(*),total_size(2)                                              
c Change the values in the array ...                                            
 do i=65536,(total_size(1)+total_size(2))/4,65536                               
 end do                                                                         
The compile command line for this demonstration program is:                     
xlf -g -o big32 big32.o -bmaxdata:0x30000000                                    
 ** If you have found this information to be informative and    **         
 ** useful, please let us know via a HONE FEEDBACK or via EQUAL.**         
 ** Make sure in your evaluation to reference the appropriate   **         
 ** INFOSYS or FLASH number.                                    **         
 ** We would like to develop as many HOWTO items as needed      **        
 ** and solicit your suggestions, comments and recommendations  **         
 ** on how to make these entries more useful. We need to know   **         
 ** the value of this information to determine whether it is    **         
 ** worth IBM time and resource to continue to create and       **         
 ** maintain HOWTO items.                                       **         
---------- ---------- ---------- --------- ---------- ----------                
This item was created from library item Q572879      3VPBZ                      
Additional search words:                                                        
3VPBZ 92                                                                        

WWQA: ITEM: RTA000022311 ITEM: RTA000022311
Dated: 06/1996 Category: RISCOSO
This HTML file was generated 99/06/24~12:43:08
Comments or suggestions? Contact us