Note: This operation is not supported by all SCSI I/O controllers.
The SCSI target-mode interface is intended to be used with the SCSI initiator-mode interface to provide the equivalent of a full-duplex communications path between processor type devices. Both communicating devices must support target-mode and initiator-mode. To work with the SCSI subsystem in this manner, an attached device's target-mode and initiator-mode interfaces must meet certain minimum requirements:
The adapter, acting as either a target or initiator device, requires its own SCSI ID. This ID, as well as the IDs of all attached devices on this SCSI bus, must be unique and between 0 and 7, inclusive. Since each device on the bus must be at a unique ID, the user must complete any installation and configuration of the SCSI devices required to set the correct IDs before physically cabling the devices together. Failure to do so will produce unpredictable results.
SCSI target mode in the SCSI subsystem does not attempt to implement any receive-data protocol, with the exception of actions taken to prevent an application from excessive receive-data-buffer usage. Any protocol required to maintain or otherwise manage the communications of data must be implemented in user-supplied programs. The only delays in receiving data are those inherent in the SCSI subsystem and the hardware environment in which it operates.
The SCSI target mode is capable of simultaneously receiving data from all attached SCSI IDs using SCSI send commands. In target-mode, the host adapter is assumed to act as a single SCSI Logical Unit Number (LUN) at its assigned SCSI ID. Therefore, only one logical connection is possible between each attached SCSI initiator on the SCSI Bus and the host adapter. The SCSI subsystem is designed to be fully capable of simultaneously sending SCSI commands in initiator-mode while receiving data in target-mode.
In the SCSI subsystem target-mode interface, the SCSI adapter device driver is responsible for managing the receive-data buffers versus the SCSI device driver because the buffering is dependent upon how the adapter works. It is not possible for the SCSI device driver to run a single approach that is capable of making full use of the performance advantages of various adapter's buffering schemes. With the SCSI adapter device driver layer performing the buffer management, the SCSI device driver can be interfaced to a variety of adapter types and can potentially get the best possible performance out of each adapter. This approach also allows multiple SCSI target-mode device drivers to be run on top of adapters that use a shared-pool buffer management scheme. This would not be possible if the target-mode device drivers managed the buffers.
Because it is possible for the attached initiator device to send data faster than the host operating system and associated application can process it, eventually the situation arises in which all buffers for this device instance are in use at the same time. There are two possible scenarios:
In both cases, the combination of the SCSI adapter device driver and the SCSI adapter must be capable of stopping the flow of data from the initiator device.
The adapter can handle both cases described previously by simply accepting the send command (if newly received) and then disconnecting during the data phase. When buffer space becomes available, the SCSI adapter reconnects and continues the data transfer. As an alternative, when handling a newly received command, a check condition can be given back to the initiator to indicate a lack of resources. The implementation of this alternative is adapter-dependent. The technique of accepting the command and then disconnecting until buffer space is available should result in better throughput, as it avoids both a request sense command and the retry of the send command.
For adapters allowing a shared pool of buffers to be used for all attached initiators' data transfers, an additional problem can result. If any single initiator instance is allowed to transfer data continually, the entire shared pool of buffers can fill up. These filled-up buffers prevent other initiator instances from transferring data. To solve this problem, the combination of the SCSI adapter device driver and the host SCSI adapter must stop the flow of data from a particular initiator ID on the bus. This could include disconnecting during the data phase for a particular ID but allowing other IDs to continue data transfer. This could begin when the number of tm_buf structures on a target-mode instance's tm_buf queue equals the number of buffers allocated for this device. When a threshold percentage of the number of buffers is processed and returned to the SCSI adapter device driver's buffer-free routine, the ID can be enabled again for the continuation of data transfer.
The SCSI device driver can optionally be informed by the SCSI adapter device driver whenever all buffers for this device are in use. This is known as a maximum-buffer-usage event. To pass this information, the SCSI device driver must be registered for notification of asynchronous event status from the SCSI adapter device driver. Registration is done by calling the SCSI adapter device-driver ioctl entry point with the SCIOEVENT operation. If registering for event notification, the SCSI device driver receives notification of all asynchronous events, not just the maximum buffer usage event.
The SCSI target-mode device-driver receive buffer routine must be a pinned routine that the SCSI adapter device driver can directly address. This routine is called directly from the SCSI adapter device driver hardware interrupt handling routine. The SCSI device driver writer must be aware of how this routine affects the design of the SCSI device driver.
First, since the receive buffer routine is running on the hardware interrupt level, the SCSI device driver must limit operations in order to limit routine processing time. In particular, the data copy, which occurs because the data is queued ahead of the user read request, must not occur in the receive buffer routine. Data copying in this routine will adversely affect system response time. Data copy is best performed in a process level SCSI device-driver routine. This routine sleeps, waiting for data, and is awakened by the receive buffer routine. Typically, this process level routine is the SCSI device driver's read routine.
Second, the receive buffer routine is called at the SCSI adapter device driver hardware interrupt level, so care must be taken when disabling interrupts. They must be disabled to the correct level in places in the SCSI device driver's lower execution priority routines which manipulate variables also modified in the receive buffer routine. To allow the SCSI device driver to disable to the correct level, the SCSI adapter device-driver writer must provide a configuration database attribute, named intr_priority, that defines the interrupt class, or priority, the adapter runs on. The SCSI device-driver configuration method should pass this attribute to the SCSI device driver along with other configuration data for the device instance.
Third, the SCSI device-driver writer must follow any other general system rules for writing a routine that must execute in an interrupt environment. For example, the routine must not attempt to sleep or wait on I/O operations. It can perform wake-up calls to allow the process level to handle those operations.
Duties of the SCSI device driver receive buffer routine include:
After the tm_buf structure has been passed to the SCSI device driver receive buffer routine, the SCSI device driver is considered to be responsible for it. Responsibilities include processing the data and any error conditions and also maintaining the next pointer for chained tm_buf structures. The SCSI device driver's responsibilities for the tm_buf structures end when it passes the structure back to the SCSI adapter device driver.
Until the tm_buf structure is again passed to the SCSI device driver receive buffer routine, the SCSI adapter device driver is considered responsible for it. The SCSI adapter device-driver writer must be aware that during the time the SCSI device driver is responsible for the tm_buf structure, it is still possible for the SCSI adapter device driver to access the structure's contents. Access is possible because only one copy of the structure is in memory, and only a pointer to the structure is passed to the SCSI device driver.
Note: Under no circumstances should the SCSI adapter device driver access the structure or modify its contents while the SCSI device driver is responsible for it, or the other way around.
It is recommended that the SCSI device-driver writer implement a threshold level to wake up the process level with available tm_buf structures. This way, processing for some of the buffers, including copying the data to the user buffer, can be overlapped with time spent waiting for more data. It is also recommended the writer implement a threshold level for these buffers to handle cases where the send command data length exceeds the aggregate receive-data buffer space. A suggested threshold level is 25% of the device's total buffers. That is, when 25% or more of the number of buffers allocated for this device is queued and no end to the send command is encountered, the SCSI device driver receive buffer routine should wake the process level to process these buffers.
The tm_buf structure is used for communication between the SCSI device driver and the SCSI adapter device driver for a target-mode received-data buffer. The tm_buf structure is passed by pointer directly to routines whose entry points have been registered through the SCIOSTARTTGT ioctl operation of the SCSI adapter device driver. The SCSI device driver is required to call this ioctl operation when opening a target-mode device instance.
The tm_buf structure contains certain fields used to pass a received data buffer from the SCSI adapter device driver to the SCSI device driver. Other fields are used to pass returned status back to the SCSI device driver. After processing the data, the tm_buf structure is passed back from the SCSI device driver to the SCSI adapter device driver to allow the buffer to be reused. The tm_buf structure is defined in the /usr/include/sys/scsi.h file and contains the following fields:
Note: Reserved fields must not be modified by the SCSI device driver, unless noted otherwise. Nonreserved fields can be modified, except where noted otherwise.
Note: The SCSI device driver must not modify this field.
Note: The SCSI device driver must not modify this field.
| SC_ADAPTER_ERROR | Indicates the tm_buf.general_card_status is valid. |
The target-mode interface provided by the SCSI subsystem is designed to handle data reception from SCSI send commands. The host SCSI adapter acts as a secondary device that waits for an attached initiator device to issue a SCSI send command. The SCSI send command data is received by buffers managed by the SCSI adapter device driver. The tm_buf structure is used to manage individual buffers. For each buffer of data received from an attached initiator, the SCSI adapter device driver passes a tm_buf structure to the SCSI device driver for processing. Multiple tm_buf structures can be linked together and passed to the SCSI device driver at one time. When the SCSI device driver has processed one or more tm_buf structures, it passes the tm_buf structures back to the SCSI adapter device driver so they can be reused.
When a send command is received by the host SCSI adapter, data is placed in one or more receive-data buffers. These buffers are made available to the adapter by the SCSI adapter device driver. The procedure by which the data gets from the SCSI bus to the system-memory buffer is adapter-dependent. The SCSI adapter device driver takes the received data and updates the information in one or more tm_buf structures in order to identify the data to the SCSI device driver. This process includes filling the tm_correlator , adap_devno , data_addr , data_len , user_flag , and user_id fields. Error status information is put in the status_validity and general_card_status fields. The next field is set to null to indicate this is the only element, or set to non-null to link multiple tm_buf structures. If there are multiple tm_buf structures, the final tm_buf.next field is set to null to terminate the chain. If there are multiple tm_buf structures and they are linked, they must all be from the same initiator SCSI ID. The tm_buf.tm_correlator field, in this case, has the same value as it does in the SCIOSTARTTGT ioctl operation to the SCSI adapter device driver. The SCSI device driver should use this field to speed the search for the target-mode instance designated by this tm_buf structure. For example, when using the value of tm_buf.tm_correlator as a pointer to the device-information structure associated with this target-mode instance.
Each send command, no matter how short its data length, requires its own tm_buf structure. For host SCSI adapters capable of spanning multiple receive-data buffers with data from a single send command, the SCSI adapter device driver must set the TM_MORE_DATA flag in the tm_buf.user_flag fields of all but the final tm_buf structure holding data for the send command. The SCSI device driver must be designed to support the TM_MORE_DATA flag. Using this flag, the target-mode SCSI device driver can associate multiple buffers with the single transfer they represent. The end of a send command will be the boundary used by the SCSI device driver to satisfy a user read request.
The SCSI adapter device driver is responsible for sending the tm_buf structures for a particular initiator SCSI ID to the SCSI device driver in the order they were received. The SCSI device driver is responsible for processing these tm_buf structures in the order they were received. There is no particular ordering implied in the processing of simultaneous send commands from different SCSI IDs, as long as the data from an individual SCSI ID's send command is processed in the order it was received.
The pointer to the tm_buf structure chain is passed by the SCSI adapter device driver to the SCSI device driver's receive buffer routine. The address of this routine is registered with the SCSI adapter device driver by the SCSI device driver using the SCIOSTARTTGT ioctl. The duties of the receive buffer routine include queuing the tm_buf structures and waking up a process-level routine (typically the SCSI device driver's read routine) to process the received data.
When the process-level SCSI device driver routine finishes processing one or more tm_buf structures, it passes them to the SCSI adapter device driver's buffer-free routine. The address of this routine is registered with the SCSI device driver in an output field in the structure passed to the SCSI adapter device driver SCIOSTARTTGT ioctl operation. The buffer-free routine must be a pinned routine the SCSI device driver can directly access. The buffer-free routine is typically called directly from the SCSI device driver buffer-handling routine. The SCSI device driver chains one or more tm_buf structures by using the next field (a null value for the last tm_buf next field ends the chain). It then passes a pointer, which points to the head of the chain, to the SCSI adapter device driver buffer-free routine. These tm_buf structures must all be for the same target-mode instance. Also, the SCSI device driver must not modify the tm_buf.user_id or tm_buf.adap_devno field.
The SCSI adapter device driver takes the tm_buf structures passed to its buffer-free routine and attempts to make the described receive buffers available to the adapter for future data transfers. Since it is desirable to keep as many buffers as possible available to the adapter, the SCSI device driver should pass processed tm_buf structures to the SCSI-adapter device driver's buffer-free routine as quickly as possible. The writer of a SCSI device driver should avoid requiring the last buffer of a send command to be received before processing buffers, as this could cause a situation where all buffers are in use and the send command has not completed. It is recommended that the writer therefore place a threshold of 25% on the free buffers. That is, when 25% or more of the number of buffers allocated for this device have been processed and the send command is not completed, the SCSI device driver should free the processed buffers by passing them to the SCSI adapter device driver's buffer-free routine.
A Typical Initiator-Mode SCSI Driver Transaction Sequence.