Linux S-Link Driver

Linux S-Link driverv1.111/t/jj

SLD

S-Link Driver voor Linux

Driver Documentation v1.1

J.E. van Grootheest

NIKHEF, Amsterdam

May 2000

1

Linux S-Link driverv1.111/t/jj

1Table of Contents

1Introduction3

1.1Purpose3

1.2Definition of terms3

2General description4

2.1Assumptions and dependencies4

3Interface description6

3.1open and close6

3.2mmap6

3.3select6

3.4ioctl6

3.5getInfoBlock and freeInfoBlock6

4Functional description7

4.1Device Driver Framework (DDF)7

4.1.1Initialization7

4.1.2open and release7

4.1.3mmap7

4.1.4poll7

4.1.5ioctl7

4.1.6Interrupt- and bottom-half entry point8

4.2HardWare Driver (HWD)8

4.2.1PCI identifications8

4.2.2powerUp and powerDown8

4.2.3startDevice and stopDevice9

4.2.4isInterrupt9

4.2.5enable- and disableInterrupt9

4.2.6handleInterrupt9

4.3BUFfer (BUF)9

4.3.1openBuf, mmapBuf and closeBuf10

4.3.2getInfoBlock and freeInfoBlock10

4.3.3getDataBuffer10

4.3.4putInfo10

1

Linux S-Link driverv1.111/t/jj

2Introduction

2.1Purpose

This document has the purpose to describe the S-Link driver for those who have to write the application making use of it and those who have to maintain the driver. Therefore the goal of this document is to explain how the driver functions, but it is not a manual which describes each interface in detail.

It is expected that the reader understands enough of the Linux kernel internals, so such things are not explained.

2.2Definition of terms

BUFBuffer module of the driver

DDFLinux Device Driver Framework

HWDHardWare Driver

InfoBlockBasic element of communication between driver and user

S-Linka definition of a hardware interface to transfer data

SLDLinux S-link Driver

3General description

The S-Link driver is a generic S-Link driver. This means that it provides only basic S-Link communication and error reporting, but nothing more. It is up to the user application to handle the data and control words that have been received and also implement the necessary error handling.

Figure 1 shows the S-Link driver and the user interfacing with it. For clarity, the linux kernel itself is not shown and interfaces between the driver and the kernel are implicit.

The user accesses the driver via regular system calls (open and ioctl, for example) and the BUFfer interface. The data, which came in on the S-Link, is retrieved via BUF together with the type of S-Link data and error information.

The driver itself consists of three parts: DDF, which implements the regular device mechanisms required by Linux, like device registration, interrupt registration and such. Module BUF manages the data buffers between the driver and the user. The HWD module implements the hardware specific device control.

3.1Assumptions and dependencies

It is expected of the user that infoblocks are released in the same order that they were retrieved

The user does not change the contents of infoblocks under its control

The driver is written such that it can handle multiple interfaces in one system and that it will also work properly in an SMP system.

There is only one real user per interface; the mmap system call will allow only one user to map the complete driver and have access to the buffers. Other users may open the driver and map the management part of the driver, for example to monitor the driver or perform ioctl calls.

The driver makes no assumption about the order of data and contol words that are received on the S-Link.

When there are no free databuffers the driver should not drop data, but halt the S-Link until another buffer has become available.

4Interface description

This chapter describes the interface between the driver and the user application the way the user experiences it.

4.1open and close

The open system call opens the device; it returns the normal file descriptor that is to be used on subsequent calls.

Close will release any resources related to this process and the device.

4.2mmap

There are two ways to use the mmap call. The first one will map all memory pages, which are used for buffers etc., in the users memory space. Only one user can do this, otherwise the buffer management will go wrong. The other way to use mmap is to only map the driver management memory. This is useful for monitoring the driver separately from the user.

The call will return an address that is to be passed to getInfoBlock and freeInfoBlock. An ioctl is provided to get the size of memory that needs to be mapped.

4.3select

Select can be used on the device to wait until data is available from the device. It works in the normal way that is expected of select.

4.4ioctl

This interface will provide all other functionality, including this list:

1.Get S-Link down status

1.Set/get S-Link return lines

1.Get the size of memory that needs to be mmaped

1.Turn on statistics; this will cause the device to put statistics in the data queue every X*100msec. X is an input parameter with the ioctl. Value 0 for X turns off statistics.

4.5getInfoBlock and freeInfoBlock

The user calls both these functions with the memory address from mmap as parameter.

GetInfoBlock will then return an infoblock with information or NULL if there is no data available. FreeInfoBlock gets as extra parameter an infoblock retrieved with getInfoBlock.

Until freeInfoBlock has been called, getInfoBlock will return the same infoblock.

5Functional description

5.1Device Driver Framework (DDF)

The DDF module is responsible for implementing all interfaces that the kernel expects and/or requires. This entails the normal file operations that any driver needs to implement and the interfaces for handling interrupts, timers and such.

Of the file operations, only open, release, mmap, ioctl and poll are implemented, as the BUF interface replaces the read and write ones.

DDF maintains private data for the complete device and passes this to BUF and HWD as needed. The private data is maintained per device, so multiple devices can be used simultaneously.

5.1.1Initialization

Both when the driver is build as a module and integrated into the kernel is it necessary to provide an initialization routine. This routine registers the driver with the kernel as a character device and checks which interfaces are present that can be handled by the driver. For each interface a memory page is allocated to keep the private data for the device.

In the case it is a module, also a cleanup function is needed. This function unregisters the driver and releases the memory pages.

5.1.2open and release

The open function uses HWD powerupReset to put the interface in a known state and initializes the BUF module. The release function inverts these actions and leaves the interface in an idle state.

These actions are only done in the case the driver is opened (closed) for the first (last) time.

5.1.3mmap

First the driver private data is mapped to the users' address space, then the DDF mmap function passes control to the BUF mmap function which does the work of mapping the databuffer and queue memory in the users memory space.

5.1.4poll

If a process wants to know if there is data available from the interface, it uses the select systemcall. This function is the back-end to that. The process is queued on the waitqueue (to be awakened by the interrupt routine) and a mask is returned to the caller indicating if there is data available.

5.1.5ioctl

The ioctl interface is normally used to offer functionality, which is either not used often, or specific to this device. Also the S-Link driver uses the ioctl interface for such purposes. This is the driver entry point for ioctl system calls.

Please refer to chapter three for a description of the possibilities.

The S-Link related calls will be forwarded to HWD and the statistics call is a function of DDF itself. To handle the statistics, a timer is started to handle the timeout. At the timeout the timer is restarted and a flag is set for BUF that causes putInfo to queue the statistics to the user.

5.1.6Interrupt- and bottom-half entry point

The driver will receive an interrupt from the device when there is a change in its status. Normally in Linux most of the handling is done in the bottom-half, so more interrupts (of other devices) can be handled while most of the processing is done. The DDF-HWD interface has been designed such that this is also possible.

In the interrupt routine the events from the device are disabled and the bottom-half is placed in the immediate queue. During the bottom-half, HWD will finish the handling of the interrupt. If HWD indicates that data was queued to the user, DDF wakes up the processes that are waiting for it.

As an interesting option it is possible not to use the bottom-half and do all the work in the interrupt routine. Normally there are no good arguments for this, but since the application of this driver is very specific, it can be allowed to deviate from the regular Linux rules.

Normally, the bottom-half should loop and check if, after handling the interrupt, there are more events present. However, due to the hardware implementation of the INCAA interface this is will only happen in very specific circumstances which are not the normal case, so that is not done here. The problem with the hardware is that it can take the interface up to 1 microsecond to react to changes in its registers.

5.2HardWare Driver (HWD)

The HWD module implements all the hardware specific functionality. These are used from DDF when, for example, an interrupt has occurred.

If S-Link datawords are expected, a DMA transfer is started into a buffer from BUF. If there is no databuffer available, a task is queued in the timer queue. This will give the user application some time to handle and free infoblocks. The task will then check if there is a free databuffer available and start a DMA transfer. If there is still no free buffer available, it will queue itself again and again until a buffer becomes available.

For S-Link controlwords it is easier. The controlwords are read from the AMCC fifo and queued to the user.

Normally controlwords are handled when the S-Link hardware indicates a type change from control to data words. However, the AMCC chip cannot generate an interrupt when there is only a controlword in the fifo. So when controlwords are expected a timer is started to take these from the fifo and queue them to the user.

5.2.1PCI identifications

HWD defines two definitions to be used by DDF for finding and registering the correct device. These two defines are the PCI Vendor and Device Ids.

5.2.2powerUp and powerDown

The powerupReset function is to be used only once: when the driver is opened. This will reset all hardware to put it into a defined state.

The powerDown function does the reverse.

5.2.3startDevice and stopDevice

These two functions do exactly what you expect them to do.

StartDevice will check what type of data is waiting in the interface and prepare the hardware such that it will start receiving that data. If there are S-Link data words present, DMA is initialized, if there are control words waiting, those will be handled. After that, the device will be handled through the interrupt and bottom-half routines.

StopDevice will do the opposite and make sure the device is in an idle state.

5.2.4isInterrupt

As the interrupt line may be shared with other devices, DDF must be able to determine whether an interrupt occured from the interface that it manages. This function from HWD tells it if it was the current interface that generated an interrupt. If the interrupt came from its interface, the events are acknowledged and the interrupts disabled to prevent getting immediately another interrupt.

5.2.5enable- and disableInterrupt

These two are used by the DDF to turn all events on and off in the interface.

5.2.6handleInterrupt

This is the workhorse function of HWD. It is called, as expected, after an interrupt has occurred to handle it. It returns to DDF an indication if data was queued in the infoblock queue.

First it'll need to determine what the cause of the interrupt was and then act upon those:

If a DMA transfer has finished put the buffer in an infoblock and queue it to the user.

If there's a change in S-Link data type, handle the change by first finishing the previous type and then preparing for the new type.
If an error occurred then report that in the infoblock with the data that is forwarded to the user.

In most other error situations an infoblock will be queued to the user describing the problem. The interface is restarted to capture more data.

5.3BUFfer (BUF)

The module BUF implements the interface between HWD and the user. So one part of this module operates in kernel space and the other part in user space.

BUF delivers the data buffers, which HWD uses to capture data into using DMA and queue it until the user is ready to receive them.

Data buffers are all the same size, which is configured by the user using the normal Linux configuration mechanisms. The number of data buffers is likewise configurable.

Infoblocks are used to maintain the data that came in over the S-Link as well as secondary information like S-Link status or statistics about the link. BUF will guess the number of infoblocks to be larger than 3.5 times the number of data buffers, since the infoblocks will be used for S-Link data and control words, but not both at the same time.

All infoblocks have the same layout with a type indicating the information stored in the block, a message which holds secondary information about the data and a third field which holds either the data directly, or a pointer to a data buffer plus the size of the data.

BUF uses circular queues for the infoblocks and data buffers, as those can be made interrupt-safe without locking mechanisms. This is necessary since there are no locking mechanisms that can be shared between kernel and user space and are also interrupt safe.

5.3.1openBuf, mmapBuf and closeBuf

These two functions serve the purpose of initializing and closing down the buffer mechanism.

On open, the necessary pages of memory will be allocated for:

1.infoblock queue,

1.data buffer queue and

1.the data buffer memory.

And the queues will be initialized to be full of empty items.

MmapBuf makes sure that the necessary pages of memory are visible in the users memory space. Items to be mapped are the infoblock queue, the data buffer queue and the data buffer memory.

5.3.2getInfoBlock and freeInfoBlock

These two function together makeup the user interface.

Using getInfoBlock the user gets an infoblock full of information, unless the queue is empty. In that case it will return NULL and the user should use the select sytem call to wait for data.

After handling the data the infoblock is released using freeInfoBlock. In the case it was information using a data buffer, also the data buffer is released.

5.3.3getDataBuffer

This is the interface for HWD to get a data buffer to DMA data into. If there are data buffers available, one is returned.

In the case that there are no data buffers, the function will return without a buffer. HWD will need to check later if there is another buffer available.

5.3.4putInfo

This is the input function to the user from HWD. HWD prepares an infoblock to be put in the queue and uses putInfo to queue it.

If there is no free infoblock available, BUF will panic() and the system needs to be reset.

If it's time to report statistics, putInfo first checks that there are a data buffer and an infoblock available. If one of these is not present, the statistics are dropped, as S-Link data is more important than statistics. In the case that both are present, a data buffer is allocated and filled with statistics data and an infoblock is prepared. These are then queued to the user to be handled there.

6User interface

The open, close, mmap, ioctl and select functions are system calls. Check with the documentation supplied with your system for their specifics.

The driver specific interfaces are (from sld.h):

infoblock_t *getFullInfoBlock( void *privData );

void freeInfoBlock( void *privData,

infoblock_t *infoBlock );

Normally th e user will want to make a loop like this to handle the S-Link interface:

do

{

infoblock_t *blockPtr;

blockPtr = getFullInfoBlock( privDataPtr );

if ( blockPtr == NULL )

{

select( sldFd );

blockPtr = getFullInfoBlock( privDataPtr );

}

handleBlock( blockPtr );

freeInfoBlock( privDataPtr );

}

while ( TRUE );

The handleBlock function is probably best implemented using a switch on the type of the infoblock (blockPtr->type), with the necessary implementations for the intended application and ignoring the rest of the infoblocks.

1