4.9.1 Description
The gtixp [5]1 device driver is the program that enables the “client” user program to read and write to the IXP card’s SDRAM. The driver does this by mapping the IXP card’s resources into host computer memory. The gtixp driver does not support DMA, so it is rather slow. (See section 5.2 for a test of its bandwidth.) If you need to map a large portion of IXP memory, the kernel has to be patched with a patch called “bigphys area” [41], it reserves physical memory for drivers at boot time. The amount of SDRAM that is mapped can be set in themain.cfile.
You need to addbigphysarea=4096to the argument line in the Linux loader you are using.
1We did not write the gtixp driver [5], we found out that some nice fellows at Georgia Tech had writ-ten one. The authors according to the source code is Himanshu Raj <rhim@cc.gatech.edu> and Ivan Ganev
<ganev@cc.gatech.edu>.
//Here we initialize the 21555 so we can use it to send data over the PCI bus //and generate interrups on the host side PCI bus that will end up in the //host kernel.
int InitHostIrq(void) {
if ((mem_fd = open("/dev/mem", O_RDWR | O_SYNC ) ) < 0) { printf("open: can’t open /dev/mem \n");
return -1;
}
//We map the memory the addresses where the 21555 is located.
//The address is assigned from the PCI system at boot.
//Do lspci -vn or cat /proc/pci on the IXP card and look for:
//Bridge: PCI device 8086:b555 (Intel Corp.) (rev 3) //In that device look for:
//Non-prefetchable 32 bit memory at 0xe0100000 [0xe0100fff]
//And you got the address that we need to map.
//Defined in logger_cfg.h or dlsystem.h as ADDRESS21555.
pci_io_mem = (unsigned char *)mmap((caddr_t) 0,
//We need to write 0x4000 0000 to the 32 bit register at address
//I21555_CSR_DS2. Since XScale is big endian and 21555 is little endian.
//We write 0x40 to byte number 4 in the register.
i21555_regs[I21555_CSR_DS2+3] = 0x40;
//Clears the IRQ mask on the 21555 to its host side by writing //a 1 to the bit for the interrupt in the
//Primary Clear IRQ Mask Register.
//To make an interrupt, the mask must be set and the interrupt register //needs to be clear.
i21555_regs[I21555_CSR_PRIMARY_CLEAR_IRQ_MASK] = 0x4; //(0xa0) return 0;
}
Figure 4.23: Code to initialize the Intel 21555 Bridge
//Makes the IRQ on the 21555 to its host side.
#define MakeHostIrq (i21555_regs[I21555_CSR_PRIMARY_SET_IRQ] = 0x4)// (0x9c)
Figure 4.24: Macro to make an interrupt
This gives you 4096 4KB pages to use for device driver memory. It is a Linux kernel patch that is not so easy to find, and you need the right one for your kernel version.
The gtixp driver seems to follow the normal procedures for a Linux kernel driver [20]. It can handle more than one IXP card, we have not tried though. It looks through the card’s resources and finds the 21555’s CSR memory region, IXP’s CSR memory region, and IXP’s SDRAM
memory region.
When you load the driver, keep an eye on /var/log/messagesas the driver will tell you if it succeeded or not. If all is good, try ls -lh /proc/driver/ixp0/ and you should get the result given in figure 4.25. The driver uses ioremap to allocate the SDRAM I/O
total 50M
dr-xr-xr-x 2 root root 0 Apr 19 16:04 . dr-xr-xr-x 3 root root 0 Apr 19 16:04 ..
-rw-rw-rw- 1 root root 4.0K Apr 19 16:04 bridgeCsr -rw-rw-rw- 1 root root 1.0M Apr 19 16:04 ixpCsr -rw-rw-rw- 1 root root 16M Apr 19 16:04 mempool -rw-rw-rw- 1 root root 0 Apr 19 16:04 reset_bridge -rw-rw-rw- 1 root root 32M Apr 19 16:04 sdram
-rw-rw-rw- 1 root root 0 Apr 19 16:04 signal
Figure 4.25: Directory listing of gtixp resources
memory region from a physical PCI address to virtual kernel space. When the client is reading or writing the sdram file, the driver usesmemcpy_[to,from]_io to perform I/O on the memory allocated by ioremap. So the client thinks it is accessing the SDRAM as a file, but the driver copies the data over the PCI bus in the background.
When the driver is loading, it scans the information from the PCI bus to see if it can find a 21555 Bridge in a PCI slot. If so, it reads its resources, to see at what addresses the resources are kept. It also registers the IRQ of the 21555 Bridge so that its IRQ handler gets called whenever there is an IRQ.
4.9.2 SDRAM PCI transfer
In figure 4.25, sdramis the file that the client program reads and writes to when transferring data between the IXP cards SDRAM and the host over the PCI bus. The user program on the host computer that has the signal file open gets a SIGUSR1 signal from the driver, when the driver gets an interrupt from the kernel. reset_bridgeis to reset the PCI bridge after resetting the card, we did not get it to work. We did not play with the other files, but the brigdeCsris the CSRs for the 21555 andixpCsris the CSR of the IXP card.
You can use the odcommand as in figure 4.26 to read the contents of SDRAM starting at 26MB. See section 4.10.3 to see how we read the file in the client.
od -x -Ax /proc/driver/ixp0/sdram -j0x01a00000
Figure 4.26: Example of reading from IXP’s SDRAM
4.9.3 IRQ
The gtixp [5]driver also handles IRQ from the 21555. It registers the interrupt that the 21555 is assigned when the host computer boots. When that interrupt is triggered, all drivers that have that IRQ are called until one of them claims it. In x86, there are few interrupts, so some devices have to share. The driver checks the 21555 interrupt register to see if it is our interrupt. If it is not, it returns IRQ_NONE, and the kernel asks the next device driver. If it is our interrupt,
the driver clears the interrupt and sends a SIGUSR1 signal to the program having the signal file open.