Bypassing I/O scheduling and linux kernel page buffering

Question

What I wanna trying to accomplish:

Developing an linux application in C language, that "exclusively" accesses a

PATA/SATA hard disk drive(HDD) to send ATA commands(in fact only those ATA commands that do not modify any byte on HDD accessed - eg. READ_SECTOR, IDENTIFY_DEVICE, SET_FEATURES, etc.).

By "exclusively", I mean that as soon as HDD is powered on(a custom hardware-which is a simple on-off switch, ensures that HDD is not powered on till the application is loaded and desires to do so), the first and only access to that HDD is only to my application. IOWs except my application, not even the linux kernel(including SCSI sub-system) nor any other application or process or human user will ever be able to access that HDD, unless when my application instructs/permits them to do so.

There is another requirement on my application: As the access to HDD is quite critical(in terms of control and not in terms of performance) in our app., so it is not desired that any I/O schedular is involved in the transactions done by application(performance on this HDD is not a constraint.). Also it is not desired that the data read from HDD is buffered by kernel buffer or page buffer. The application will read in block size of 512 bytes or a multiple of it only.

Now the problem that I'm facing is:

The SCSI sub-system resides below(and written to work with) I/O scheduler and kernel buffer or page buffer cache.

Although 'sg-driver' is provided by the SCSI subsystem to directly send commands(- Linux SCSI sub-system commands, not ATA or SCSI commands directly - which are then translated by libata to actual ATA commands. Am I right here?) to the HDD,but that is a I/O approach - you give i/p and get o/p, i.e. you have no control over the process of data transfer protocol (eg PIO, DMA and ATA status and Error registers, etc.) and device configuration(via Set Features ATA command.).

Also the error reporting mechanism must be sound and is specific to ATA protocol and not simply Linux SCSI subsystem error codes. IOWs my application need to have access to ATA error register and ATA status register on PATA/SATA HDDs.

What my application demands is exclusive control of HDD - eg. issuing a READ_SECTOR ATA cmd and then retrieving the data itself from HDD directly via reading i/o ports or via 'libata' with the above requirements must be satisfied.

What I can't do ?

I'm not going to write a PATA/SATA HBA device driver or every HBA available in market, as those are already included in kernel for libata.

What I learned till now ?

To accomplish the desired task, I may(or may not?) need to write a block device driver that interacts directly with VFS layer(or is there any way to bypass VFS even, so that my app. can directly communicate with this block driver) w/o involving/messing with kernel buffer or page buffer and I/O scheduler. This block driver will communicate directly with libata(bypassing SCSI subsystem upper layer ), which then communicates with PATA/SATA HBA driver.

Is it possible to write such a driver in a cpu architecture independent way?

Is it a feasible approach? If yes then would it affect I/O performance of other HDDs attached that are not accessed by my app. in this way. Do I need to write a system call over VFS(or bypassing it if possible)in this case for my application to communicate wih my block driver? Please enlighten me regarding this approach.

or is it possible for my block device driver to directly communicate with PATA/SATA HBA driver written for libata, But again would this approach affect I/O performance of other HDDs attached that are not accessed by my app. in this way. Also how would my application communicate with this block device driver?

Please enlighten me.

Also I wanna know about the same scenario for my application, with one difference - instead of PATA/SATA drives what if I have SCSI hard drives and its variants - specifically, SAS, Fibre channel and USB. And of course this time i'll not be using libata and ATA commands, but SCSI protocol commands.

Would u like to suggest a live cd distro as my application host, that contains PATA/SATA HBA libata drivers(- not for IDE sub-system as I will not be using it for it is depreciated now and hence might not be updated with regards for HBA drivers.) for most HBAs.

In short, what is the most direct way for an linux application to access a PATA/SATA or SCSI/SAS/Fibre Channel HDD.

I hope, I have provided sufficient info regarding my question, but if u wanna get more info or more clarification, please feel free to ask.

Update1 (on 27/6/2012):
With the useful discussion with Chris(see below) and my research, I reached the following conclusions:

1. that a readymade USB-to-PATA/SATA adapter will not solve my purpose, becoz it does not allow my app. or driver to change the data transfer mode(PIO vs DMA) on-the-fly, it does not allow my app. or driver to read ATA registers.

2. that a custom made USB-to-PATA/SATA adapter might help, but that will require either a embedded processor which need to implement ATA protocol, or an FPGA chip that implements the whole ATA protocol. But the embedded processor solution involves GPIO and is not good for SATA as it will require specialized transceivers, and the I/O performance will be an issue for both PATA and SATA- too slow for my application.

Such a adapter will talk to my linux-kernel driver (or via libusb) to my app. by a custom protocol that helps in communication b/w my app. and ATA protocol on embedded processor. In case of FPGA chip solution, I need to implement this protocol in FPGA itself alongwith the ATA protocol.

But at this point it is infeasible in terms of labour, time and money for me to implement FPGA solution and embedded processor solution. So I'm stuck with software only solution.

Finally, it seems that I'm probably going to have to duplicate and modify everything down to the hardware inteface layer to meet by requirements as said by Chris.

So, between VFS layer and HBA driver or libata layer, exactly how should I proceed. What things need to be implemented and what not?

Can someone throw light on this issue? Any ideas??

Update2 (on 1/7/2012):
I'm struggling with the issue. Is there someone on SO, who can enlighten me?

Answer 1

Realistically, if you want this level of detail control you are going to end up having to write your own low-level drivers.

Your constraint about avoiding I/O buffering and scheduling may be particularly challenging - you might avoid DMA, but a modern processor has its I/O rather decoupled from internal operations for performance reasons. Perhaps if you can fully disable all interrupts you might at least be able to timestamp when you do things. You will probably want the drive on it's own interface adapter, certainly not sharing with a running filesystem.

Doing things from userspace would probably require working by means of proxies in the kernel - you'll need to do your timing critical things on the kernel side though.

A far simpler solution, if it would meet your needs, would be to use a USB-to-SATA or PATA adapter. You can tell the existing kernel drivers to ignore the VID/PID using the quirks mode of modprobe, then talk to the device via libusb from userspace. However, there will certainly be latency there.

For the finest level of control, you probably need to connect the drive to an embedded processor which has no I/O buffering, or perhaps even an FPGA. This was not particularly hard to do for low data rates with PATA, SATA probably requires specialized transceivers but may not be out of the realm of possibility (or perhaps you can work through one of the adapters). You would probably end up connecting this custom peripheral to a PC via USB or even a serial port, and using that to issue tasks and obtain results (would be convenient if you set it up so the PC can download the device's firmware/bitstream, so you have flexibility).