How can I create a pyserial web service?

Question

I have written a web application that interacts with a serial device on /dev/ttyS02. The problem is my current messaging and queuing solution. Please read below.

Heres the communication bridge between the application and pyserial:

• My web application inserts a request record via PHP into a d_requests table in MySQL. The inserted record's processed column is set to 0. The inserted record's id is saved in an $id variable and the PHP application enters a loop state where it will continually check if the d_requests[processed] column = 1 using the $id as lookup reference.

• I have a python daemon service that checks every second for records in the d_requests table where the processed column = 0. This is considered a new request.(refer to source code - python service)

• The python service then uses the record's information to connect to the port via pyserial.

• The requested action is performed. The record's processed column is then updated to 1 and a few other fields are updated as well. This marks the record as processed.

• The PHP control block then exits the loop (Point 1.) and returns the result as json to the JS application.Where it is presented to the user.

Some points of note

• The serial device is capable of processing 1 request every 250 ms.
• The python daemon service monitors the d_requests table for records where the processed column = 0 every 1 second.
• The only communication my web application has with the python daemon service is the MySQL
  DB by inserting requests records in the d_requests table.
• I use PHP block code to lookup the request using the inserted id every second to check if the processed column has been updated to 1.

My Concerns

Single point of failure

When the daemon service is not running serial requests cannot take place

Extreme Resource Usage

I expect about 4-5 requests per second to a serial device. Using the current implementation for handling messages the db will be working overtime and CPU usage will be high, since the PHP application and python daemon/service will connect and perform queries on the DB and there will be delays with the processing of requests.

Conclusion : Is there a better way to improve my current messaging and queuing solution? I think a pyserial web service will work great in this case, where the serial port for eg. is attached to a web socket eg. host:<7000> and I can just send a request via PHP to it and wait for the response back from the web service. Unfortunately I dont know how to do this.

Any ideas?

Thank You

Source code

python service

    import sys, time
    from daemon import Daemon
    import MySQLdb 

#Database parameters
config = {"host":"localhost","username":"root","password":"cake","database":"mydb"}

#Check if MySQLdb library is present
try:
    conn = MySQLdb.connect(config['host'],config['username'],config['password'],config['database'])
except MySQLdb.Error, e:
    print "Error %d: %s" % (e.args[o], e.args[1])
    sys.exit(1);

#Check if pyserial library is present
try:
    import serial
except ImportError:
    print "Error,pySerial module not installed"
    sys.exit(1);

#Create DB cursor  
#cursor = conn.cursor(cursorclass=MySQLdb.cursors.DictCursor)
#Declare global variables here
class MyDaemon(Daemon): 
    def run(self):
        while True:
            time.sleep(2)
            cursor = conn.cursor(cursorclass=MySQLdb.cursors.DictCursor)
            data = ''
            try:
                cursor.execute ("""SELECT * FROM d_requests where processed = 0""")
                rows=cursor.fetchall()
                print "Waiting for requests..."
            except MySQLdb.Error as detail:
                print "MySQL Error,",detail
            if len(rows) == 0:
                cursor.close()
                print "No request found..."
                continue
            for row in rows:
                try:                
                    print "Processing request..."                   
                    ser = serial.Serial(port=row['port'],
                    baudrate = row['baud'],
                    bytesize = row['bytesize'], #8
                    parity = row['parity'], #serial.PARITY_NONE or N or C
                    stopbits = row['stopbits'], #1
                    timeout = row['wait_for_reply'], #0.5
                    xonxoff = row['sw_flowcontrol'], #0
                    rtscts = row['hw_flowcontrol']) #0                  
                    #Send command to device
                    ser.write(row['request_string'] + "\r")
                    #Read device response                   
                    data = ser.read(100)#TBD:This value needs to be changeable,not always 100 bytes
                    ser.close()
                    print "RESULT : " + data                    
                except (serial.SerialException, AttributeError, NameError) as detail:
                    data = "Error, could not open port"
                    print data                  
                except serial.SerialTimeoutException as detail:
                    data = "Error, port connection timeout" #Error ,detail
                    print data
                except:
                    data = "Error,Unexpected error"
                    print data              
                finally:
                    #ser.close()
                    try:
                        cursor.execute("""UPDATE d_requests SET processed = %s, result_string = %s WHERE id = %s""",(1,data,row['id']))
                    except MySQLdb.Error as detail:
                        print "MySQL Error,",detail
                #cursor.commit() for innoDB table engines
            cursor.close()
if __name__ == "__main__":
    daemon = MyDaemon('/tmp/daemon-example.pid')
    if len(sys.argv) == 2:
        if 'start' == sys.argv[1]:
            daemon.start()          
        elif 'stop' == sys.argv[1]:
            daemon.stop()
        elif 'restart' == sys.argv[1]:
            daemon.restart()
        elif 'foreground' == sys.argv[1]: #this runs the daemon in the foreground
            daemon.run()
        else:
            print "Unknown command"
            sys.exit(2)
        sys.exit(0)
    else:
        print "usage: %s start|stop|restart" % sys.argv[0]
        sys.exit(2)

Answer 1

Researching similar need. Found "ser2net" and "termnetd" daemons helpful so far.

Answer 2

Is there a better way to improve my current messaging and queuing solution?

You bet! They're called message queues, and they are awesome.

My favorite is Gearman, written by the the same team that brought us memcached. It has PHP and Python bindings. It's not actually a message queue so much as an RPC service. Regardless, it will let you call methods from one of your environments and handle them in the other.

In this case, you'd want to write your serial interface code in Python, and have it expose all the things it can do as Gearman functions. It will sit open as a daemon. Your PHP code can call those functions through Gearman.