Synchronization between python and arduino using serial communication

Question

I have an application where I need to:

1. move a servo motor using arduino to a position and stop at that position
2. have a camera controlled by python acquire an image at that position
3. when the image is acquired, the servo should move to a symmetrical position

The sequence is repeated a number N of time

So I tried to synchronize the arduino and the python code using serial communication. On the arduino side, when the servo reaches a position, it sends a string to the python code using serial communication. The string is either "Cross", or "Co" depending of the position reached. The arduino should wait for a string "Ok" to be sent by serial communication by the python code (after image acquisition). After receiving this string, the arduino should actuate the servo for it to move to the other position.

On the python code side, I read serial data and depending on the string received (Cross or Co):

1. a string name is defined
2. An image is acquired using the camera
3. the image is saved or append to a list
4. The string "Ok" is sent to arduino.

the codes are attached below.

The problem is that I do not manage to synchronize correctly the servo positions and the image acquisition. The servo just run back and forth between the two positions and does not seem to be reading any string from the serial communication. It then never stops really to a position. The arduino code however does send "Cross" and "Co" to the python code, and the python code does manage to read them and acquire and save images but often with the wrong names. Having the arduino code waits long enough at each position is not a solution, since I need a decent frequency of image acquisition.
So I would like to know what is the best way to synchronize the two codes and be sure, that the right name of my images will correspond to the right position of the servo?

Thanks in advance for any link or ideas.

Greg

arduino code `

#include <Servo.h>

//servo
Servo myservo;  // create servo object to control a servo
// twelve servo objects can be created on most boards
int pos = 0;    // variable to store the servo position

//camera
const int CameraPin =  53;      // the number of the camera trigg
int CameraState = LOW;             // ledState used to set the LED
const int ledPin =  13;      // the number of the LED pin
String Str = "c";

void setup() {
  myservo.attach(9);  // attaches the servo on pin 9 to the servo object
  // set the digital LED pin as output:
  pinMode(CameraPin, OUTPUT);
  // set the digital camera pin as output:
  pinMode(ledPin, OUTPUT);
  //serial communication 
  Serial.begin(9600);
  Serial.println("Ready");
      }
void loop() {

Serial.flush();
// go to Co position and wait
ServoCo(15); // go to Co position
Serial.println("Co"); //send signal Co to python
while(!Serial.available()) {} // wait for python to send data acquired
while ((Serial.available()<2)) // Test on the length of the serial string   
{ 
  delay(1);
  String Str = Serial.readStringUntil('\n');
  Serial.println(Str);
}

// go to cross position and wait
ServoCross(15); // go to Cross position
Serial.println("Cross");
while(!Serial.available()) {}
  while ((Serial.available()<2))
{
  delay(1);
  String Str = Serial.readStringUntil('\n');
  Serial.println(Str);
}
}
delay(100);
}

void ServoCross(int ServoDelay)
{
  for (pos = 105; pos >= 75; pos -= 1) { // goes from 0 degrees to 180 degrees
    // in steps of 1 degree
    myservo.write(pos);              // tell servo to go to position in variable 'pos'
    delay(ServoDelay);  
  }
}
  void ServoCo(int ServoDelay)
{
  for (pos = 75; pos <= 105; pos += 1) 
  { // goes from 0 degrees to 180 degrees
    // in steps of 1 degree
    myservo.write(pos);              // tell servo to go to position in variable 'pos'
    delay(ServoDelay);  
  }`

Python code:

from time import sleep
import serial
import scipy
ser = serial.Serial('COM10', 9600) # Establish the connection on a specific port

Nb_total_image = 100
imagergb_cross = np.zeros((480,640))
imagergb_co = np.zeros((480,640))
counter_image = 0;

ser.write('start')
while counter_image<Nb_total_image:
    ser.flush()
    ReadArduino = ser.readline().split('\r')[0] # Read the newest output from the Arduino

    print(ReadArduino)
    if ReadArduino == 'Cross':
        nameImage = 'CrossImage' + str(counterImageCross)
        cam.reset_frame_ready()                 # reset frame ready flag
        # send hardware trigger OR call cam.send_trigger() here
        cam.send_trigger()
        cam.wait_til_frame_ready(1000)              # wait for frame ready due to trigger        
        imageRawcross = cam.get_image_data()
        ser.write("Ok\n")

    else:
        nameImage = 'CoImage' + str(counterImageCo)
        cam.reset_frame_ready()                 # reset frame ready flag
        # send hardware trigger OR call cam.send_trigger() here
        cam.send_trigger()
        cam.wait_til_frame_ready(1000)              # wait for frame ready due to trigger                
        ImageRawCo = cam.get_image_data()
        ser.write("Ok\n")
    imagergb = imageRawCo-imageRawcross   
    counter_image = counter_image + 1

Answer 1

In your loop() function, you have implemented your software as that function will be performed once as the main() in a Console software.

Because Arduino (C langage) has to communicate through the serial link with your computer (Python langage), one efficient solution is to use state-machine principles.

Step 1 - define the list of states needed and also the number of positions to reach.

A simple enum e_State allow to define the list.

typedef enum e_State {
    STATE_START = 0, // first state to initialize
    STATE_MOVE_POS,  // moving Servo to the selected position
    STATE_SEND_CMD,  // sending message when position reached
    STATE_WAIT_ACK   // waiting acknowledge from message
} eState;

For the 2 positions to reach, enum e_Pos is used:

typedef enum e_Pos {
    FIRST_POS = 0,
    SECOND_POS
} ePos;

Step 2 - define the starting parameters

To store persistent data between loop() calls, static variables are used:

static eState LoopState = STATE_START; // to store the current state
static ePos ServoPos = FIRST_POS;  // to store the selected position

And to store temporally the next state, add the eState NextState.

At the entry of the loop() function, NextState = LoopState; to maintain the same state by default.

Step 3 - define the state-machine algorithm.

void loop() {

static eState LoopState = STATE_START; // to store the current state
static ePos ServoPos = FIRST_POS;  // to store the selected position

    eState NextState = LoopState;

    switch (LoopState) {
    case STATE_START:
        ServoPos = FIRST_POS;
        NextState = STATE_MOVE_POS;
        break;
    case STATE_MOVE_POS:
        NextState = STATE_SEND_CMD;
        break;
    case STATE_SEND_CMD:
        NextState = STATE_WAIT_ACK;
        break;
    case STATE_WAIT_ACK:
        // NextState = STATE_MOVE_POS;
        break;
    default:
        // when undefined state, restart
        NextState = STATE_START;
    }
    // define the state for the next loop
    LoopState = NextState;
}

Step 4 - manage the action of STATE_MOVE_POS with selected ServoPos.

switch (ServoPos) {
case FIRST_POS:
    ServoCo(15); // go to 'Co' position
    break;
case SECOND_POS:
    ServoCross(15); // go to 'Cross' position
    break;
};
NextState = STATE_SEND_CMD; // when reached, send serial message

Step 5 - manage the action of STATE_SEND_CMD based to the reached ServoPos.

Serial.flush();   // clear the serial buffer
if (ServoPos == FIRST_POS) {
    Serial.println("Co"); //send signal 'Co' to python
}
else {
    Serial.println("Cross"); //send signal 'Cross' to python
}
NextState = STATE_WAIT_ACK;

Step 6 - manage action of STATE_WAIT_ACK by looking for "OK" acknowledge.

Optionally, add the comparison if (Str == "OK") to be sure that computer answers correctly.

if (Serial.available()) { // no state change while no acknowledge
    String Str = Serial.readStringUntil('\n');
    Serial.println(Str);
    if (Str == "OK") {
        // ack is OK, select the next position and continue
        ServoPos = (ServoPos == FIRST_POS)?(SECOND_POS):(FIRST_POS);
        NextState = STATE_MOVE_POS;
    }
    else {
        // ack is KO, restart from first position
        NextState = STATE_START;
    }
}