CODEWITHSUNDEEP
pythonazurekinect
Junaid
Junaid

Reputation: 19

Syncing two cameras (Azure Kinect DK and Pupil Core)

I am attempting to resolve a sync issue that I have, my pupil core is a second behind my azure kinect dk camera, I trying to add delays to manually sync these clips however I have not been successful, I am using Pupil Core Network API to send commands to Pupil Capture.

import zmq
import msgpack as serializer
import subprocess
import threading
import time

def record_video():
    # Path to k4arecorder
    k4a_recorder_path = r"C:\Program Files\Azure Kinect SDK v1.4.1\tools\k4arecorder.exe"

    # Command arguments for k4arecorder
    duration = 30
    output_path = "master1.mkv"

    # Construct the command
    command = f'"{k4a_recorder_path}" --external-sync master -l {duration} {output_path}'

    # Call k4arecorder command to record video
    subprocess.call(command, shell=True)

def record_sound():
    # Path to Python interpreter and sound recording script
    python_command = r"C:\Users\scrim\AppData\Local\Programs\Python\Python311\python.exe"
    sound_recording_script = "sound_recording.py"

    # Command arguments for sound recording script
    duration = 30
    output_file = "sound_recording.wav"

    # Construct the command
    command = [python_command, sound_recording_script, "--duration", str(duration), "--output", output_file]

    # Call sound recording script
    subprocess.call(command)

if __name__ == "__main__":
    # Setup zmq context and remote helper
    ctx = zmq.Context()
    socket = zmq.Socket(ctx, zmq.REQ)
    socket.connect("tcp://127.0.0.1:50020")

    # Measure round trip delay
    t = time.time()
    socket.send_string("t")
    print(socket.recv_string())
    print("Round trip command delay:", time.time() - t)

    # Set current Pupil time to 0.0
    socket.send_string("T 0.0")
    print(socket.recv_string())

    # Start recording
    time.sleep(1)
    socket.send_string("R")
    print(socket.recv_string())

    # Create separate threads for audio and video recordings
    video_thread = threading.Thread(target=record_video)
    audio_thread = threading.Thread(target=record_sound)

    # Start the threads
    video_thread.start()
    audio_thread.start()

    # Wait for both threads to complete
    video_thread.join()
    audio_thread.join()

    # Send notification:
    def notify(notification):
        """Sends ``notification`` to Pupil Remote"""
        topic = "notify." + notification["subject"]
        payload = serializer.dumps(notification, use_bin_type=True)
        socket.send_string(topic, flags=zmq.SNDMORE)
        socket.send(payload)
        return socket.recv_string()

Any help would be greatly appreciated.

I have attempted to add sleep commands before recording pupil capture, after calling pupil capture. However this doesn't work.

Upvotes: 0

Views: 180

Answers (1)

LeelaRajesh_Sayana
LeelaRajesh_Sayana

Reputation: 640

I haven't tried syncing a different device such as Pupil core with Azure Kinect DK. The official documentation on Azure Kinect Dk provides steps on How to synchronize multiple Azure Kinect devices. However, it does not mention best practices or approaches on syncing with other devices. This is an interesting use case.

Inspecting the code, I would propose to add a delay before you make a call to subprocess.call(command, shell=True). I believe this should help with sync issue. You might need to tune the delay based on your observations. Please find the below code with the modifications.

def record_video():
    # Path to k4arecorder
    k4a_recorder_path = r"C:\Program Files\Azure Kinect SDK v1.4.1\tools\k4arecorder.exe"

    # Command arguments for k4arecorder
    duration = 30
    output_path = "master1.mkv"

    # Construct the command
    command = f'"{k4a_recorder_path}" --external-sync master -l {duration} {output_path}'

    # Add delay before starting recording
    time.sleep(1)

    # Call k4arecorder command to record video
    subprocess.call(command, shell=True)

Hope this helps. Cheers!

Upvotes: 0

Related Questions