USB HID multitouch serial reporting

Question

I am creating an Arduino device that connects to a Windows computer and simulates a touchscreen on the computer. The screen does not belong to the Arduino, but rather the Arduino relays HID touch messages to the Windows 10 computer via USB.

The first thing I did was to relay AbsoluteMouse events instead of Digitizer events. This worked fine for me. However, as my device supports multiple touches, I would like to use multi-touch HID messages instead of AbsoluteMouse. As a proof of concept, I am starting with relaying only one finger's data. (I plan to add more fingers after I get this to work). I am taking the SerialReport approach, as it seems to be the easiest to implement:

I have found many examples of using AbsoluteMouse, so basically I didn't have to do much more than copy-paste the HID reports. However, with touchscreens it is very hard to come by code samples. (I have only found the synaptic touchpad sample which is not a touchscreen). As such I have tried to implement my own code to report touch messages. But this fails. I think I understand how to interpret the HID descriptor, but this is my first time, so I am not sure... Is there anything wrong with my message structure?

#define REPORTID_MTOUCH                 1

static const uint8_t _hidSerialMultiTouchDescriptor[] PROGMEM = {
    // https://learn.microsoft.com/en-us/windows-hardware/design/component-guidelines/sample-report-descriptor--serial-reporting-mode-
    // Linked from: https://learn.microsoft.com/en-us/windows-hardware/design/component-guidelines/selecting-packet-reporting-modes-in-multitouch-drivers
    0x05, 0x0d,                         // USAGE_PAGE (Digitizers)
    0x09, 0x04,                         // USAGE (Touch Screen)
    0xa1, 0x01,                         // COLLECTION (Application)
    0x85, REPORTID_MTOUCH,              //   REPORT_ID (Touch)
    0x09, 0x22,                         //   USAGE (Finger)
    0xa1, 0x00,                         //   COLLECTION (Physical)
    0x09, 0x42,                         //     USAGE (Tip Switch)
    0x15, 0x00,                         //     LOGICAL_MINIMUM (0)
    0x25, 0x01,                         //     LOGICAL_MAXIMUM (1)
    0x75, 0x01,                         //     REPORT_SIZE (1)
    0x95, 0x01,                         //     REPORT_COUNT (1)
    0x81, 0x02,                         //     INPUT (Data,Var,Abs)
    0x95, 0x03,                         //     REPORT_COUNT (3)
    0x81, 0x03,                         //     INPUT (Cnst,Ary,Abs)
    0x09, 0x32,                         //     USAGE (In Range)
    0x09, 0x47,                         //     USAGE (Confidence)
    0x95, 0x02,                         //     REPORT_COUNT (2)
    0x81, 0x02,                         //     INPUT (Data,Var,Abs)
    0x95, 0x0a,                         //     REPORT_COUNT (10)
    0x81, 0x03,                         //     INPUT (Cnst,Ary,Abs)
    0x05, 0x01,                         //     USAGE_PAGE (Generic Desk..
    0x26, 0xff, 0x7f,                   //     LOGICAL_MAXIMUM (32767)
    0x75, 0x10,                         //     REPORT_SIZE (16)
    0x95, 0x01,                         //     REPORT_COUNT (1)
    0x65, 0x00,                         //     UNIT (None)
    0x09, 0x30,                         //     USAGE (X)
    0x81, 0x02,                         //     INPUT (Data,Var,Abs)
    0x09, 0x31,                         //     USAGE (Y)
    0x46, 0x00, 0x00,                   //     PHYSICAL_MAXIMUM (0)
    0x81, 0x02,                         //     INPUT (Data,Var,Abs)
    0x05, 0x0d,                         //     USAGE PAGE (Digitizers)
    0x09, 0x48,                         //     USAGE (Width)
    0x09, 0x49,                         //     USAGE (Height)
    0x95, 0x01,                         //     REPORT_COUNT (2)
    0x81, 0x02,                         //     INPUT (Data,Var,Abs)
    0x81, 0x03,                         //     INPUT (Cnst,Ary,Abs)
    0x09, 0x51,                         //     USAGE (Contact Identifier)
    0x75, 0x10,                         //     REPORT_SIZE (16)
    0x95, 0x02,                         //     REPORT_COUNT (1)
    0x81, 0x02,                         //     INPUT (Data,Var,Abs)
    0x09, 0x55,                         //    USAGE(Maximum Count)
    0x15, 0x00,                         //   LOGICAL_MINIMUM (0)
    0x25, 0x08,                         //   LOGICAL_MAXIMUM (255)
    0x75, 0x08,                         //   REPORT_SIZE (8)
    0x95, 0x01,                         //   REPORT_COUNT (1)
    0xb1, 0x02,                         //   FEATURE (Data,Var,Abs)
    0xc0,                               //   END_COLLECTION
    0xc0                               // END_COLLECTION
};

This is my implementation of said message:

typedef union {
  struct{
    uint8_t TipSwitchLsb4Bits_InRange_Confidence_Empty2Bits;
    uint8_t Padding8_10Minus2;
    int16_t xAxis;
    int16_t yAxis;
    int16_t width;
    int16_t height;
    int16_t ContactId;
    uint8_t MaximumCount;
  };
} HID_SerialTouchReport;

void reportTouch(unsigned int x, unsigned int y) {
  HID_SerialTouchReport report;
  report.TipSwitchLsb4Bits_InRange_Confidence_Empty2Bits = 0xff;
  report.xAxis = x;
  report.yAxis = y;
  report.width = 4;
  report.height = 4;
  report.ContactId = 0;
  report.MaximumCount = 1;
  HID().SendReport(REPORTID_MTOUCH, &report, sizeof(report));
}

...
  static HIDSubDescriptor node(_hidSerialMultiTouchDescriptor, sizeof(_hidSerialMultiTouchDescriptor));
HID().AppendDescriptor(&node);
...

//Move the pointer diagonally across the screen:
for(unsigned int x=0; x<32767; x+=1000) {
  reportTouch(x, x);
  delay(50);
}

The test code is supposed to draw a diagonal line from top left to bottom right on Microsoft Paint.

When I test this with an AbsoluteMouse descriptor/Message everything works. So I presume the problem is in my definition of HID_SerialTouchReport.

Some things that have grabbed my attention:

Why does "PHYSICAL_MAXIMUM (0)" come while still defining "USAGE (Y)". Is it somehow relevant to Y and not X? And why is it "0"?
What is the significance of "Ary" in "INPUT (Cnst, Ary, Abs)" for height and width?
I am sort of "fudging" things with: "report.TipSwitchLsb4Bits_InRange_Confidence_Empty2Bits=0xff;". However, this seems alright to me, because I think all the non-padded bits need to be set to "1" anyway.
I am not sure what I should set report.MaximumCount to be. I have tried experimenting with different Count & Id values, but all had no effect.
My Arduino does show up as a "composite device" in Device Manager, so I figure not all is wrong.
In the article I linked "Serial Packet" mode is for Legacy Windows 7, and I am using Windows 10. But I figure this should not be a problem.
While I did copy the HID descriptor directly from the Microsoft site, I am not sure I properly constructed the message structure. (Did I forget a few bits?)

Or is it something entirely other which is causing my code to fail?

USB HID multitouch serial reporting

Answers (1)

Related Questions