Ada Baumann
Ada Baumann

Reputation: 31

Injecting Touchpad input on Windows

Background

I'm working on a program that shares input devices over a network to pass them into virtual machines. Currently, it works for Linux hosts and Linux guests. It uses the evdev interface, which is native to Linux, to read and write input events. I now want to write a Windows client, that translates the evdev events into input events on Windows.

Problem

I found a Windows API for input Injection that lets you inject Mouse, Gamepad, Keyboard, Pen, Pointer, and Touch input. But I haven't been able to find a method to inject Touchpad input. Here is an example of injecting Mouse input in C#:

using System;
using System.Collections.Generic;
using System.Data;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Runtime.InteropServices;
using System.Runtime.InteropServices.WindowsRuntime;
using Windows.Devices.Bluetooth.Advertisement;
using Windows.Foundation;
using Windows.Foundation.Collections;
using Windows.UI.Input.Preview.Injection;
using Windows.UI.ViewManagement;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Controls.Primitives;
using Windows.UI.Xaml.Data;
using Windows.UI.Xaml.Input;
using Windows.UI.Xaml.Media;
using Windows.UI.Xaml.Navigation;

// The Blank Page item template is documented at https://go.microsoft.com/fwlink/?LinkId=402352&clcid=0x409

namespace InputInjectorGUI
{
    public sealed partial class MainPage : Page
    {
        public MainPage()
        {
            this.InitializeComponent();
            ApplicationView.PreferredLaunchViewSize = new Size(600, 200);
            ApplicationView.PreferredLaunchWindowingMode = ApplicationViewWindowingMode.PreferredLaunchViewSize;

            Mouse mouse = new Mouse();

            // usage examples
            //mouse.MoveRel(100, 100);
            //mouse.MoveAbs(32768, 32768);
            //mouse.LeftDown();
            //mouse.LeftUp();
            //mouse.MoveRelOnVirtualDesk(10000, 10000);
            //mouse.Wheel(-500);
            //mouse.HWheel(500);
            mouse.EvaluateEvents();
        }

        public abstract class InputType<T, O>
        {
            public List<T> Data { get; set; }
            public List<O> Options { get; set; }
            public InputInjector InputInjectorInstance { get; set; }
            
            public InputType()
            {
                Data = new List<T>();
                Options = new List<O>();
                InputInjectorInstance = InputInjector.TryCreate();
                if (InputInjectorInstance != null)
                {
                    Debug.WriteLine("Successfully created input injector");
                } else
                {
                    throw new Exception("Could not create input injector");
                }
            }

            public O OptionsOr()
            {
                if (Options.Count == 0)
                {
                    return default;
                }
                dynamic Option = default(O);
                
                foreach (O o in Options)
                {
                    Option |= o;
                }
                return Option;
            }

            public void EvaluateEvents()
            {
                if (Data.Count >= 1)
                {
                    InjectEvents(Data);
                }
                Data.Clear();
            }

            public abstract void InjectEvents(List<T> Events);
        }

        public class Mouse: InputType<InjectedInputMouseInfo, InjectedInputMouseOptions>
        {
            public override void InjectEvents(List<InjectedInputMouseInfo> Events)
            {
                this.InputInjectorInstance.InjectMouseInput(Events);
            }

            public void Action(InjectedInputMouseOptions TempOption, int DeltaX, int DeltaY, int Data)
            {
                this.Options.Add(TempOption);
                this.Data.Add
                (
                    new InjectedInputMouseInfo
                    {
                        DeltaX = DeltaX,
                        DeltaY = DeltaY,
                        MouseData = unchecked ((uint)Data),
                        MouseOptions = this.OptionsOr(),
                        TimeOffsetInMilliseconds = 0,
                    }
                );
                this.Options.Remove(TempOption);
            }

            public void MoveRel(int x, int y)
            {
                Action(InjectedInputMouseOptions.Move, x, y, 0);
            }
            public void MoveRelNoCoalesce(int x, int y)
            {
                Action(InjectedInputMouseOptions.MoveNoCoalesce, x, y, 0);
            }
            public void MoveRelOnVirtualDesk(int x, int y)
            {
                Action(InjectedInputMouseOptions.VirtualDesk, x, y, 0);
            }
            public void MoveAbs(int x, int y)
                // normalized between (0, 0) and (65535, 65535) on primary monitor
            {
                Action(InjectedInputMouseOptions.Absolute, x, y, 0);
            }
            public void LeftDown()
            {
                Action(InjectedInputMouseOptions.LeftDown, 0, 0, 0);
            }
            public void LeftUp()
            {
                Action(InjectedInputMouseOptions.LeftUp, 0, 0, 0);
            }
            public void RightDown()
            {
                Action(InjectedInputMouseOptions.RightDown, 0, 0, 0);
            }
            public void RightUp()
            {
                Action(InjectedInputMouseOptions.RightUp, 0, 0, 0);
            }
            public void MiddleDown()
            {
                Action(InjectedInputMouseOptions.MiddleDown, 0, 0, 0);
            }
            public void MiddleUp()
            {
                Action(InjectedInputMouseOptions.MiddleUp, 0, 0, 0);
            }
            public void Wheel(int Amount)
            {
                Action(InjectedInputMouseOptions.Wheel, 0, 0, Amount);
                // Amount maybe has to be y instead
            }
            public void HWheel(int Amount)
            {
                Action(InjectedInputMouseOptions.HWheel, 0, 0, Amount);
                // Amount maybe has to be x instead
            }
            public void XDown()
            {
                Action(InjectedInputMouseOptions.XDown, 0, 0, 0);
            }
            public void XUp()
            {
                Action(InjectedInputMouseOptions.XUp, 0, 0, 0);
            }
        }
        public void MovePointer()
        {
            InputInjector _inputInjector;
            _inputInjector = InputInjector.TryCreate();

            if (_inputInjector != null)
            {
                List<InjectedInputMouseInfo> mouseData = new List<InjectedInputMouseInfo>
                {
                    new InjectedInputMouseInfo
                    {
                        DeltaX = 100,
                        DeltaY = 100,
                        MouseData = 1,
                        MouseOptions = InjectedInputMouseOptions.Move,
                        TimeOffsetInMilliseconds = 0,
                    }
                };
                _inputInjector.InjectMouseInput(mouseData);
                Debug.WriteLine("Successfull input injection");
            }
            else
            {
                Debug.WriteLine("Failure injecting input");
            }
        }
    }
}

I also found the Win32 SendInput function, but it only seems to work with Mouse and Keyboard inputs. There is the possibility to give Hardware inputs for all devices that aren't a Mouse or a Keyboard but I wouldn't know how to approach that.

The python pynput module also only provides an interface to Mouse and Keyboard. It is an abstraction on top of the Win32 SendInput function through ctypes.

One might be able to write a HID source driver but that seems way overkill for such a simple problem.

The programming language doesn't matter, as long as it can inject Touchpad input. Preferred languages would be Python, C, C++, C#, or Rust though.

Edit:

Looking into using a Python script to interact with usbip to simulate a USB touchpad over a network as suggested by this blog post. There seems to be a decent amount of documentation on HID touchpads by Microsoft that I might be able to utilize.

Edit 2:

After trying to use the usbip virtual device python script, I noticed that

  1. It would probably be possible to get it working with everything but
  2. It was slow and would not lead to a native-like experience, since it seems to only be able to process a few events per second. I figured it might be possible to run multiple virtual device clients to get a native-like experience but that seemed way too hacky and there must be a better way.

I guess the best shot at solving the problem would be to write a Virtual HID source driver. Through this issue I was able to find this example for a HID Injector. I deem this example necessary because the documentation on the virtual HID framework doesn't make it apparent how to exactly implement a virtual HID driver, or at least not to people without previous experience with writing Windows drivers. As a pointer here is how to write a "Hello World" KMDF driver. Now I just have to understand how exactly the HID Injector works so that I can implement my own version using the aforementioned HID Touchpad documentation. This approach seems pretty complicated and complex, but I think it will yield the best results.

Edit 3:

Got the example HIDInjector project running. It looks like a promising base to build off of.

Upvotes: 3

Views: 347

Answers (0)

Related Questions