CODEWITHSUNDEEP
c#
user12868521
user12868521

Reputation:

List of all Possible Input Characters from specific language keyboard including dead key character

I want to get all possible characters that can be enter by keyboard. For example, if language is English(United State) then list of all character that can be input by keyboard.Here is my code(C#):

Enumerable.Range(char.MinValue, char.MaxValue)
                .Select(n => (char)n)
                .Where(c => IsRepresentable(c, 
keyboardLayout));

But it missing dead keys characters. Is there any solution to get all characters list for specific keyboard? Here are few dead keys characters: ò, Ò ,ô ,Ô, ß, ä, Ä

Upvotes: 1

Views: 1366

Answers (1)

wardies
wardies

Reputation: 1259

This is not a trivial problem. I believe there are at least two ways to approach this and both require use of Windows APIs, whether Win32 or individual keyboard DLLs. I have fleshed out the second (easier?) option because the first option seems incredibly complicated to me:

1. Query the keyboard layout DLL

Each keyboard layout has its own DLL describing the combinations of keys that are required to produce each character. By calling methods in the DLL you can query the layout table to get the full details. The main benefit is you are guaranteed a completely accurate list of key combinations. A secondary benefit is that the keyboard layout does need to be installed and set to default for the querying application.

The Windows registry holds the entire list of available layouts in individual Keys under HKLM\SYSTEM\CurrentControlSet\Control\Keyboard Layouts. Therein you can find entries for the DLL file name and display name. All the keyboard layout DLL files are stored under the %SystemRoot%\system32 folder.

For further information on this complicated topic, please see Michael S. Kaplan's blog archive. A good starting point is the %WINDIR%\system32\kbd*.dll post.

2. Exhaustively try each key combination you can imagine

This is frankly a much easier solution from my point of view. We can use the Win32 API calls GetKeyboardLayout, MapVirtualKey and ToUnicodeEx available in User32.dll to simulate pressing key combinations and see what character is produced.

We start by trying all the normal keys without trying dead key prefixes:

TestDeadKeyCombinations(Keys.None);

Then we try prefixing potential dead keys we've identified, e.g. Keys.Oem1 or Keys.Question:

for (int deadKey = (int)Keys.OemSemicolon; deadKey <= (int)Keys.Oem102; deadKey++)
{
    TestDeadKeyCombinations((Keys)deadKey);
}

Note that the number row symbols (e.g. Shift+5) can be dead keys too:

for (int deadKey = (int)Keys.D0; deadKey <= (int)Keys.D9; deadKey++)
{
    TestDeadKeyCombinations((Keys)deadKey);
}

Then our TestDeadKeyModifiers method tries three combinations of dead key combinations:

  1. Pressing the dead key followed by the number row keys 0-9 or the A-Z keys:

    for (int key = '0'; key <= 'Z'; key++)
{
    TestDeadKeyModifiers(deadKey, (Keys)key);
}

  2. Pressing the dead key twice:

    TestDeadKeyModifiers(deadKey, deadKey);

  3. Pressing the dead key followed by the space bar:

    TestDeadKeyModifiers(deadKey, Keys.Space);

The TestDeadKeyModifiers method iterates through all the combinations of dead key presses followed by normal key presses while trying, in addition, all variations of Shift and AltGr.

void TestDeadKeyModifiers(Keys deadKey, Keys key)
{
    foreach (var mod1 in _modifierCombinations)
    {
        foreach (var mod2 in _modifierCombinations)
        {
            if (TestKeypress(deadKey, mod1, true))
            {
                TestKeypress(key, mod2, false);
            }
        }
    }
}

And our TestKeypress method does the Win32 API calls to simulate the keypress:

public bool TestKeypress(Keys key, List<Keys> modifiers, bool deadKey)
{
    if (deadKey && key == Keys.None)
    {
        // Try the special case of key on its own (without prior dead key).
        return true;
    }

    byte[] keyboardState = new byte[255];

    foreach (var holdKeyCombination in modifiers)
    {
        keyboardState[(int)holdKeyCombination] = 0x80;
    }

Now we've prepared the simulated keyboard and have simulated holding down the specified modifiers keys by setting the high bit of the byte. Next we map the requested key to the keyboard scan code using MapVirtualKey. If we're trying to simulate a dead keypress, then we set uMapType to 2.

    uint virtualKeyCode = (uint)key;

    uint scanCode = MapVirtualKey(virtualKeyCode, (deadKey ? 2U : 0U));

We request the input locale for the current thread and this information (both the input language and the physical layout of the keyboard) is passed to the ToUnicodeEx API call along with everything else:

    IntPtr inputLocaleIdentifier = GetKeyboardLayout(0);

    StringBuilder output = new StringBuilder();

    int result = ToUnicodeEx(virtualKeyCode, scanCode, keyboardState, output,
        (int)5, (uint)0, inputLocaleIdentifier);

Finally, we check the result and if we got a single character in the output string, count this as a success and add it to our list of characters along with the key combination used to obtain it. We also keep track of the last dead key press outside the method so we can reuse this function for the next keypress:

    if (result != -1 && output.Length == 1)
    {
        AddSuccess(output[0], key, modifiers);

        _lastDeadKey = Keys.None;
    }
    else if (deadKey && result == -1)
    {
        _lastDeadKeyModifiers = modifiers;
        _lastDeadKey = key;
    }
    else if (!deadKey)
    {
        _lastDeadKey = Keys.None;
    }

    return (result == -1 || output.Length == 1);
}

If no character (or more than one character) is produced, then we ignore the result. E.g. on the United States-International keyboard, pressing the Shift+6 combination once acts as a dead key and a second time gives us two circumflexes ^^. We don't want to record this since Shift+6 followed by Space gives us the single circumflex we want.

Note: whilst it is possible that for some keyboards (e.g. United States-International) this will find all the characters you need, it is not guaranteed to do so. I am fairly certain most keyboard layouts will use additional keys as dead keys, or as normal keys, or distinguish between right and left modifiers, and these would need to be catered for.

And of course, every Unicode character is technically available by use of the Alt+Numpad keyboard combination to retrieve the character by its decimal Unicode value.

The full LinqPad script is here along with additional code to track which key combination produced each character:

[DllImport("user32.dll", CharSet = CharSet.Unicode)]
private static extern int ToUnicodeEx(uint virtualKeyCode, uint scanCode,
    byte[] keyboardState,
    [Out, MarshalAs(UnmanagedType.LPWStr, SizeParamIndex = 4)] StringBuilder receivingBuffer,
    int bufferSize, uint flags, IntPtr dwhkl);

[DllImport("user32.dll")]
static extern uint MapVirtualKey(uint uCode, uint uMapType);

[DllImport("user32.dll")]
static extern IntPtr GetKeyboardLayout(uint idThread);


HashSet<char> _typeable = new HashSet<char>();
Dictionary<char, string> _howToObtain = new Dictionary<char, string>();

List<List<Keys>> _modifierCombinations = new List<List<Keys>> {
    new List<Keys>{ },
    new List<Keys>{ Keys.ShiftKey, Keys.LShiftKey },
    new List<Keys>{ Keys.ShiftKey, Keys.RShiftKey },
    new List<Keys>{ Keys.ControlKey,Keys.Menu, Keys.LControlKey, Keys.RMenu },
    new List<Keys>{ Keys.ShiftKey, Keys.LShiftKey,
        Keys.ControlKey, Keys.Menu, Keys.LControlKey, Keys.RMenu }
};

private List<Keys> _modifierKeys = new List<Keys> {
    Keys.ControlKey,
    Keys.ShiftKey,
    Keys.Menu,
    Keys.LShiftKey,
    Keys.RShiftKey,
    Keys.LControlKey,
    Keys.RControlKey,
    Keys.LMenu,
    Keys.RMenu
};

Keys _lastDeadKey = Keys.None;
List<Keys> _lastDeadKeyModifiers;

void Main()
{
    TestDeadKeyCombinations(Keys.None);

    for (int deadKey = (int)Keys.OemSemicolon;
            deadKey <= (int)Keys.Oem102;
            deadKey++)
    {
        TestDeadKeyCombinations((Keys)deadKey);
    }

    for (int deadKey = (int)Keys.D0; deadKey <= (int)Keys.D9; deadKey++)
    {
        TestDeadKeyCombinations((Keys)deadKey);
    }

    // _typeable.OrderBy(x => x).Dump("Typeable characters");

    _howToObtain.OrderBy(x => x.Key).Dump(
        "Characters available through keyboard combinations");

    // Enumerable.Range(32, ((int)char.MaxValue) - 31)
    //  .Select(e => (char)e)
    //  .Except(_typeable)
    //  .OrderBy(x => x)
    //  .Select(x => $"{x} (0x{((int)x).ToString("x4")})")
    //  .Dump("Non-typeable characters");
}

void TestDeadKeyCombinations(Keys deadKey)
{
    for (int key = '0'; key <= 'Z'; key++)
    {
        TestDeadKeyModifiers(deadKey, (Keys)key);
    }

    TestDeadKeyModifiers(deadKey, deadKey);

    TestDeadKeyModifiers(deadKey, Keys.Space);
}

void TestDeadKeyModifiers(Keys deadKey, Keys key)
{
    foreach (var mod1 in _modifierCombinations)
    {
        foreach (var mod2 in _modifierCombinations)
        {
            if (TestKeypress(deadKey, mod1, true))
            {
                TestKeypress(key, mod2, false);
            }
        }
    }
}

public bool TestKeypress(Keys key, List<Keys> modifiers, bool deadKey)
{
    if (deadKey && key == Keys.None)
    {
        // Try the special case of key on its own (without prior dead key).
        return true;
    }

    byte[] keyboardState = new byte[255];

    foreach (var holdKeyCombination in modifiers)
    {
        keyboardState[(int)holdKeyCombination] = 0x80;
    }

    uint virtualKeyCode = (uint)key;

    uint scanCode = MapVirtualKey(virtualKeyCode, (deadKey ? 2U : 0U));
    IntPtr inputLocaleIdentifier = GetKeyboardLayout(0);

    StringBuilder output = new StringBuilder();

    int result = ToUnicodeEx(virtualKeyCode, scanCode, keyboardState, output,
        (int)5, (uint)0, inputLocaleIdentifier);

    if (result != -1 && output.Length == 1)
    {
        AddSuccess(output[0], key, modifiers);

        _lastDeadKey = Keys.None;
    }
    else if (deadKey && result == -1)
    {
        _lastDeadKeyModifiers = modifiers;
        _lastDeadKey = key;
    }
    else if (!deadKey)
    {
        _lastDeadKey = Keys.None;
    }

    return (result == -1 || output.Length == 1);
}

void AddSuccess(char outputChar, Keys key, List<Keys> modifiers)
{
    if (_typeable.Add(outputChar))
    {
        // This is the first time we've been able to produce this character,
        // so store the first (simplest) key combination that produced it
        string hto = string.Empty;

        if (_lastDeadKey != Keys.None)
        {
            hto = ExplainKeyCombo(_lastDeadKey, _lastDeadKeyModifiers);
            hto += " ";
        }

        hto += ExplainKeyCombo(key, modifiers);

        _howToObtain.Add(outputChar, hto);
    }
}

string ExplainKeyCombo(Keys key, List<Keys> modifiers)
{
    string explain = string.Empty;

    if (modifiers.Intersect(
        new List<Keys>{
            Keys.ShiftKey, Keys.LShiftKey
        }).Count() == 2)
    {
        explain += "Shift+";
    }

    if (modifiers.Intersect(
        new List<Keys> {
            Keys.ControlKey, Keys.Menu, Keys.LControlKey, Keys.RMenu
        }).Count() == 4)
    {
        explain += "AltGr+";
    }

    explain += key.ToString();

    return explain;
}

Upvotes: 2

Related Questions