Why do some emoticons cause Python/Idle to crash on Ubuntu?

Question

I have run into a problem printing strings containing emoticons using Python code running via Idle under Ubuntu 20.04LTS. The code works fine under Windows 10, but when I try to run it under Ubuntu, trying to view some strings cause Python/Idle to crash without showing any helpful error messages. The emoticons themselves appear to be the problem, but not all emoticons. For example, 128516 (grinning face with smiling eyes) is ok whereas 128077 (thumbs up) causes the crash. Both PCs are running Python 3.8.6 and Idle shows the same version number.

I have created a code snippet to demonstrate the problem.

def make_uchr(code: str):
    return chr(int(code.lstrip("U+").zfill(8), 16))

def calc1char(intdecimal):   
   # print()  128516 is ok...  128077 crashes on Ubuntu
          
    strhex = hex(intdecimal)[2:]
    strUhex  = "U+"+strhex
    stroutput = make_uchr(strUhex)
    return stroutput
    
def main():

    while True:
        try:
            print("Enter a decimal integer or ctrl+C to exit...")
            print("Try 128516, then try 128077.")
            strdecimal = input('Enter your choice: ')
            intdecimal = int(strdecimal)
            stroutput = calc1char(intdecimal)
            print("The corresponding character is:")
            print(stroutput)
            print()
        except KeyboardInterrupt:
            break

if __name__ == '__main__':
    main()

When under Windows 10, this code runs without seeing any problems both when run via Idle and if run from the command prompt. Under Ubuntu, when it is run using Idle, Idle crashes for some codes (e.g. 128077) without any visible error messages. When run from the terminal or using Pycharm, the expected emoticon is printed to the console. I recently upgraded from Ubuntu 18.04LTS and the problem was showing there as well.

In the code where the problem became apparent, the strings with emoticons will export to a text file correctly and I do have a workaround to cleanse the viewed string when running under Ubuntu, i.e.

if platform.system() == "Linux":  
    strmsg = (strmsg.encode("utf-8", errors='replace'))
    strmsg = str(strmsg.decode("ascii", errors='replace')) 

This gives question marks in place of other characters with codes greater than 127.

However I am curious whether there is a fix for the problem rather than a workaround. Any suggestions would be appreciated.

---

Here is another variant using a GUI example.

from tkinter import *
import tkinter as tk

def make_uchr(code: str):
    return chr(int(code.lstrip("U+").zfill(8), 16))

def calc1char(intdecimal):   
          
    strhex = hex(intdecimal)[2:]
    strUhex  = "U+"+strhex
    stroutput = make_uchr(strUhex)
    return stroutput

def printtoconsole():

    intdecimal = int(entry_decimal.get())
    stroutput = calc1char(intdecimal)
    print(stroutput)

def printtolabel():

    intdecimal = int(entry_decimal.get())
    stroutput = calc1char(intdecimal)    
    label_output.configure(text=stroutput)

def printtoentry():

    intdecimal = int(entry_decimal.get())
    stroutput = calc1char(intdecimal)
    entry_output.delete(0,len(entry_output.get()))
    entry_output.delete(0,len(entry_output.get()))
    entry_output.insert(INSERT, stroutput)
    
def printtotext():
    intdecimal = int(entry_decimal.get())
    stroutput = calc1char(intdecimal)
    text_box.delete("1.0", END)
    text_box.insert(INSERT,stroutput)    
    text_box.pack()
    
def main():

    window = tk.Tk()
    window.geometry("300x300")
    window.title("Emoticons  !")

    decimal_var = tk.StringVar()
    strentryoutput_var = tk.StringVar()
    strdecimal = "128516"

    decimal_var.set(strdecimal)

    global entry_decimal, text_box, label_output, entry_output

    button1=Button(window, text="Print()",   command=printtoconsole)
    button2=Button(window, text="To Label",   command=printtolabel)
    button3=Button(window, text="To Entry",   command=printtoentry)
    button4=Button(window, text="To Text",   command=printtotext)

    entry_decimal = Entry(window, textvariable = decimal_var)
    label_decimal = Label(window, text = "Enter decimal")
    label_output = Label(window, text = "label to show output")
    entry_output = Entry(window, textvariable = strentryoutput_var)
    text_output = Text(window)
    text_box = Text(text_output)

    button1.place(x=10, y=50)
    button2.place(x=10, y=100)
    button3.place(x=10, y=150)
    button4.place(x=10, y=200)

    label_decimal.place(x=10, y=10) 
    entry_decimal.place(x=100, y=10)
    label_output.place(x=100, y=100)
    entry_output.place(x=100, y=150, width=50)
    text_output.place(x=100, y=200, width=50, height=30)


if __name__ == '__main__':
    main()

This new code uses a GUI to give 4 buttons to either print the emoticon to the console, to a label widget, to an entry widget or a text widget. Taking the decimal range 128547 to 128549, on Windows 10, emoticons print from each button without problems. On Ubuntu 20.04, all buttons work correctly for 128547 and 128549. However for 128548, the print button gives no output, but does not fail. The label, entry and text buttons cause the GUI to close, but leave the console open and reset showing "=== Restart: Shell === >>>". Restarting from the editor window using F5 is then possible. On Ubuntu, the entry button needs clicking twice to update the emoticon, whereas under Windows, one click is sufficient. Curious, but perhaps a different matter...

Answer 1

You can fix it by installing ttf-ancient-fonts-symbola on Ubuntu.

sudo apt-get install -y ttf-ancient-fonts-symbola

Answer 2

Further to comments on Issue42225 Python Issues (https://bugs.python.org/issue42225), the following code run in a terminal window is given as a simple test for Linux and MacOS systems to show whether a character will give a problem.

    $ wish
    % label .l -text 'paste an emoticon graphic here'

If the character would print, the following is output: ".l"

For a troublesome character, following error is typical.

% X Error of failed request:  BadLength (poly request too large or internal Xlib length error)
  Major opcode of failed request:  139 (RENDER)
  Minor opcode of failed request:  20 (RenderAddGlyphs)
  Serial number of failed request:  599
  Current serial number in output stream:  599 

It is also implied that the font package "fonts-noto-color-emoji" is involved in the crashing of Tk windows on Ubuntu. When I uninstalled this font the crashing behaviour in Ubuntu stopped - although several emoji characters are no longer available.

Further internet searching around the "RenderAddGlyphs" error message found the following links on a related topic. https://forum.manjaro.org/t/xorg-problem-while-running-tkinter-program-on-manjaro-python/37324/7 and https://bugs.launchpad.net/ubuntu/+source/git/+bug/1852985 One suggestion from this second link referred to installing the "Unifonts" package. Although this is not the most detailed representation of emoji characters - and also monochrome, it does have the significant benefit that Tk (version: 8.6.10) running on Ubuntu 20.04 no longer crashes when one of the previously troublesome characters is encountered. So far, this has proved a good enough workaround for my original problem...