CODEWITHSUNDEEP
pythonpython-3.xpython-asynciouvloop
elaspog
elaspog

Reputation: 1709

Error with Observer while exiting async function only when using uvloop as event loop

I can't figure out, why I'm getting error while shutting down when using uvloop, and not getting the same error when going without it.

The error:

ImportError: sys.meta_path is None, Python is likely shutting down

I need to use an Observer to watch config files and reconfigure the behaviour. I'm getting the error only when I'm calling observer.stop() and observer.join() AND using uvloop. When I'm not calling them, there is no exception in the output. It doens't matter if I'm calling the stop() and join() functions from async code (see Option 1) or from the synchronous code (see Option 2 (preferred)).

if sys.platform != 'win32':
    import uvloop
    uvloop.install()


async def main(loop_container, connector, processor, observer):

    loop_container.set_current_event_loop()

    try:
        connector.connect_exchange()
        asyncio.create_task(processor.process_queue_events())
        is_connected = await connector.is_connected()
        await connector.start_symbol_watchings()

        retry_delay = 1
        while True:
            await asyncio.sleep(retry_delay)    # keepalive
            # retry logic (...)

    except asyncio.CancelledError:
        logging.error(f'{Utils.get_current_datetime()}  Execution has been cancelled.')

    except Exception as exc:
        logging.error(f'Unknown Error:  {type(exc).__name__}  {str(exc)}')

    finally:
        await connector.disconnect_exchange()

        # Option 1
        #observer.stop()
        #observer.join()

if __name__ == "__main__":

    try:

        tick_data_queue = asyncio.Queue()
        observer        = Observer()
        loop_container  = EventLoopContainer()

        config_abs_path = os.path.abspath(CONFIG_FILE_NAME)
        output_abs_path = os.path.abspath(OUTPUT_FILE_NAME)
        config_reader   = ConfigReader(config_abs_path)
        state_persister = StatePersister(output_abs_path, config_reader)

        connector       = ExchangeConnector(loop_container, tick_data_queue, config_reader, state_persister)
        processor       = TickProcessor(loop_container, tick_data_queue, config_reader, state_persister, connector)
        event_handler   = FileChangeHandler(loop_container, config_reader, state_persister, connector, processor)

        observer.schedule(
            event_handler,
            path        = os.path.dirname(config_abs_path),
            recursive   = False,
        )
        observer.start()

        main_coro = main(loop_container, connector, processor, observer)
        asyncio.run(main_coro)

        # Option 2 (preferred)
        #observer.stop()
        #observer.join()

    except Exception as exc::
        logging.critical('Terminated.')

I'm not getting this error when:

What I can understand, that from the thread used by the Observer class somehow conflicts with the async loop.

Does anyone has any idea how to resolve this issue?

^C
2024-11-14|17:25:17.996  Execution has been cancelled.
2024-11-14|17:25:18.001  [symbol='ETH/USDT:USDT']  Connection error: ExchangeClosedByUser : Connection closed by the user.
2024-11-14|17:25:18.002  [symbol='BTC/USDT:USDT']  Connection error: ExchangeClosedByUser : Connection closed by the user.
2024-11-14|17:25:18.253  Disconnected from the exchange.
--- Logging error ---
--- Logging error ---
--- Logging error ---
--- Logging error ---
Exception ignored in: <bound method Loop.call_exception_handler of <uvloop.Loop running=False closed=True debug=False>>
Traceback (most recent call last):
  File "uvloop/loop.pyx", line 2429, in uvloop.loop.Loop.call_exception_handler
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1548, in error
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1664, in _log
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1680, in handle
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1736, in callHandlers
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1026, in handle
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/handlers.py", line 83, in emit
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1075, in handleError
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 129, in print_exception
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 1044, in __init__
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 492, in _extract_from_extended_frame_gen
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 369, in line
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 350, in _set_lines
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/linecache.py", line 25, in getline
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/linecache.py", line 41, in getlines
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/linecache.py", line 88, in updatecache
ImportError: sys.meta_path is None, Python is likely shutting down
--- Logging error ---
--- Logging error ---
--- Logging error ---
--- Logging error ---
Exception ignored in: <bound method Loop.call_exception_handler of <uvloop.Loop running=False closed=True debug=False>>
Traceback (most recent call last):
  File "uvloop/loop.pyx", line 2429, in uvloop.loop.Loop.call_exception_handler
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1548, in error
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1664, in _log
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1680, in handle
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1736, in callHandlers
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1026, in handle
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/handlers.py", line 83, in emit
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/logging/__init__.py", line 1075, in handleError
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 129, in print_exception
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 1044, in __init__
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 492, in _extract_from_extended_frame_gen
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 369, in line
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/traceback.py", line 350, in _set_lines
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/linecache.py", line 25, in getline
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/linecache.py", line 41, in getlines
  File "/home/ubuntu/.pyenv/versions/3.13.0/lib/python3.13/linecache.py", line 88, in updatecache
ImportError: sys.meta_path is None, Python is likely shutting down

Upvotes: 1

Views: 29

Answers (0)

Related Questions