How are entrypoints of the Python standard library registered?

Question

Regular Python packages register entrypoints in their setup.py or specify them in a pyporject.toml file. The latter is the case, if they don't use distutils/setuptools for packaging. As an example, setuptools' own setup.py has this as one of the arguments to setuptools.setup():

    entry_points={
        "setuptools.finalize_distribution_options": [
            "parent_finalize = setuptools.dist:_Distribution.finalize_options",
            "features = setuptools.dist:Distribution._finalize_feature_opts",
            "keywords = setuptools.dist:Distribution._finalize_setup_keywords",
            "2to3_doctests = "
            "setuptools.dist:Distribution._finalize_2to3_doctests",
        ],

Question:

How do standard library packages register their entrypoints, when they come bundled with python?

For example distutils itself has the setup_keywords group of entrypoints.

>>> for ep in pkg_resources.iter_entry_points(group='distutils.setup_keywords'):
>>>    print(ep)

convert_2to3_doctests = setuptools.dist:assert_string_list
dependency_links = setuptools.dist:assert_string_list
eager_resources = setuptools.dist:assert_string_list
exclude_package_data = setuptools.dist:check_package_data
extras_require = setuptools.dist:check_extras
include_package_data = setuptools.dist:assert_bool
install_requires = setuptools.dist:check_requirements
namespace_packages = setuptools.dist:check_nsp
package_data = setuptools.dist:check_package_data
# ... and so on ...

Where do they come from?

How do I list all entrypoints for all the standard library packages? I.e. without knowing the group name for pkg_resources.iter_entry_points(group='<groupname>')?

Answer 1

They aren't. The standard library registers no entry points.

It might use entry points, but it is third party libraries who are expected to register objects in those keys.

Just create an empty virtualenv and run

import importlib.metadata

for (group, epoints) in importlib.metadata.entry_points.items():
    for ep in epoints:
       print(ep.value)

and you will see only objects coming from setuptools, wheel and pip have been registered. None of those are part of the standard library.

---

If you meant to ask how would the standard library register groups where external libraries would register objects...

It neither does. Groups are not registered whatsoever.

As an external library developer you can announce objects in whatever entry point group name you want with only an expectation that it will be used by another application. There is no actual mechanism to register "the willing to use an entry point group" besides documentation for humans.

Answer 2

The entry points are listed in entry_points.txt files for each of the packages. For example:

$ find /path/to/envs/py38/lib/python3.8/ -name "entry_points.txt"
/path/to/envs/py38/lib/python3.8/site-packages/mypy-0.761.dist-info/entry_points.txt
/path/to/envs/py38/lib/python3.8/site-packages/Pygments-2.5.2.dist-info/entry_points.txt
/path/to/envs/py38/lib/python3.8/site-packages/pip-20.0.2-py3.8.egg-info/entry_points.txt
/path/to/envs/py38/lib/python3.8/site-packages/setuptools-45.2.0.post20200210-py3.8.egg-info/entry_points.txt
/path/to/envs/py38/lib/python3.8/site-packages/ipython-7.12.0.dist-info/entry_points.txt
/path/to/envs/py38/lib/python3.8/site-packages/wheel-0.34.2-py3.8.egg-info/entry_points.txt

As we can see distutils doesn't advertise its own entry points, this is actually done by setuptools:

$ cat /path/to/envs/py38/lib/python3.8/site-packages/setuptools-45.2.0.post20200210-py3.8.egg-info/entry_points.txt
[console_scripts]
easy_install = setuptools.command.easy_install:main

[distutils.commands]
alias = setuptools.command.alias:alias
bdist_egg = setuptools.command.bdist_egg:bdist_egg
bdist_rpm = setuptools.command.bdist_rpm:bdist_rpm
bdist_wininst = setuptools.command.bdist_wininst:bdist_wininst
build_clib = setuptools.command.build_clib:build_clib
build_ext = setuptools.command.build_ext:build_ext
build_py = setuptools.command.build_py:build_py
develop = setuptools.command.develop:develop
dist_info = setuptools.command.dist_info:dist_info
easy_install = setuptools.command.easy_install:easy_install
egg_info = setuptools.command.egg_info:egg_info
install = setuptools.command.install:install
install_egg_info = setuptools.command.install_egg_info:install_egg_info
install_lib = setuptools.command.install_lib:install_lib
install_scripts = setuptools.command.install_scripts:install_scripts
rotate = setuptools.command.rotate:rotate
saveopts = setuptools.command.saveopts:saveopts
sdist = setuptools.command.sdist:sdist
setopt = setuptools.command.setopt:setopt
test = setuptools.command.test:test
upload_docs = setuptools.command.upload_docs:upload_docs

[distutils.setup_keywords]
convert_2to3_doctests = setuptools.dist:assert_string_list
dependency_links = setuptools.dist:assert_string_list
eager_resources = setuptools.dist:assert_string_list
entry_points = setuptools.dist:check_entry_points
exclude_package_data = setuptools.dist:check_package_data
extras_require = setuptools.dist:check_extras
include_package_data = setuptools.dist:assert_bool
install_requires = setuptools.dist:check_requirements
namespace_packages = setuptools.dist:check_nsp
package_data = setuptools.dist:check_package_data
packages = setuptools.dist:check_packages
python_requires = setuptools.dist:check_specifier
setup_requires = setuptools.dist:check_requirements
test_loader = setuptools.dist:check_importable
test_runner = setuptools.dist:check_importable
test_suite = setuptools.dist:check_test_suite
tests_require = setuptools.dist:check_requirements
use_2to3 = setuptools.dist:assert_bool
use_2to3_exclude_fixers = setuptools.dist:assert_string_list
use_2to3_fixers = setuptools.dist:assert_string_list
zip_safe = setuptools.dist:assert_bool

[egg_info.writers]
PKG-INFO = setuptools.command.egg_info:write_pkg_info
dependency_links.txt = setuptools.command.egg_info:overwrite_arg
depends.txt = setuptools.command.egg_info:warn_depends_obsolete
eager_resources.txt = setuptools.command.egg_info:overwrite_arg
entry_points.txt = setuptools.command.egg_info:write_entries
namespace_packages.txt = setuptools.command.egg_info:overwrite_arg
requires.txt = setuptools.command.egg_info:write_requirements
top_level.txt = setuptools.command.egg_info:write_toplevel_names

[setuptools.finalize_distribution_options]
2to3_doctests = setuptools.dist:Distribution._finalize_2to3_doctests
features = setuptools.dist:Distribution._finalize_feature_opts
keywords = setuptools.dist:Distribution._finalize_setup_keywords
parent_finalize = setuptools.dist:_Distribution.finalize_options

[setuptools.installation]
eggsecutable = setuptools.command.easy_install:bootstrap

Discovering all entry points

Starting with Python 3.8 you can use importlib.metadata.entry_points for listing the entry points of all installed packages:

>>> from importlib.metadata import entry_points
>>> from pprint import pprint
>>> pprint(entry_points())
{'console_scripts': (EntryPoint(name='dmypy', value='mypy.dmypy.client:console_entry', group='console_scripts'),
                     EntryPoint(name='mypy', value='mypy.__main__:console_entry', group='console_scripts'),
                     EntryPoint(name='stubgen', value='mypy.stubgen:main', group='console_scripts'),
                     EntryPoint(name='pygmentize', value='pygments.cmdline:main', group='console_scripts'),
                     EntryPoint(name='pip', value='pip._internal.cli.main:main', group='console_scripts'),
                     EntryPoint(name='pip3', value='pip._internal.cli.main:main', group='console_scripts'),
                     EntryPoint(name='pip3.8', value='pip._internal.cli.main:main', group='console_scripts'),
                     EntryPoint(name='easy_install', value='setuptools.command.easy_install:main', group='console_scripts'),
                     EntryPoint(name='iptest', value='IPython.testing.iptestcontroller:main', group='console_scripts'),
                     EntryPoint(name='iptest3', value='IPython.testing.iptestcontroller:main', group='console_scripts'),
                     EntryPoint(name='ipython', value='IPython:start_ipython', group='console_scripts'),
                     EntryPoint(name='ipython3', value='IPython:start_ipython', group='console_scripts'),
                     EntryPoint(name='wheel', value='wheel.cli:main', group='console_scripts')),
 'distutils.commands': (EntryPoint(name='alias', value='setuptools.command.alias:alias', group='distutils.commands'),
                        EntryPoint(name='bdist_egg', value='setuptools.command.bdist_egg:bdist_egg', group='distutils.commands'),
                        EntryPoint(name='bdist_rpm', value='setuptools.command.bdist_rpm:bdist_rpm', group='distutils.commands'),
                        EntryPoint(name='bdist_wininst', value='setuptools.command.bdist_wininst:bdist_wininst', group='distutils.commands'),
                        EntryPoint(name='build_clib', value='setuptools.command.build_clib:build_clib', group='distutils.commands'),
                        EntryPoint(name='build_ext', value='setuptools.command.build_ext:build_ext', group='distutils.commands'),
                        EntryPoint(name='build_py', value='setuptools.command.build_py:build_py', group='distutils.commands'),
                        EntryPoint(name='develop', value='setuptools.command.develop:develop', group='distutils.commands'),
                        EntryPoint(name='dist_info', value='setuptools.command.dist_info:dist_info', group='distutils.commands'),
                        EntryPoint(name='easy_install', value='setuptools.command.easy_install:easy_install', group='distutils.commands'),
                        EntryPoint(name='egg_info', value='setuptools.command.egg_info:egg_info', group='distutils.commands'),
                        EntryPoint(name='install', value='setuptools.command.install:install', group='distutils.commands'),
                        EntryPoint(name='install_egg_info', value='setuptools.command.install_egg_info:install_egg_info', group='distutils.commands'),
                        EntryPoint(name='install_lib', value='setuptools.command.install_lib:install_lib', group='distutils.commands'),
                        EntryPoint(name='install_scripts', value='setuptools.command.install_scripts:install_scripts', group='distutils.commands'),
                        EntryPoint(name='rotate', value='setuptools.command.rotate:rotate', group='distutils.commands'),
                        EntryPoint(name='saveopts', value='setuptools.command.saveopts:saveopts', group='distutils.commands'),
                        EntryPoint(name='sdist', value='setuptools.command.sdist:sdist', group='distutils.commands'),
                        EntryPoint(name='setopt', value='setuptools.command.setopt:setopt', group='distutils.commands'),
                        EntryPoint(name='test', value='setuptools.command.test:test', group='distutils.commands'),
                        EntryPoint(name='upload_docs', value='setuptools.command.upload_docs:upload_docs', group='distutils.commands'),
                        EntryPoint(name='bdist_wheel', value='wheel.bdist_wheel:bdist_wheel', group='distutils.commands')),
 'distutils.setup_keywords': (EntryPoint(name='convert_2to3_doctests', value='setuptools.dist:assert_string_list', group='distutils.setup_keywords'),
                              EntryPoint(name='dependency_links', value='setuptools.dist:assert_string_list', group='distutils.setup_keywords'),
                              EntryPoint(name='eager_resources', value='setuptools.dist:assert_string_list', group='distutils.setup_keywords'),
                              EntryPoint(name='entry_points', value='setuptools.dist:check_entry_points', group='distutils.setup_keywords'),
                              EntryPoint(name='exclude_package_data', value='setuptools.dist:check_package_data', group='distutils.setup_keywords'),
                              EntryPoint(name='extras_require', value='setuptools.dist:check_extras', group='distutils.setup_keywords'),
                              EntryPoint(name='include_package_data', value='setuptools.dist:assert_bool', group='distutils.setup_keywords'),
                              EntryPoint(name='install_requires', value='setuptools.dist:check_requirements', group='distutils.setup_keywords'),
                              EntryPoint(name='namespace_packages', value='setuptools.dist:check_nsp', group='distutils.setup_keywords'),
                              EntryPoint(name='package_data', value='setuptools.dist:check_package_data', group='distutils.setup_keywords'),
                              EntryPoint(name='packages', value='setuptools.dist:check_packages', group='distutils.setup_keywords'),
                              EntryPoint(name='python_requires', value='setuptools.dist:check_specifier', group='distutils.setup_keywords'),
                              EntryPoint(name='setup_requires', value='setuptools.dist:check_requirements', group='distutils.setup_keywords'),
                              EntryPoint(name='test_loader', value='setuptools.dist:check_importable', group='distutils.setup_keywords'),
                              EntryPoint(name='test_runner', value='setuptools.dist:check_importable', group='distutils.setup_keywords'),
                              EntryPoint(name='test_suite', value='setuptools.dist:check_test_suite', group='distutils.setup_keywords'),
                              EntryPoint(name='tests_require', value='setuptools.dist:check_requirements', group='distutils.setup_keywords'),
                              EntryPoint(name='use_2to3', value='setuptools.dist:assert_bool', group='distutils.setup_keywords'),
                              EntryPoint(name='use_2to3_exclude_fixers', value='setuptools.dist:assert_string_list', group='distutils.setup_keywords'),
                              EntryPoint(name='use_2to3_fixers', value='setuptools.dist:assert_string_list', group='distutils.setup_keywords'),
                              EntryPoint(name='zip_safe', value='setuptools.dist:assert_bool', group='distutils.setup_keywords')),
 'egg_info.writers': (EntryPoint(name='PKG-INFO', value='setuptools.command.egg_info:write_pkg_info', group='egg_info.writers'),
                      EntryPoint(name='dependency_links.txt', value='setuptools.command.egg_info:overwrite_arg', group='egg_info.writers'),
                      EntryPoint(name='depends.txt', value='setuptools.command.egg_info:warn_depends_obsolete', group='egg_info.writers'),
                      EntryPoint(name='eager_resources.txt', value='setuptools.command.egg_info:overwrite_arg', group='egg_info.writers'),
                      EntryPoint(name='entry_points.txt', value='setuptools.command.egg_info:write_entries', group='egg_info.writers'),
                      EntryPoint(name='namespace_packages.txt', value='setuptools.command.egg_info:overwrite_arg', group='egg_info.writers'),
                      EntryPoint(name='requires.txt', value='setuptools.command.egg_info:write_requirements', group='egg_info.writers'),
                      EntryPoint(name='top_level.txt', value='setuptools.command.egg_info:write_toplevel_names', group='egg_info.writers')),
 'pygments.lexers': (EntryPoint(name='ipython', value='IPython.lib.lexers:IPythonLexer', group='pygments.lexers'),
                     EntryPoint(name='ipython3', value='IPython.lib.lexers:IPython3Lexer', group='pygments.lexers'),
                     EntryPoint(name='ipythonconsole', value='IPython.lib.lexers:IPythonConsoleLexer', group='pygments.lexers')),
 'setuptools.finalize_distribution_options': (EntryPoint(name='2to3_doctests', value='setuptools.dist:Distribution._finalize_2to3_doctests', group='setuptools.finalize_distribution_options'),
                                              EntryPoint(name='features', value='setuptools.dist:Distribution._finalize_feature_opts', group='setuptools.finalize_distribution_options'),
                                              EntryPoint(name='keywords', value='setuptools.dist:Distribution._finalize_setup_keywords', group='setuptools.finalize_distribution_options'),
                                              EntryPoint(name='parent_finalize', value='setuptools.dist:_Distribution.finalize_options', group='setuptools.finalize_distribution_options')),
 'setuptools.installation': (EntryPoint(name='eggsecutable', value='setuptools.command.easy_install:bootstrap', group='setuptools.installation'),)}