Building a python package to publish in pypi

Question

I am greatly confused with the process of building a python package that I want to distribute on pypi.

There are some specific, basic things that I did not understand:

• What exactly is that gets published? Binaries? Source code? How do I do one or the other?
• How do I build multiple platform-specific, os-specific build from the same codebase?
• How do I build a the package for multiple versions of Python from the same codebase? Is it necessary if I want to support many python versions?

I am using a .toml file for the setup configuration. I found some answers only, but all refer to procedures with either a setup.py or a setup.cfg.

Answer 1

What exactly is that gets published? Binaries? Source code?

Yes, and yes. It depends on details of your project and your package config. Arbitrary commands can be run during a package build. You might, for example, run a fortran compiler locally and ship binaries, or you might insist that each person installing the package run their own local fortran compiler.

We usually expect full *.py source code will appear on pypi.org. "Binaries" here will usually describe compiled machine code, and not *.pyc bytecode files.

How do I build multiple platform-specific, os-specific build from the same codebase?

I have only done this via git pull on a target platform, followed by a local build, but there is certainly support for cross target toolchains if you need that.

How do I build a the package for multiple versions of Python from the same codebase?

Same as above -- do a build under each separate target version.

Is it necessary if I want to support many python versions?

Typically the answer is "no". Pick a minimum required interpreter version, such as 3.7, and do all your development / testing / release work in that.

Backward compatibility of interpreters is excellent. Folks running 3.8 or 3.11 should have no trouble with your package.

There can be a fly in the ointment. Suppose your project depends on library X, or depends on X which depends on Y. And one of them stopped being updated a few years ago, or went through a big change like a rename. Your users who are on 3.11 might find it inconvenient to obtain a compatible version of X or Y. This might motivate you to do split releases, for example via major version number or by slightly altering your project name. Clearly you haven't crossed that bridge quite yet.

---

The poetry ecosystem is reasonably mature. It has tried to fix many of the rough edges surrounding the python packaging practices of the last few decades. I recommend that you prefer modern over ancient practices, and that you adopt poetry for your project.

If that won't fly for some reason, and especially if binaries are a big deal for your project, consider publishing via conda. There are many pip pitfalls with target system needing compilers and libraries. Conda does an excellent job of ensuring that conda install ... will Just Work.