Use custom Python package

Question

I am fairly new to Python and I don't really know of how to use a custom package or its development life-cycle.

I've been developing a Python library meant to be used from many scripts located on a specified folder in a production environment.

Currently the library and the scripts that use it are heavily coupled from a development life-cycle perspective as it development is still in an early phase.

Question is, how can I test the library works fine when used from the scripts?

So far, in order to be able to import the custom library from the scripts, I've considered 3 different alternatives:

1. As an installed package with pip or so

2. Somehow downloading the package to a directory and alter the pythonpath

3. As a directory within the scripts folder

From my experience with other languages, option 1 would be the best, since it would probably be easier to handle package releases and everything would be in the 'standard' way, however in my case can be somehow difficult as the package source is hosted in a private Gitlab server isolated from the production environment.

Options 2 and 3 seem to me like workarounds and not much like the 'good way' of using a custom library.

Are there any other alternatives in Python to effectively use a package?

Answer 1

Starting with how to use a custom package

you have to define what is the purpose of that package. I mean, it is something that will be useful for everyone or only for you. If the first case, then the best way is publishing it on PyPI (In that case follow all the steps). If it is the second purpose, then omit the step 5.

Then, whatever your package purpose is, the first steps are:

1. Develop the package (try to document and test it). PyCharm it is a good IDE that can help to create unit test. I recommend you to take a look at the following link: Creating-and-running-Python_Test

   A Project Test folder structure that I have been working on

2. The second step is to create a setup.py file. There are a lot of fields a setup.py file can contain. Here you have an example:

     from setuptools import setup
   
   
     setup(
     name='mdutils',
     version=1.0,
     license='BSD',
     author='Didac Coll',
     author_email='didaccoll_93@hotmail.com',
     maintainer='Dídac Coll',
     maintainer_email='didaccoll_93@hotmail.com',
     description='A package, useful to create Markdown files while executing python code.',
     long_description=open('README.md').read(),
     platforms=['Python 3.6'],
     packages=['mdutils', 'mdutils.tools', 'mdutils.fileutils'],
     include_package_data=True,
     zip_safe=False,
     classifiers=['Development Status :: 4 - Beta',
                  'Operating System :: OS Independent',
                  'Programming Language :: Python',
                  'Programming Language :: Python :: 3.6',
              'Topic :: Utilities',
              'License :: OSI Approved :: BSD License'])
   

3. Once you have created your package, you have to test the distribution with the following command python setup.py develop

4. If all is ok, now you can create your distribution with the following command python setup.py sdist. This will generate a new folder called dist where you will find a tar.gz file. If you are interested in creating a distribution for windows I recommend you to use the next command python setup.py sdist --format=zip. In this manner, you are going to create a zip file to dist folder.

   Now you have your own package distribution in a compressed file. So if you want to install it to a project, only needs to use pip: pip install mdutils- 1.0.zip. You can also upgrade it using the following python command: python install --upgrade mdutils-2.0.zip.

   Executing the previous commands will install your package to your site-package folder.

   Here, you will find a lot of information about packaging and distributing: https://packaging.python.org/tutorials/distributing-packages/

Here, there is a good tutorial, too: https://python-packaging.readthedocs.io/en/latest/

5. Finally, if your project works properly you can publish it on PyPi. You have to create an account. Once, you have it, use the following command to register your package: python setup.py register. I strongly recommend you to test it on TestPyPI before trying to publish on PyPI. Create another account here: https://test.pypi.org/ and type this command on your PyCharm terminal: twine upload --repository-url https://test.pypi.org/legacy/ dist/* (More information about TestPyPI here).

Answer 2

By far the best way to distribute a package is via pip. The function is easy to use and there are no compiler issues for the end user has already been taken care of when distributing. pip is particularly useful if your package contains C extensions, since compiling on OSes (Windows in particular) can be a nightmare.

Another way (similar to 2) is to install from the source, i.e. a setup.py file and not add the pythonpath, just install it as you would with pip:

You would need a setuptools setup script. Once you are ready to build run:

python setup.py build

OR python ./setup.py build

if you run in PowerShell.

This will produced a compressed file containing the source (compiled in the case of extensions) and you could distribute that.

The end user just has to run:

python setup.py install 

OR python ./setup.py install on PowerShell to install the package.