CODEWITHSUNDEEP
pythonbuildoutpypitipfy
Sudhir Jonathan
Sudhir Jonathan

Reputation: 17526

Using custom packages on my python project

I'm doing a few projects in python right now, and I'm trying to figure out how to work with my own versions of existing open source packages.

For instance, I'm using tipfy with zc.buildout, and I've added in the 'paypal' package. Unfortunately it doesn't have a feature I need, so I've forked it on github and added the feature. I will send the original package maintainers a pull request, but whether they accept my additions or not, I'd like to use my version of the package and keep the convenience of having zc.buildout manage my dependencies. How do I do this?

Do I upload my own take on the library to PyPI and prefix it with my name? Wouldn't that unnecessarily pollute the index?

Or should I make and maintain my own index and package repo? Where do I find the format for this? And is it against to terms of the OSS licenses to host my own repo with modified packages with the same names? (I'd rather not modify every file in the project with new namespaces)

I'm sure this problem comes up quite a lot, and not just with python. I can see this happening with Maven and SBT, too... what do people usually do when they want to use their own versions of popular packages?

Upvotes: 6

Views: 1585

Answers (6)

Petri
Petri

Reputation: 5006

It makes perfect sense to have your own custom PyPI server for your proprietary, private, forked or development packages.

For example pypiserver is lightweight and easy to set up.

Upvotes: 0

S.Lott
S.Lott

Reputation: 392010

Do I upload my own take on the library to PyPI and prefix it with my name?

No.

Wouldn't that unnecessarily pollute the index?

Obviously. (Also, you're assuming that your extension is actually useful to others. It might not be all that useful to anyone but you. Indeed, your extension might indicate a failure to understand the package.)

Or should I make and maintain my own index and package repo?

Never. That's completely silly. You have the modified package. You don't need a repository unless you're going to complete against PyPI. You could try, but why bother?

Here's what you do.

extend

You can easily extend a given package without modifying or forking.

my_paypal.py

from paypal import *

class MyFancyExtension( SomeExistingClass ):
    def override( self, ... )

def my_other_extension( ... ):
    this()
    that()

It's quite easy. It's often better than forking because you preserve the original package untouched by your extensions.

Upvotes: 0

Jason Baker
Jason Baker

Reputation: 198797

It's been a while since I've used buildout, but if I recall correctly, there is a pip recipe that allows you to use pip requirements files. You can create a requirements file that contains something like this:

-e git+http://<github url>

That will check the package out locally when installing.

Upvotes: 2

Martijn Pieters
Martijn Pieters

Reputation: 1124548

There are two ways you could go about this. I use both, depending on the context the buildout is being used for:

  1. Use mr.developer to include packages from a version control system (mr.developer support a wide range of systems, including git). I use this when developing.

  2. Build a private package repository. A simple directory listing in Apache is enough. Add the URL to your private repository as a find-links entry:

    [buildout]
...
find-links =
    ...
    http://username:[email protected]/projectname

    In this example I also included a username and password (buildout then will authenticate) and a path on the server that is project specific. You can also just build one big private repository for all your projects, of course.

    In that repository you either put complete eggs, or just a tarball of the package. Repositories named in find-links are searched before PyPI.

    I use this method for deployment buildouts. This way the software will use released packages, which makes release management much clearer and simpler.

Hosting your own forks of OSS packages is perfectly fine! This is one of the freedoms you get when using OSS. Note that when you fork GPL code like this, and are distributing it to a third party, you need to make your changes available to them. A package repository is one way of doing complying with this.

Upvotes: 6

Eli Bendersky
Eli Bendersky

Reputation: 273756

To keep your headache in check, I would really recommend just bundling all such custom code with your package. Say you made a fork of some package. As long as its license allows it, I would just bundle the modified version of package with my code, as if it's just another directory. You can place it locally under package so it will be easily found. Once the developers of package fix what you need, just remove this directory and make it a dependency on an online package once again.

An added bonus of this approach is making distribution to users/customers easier.

Upvotes: 2

Dan
Dan

Reputation: 23707

Any paths you put in dependency_links in setup.py of your project will be searched first. So just drop your package in a path and set dependency_links to that path.

local_packages = ['file:///path/to/packages']

setup(
 ...
 dependency_links=local_packages,
...)

Setuptool docs cover a little bit more on how it all works, but that's really all there is to it.

Edit: zc.buildout uses the setuptools dependency_links var by default. If you want to turn it off.

Upvotes: 0

Related Questions