CODEWITHSUNDEEP
pythonartifactory
Xabs
Xabs

Reputation: 4664

Artifactory PyPi repo layout with build promotion

Q1: I have an Artifactory PyPi enabled repo my-pypi-repo where I can publish my packages. When uploading via python setup.py -sdist, I get a structure like this:

my-pypi-repo|
            |my_package|
                       |x.y.z|
                             |my_package-x.y.z.tar.gz

The problem is this structure will not match any "allowed" repo layout in Artifactory, since [org] or [orgPath] are mandatory:

Pattern '[module]/[baseRev]/[module]-[baseRev].[ext]' must at-least contain the tokens 'module', 'baseRev' and 'org' or 'orgPath'.

I managed to publish to a path by 'hacking' the package name to myorg/my_package, but then pip cannot find it, so it's pretty useles.

Q2: Has anyone tried the "ci-repo" and "releases-repo" with promotion for Python using Artifactory?

What I would like to achieve:

CI repo: my_package-1.2.3+build90.tar.gz When this artifact gets promoted build metadata gets dropped

Releases repo: my_package-1.2.3.tar.gz

I can achieve this via repo layouts (providing I resolve Q1). The problem is how to deal with the "embedded" version inside my Python script, hardcoded in setup.py.

I'd rather not rebuild the package again, for best practices.

Upvotes: 3

Views: 3090

Answers (2)

drohm
drohm

Reputation: 224

I am running into the same issue in regards to your first question/problem. When configuring my system to publish to artifactory using pip, it uses the format you described.

As you mentioned, the [org] or [orgPath] is mandatory and this basically breaks all the REST API functionality, like searching for latest version, etc. I'm currently using this as my Artifact Path Pattern:

[org]/[module]/[baseRev].([fileItegRev])/[module]-[baseRev].([fileItegRev]).[ext]

The problem is that pip doesn't understand the concept of [org] in this case. I'm temporarily using a python script to publish my packages to Artifactory to get around this. Hopefully this is something that can be addressed by the jFrog team.

The python script simply uses Artifactory's REST API to publish to my local pypi repository, tacking on a few properties so that some of the REST API functions work properly, like Artifact Latest Version Search Based on Properties.

I need to be able to use that call because we're using Chef in-house and we use that method to get the latest version. The pypi.version property that gets added when publishing via python setup.py sdist upload -r local doesn't work with the REST API so I have to manually add the version property. Painful to be honest since we can't add properties when using the upload option with setup.py. Ideally I'd like to be able to do everything using pip, but at the moment this isn't possible.

I'm using the requests package and the upload method in the Artifactory documentation here. Here is the function I'm using to publish adding a few properties (feel free to add more if you need):

def _publish_artifact(name, version, path, summary):
    base_url = 'http://server:8081/artifactory/{0}'.format(PYPI_REPOSITORY)
    properties = ';version={0};pypi.name={1};pypi.version={0};pypi.summary={2}'\
             .format(version, name, summary)
    url_path = '/Company/{0}/{1}/{0}-{1}.zip'.format(name, version)
    url = '{0}{1}{2}'.format(base_url, properties, url_path)

    dist_file = r'{0}\dist\{1}-{2}.zip'.format(path, name, version)
    files = {'upload_file': open(dist_file, 'rb')}
    s = requests.Session()
    s.auth = ('username', 'password')
    reply = s.put(url, files=files)
    logger.info('HTTP reply: {0}'.format(reply))

Upvotes: 3

JBaruch
JBaruch

Reputation: 22893

A1: Artifactory layouts aren't enforcive, you can deploy any file under any path to any repo. Some layout-related features, like snapshots cleanup won't work then, but I don't think you need them anyway.

A2: The best solution will be to code your promotion in a promotion user plugin. Renaming artifacts on the fly during their promotion to another repo is one of the most popular scenarios of this kind of plugin.

Upvotes: 1

Related Questions