Is there an 'official' CPAN specification document?

Question

Is there a authoritative and/or de-facto document detail how CPAN works, in enough detail that anyone could use it to rebuild CPAN from scratch without relying on oral-tradition and hearsay? Something documenting the directory structure, files that are supposed to be in a distribution, how version numbers really work, who can upload what and what exactly gets listed on the search.CPAN.org website, etc?

Answer 1

Your question touches on a lot of different things...

"A" CPAN is just a directory with a certain structure and a few special files. The structure and file formats are not formally defined, but it is fairly simple. There are several modules (mostly in the CPAN:: namespace) that can create those files for you. The most important file is the modules/02packages.details.txt.gz file (a.k.a. "the index"), which maps module names to the paths of the distribution archives that contain them. The paths in the index are assumed to be relative to the authors/id directory. And that's pretty much it. Other files like authors/01mailrc.txt.gz and modules/03modlist.data.gz must be present and have a "valid" format but don't require any real data in them -- they are not essential to the basic operation of module installers.

"The" CPAN is a central CPAN where the community shares their distributions. Authors can put their work into "the" CPAN via PAUSE, which does the work of analyzing the distribution and generating the index. The About page describes how PAUSE works at a high level. "The" CPAN is actually mirrored to several different locations, so you can point your installer at any of them. You can easily create your own mirror too. search.cpan.org and metacpan are searching and browsing services that are built on top of "the" CPAN. So if you upload a distribution to PAUSE, it will (eventually) show up on those sites too.

How you actually make a distribution is a bit more complicated though. Again, there isn't a formal definition of what a distribution should look like. PAUSE will actually accept just about anything and will try very hard to make sense of it. But nearly all authors use a tool like Module::Build or ExtUtils::MakeMaker to do most of the work. These tools will prescribe the conventions for organizing your code and can generate the files that help PAUSE and other systems understand your distribution. At an even higher level, tools like Module::Starter and Dist::Zilla will generate even more infrastructure for you. You don't have to use any of them, but you should use at least one of them.

If your goal is to construct your own CPAN with the distributions of your choosing (perhaps your own proprietary modules plus some open source modules from "the" CPAN) then Pinto might be what you're looking for. Pinto will construct a local CPAN-like repository with whatever distributions you want. Pinto knows how to work with dependencies and fetch them from an upstream repository (usually "the" CPAN). Pinto has tools to help you manage change as your dependencies evolve. And it is all compatible with the existing Perl module installers, so all you have to do is "point" them at your local repository.

Answer 2

There are various projects, such as MyCPAN, Pinto, and StratoPAN that create CPAN-like repositories with the index files that PAUSE provides and that the clients expect. What are you trying to accomplish?

Answer 3

CPAN is nothing more than a mirrored file repository. Files are uploaded using PAUSE. About PAUSE answers most if not all of your questions.

You also mentioned search.cpan.org, but that's not part of CPAN. It's just one of many tools that accesses CPAN. Web site https://metacpan.org/ and command line tools cpan and cpanm are others.