CODEWITHSUNDEEP
modulepackagejulia
user2138149
user2138149

Reputation: 17170

What relation, if any, do files and directory paths have in relation to modules and packages?

I have recently started learning Julia. I am trying to figure out how modules and packages are intended to be used.

At the moment, I'm not sure if directory structure and files have any relation to these concepts.

So:

I am thinking about Julia packages and modules from the context of Python.

In Python, a package is a directory. Now that we have anonymous packages, the empty __init__.py file is no longer required. This file used to be required to mark a directory as being a Python package, now it is optional.

In Python, each file is itself a module. The name of the module is the same as the name of the file, with the .py extension removed. If files are organized in a hierarchical directory structure, then the subdirectory names themselves become sub-packages.

There is not much difference between a package, a sub-package and a module. They all define namespaces. Of course, being files, modules can contain actual code.

In Python there is no explicit way to declare a module or namespace. They are implied by the file and directory names and structure.

For this reason, to import Python code, a single statement like import X is required. In Julia, code first has to be include'd before it can be imported with import or using. In Python, the include'ing is automatic because module names are defined by filenames. So there is a 1:1 correspondence.

In Rust, if my memory serves me correctly, directory structure is also important for defining package and module structure. I don't remember the complete, precise details, but I think it's pretty similar to Python.

On the other hand, Julia is more like C or C++.

So - in the case of Julia, do any of the following concepts have any influence on how packages and modules work?

I hope the question is clear - I know what I'm asking is a bit abstract. You don't tend to think too much about why packages and modules behave a certain way in a particular language, you usually just figure out what works and continue to use it.

Upvotes: 0

Views: 59

Answers (1)

JuRien
JuRien

Reputation: 21

I'll try to answer with what I've learned while contributing to a Julia package and creating one from scratch for personal use.

In Python REPL, import searches python packages in PATH environment variable (accessible with sys.path in Python). Julia REPL's equivalents (using and import) does similarly, but have a different search pattern (accessing folders in LOAD_PATH and DEPOT_PATH, in Julia). Otherwise, both REPL are unfazed by the folder structure, everything starts from the current working directory (e.g. with Julia's include() or Python execfile()). In other words, you can run code from everywhere from a REPL, as long as you have the path. When a package/module is installed, you should have access to it through the language's base package manager.

Where Python considers a directory as a module, the analog in Julia would be how the Pkg package handles packages (that's a fun thing to say), which is also in a directory, where the structure is rather rigid. Since Pkg is the main package manager in Julia, we can expect it to be the authority on how to structure packages.

Let's dive a bit in a Julia package structure. A package begins with a Project.toml file in a directory <1>, where the package name will be held, say SomeName. A Project.toml file looks like this:

name = "SomeName"
uuid = "a5927f71-4e94-4484-9e8d-767c99d7f0c4"
authors = ["yourname <[email protected]>"]
version = "0.1.0"

[deps]
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"

The Project.toml file, along with the manifest, is central to how packages are handled, as it specifies package name, dependencies (under [deps] above), versioning, etc. The package name in Project.toml must correspond to the name of a julia file in the src directory on the same level as Project.toml <2>, namely src/SomeName.jl. That file must contain the module name in its definition, such as:

module SomeName
...
end

While developing in Julia, the name of the base directory of the package you're working on is not critical (but should match the name of the package if you wish to publish). After a package is installed (e.g. in ~\.julia\packages\) the name of the package's directory must fit with the name of the package in Project.toml and the main module file name (found in src\). Otherwise using won't find it. I tried to mess with the following and they all broke using:

  • changing installed package directory name
  • changing package name in Project.toml
  • changing the main module file name (e.g. src/PkgName.jl)

I could probably try to change the module name in the .jl module file, etc. My bet is that it's going to break the using also. (I'll leave as an exercise to the reader ;) )

In summary and in answer to:

So - in the case of Julia, do any of the following concepts have any influence on how packages and modules work?

  • directory names
  • file names
  • the hierarchical structure of a filesystem tree

In a package:

  • Directory names: Base directory name of installed packages must match package name. During development, it changes nothing.
  • File names: Some file names are important. In a package, Project.toml and src/PkgName.jl must exist, for example.
  • hierarchical structure: Folder src is necessary in a package directory.

Otherwise, in the REPL: adding packages, activating environments and including files is relative to working directory.

To illustrate all this, we can make a minimal working example to poke around how Pkg handles loading a local package.

Let's choose a working directory and start julia. I assume linux, but then it should be easy to do the same on Windows, etc.

cd ~/a/path/of/choice
julia

We generate a base template according to Pkg rules:

julia> ] generate MyTestPkg

This will create a folder MyTestPkg and two files, MyTestPkg/Project.toml and MyTestPkg/src/MyTestPkg.jl.

To load the package, use:

julia> ] activate MyTestPkg

Note here that if you were in folder PARENT and you move one level up (cd ..), you would have to activate with:

julia> ] activate PARENT/MyTestPkg

So what you are passing to activate is a relative folder path.

Once activated, you can use what's in the package with lines

julia> using MyTestPkg
julia> MyTestPkg.greet() # or any function you added in the module

You can play with where the Project.toml and MyTestPkg.jl files are. I tried to rename the folder MyTestPkg. As long as I specify the folder name, it still found the package I want. If you move the MyTestPkg.jl one level up (out of src\) or down (into a subfolder in src\), Pkg fails to instantiate the package, and will fail when calling using MyTestPkg.

<1> https://pkgdocs.julialang.org/v1/toml-files/

<2> https://pkgdocs.julialang.org/v1/creating-packages/#Code-structure

Other good reads:

Upvotes: 2

Related Questions