Organize local code in packages using Go modules

Question

I can not find a way to factor out some code from main.go into a local package when using Go modules (go version >= 1.11) outside of $GOPATH.

I am not importing any external dependencies that need to be included into go.mod, I am just trying to organize locally the source code of this Go module.

The file main.go:

package main

// this import does not work
import "./stuff"

func main() {
    stuff.PrintBaz()
}

The file ./stuff/bar.go (pretending to be a local package):

package stuff

import "log"

type Bar struct {
    Baz int
}

func PrintBaz() {
    baz := Bar{42}
    log.Printf("Bar struct: %v", baz)
}

The file go.mod (command go mod init foo):

module foo

go 1.12

When executing go run main.go:

• If I import "./stuff", then I see build command-line-arguments: cannot find module for path _/home/<PATH_TO>/fooprj/stuff.
• If I import "stuff", then I see build command-line-arguments: cannot load stuff: cannot find module providing package stuff.
• If I import stuff "./stuff" with a package alias, then I see again: build command-line-arguments: cannot find module for path _/home/<PATH_TO>/fooprj/stuff.

I can not find a way to make local packages work with go modules.

• What's wrong with the code above?
• How can I import local packages into other Go code inside a project defined with Go modules (file go.mod)?

Answer 1

First you have to choose a name for your project and write it to go.mod file. This name belongs to root directory of the project. Each new package you create must be located inside its own subdirectory and its name should match directory name.

go.mod:

module myprojectname

or (preferred way, see @thepudd's answer for details)

module github.com/myname/myprojectname

Then import your project's packages like:

import myprojectname/stuff

or

import github.com/myname/myprojectname/stuff

Files of package stuff should be located inside project's stuff directory. You name these files as you like.

Also it's possible to create deeper project structure. For instance, you decided to separate source code files from other ones (like app configs, docker files, static files, etc...). Let's move stuff directory inside pkg, every go file inside pkg/stuff still have stuff package name. To import stuff package just write:

import myprojectname/pkg/stuff

Nothing stops you from creating more levels in the hierarchy like github.com/myuser/myprojectname/pkg/db/provider/postgresql, where:

• github.com/myuser/myprojectname - project name.
• postgresql - package name.
• pkg/db/provider/postgresql - path to the package relative to project's root.

You can read more about go modules here: https://github.com/golang/go/wiki/Modules

Check out this repository to get useful information about various patterns used in project organizing: https://github.com/golang-standards/project-layout If you go inside pkg directory you will find out which open source projects use pkg directory in their structure.

Answer 2

Module structure

The most common and easiest approach is:

• Use a single go.mod per repository, and
• Place the single go.mod file in the repository root, and
• Use the repository name as the module path declared in the module line in the go.mod
  • (If you are using a custom import path such as me.io/mymod rather than using a VCS host based import path, then you would use the custom import path instead of the repository name in your go.mod).

For example, if your repo is github.com/my/repo, then you would place a single go.mod in the repo root, with the first line reading module github.com/my/repo. That can be created by cd'ing to the repo root and running go mod init github.com/my/repo. When working with the repo locally, the repo directory can be located wherever is convenient on your filesystem. (If you do not have a repo, see below).

Following this helps you stay on the happy path with modules, and it avoids multiple subtleties.

Russ Cox commented in #26664:

For all but power users, you probably want to adopt the usual convention that one repo = one module. It's important for long-term evolution of code storage options that a repo can contain multiple modules, but it's almost certainly not something you want to do by default.

There is much more about multi-module repositories in the "Multi-module Repositories" FAQ section on the modules wiki. Those 6 or so FAQs in that section should be read in their entirety by anyone considering veering off the recommendation above.

Arranging packages within a module

Once you have set up your go.mod, you can arrange your packages in directories however you see fit in directories underneath the directory containing the go.mod, as well as in the directory with the go.mod. Three good articles about how to arrange your code in packages:

• https://rakyll.org/style-packages/
• https://medium.com/@benbjohnson/standard-package-layout-7cdbc8391fc1#.ds38va3pp
• https://www.goinggo.net/2017/02/design-philosophy-on-packaging.html

Those articles are classics that pre-date the introduction of modules, but the philosophies in them still apply to how to arrange your packages within a module.

Importing other packages in the same module

When importing another package with modules, you always use the full path including the module path. This is true even when importing another package in the same module. For example, if a module declared its identity in its go.mod as module github.com/my/repo, and you had this organization:

repo/
├── go.mod      <<<<< Note go.mod is located in repo root
├── pkg1
│   └── pkg1.go
└── pkg2
    └── pkg2.go

Then pkg1 would import its peer package as import "github.com/my/repo/pkg2". Note that you cannot use relative import paths like import "../pkg2" or import "./subpkg". (This is part of what OP hit above with import "./stuff").

Modules vs. repositories vs. packages vs. import paths

A Go module is a collection of related Go packages that are versioned together as a single unit. Modules record precise dependency requirements and create reproducible builds.

Summarizing the relationship between repositories, modules, and packages:

• A repository contains one or more Go modules (most often exactly one module in the repository root).
• Each module contains one or more Go packages.
• Each package consists of one or more Go source files that all reside in a single directory.
• Each Go source code file:
  • declares its own package with a package foo statement, which must be consistent for all Go files in a single package directory.
  • automatically has access to other Go source code in the same package, without explicitly importing its own package.
  • imports code from another package via an import path supplied in an import statement such as import "github.com/my/repo/pkg1". The import path always starts with the module path of that package, regardless of whether that package is in the same module or a different module.

A single module can be used to build multiple executable binaries, but the func main() {...} entry point for each binary must be in its own package main, with each package main in a distinct directory.

Local-only modules

You can create, build and run modules without ever publishing them or placing them on a VCS server.

A module is defined by a tree of Go source files with a go.mod file in the tree's root directory, so you can pick a directory on your local filesystem to serve as the root of your module and place your go.mod there. Everything described above still applies, except there does not need to be a repo.

For example:

some-directory/
├── go.mod     <<<< go.mod at module root declaring 'module example.com/foo'
├── pkg1
│   └── pkg1.go
└── pkg2
    └── pkg2.go

If the module path in the first line of that go.mod reads module example.com/foo, then pkg1 would import its peer package as import "example.com/foo/pkg2".

Of course, you can start with a local-only module without a repo, then later start tracking it in a local repo or publish it to a VCS server.

Even for local-only modules, it is a good idea to give the module a globally unique name, such as go mod init my.org/foo, or go mod init github.com/some/repo as if it will be published someday, even if that repo does not yet exist. (If you are in a hurry, it is possible to use a shorter module name without a dot or leading hostname component, such as go mod init temp or go mod init m, but it won’t be fetchable by the go command, and you run the risk of colliding with a current or future standard library package or otherwise seeing confusing error messages later).

If you have multiple local modules, you can point them at each other via replace directives or via the Go 1.18+ workspace feature, though those are more complex workflows than placing everything for a given project in a single module.

Next steps

If you are new to Go modules, it is worthwhile to go through the official "Create a Go Module" tutorial from the Go project. This will likely save you time overall.

If you are stuck right now with an error message related to modules or import paths, there's a very good chance the error will make much more sense after completing the tutorial.