Spoiler Alert: There is no such thing. The optimal Go project layout depends on your exact use case.
Design the architecture, name the components, document the details.
This quote is one of the Go proverbs.
Rob Pike presented the Go proverbs at a talk at Gopherfest 2015. They are available as a list since then, or recently also as (AI-generated) Limericks if you prefer.
While Rob Pike focuses on the naming aspect of this proverb, I want to talk about architecture, or rather, one particular part of it:
How to lay out a Go project.
This topic is frequently discussed among Gophers, mainly for one reason:
There is no such thing as a standard Go project layout.
If you think about it, there cannot be a single way of laying out a Go project because different project types have different needs.
Whenever I come across an article or a repo that promotes a particular Go project layout as a “standard,” I think: “framework.”
Frameworks are unpopular in Go because they impose a rigid structure on a project that might or might not match the project's specific needs.
The same applies to the files-and-directories layout of a project. No single layout fits all purposes.
So how to lay out a Go project then?
By following best practices.
Best practices are a strong guidance without being dogmatic. Apply common sense when using them.
If you start a new project and don't know how large it will become over time, use the simplest layout possible, and add structure only if required.
Avoid grab-bag package names like
helpers, or similar. (This is the “name your components” part of the proverb.) Rather, group your code by functionality or responsibility.
While you are free to choose arbitrary directory names for your project layout, you'll want to know and memorize these special directory names:
internal/: Packages inside the
internaldirectory are not accessible outside the Go module it belongs to. Even the Go toolchain follows this convention. If you expose a package to the public, it becomes a public API, so you have to ensure it follows semantic versioning. And you will receive issues and pull requests from users of this package, even if the package was only meant for use inside your project. Use the
internaldirectory to avoid this situation.
testdata/: Put ancillary test data inside this directory. The Go toolchain ignores anything inside. (Side node: in addition to
testdata, the Go toolchain also ignores any directory starting with
vendor/: While the Go proxy server does already a good job of helping to ensure consistent builds, you can go one step further and download all 3rd-party dependencies into a
go mod vendorexpects this directory name.
Do not use a
src directory. It simply makes no sense because a Go project is plain source code.
Moreover, every extra directory makes your import paths longer. Strive to keep the import paths short by omitting directories that do not add value.
Best practices and conventions alone are not sufficient to define how useful project layouts look like. So let me list a few examples.
Command-line tools do not need much structure. The simplest ones require only a single directory. This directory is the root of the repository.
Let's assume you want to write a data compression tool. You name the root directory
compress because Go picks the main package's directory name as the name of the binary by default. All your
.go files reside in the
compress directory, no subdirectories required.
compress/ +-- main.go go.mod go.sum
Assuming the project repository has been published to
github.com/your/compress, users can install your tool directly through
go install github.com/your/[email protected]
(although I always recommend pre-packaging public binaries for package managers. This is easier than you might think.)
Chances are that your tool includes one or more library packages for its own use. Add them to the
internal directory, for the reasons stated above.
compress/ +-- main.go go.mod go.sum internal/ +-- deflate/ +-- deflate.go
I'll come to public library packages later.
A library project with no auxiliary CLI tools can start with a single directory, just like the simple CLI tool project.
A library for compressing data, for example, would look like so:
compress.go go.mod go.sum
Additional (public) packages, such as encoding and decoding packages, get separate folders at the root level.
compress/ +-- compress.go go.mod go.sum encode/ +-- encode.go decode/ +-- decode.go
The import paths are straightforward:
import "github.com/your/compress" import "github.com/your/compress/encode" import "github.com/your/compress/decode"
The next two project types share the same resulting layout.
The convention for both cases is to have the library code at the project's root and put code for command-line tools in a
So if we combine our
compress tool and
compress library into one, that's what we get:
compress/ +-- compress.go go.mod go.sum cmd/ +-- compress/ +-- main.go
For library users, the import path remains as short as in the pure library example.
Tool users would install the tool as
go install github.com/your/compress/cmd/[email protected]
...which is a bit longer than before, but that's not a deal-breaker.
If we throw all of the above together—a library with sub-packages, internal packages, and command-line tools, we do not need to change any of the previous approaches. They fit nicely together.
compress/ +-- compress.go go.mod go.sum encode/ +-- encode.go decode/ +-- decode.go internal/ +-- deflate/ +-- deflate.go cmd/ +-- compress/ +-- main.go
Note that even though the internal
deflate package belongs to the command, placing the
internal directory at the root is still better because the resulting import path is shorter.
Move forward to large application projects that add non-code artifacts, other languages, documentation, DB migration files, build and deploy scripts, or Website assets. To avoid re-inventing the wheel every time you start a new project, you'll want to have consistent places for these items.
This is where project templates like the self-proclaimed “Standard Go Project Layout” come into play. “Self-proclaimed” because this template is anything but an official standard layout. Still, it is a good showcase of how a full-featured project layout can look like.
Imagine that our
compress project has grown into a full-blown Web app. If we follow the Standard Go Project Layout, our project might look like this:
compress/ +-- go.mod go.sum README Makefile api/ assets/ build/ cmd/ ... pkg/ vendor/ web/ website/
Despite the size of such a project, the layout remains clear at the top level because everything is tucked away under a specific directory.
Even the Go library packages.
They live inside a directory named
pkg. This layout change is nontrivial because it is incompatible with the above project layouts.
pkgwill contain a
The Standard Go Project Layout's README says about the
This is a common layout pattern, but it's not universally accepted and some in the Go community don't recommend it.
It's ok not to use it if your app project is really small and where an extra level of nesting doesn't add much value (unless you really want to :-)). Think about it when it's getting big enough and your root directory gets pretty busy (especially if you have a lot of non-Go app components).
“But Christoph, doesn't the Go community frown upon the use of a
pkg directory?” Yes and no. What I see is that one part of the Go community is happy with using
pkg, and another part is happy without.
Both sides have good reasons.
Among the various arguments in favor of using a
pkg directory, three major benefits stick out.
pkgis a clear sign that these packages are meant for public use. A dedicated directory for public packages is a clear sign for the project users: “Here you can find our public packages.”
package webinside a
webdirectory at the root level. Then you want to add a directory for Website assets. The name
webwould be the well-known standard name for this, but alas, it's already taken! A
pkgdirectory prevents this from happening.
Proponents of using
Peter Bourgon: Go best practices, six years in. The Industrial Programmer suggests using
pkg as a minimal layout. "All of your artifacts remain go gettable. The paths may be slightly longer, but the nomenclature is familiar to other Go developers. And you have space and isolation for non-Go assets." However, Peter clarifies that while this layout may be a good fit for many types of projects, there is no single best repo structure.
Travis Jeffery: I'll take pkg over internal. Travis criticizes the
internal directory because it diminishes the use of public projects. I disagree. Any package outside
internal must meet the elevated standards of a public package API. Hence, a project owner is entitled to declare a package for internal use only. But he has a couple of very reasonable arguments towards using
pkg. For example: "...the directory is useful boilerplate that clarifies the project’s layout for people. Useful boilerplate for clarity sounds like a Go tagline."
Kyle C. Quest: Go Project Layout. Kyle observes that the
pkg pattern is "pretty popular" but admits that it can confuse Go newcomers because of the
pkg directory inside
GOPATH that has nothing but the name in common with the
pkg directory in repository layouts.
Not using a
pkg directory has also several benefits.
pkgdirectory is only a “pass-through” step to package subdirectories. It contains no packages directly, hence a
/pkg/element inside an import path is a pointless no-op that provides no useful grouping.
internaldirectory is public anyway. A
pkgdirectory does not prevent the users of a project from looking for public libraries elsewhere. If you want to avoid maintaining a package for public use (which is always more effort than maintaining an internal-only package), then you should better put it inside
pkgdirectory as a standard might make Go newcomers believe that even the smallest projects require a
pkgdirectory, where it would, in fact, add no value at all.
internal), these packages should be moved to a separate library project.
Proponents of a life without
Eli Bendersky: Simple Go project layout with modules. Eli argues that
pkg is useful only in large application projects with lots of non-Go stuff around, but such application projects most certainly contain packages for their own use only, and hence the packages should better live in
internal. (Similar to #2 above.) And pure library projects do not have the problem of isolating Go code from a truckload of non-Go assets.
Xe Iaso: The Within Go Repo Layout. Xe points out possible confusion with
$GOPATH/pkg. Since Go Modules, however, the
GOPATH directory has become much less visible, and I think the likeliness of confusing newcomers has diminished drastically. Xe also points out that not imposing a
pkg directory leaves the development team more degrees of freedom in naming things. And if an application project also exposes packages, then those packages might as well go into a separate, pure library project. (#4 above)
My conclusion from all the aspects I laid out above is this:
Projects that contain only library packages do not require any isolation from non-Go assets. Do not use
pkg and let your users enjoy shorter, more readable import paths. Even if the library happens to require a few non-Go assets, like a Web UI library that uses HTML and CSS files, there is usually no point in adding a
pkg directory. After all, the Go packages are first-class citizens in a Go library project.
Even when tools and libraries live side-by-side, adding
pkg has no benefit. The
cmd directory is enough. Keep the import paths clean.
In a large application, possibly with lots of non-Go assets, isolating packages from the rest of the project through a
pkg project might make sense.
But if you do so, be aware of the consequences.
To make your life easier, you might want to keep public library packages separate from your application project.
Dividing a project into library packages and commands is only the first step. At the next level, you'll want to think about how to organize your code into packages. Should you use a monolithic package, or a Rails-style scheme with handler, controller, and model packages?
Ben Johnson might have an answer for you: Standard Package Layout
The Go team also shared some insights about Organizing Go code, but be aware that the article is from 2012, where GOPATH as the single Go workspace was still a thing. Apart from that, their advice still holds.
Applied Go Courses helps you get up to speed with Go without friction. Our flagship product, Master Go, is an online course with concise and intuitive lectures and many practices and exercises. Rich graphic visualizations make complex topics easy to understand and remember. Lectures are short and numerous to save your precious time and serve as quick reference material after the course. Learn more at https://appliedgo.com.
Categories: Best Practices, Ecosystem