Golang is often touted as a “simple” language by design, deliberately eschewing features that would add complexity. The language design is openly opinionated but avoids baking those opinions into the spec. It’s why core design patterns like errors and context objects are implemented in the standard library instead of being built into the language. While special syntax for these types could decrease boilerplate, it would also make the language more complex in a way that violates the design philosophy of the language.

The language designers would argue that there’s elegance in simplicity. That Golang doesn’t need special cases.

But that’s not the complete story. It turns out that Golang actually does have “special cases” in order to allow for behavior that can’t otherwise be expressed in the language.

Today we’re going to look at some of those special cases.

The init function#

Sometimes, a package needs to run code on startup. Typically, this is a sign of global state that should be avoided, but sometimes it’s necessary. For instance, a package might need to parse environment variables, or setup logging, or register itself with another package, like how SQL drivers register with the "database/sql" package in the standard library. At Dolt, we end up needing to do all of these things at some point.

In order to allow this, you can define a function named init. Golang gives special meaning to these functions:

• If a package has function named init, it will be invoked when the package is imported.
• If a function is named init, it must take no parameters and have no return value.
• The init function can be declared multiple times in the same package, and each one will be executed when the package is imported. This is the only function that is allowed to be redeclared.

The last point has some interesting consequences. Because the smallest code unit in Golang is packages, not source files, this means that you can also declare multiple init functions in the same file.

When I read this, I immediately wondered what would happen if I tried to manually call init() while there were multiple definitions in my package. It turns out you’re simply not allowed to manually invoke init, even if you only define it once. If you try, you’ll get a compiler error that reads undefined: init.

There’s no mention of the function’s special properties in the error message. I can only imagine the confusion of the poor sap who names their function init without being aware of its special properties, then tries to understand why the compiler is telling them the function isn’t defined.

internal import paths#

You probably already know that Golang uses casing to determine the visibility of symbols: symbols that begin with an uppercase letter are public and can be accessed from outside their package, while symbols that begin with a lowercase letter are private and can only be accessed from within their package.

This suggests that there are only two options: globally-visible, and package-private. But there’s actually a way to make a symbol visible to some packages but not others. All you have to do is put the source file in a directory named internal.

For example, if you place a package in the directory .../a/b/c/internal/d/e/f, then that package can only be imported by a package in .../a/b/c or one of its sub-directories.

This applies to the entire package, and can’t be applied only to specific symbols. But that’s not a huge problem: if you have a package foo that has symbols that you only want to be accessed by foo’s subpackages, you can create foo/internal for just those symbols.

Curiously, if you try to look up this behavior in the Go language specification, you won’t find it. You can find it described in the Go User Manual and in the Go 1.4 Release Notes, but there’s no mention of internal import paths in the spec itself. And that’s because this behavior isn’t actually part of the language spec, it’s a decision enforced by the Go toolchain.

What the spec does say is this:

The interpretation of the ImportPath is implementation-dependent but it is typically a substring of the full file name of the compiled package and may be relative to a repository of installed packages.

This has implications for more than just internal visibility. Let’s give it its own section:

Implementation-dependent import behavior#

There’s only one implementation of the Go toolchain. So why would the language designers decide that resolving import paths is implementation-defined?

Firstly, because there are other build systems that can compile Go code without using the Go toolchain, such as Bazel. Bazel has its own rules for visibility and package management, and making this process implementation-dependent means that Bazel can use its own rules instead of being forced to use Go’s.

But even if tools like Bazel didn’t exist, making the import process deliberately unspecified isn’t actually that crazy of an idea. It means that the Go toolchain can evolve how it resolves imports without needing to change the spec. And we’ve seen this happen, when Go 1.4 added the ability to declare “canonical names” for packages to make it harder to accidentally import the same package from two different repositories. By leaving the process for resolving import paths unspecified, the Go toolchain can also add the ability to support additional resolution methods, such as loading packages from a remote service instead of looking for them locally.

This is all standard practice: for instance, Java and Python provide the ability to customize the package resolution process. So it makes sense to not define the process too rigidly in the spec.

In practice, the way the Go toolchain imports packages is to (more-or-less) treat the import path as a directory path relative to the $GOPATH environment variable. It then finds all the source files in that directory, verifies that they all declare the same package name, and then exports them, making them available to the importing code with the package name as the identifier.

But because this is implementation-defined behavior, it lets the Go toolchain include some special cases. Internal import paths are one such case. Test packages are another.

Test packages#

Ordinarily, the Go toolchain requires that multiple packages cannot exist in the same directory. But it makes an exception for packages whose names end in _test.

Instead, Go will allow a directory to both contain files that declare (eg) package foo and files that declare package foo_test, provided that every file in the foo_test package has a filename that begins with test_. This allows for test packages that live alongside the package being tested. Using this directory as an import path will always yield the non-test package, making it impossible to import the test package. Instead, the test package can be used with the go test command.

Import weirdness#

This isn’t a “special case”, but it’s a consequence of Go’s import behavior and it deserves a mention.

What makes Go unusual here isn’t the fact that import behavior is implementation-defined, it’s the odd relationship between import paths and package names.

Namely that there isn’t one.

Although a directory can only contain a single package (except for test packages), there’s no requirement for a package to have the same name as its directory. Every style guide says that you should, but you don’t have to.

This means that if you have a source file that looks like this:

import (
    "fmt"
    
    "dependency/one"
    "dependency/two"
)

func main() {
    fmt.Println(one.Foo)
}

It’s not actually possible to confidently determine which import path corresponds to the imported symbol one. Convention tells us that the package one came from the import path dependency/one, but this isn’t actually enforced.

If we examine the imported packages, we might discover that the package names are swapped with regard to the paths:

// File: dependency/one/a.go

package two

var Foo = "This string isn't going to be printed."

// File: dependency/two/a.go

package one

var Foo = "But this one will! Greetings from dependency/two!"

Even though packages are extremely unlikely to do this, it can cause issues for IDEs, since it means that every import path must be analyzed before any symbol in the source code can be resolved.

You can avoid this by always using explicit identifiers for imports, like so:

import (
    fmt "fmt"
    
    one "dependency/one"
    two "dependency/two"
)

func main() {
    // The symbol |one| here is guarenteed to resolve to the package imported from "dependency/one"
    fmt.Println(one.Foo)
}

Given that the odds of this happening are low, it’s probably not worth refactoring a project to always use explicit imports. But for new projects, there’s not really any reason to favor implicit imports over explicit ones.

For comparison, Golang has a feature called “dot imports”, where all of the symbols from an imported package are added to the importer’s namespace instead under a package identifier. It looks like this:

import (
    . "fmt"
)

func main() {
    // this is the same as fmt.Println
    Println("Hello, world!")
}

Use of this feature is discouraged because it obscures the origin of top-level identifiers and makes code harder to read. But implicit imports aren’t guarenteed to be free of this problem either.

The reason why other languages don’t have this problem is because even though the import process is not rigidly defined, import statements in languages like Java or Python work by accepting a fully-qualified package name, and produce a package with that fully-qualified package name.

But packages in Go don’t have fully-qualified package names, just simple identifiers. That’s why import statements must use paths in order to disambiguate them. But the name of the package isn’t tied to the path, and is mostly just used to determine the implicit identifier used when the package is imported, and for documentation generated by the godoc tool. There is no hierarchical namespace of package names, or even a global namespace of package names.

Conclusion#

There’s a comment on ycombinator that I like to throw around.

Go tries to be a simple language, but it doesn’t always succeed. Sometimes real-world requires resist simplicity, and aiming for simplicity in one place results in moving that complexity somewhere else. Every real system is going to have complexities. Is it better for that complexity to be upfront, or hidden?

Honestly, most of the things I pointed out here are curiosities, not real issues. If they’re warts, they’re warts that most programmers won’t ever actually see. Is that better than a language that delivers complexity up-front?

I’m curious what you think. Feel free to join our Discord and let me know how I’m wrong about Go.