We're using Go to write Dolt, the world's first version-controlled SQL database. We build release binaries for all three major operating systems: Linux, OS X, and of course Windows. (Sorry Plan9). And in fact we have customers on all of these operating systems, so they actually have to all work. Go makes it very easy to compile releases for different platforms (from any platform) via the GOOS and GOARCH environment variables, and GitHub actions makes it easy to run all our tests on every platform.

All that automation is great. Mac and Linux are close enough in practice that using a Macbook gives us pretty good assurance that Linux works as well. And of course, we run Dolt on Linux ourselves to power DoltHub and our other web properties. But: somebody on the team still needs to run Windows on a daily basis to find compatibility issues as they arise, and to debug customer issues that occur only on that platform. As in, they need to run it as their personal development machine while they work on the database. If nobody works on Windows, if everyone uses Apple laptops every day, then it's inevitable that you're letting your customers find Windows issues for you.

Look, we all know that Apple makes the best laptops. It's basically an objective fact, totally beyond dispute to all but crazy people. Not only do Apple machines look better, perform better, sleep and wake better, and have longer battery lives, they even cost less for comparable specs. Nobody wants to give that up. But somebody has to.

Friends, I drew the black egg. I took one for the team and passed up shiny Apple silicon in favor of a stinky ThinkPad, all so our Windows customers get a better product.

This is a blog about the ups and downs of being a Go developer on Windows: what works well, what is difficult, and what to watch out for.

Installing and running Go: easy

Go is easy to get on Windows, and even has a full installer program. Upgrades are also very easy, and uninstall the older version of Go automatically.

Both VS Code and GoLand run natively on Windows just fine. The entire Go toolchain, like the dlv debugger, work more or less seamlessly.

Using Go on the command line: use WSL

Life in the Windows world got a lot more comfortable for Linux natives with the adoption of WSL, the Windows Subsystem for Linux. WSL gives you the ability to run Linux binaries on an emulated or virtualized Linux machine, from within Windows. You can run Windows executables from within a bash shell, you can use the Windows file explorer to examine the Linux file system, and Linux binaries can read and modify the Windows file system. All the admin tools you know like apt, cron and ssh work exactly as you expect.

There are two variants of WSL, and they work pretty differently.

• WSL 1 is an emulation layer. It works very well despite the lack of true virtualization, but there are certainly some compatibility problems. The main reason to use it despite these issues is excellent performance across the file system boundary.
• WSL 2 is true virtualization. You're running Linux in a VM on your Windows machine. The main drawback is poor performance across the file system boundary.

I use WSL 1 despite the occasional compatibility problem, just for the performance benefits. I use the Ubuntu command line for basically everything except editing and debugging Go code, which I do in my IDE. So I have my project code on the Linux partition, and access it in GoLand from the Windows partition. I need this to be fast, so WSL 1 it is.

But because of the limitations, I often resort to using Windows binaries from the Ubuntu shell, and I have scripts to automate this. Here's how I build Dolt from my Linux shell, for example.

dolt_install() {
    cd $GOPATH/src/github.com/dolthub/dolt/go/cmd/dolt
    go.exe build
    mv dolt.exe $GOPATH/bin
    /usr/local/go/bin/go install
    popd
}

So here I'm building two binaries: a Windows binary with go.exe, and a Linux one with /usr/local/go/bin/go. Yes, I have two Go binaries installed, one for each operating system. For the most part, I work with go.exe because that's what my IDE uses, and I want things to stay in sync. But every now and then I want to use the Linux binary, mostly because I've learned from painful experience that mixing Windows and Linux binaries in the same script leads can cause a lot of inscrutable issues.

Overall I love WSL and find that it makes development on Windows much more bearable. It can't do anything about the inferior hardware, but at least I get a real shell environment and standard Linux toolchain.

One annoying issue wih WSL is that certain syscall operations won't work as expected, so for example the standard CPU profiling built into Go doesn't work. In practice this almost never comes up, but it's very irritating and mysterious when it does. For these times, it's nice to have the Windows executable of your program as a workaround.

The rest of the toolchain: msys64

If you want to use features like Cgo, you'll need something approximating the gcc build toolchain, which obviously Windows does not have. You can't use the Linux gcc toolchain to build Windows binaries that use Cgo, they need to be Windows-native executables. Getting this set up on Windows is kind of irritating, but it's a one-time cost and thereafter more or less painless.

There are a few ways to do this, but the one that seems to be easiest and most bulletproof is MSYS2. Follow the instructions, and make sure that you put C:\mysys64\bin on your %PATH%, and everything should just work.

At some point we updated a dependency involved in the Cgo portion of the product and it wouldn't build anymore. I needed to set a new environment variable to get things moving again:

CGO_LDFLAGS=-lssp

For whatever reason, this only happened on Windows, with WSL Linux builds unaffected.

Standard tricky bits: file paths

Everyone knows that Bill Microsoft made a terrible decision when he decided to use \ as the standard path separator in the DOS file system. 50 years later we're still stuck with the result of that decision, and it's the source of a lot of platform compatibility issues whenever a program has to touch the filesystem.

For its part, Go is pretty accommodating of using Linux path separators (/) in paths on any platform. So code like this tends to work fine on Windows:

os.Create(fmt.Sprintf("%s/%s", dir, filename)

The more platform agnostic way to write this uses filepath.Join instead:

os.Create(filepath.Join(dir, filename)

But both ways work. The times when you get yourself in trouble tend to be whenever you're doing any parsing or string munging on file paths. Code like this is pretty common in my experience:

fileName := path[strings.LastIndex(path, "/")+1:]

Of course nobody reading this would ever hard-code the expectation of a path separator into their program logic like this. But people do. The filepath package contains lots of functions for dealing with these platform-specific issues. The above code should probably be written like this:

fileName := filepath.Base(path)

And if you really need to write your own parsing logic for something, make sure you're using the path separators for your platform when appropriate.

fileName := path[strings.LastIndex(path, filepath.Separator)+1:]

This gets even trickier when you're dealing with file URLs. Absolute file URLs on Windows contain a drive label, something like this:

file://C:/Users/zachmu/My%20Documents/upload.png

My advice here is to proceed with extreme caution. Yes, use the standard URL libraries in Go. But also test what you're doing exhaustively, on Windows, including any third-party libraries that might handle the URLs you construct. Something about the extra : in the URL makes this painfully easy to screw up.

Deleting or moving files: close your files, and be prepared to retry

Modern versions of Windows are very particular about when a file or directory can be deleted. Unlike Linux, where the file system is an independent entity and will happily delete a file currently being read by another process ("oops, were you using that?"), Windows tracks file ownership very jealously and will rebuff your effort to delete it if it's still considered in use. This code works fine on Linux and fails on Windows.

	f, _ := os.Open("myFile")
	r := bufio.NewReader(f)
	for {
		line, _, err := r.ReadLine()
		if err != nil {
			// cleanup the temp file
			os.Remove("myfile")
		}
		processLine(line)
	}

On windows, you need to always, always call Close() on file handles before attempting to delete them. This is easy enough to do when the deletion happens near the access like above. But when you're writing a program that touches many thousands of files in a long-lived process, it's very easy to accidentally lose track of one or more of them. This is effectively a resource leak, and while it's not ideal on Linux, it won't prevent cleanup operations like the above. But on Windows it leads to unexpected errors and lots of crud accumulating on disk.

Even worse, Windows tracks file usage somewhat asynchronously. It's possible that a call to Close() immediately followed by a call to os.Remove() will fail because Windows hasn't yet updated its internal lock tracking. So when you absolutely need a file to be deleted, you need to try more than once.

for i :0; i < closeAttempts; i++ {
    err = os.Remove(file)
    if err == nil {
        break
    } else {
        time.sleep(10 * time.Milliseconds)
    }
}

Yes, this is 100% as bad as it looks. In recent releases Go's standard libraries will even do this for you in some cases, but in general it's up to you to make sure that Windows delete or rename operations actually succeed, and retry as necessary.

Socket files and other nice things we can't have

Linux environments offer quite a few quality of life features that just aren't available on Windows. The precise details depend on what you're trying to accomplish, but the upshot is that sometimes you'll need to write your code twice: once for Windows and once for Linux.

Luckily Go makes this easy. There are a few methods, but the one we use most is to suffix the name of a file with _unix.go or _windows.go. The Go compiler automatically excludes or includes files with this naming scheme depending on the target platform.

For example, we have some socket logic in our database connection handler.

// net_listener_unix.go

func newNetListener(protocol, address string) (net.Listener, error) {
	lc := net.ListenConfig{
		Control: func(network, address string, c syscall.RawConn) error {
			var socketErr error
			err := c.Control(func(fd uintptr) {
				err := unix.SetsockoptInt(int(fd), unix.SOL_SOCKET, unix.SO_REUSEADDR, 1)
				if err != nil {
					socketErr = err
				}

				err = unix.SetsockoptInt(int(fd), unix.SOL_SOCKET, unix.SO_REUSEPORT, 1)
				if err != nil {
					socketErr = err
				}
			})
			if err != nil {
				return err
			}
			return socketErr
		},
	}
	return lc.Listen(context.Background(), protocol, address)
}

On Windows, this looks very different. It's simpler, but not in a good way: no socket file support.

// net_listener_windows.go

func newNetListener(protocol, address string) (net.Listener, error) {
	return net.Listen(protocol, address)
}

Note that both functions have the same name. Go chooses one to build into the binary at compile time.

Conclusion

Developing on Windows can make you feel like a second-class citizen. It's not just that you have objectively worse hardware to work on, it's that you're subject to a whole litany of problems that your Mac and Linux colleagues just never encounter. Go has ways to get around most of them, but you have to know they're there and use them. You have to have tests. And yes, if you're serious about supporting Windows customers, somebody on your team has to work on Windows.

Have questions or comments about Go on Windows? Or maybe you're curious about Dolt, the world's only version-controlled SQL database? Come by our Discord to talk with our engineering team.