At DoltHub, we love Go! We're using Go to build DoltDB, an open-source SQL database engine with Git-like distributed version control. Dolt lets you branch, fork, merge, and diff your relational tables, just like Git does for your source code files. Heck, we even added interactive rebase support earlier this year!

We love developing with Go and we publish a new article in our Golang blog series every three weeks. With Go 1.22 expected to be released next month, we thought it was time to take a look through the latest release candidate, explore the new changes, and share what we're excited about.

As a reminder, the Go team makes it very easy to try out release candidates. You can easily download the Go 1.22 RC1 build for your platform by running: go install golang.org/dl/go1.22rc1@latest. On my system, this installed the release candidate at: ~/go/bin/go1.22rc1. IDEs like GoLand make it equally easy to download Go release candidates and start using them in your project, too. Just make sure your project settings reference the release candidate and that your go.mod file declares that your code uses go 1.22.

Loop Variable Reallocation

It's here! Go 1.22 changes the default behavior of loop variables in for loops so that they are no longer reused, and instead a new variable is allocated each time through the loop. Reusing the loop variable was a frequent cause of bugs in Go programs when the loop variable was passed to a go routine and then by the time the goroutine started up, the variable had been set to a new value.

We won't spend too much time on this change, since we already covered it in depth in our blog on the Go 1.21 RC. Go check out that blog if you want to see more details on how the Go team rolled this change out, or read about some interesting examples where this behavior led to some pretty serious bugs.

We did a quick egrep through some of our source code, and found over a dozen places where we needed to manually capture a loop variable's value into a new variable, so we could safely pass it in to a goroutine. It's fair to guess that for most of these, we probably didn't remember this Go quirk the first time around and had to go back and fix it.

egrep -R '\t(\w+) := \1$' . 
./cmd/dolt/commands/sqlserver/server_test.go:        test := test
./libraries/doltcore/doltdb/root_val.go:        col := col
./libraries/doltcore/doltdb/hooksdatabase.go:            hook := hook
./libraries/doltcore/sqle/cluster/mysqldb_persister.go:        r := r
./libraries/doltcore/sqle/cluster/controller.go:        client := client
./libraries/doltcore/sqle/cluster/branch_control_replica.go:        r := r
./libraries/doltcore/sqle/dsess/transactions.go:        f := f
./libraries/doltcore/sqle/dsess/transactions.go:        i := i
./store/types/list_editor.go:            edit := edit
./store/types/set_editor.go:            edit := edit
./store/nbs/store.go:        h := h
./store/nbs/store.go:        c := c
./store/nbs/table_reader.go:        i := i
./store/blobstore/oci.go:                u := u
./store/blobstore/oci.go:            u := u

Integers on the Range

Speaking of for loops... there's one more small change to the Go language currently slated for Go 1.22 – the ability to range over an integer in a for loop. A draft of the updated Go language spec explains how this new behavior works with for loops using a range clause over an integer value: For an integer value n, the iteration values 0 through n-1 are produced in increasing order, with the same type as n. If n <= 0, the loop does not run any iterations.

This is just a little bit of syntactic sugar, since you could already write for loops over integers by using the for i := 0; i < 10; i++ { syntax, but it's still a nice addition to the language, and the code definitely looks much cleaner when using this new syntax. Here's a quick example where we use this new syntax to spin up a set number of background processes. (Note that this example also illustrates the sharp edge with loop variable reuse that we just discussed above.)

func main() {
    const processCount = 10
    var wg sync.WaitGroup
    for i := range processCount {
        wg.Add(1)
        go func() {
            defer wg.Done()
            startBackgroundProcessor(i)
        }()
    }
    
    wg.Wait()
}

func startBackgroundProcessor(id int) {
    fmt.Printf("Starting background processor: %d\n", id)
}

The Rangefunc Experiment 🧪

Continuing on with the theme of loops... Go 1.22 RC1 contains one experiment for a new Go language feature that allows you to range over function iterators. This is a preview of a language change being considered, so it will likely change, and there's no set timeline for if or when it would be officially added to the language. Note that this is still an opt-in feature, so if you want to try this code out yourself with the Go 1.22 release candidate, you'll need to make sure you set the GOEXPERIMENT environment variable to rangefunc before building and running your code.

Basically, this feature lets you write APIs that return a function that gets invoked once per loop iteration, and on each invocation, Go will pass in the contents of your for loop block to that function. That enables your function iterator function to choose the order to iterate over elements and how to execute the loop body, while allowing the calling site to specify the loop body code. The generic syntax and the function-taking-a-function syntax may look a little odd at first, but it's actually pretty straightforward once you work through it.

Take a look at the concrete example below, where we create a new function called SortIterator, with a function signature that meets the requirements defined in the rangefunc spec for being used as a function iterator argument to range. The function must return another function that the Go runtime can repeatedly call to iterate over the values. Each time it calls that function, it passes in another function that is implicitly created by Go from the contents of your for loop block. Neat, right?

// SortIterator makes a new copy of the specified slice |s| and sorts it (useful when you can't sort the slice in place), 
// then returns a function that is compatible for use with the range keyword, to iterate over the sorted slice and 
// execute the body of a for loop for each element in the sorted slice. 
func SortIterator[E cmp.Ordered](s []E) func(func(int, E) bool) {
	sortedSlice := make([]E, len(s))
	copy(sortedSlice, s)
	slices.Sort(sortedSlice)

	return func(yield func(int, E) bool) {
		for i, v := range sortedSlice {
			if !yield(i, v) {
				return
			}
		}
	}
}

func main() {
	s := []string{"xyz", "abc", "mno", "jkl"}
	for i, x := range SortIterator(s) {
		fmt.Println(i, x)
	}
}

This isn't the easiest syntax to understand initially, but it's a pretty neat feature. You may have seen a somewhat similar pattern where an Iter function controls iteration over a collection and a function is passed in as an argument to the Iter function that contains the logic to execute on each iteration. This new rangefunc experiment provides another way to achieve a similar result, while keeping the loop body in the calling code and avoiding the noise of explicitly having to wrap the loop body code in a function.

Performance Improvements

There are two notable changes in Go 1.22 RC1 that can improve the performance of Go applications. The first is improvements to the Go runtime that more efficiently organizes type-based garbage collection metadata, positioning it nearer to each heap object. This improves CPU performance by approximately 1% to 3%. It also reduces memory overhead by approximately 1% since redundant metadata has been deduplicated.

The compiler in Go 1.22 also contains performance improvements for Profile-guided Optimization (PGO). PGO is still a new Go feature and was originally released last year in Go 1.20. The Go team has been iterating on it and Go 1.22 contains changes that allow PGO to devirtualize a higher proportion of calls than it previously could. The compiler now interleaves devirtualization and inlining, so interface method calls are more optimized. The Go team reports that with these improvements, PGO can now improve performance of Go programs by 2% to 14%.

We haven't dug in deep to these changes yet, but they definitely caught our eye. We've been curious about PGO since it came out in Go 1.20, so stay tuned for a future blog post where we explore turning PGO on in our build process and see what kind of performance changes we see for Dolt.

Trace Improvements and Web UI Refresh

The Go team has given the trace tool some needed love in this release. The execution tracer has been completely overhauled, fixing many long-standing bugs, and the trace tool's web UI has been refreshed. The most interesting change to the trace tool is that it has added support for a thread-oriented view of trace results. You can see an example of this new thread view in the screenshot below.

Note that most of these improvements rely on new data in the trace files that is currently only collected in the Go 1.22 release candidate, so don't expect to load a trace file created with an older version of Go and see the same improvements.

Standard Library Updates

The biggest changes to Go's standard library in 1.22 RC1 are a new math/rand/v2 package and changes to HTTP routing. In addition to those larger changes, there are also several small API improvements that are worth knowing about.

The new math/rand/v2 has some nice incremental improvements over math/rand, such as removing some deprecated functions (e.g. rand.Read), cleaning up some function naming for better consistency (e.g. Intn → IntN, Int31 → Int32), but the most exciting updates are improvements to the random number algorithms used. Several APIs have been updated to use faster random number generator algorithms that weren't possible to change before, since they changed the output streams. btw... the new math/rand/v2 package is the very first v2 package in Go's standard library. Making it 14 years before having to make backwards incompatible changes and fork a v2 of a standard library package isn't too shabby at all.

The changes to HTTP routing make defining HTTP routes more expressive by allowing you to specify HTTP methods and wildcard matching with net/http.ServeMux. For example, registering a handler for "POST /items/create" allows you to limit that route to match only for POST HTTP operations. If multiple routes exist for the same path, the route with the most specific matching HTTP method will be given precedence. The one special case with HTTP methods is that registering the GET method for a handler automatically registers the HEAD method, too. Wildcard patterns have also been introduced, so you can register a route for "/items/{id}" and then extract the specified ID from the path using Request.PathValue.

There are also several smaller changes to Go's standard library, such as the new cmp.Or function that takes a variable number of arguments and returns the first value that is not equal to the zero value for its type.

Being a database company, any changes to the database/sql package naturally catch our eye, and there's currently a new Null type that uses generics to provide a slightly simpler way to scan nullable column values for any column type. We're happy to see the standard library slowly, but surely, incorporating more use of generics to make our lives a little bit easier. Before this new generic Null type, you needed to use a type-specific type in order to scan a value from a nullable column (e.g. the NullString type). The resulting code doesn't actually change much (mostly just from var value sql.NullString → var value sql.Null[string]), but it's still nice to have a generic option and have fewer concrete types to deal with. The example below shows how to use this new Null[T] type (note that error handling is omitted for brevity):

	// Query a column that can contain null values
	rows, _ := db.Query("SELECT aNullableStringColumn from myTable;")
	for rows.Next() {
		var value sql.Null[string]
		rows.Scan(&value)
		if value.Valid {
			fmt.Printf("Value: %s\n", value.V)
		} else {
			fmt.Printf("Value: NULL\n")
		}
	}

Wrapping Up

Thanks for reading our take on the changes in the Go 1.22 release candidate. It's always fun to see a preview of what new goodies will be rolling out in the next Go release. We still expect more small changes before Go 1.22 is officially released next month, and the Go team has been known to remove features from the release candidates before the official release, but this RC should give us a pretty good idea of what we'll have available in Go 1.22 next month.

We'd love to hear what you think of the changes in this release candidate! Did we miss anything from the Go 1.22 draft release notes that you think is an important feature? Swing by our Discord and let us know what you think! The dev team for DoltHub and DoltDB all hang out on Discord and we love when people come by to talk with us about databases, versioning, and Golang! 🤓