Is Go 1.21 already on your upgrade list? It should be!

In the tradition of Go, release 1.21 does not deliver loads of new language features but many improvements to the toolchain and the ecosystem.

What excites me: Loopvar experiment

This behavior has certainly confused many Go newcomers around the globe, myself included.

Up to now, Go has single-instance loop variables

The loop counter of a for loop like, for example, for i : = 0; i < 10; i++ , is a single instance. In every loop iteration, code inside the loop body reads the current value of i from the same instance of i.

In most scenarios, there is absolutely no problem with this. The loop

for i := 0; i < 3; i++ {
    fmt.Println(i)
}

prints out the numbers 0 to 2, as expected and intended.

The following code also captures the address of i in each iteration:

for i := 0; i < 3; i++ {
    fmt.Println(&i, i)
}

Output:

0x140000a4040 0
0x140000a4040 1
0x140000a4040 2

All iterations capture the same address of i. This proves that i is a single instance.

So what's the problem? Why should I care about the address of i?

Things start becoming weird when the loop body implicitly captures the address of the loop variable.

Deferred evaluation of function arguments

Consider this code (taken from LoopvarExperiment · golang/go Wiki):

var prints []func()

for i := 1; i <= 3; i++ {
    prints = append(prints, func() { fmt.Println(i) })
}

for _, print := range prints {
    print()
}

The loop fills a slice with functions that print i. However, rather than printing the value that i had when appending the function to the array, all functions print the value i has after the loop finishes.

(Playground link- ensure the Go version is set to Go 1.21 or lower. If you read this article when Go 1.23 is out, then it's probably too late. If all goes well, the loopvar behavior will be/has been moved out of experimental status in Go 1.22.)

4
4
4

The code does not obviously capture the address of i, because it happens implicitly when appending a new func value to the slice. The function argument is not evaluated until the function is called, and at that time, i evaluates to the value it has when the loop finished.

Deferred evaluation by goroutine closures

If you spawn goroutines that are defined through a closure inside a loop, the closure holds a reference to the loop variable.

for i := 0; i < 3; i++ {
go func() {
        fmt.Println("I am goroutine", i)
    }()
}

This will print:

I am goroutine 3
I am goroutine 3
I am goroutine 3

If you pass i to the goroutine as a function argument,

for i := 0; i < 3; i++ {
go func(i int) {
        fmt.Println("I am goroutine", i)
    }(i)
}

the output is similar to

I am goroutine 0
I am goroutine 2
I am goroutine 1

(with a varying sequence of 0, 1, and 2) because when a function is spawned as a goroutine, the function arguments are always evaluated before spawning the goroutine. In this case, this is right inside the loop iteration.

I wrote a whole article about that phenomenon, so I will not go into the details here.

The point is, references to loop variables can occur implicitly. As a result, loops can behave in an unexpected way and cause bugs that are difficult to track down.

How Go 1.21 addresses this behavior

Go 1.21 contains the Loopvar experiment.

If you set the environment variable

GOEXPERIMENT=loopvar

then the Go compiler creates separate instances of the loop variable in every loop iteration. This way, even if the loop variable evaluation is deferred for one of the two reasons outlined above, the deferred evaluation receives the value the loop variable had in the particular loop iteration when the loop variable was formally read.

The functions in the slice from above then prints what most developers would expect:

1
2
3

The pointers slice prints different addresses, proving that each iteration sees a separate instance of the loop variable:

[0x140000a4050 0x140000a4058 0x140000a4060]

Why is this an experiment?

The Loopvar experiment fixes an obviously wrong behavior, so why wasn't the new loop behavior enabled in Go 1.21 by default?

As always, the Go team is very cautious about introducing changes that may break existing code. The loopvar experiment can indeed break existing code, but the Go team expects that the change should virtually never break real programs.

Au contraire, the new semantics can even help fix buggy code.

What if we cannot upgrade to Go 1.21 yet?

The good news first: Go's backward and forward compatibility is further improved with Go 1.21 (see the toolchain section below), so maybe you can upgrade even if you don't know it yet.

But if you absolutely cannot upgrade and want to use the loopvar behavior in your code, you can simply copy the loop variable into a per-iteration instance.

Example:

for i := 0; i < 3; i++ {
i := i  // copy the loop variable into a per-iteration variable of the same name
go func() {
        fmt.Println("I am goroutine", i)
    }()
}

Using the same name as the loop variable might look weird, but this way you don't have to go through the loop body and update all the occurrences of the loop variable to a different name. (Side note: Using the same name is possible because the loop variable's scope is outside the loop body, and therefore the per-iteration variable shadows the loop variable. No name conflict here.)

Find out more on the LoopvarExperiment · golang/go Wiki page.

Go toolchain or Go toolchains?

Stop talking about "the Go toolchain." Go 1.21 has separated the Go command from the rest of the tooling. When you install Go 1.21.0, you start with a single Go 1.21.0 toolchain. Depending on several aspects, the Go command may download a toolchain of a different version.

For example, assume that Go 1.21.3 is already out buy you are still using Go 1.21.0. If the main go.mod or go.work file contains a go directive that specifies Go 1.21.3, the Go command downloads the Go 1.21.3 toolchain to compile the module.

If the go directive specifies an older version, say, Go 1.19, then your installed Go command will compile the module with Go 1.21.0 because the go line in a go.mod file is a minimum requirement that is satisfied by newer Go versions.

But what if a module wants to enforce a specific toolchain?

It can do so through the toolchain directive. Independent of the go directive, it pins the toolchain to a particular version. So if a go.mod or go.work file contains toolchain 1.19, your Go 1.21 command will download the Go 1.19 toolchain to compile the module.

In summary:

  • You have Go 1.21.0 installed.
  • A go.mod file contains the line go 1.19.1.
  • Your Go command compiles the module with the GO 1.21.0 toolchain.
  • Another go.mod file contains two lines:
    • go 1.19.1
    • toolchain 1.19.1
  • Then your Go command will use the 1.19.1 toolchain to compile the module.

Since dependency management is inherently complex, there is much more to say about the new toolchain mechanism. Find all the nitty-gritty details in Go Toolchains - The Go Programming Language.

The thing about type inference

The good news is that the compiler got better at type inference, but the description in the release notes is so terse and bare of any context that it is difficult to follow. I was too lazy busy to decipher the release notes and find or construct examples to illustrate how these type inferences work and make our lives better, and so I was delighted to see that André Eriksson did the hard work to dig through all five cases and come up with sample code because, you know, a code snippet says more than a thousand words.

Understanding Go 1.21 generics type inference – Encore Blog

Activating tracing is not a big issue anymore...

...at least if your app runs on amd64 or arm64 architectures, for which the release notes claim a tenfold improvement on CPU cost over the previous release.

Felix Geisendörfer started this optimization off by creating an experimental patch for the runtime to implement "frame pointer unwinding". (I am not going to explain this here, if you are curious, read this article.) In this (pre-Go 1.21) article, he summarizes all the contributions that finally have led to this massive speedup: Waiting for go1.21: Execution Tracing with < 1% Overhead

Profile-Guided Optimization

Profile-Guided Optimization (PGO) was already available in Go 1.20 as a preview; now it has become a standard feature.

By collecting and analyzing performance profiles, the compiler can identify bottlenecks that do not show in static analysis. Most Go programs should see a performance improvement between 2% and 7% from enabling PGO. The Go 1.21 compiler itself was built with PGO, which resulted in a 6% speedup for build times.

The PGO user guide contains all you need to know to get started: Profile-guided optimization - The Go Programming Language

New packages: slog, maps, slices, cmp

slog: logging with style structure

Go's log package reflects the minimalistic approach that Go has since its beginnings. That's fine for generating human-readable log output that developers use for troubleshooting. Large software deployments, however, need a log file format that can be parsed by monitoring tools.

Structured logging addresses this need. Every log message is a record containing key-value pairs that represent a log message, a severity level, the time the record was created, and other useful attributes.

Again, I left the tedious work to others. Redowan Delowar gets you going on Go structured logging with slog | redowan's reflections.

slices

Good news for fans of slices and convenience functions: the new slices package delivers over two dozen functions for inspecting and manipulating slices.

See for yourself: slices package - slices - Go Packages

maps

The new maps package contains considerably fewer functions than slices, but the functions Clone, Copy, and Equal alone ensure a decent productivity boost for all map afficionados. DeleteFunc uses a function to determine which elements to delete, and EqualFunc uses the same approach to allow comparing maps where a simple == does not work.

Short but sweet: maps package - maps - Go Packages

cmp

Two generic functions: Compare(x, y T) int and Less(x, y T) bool.

Compare returns either of -1, 0, or +1 depending on whether x is less than, equal to, or greater than y.

Less returns true if x is less than y.

For both functions, the arguments must be ordered, for which the package provides the type constraint Ordered.

Comparatively concise: cmp package - cmp - Go Packages

New built-in functions

Min, max, clear.

I bet you read about them in four or five places at least. And they are easy to grasp. Hence you'll surely understand that I only mention them briefly.

Min and max: minimum and maximum, respectively, of their arguments

var a, b, c int // use any orderable type
mi := min(a, b) // the minimum of a and b
ma := max(a, b, c) // the maximum of a, b, c

Clear: easily empty a map, a slice, or a type parameter whose types in its type set are maps or slices. (Uh, what does the last part mean?! – See the example below.)

// clear a map
m := map[int]int{1:2, 3:4}
clear(m) // m is now an empty map, len(m) == 0

// before Go 1.21
for k := range m {
  delete(m, k)
}

// clear a slice
s := []string{"a", "b", "c"}
clear(s) // all entries of s now have their zero value

// before Go 1.21
for i := range s {
  s[i] = ""
}

// wipeSlice()'s parameter t must be a slice or a type derived from a slice
func wipeSlice [T ~[]E, E any](t T) T{
    clear(t)
    return t
}

// wipeMap()'s parameter t must be a map or a type derived from a map
func wipeMap[T ~map[K]V, K comparable, V any](t T) T {
    clear(t)
    return t
}

(Playground)

Maybe you have noticed the comment about clear(s) in the code block above: "all entries of s now have their zero value." Wait a second—why doesn't clear empty the slice completely?

Slices are structs that contain the length and capacity of the slice and a pointer to the actual array holding the data. Structs are passed to functions by value. A function that receives a slice only owns a copy of the slice header. Setting the length inside this header to zero has no effect on the caller's copy of the slice header. But setting the array elements to zero does.

Anton Zhiyanov explains this in more detail and with sample code in Built-in functions in Go 1.21.

Experimental port to WASI

Go 1.21 supports new OS and architecture targets (GOOS=wasip1, GOARCH=wasm) to compile a Go binary as a WebAssembly module.

Try this:

  • Install wazero (Hint: Homebrew has a wazero formula & can keep it updated.)
  • Create a new Go project and add some code:
package main

import "fmt"

func main() {
    fmt.Println("Hello WASI world!")
}
  • Run GOOS=wasip1 GOARCH=wasm go build -o hello.wasm
  • Result: Hello WASI world!

More articles

What flag.Func and the constraints package have in common

Not much, functionality-wise. But there is a connection between the two that will become clear once Carl M. Johnson explains What he worked on for Go 1.21. (Spoiler: he worked on adding flag.Func and on dropping the constraints package.)

Tired of reading lengthy articles like this?

Too late. But for an instant overview of Go 1.21, see Jan Stramer's Miro map: Go 1.21 All You Need to Know, Online Whiteboard for Visual Collaboration

Background photo for cover image by Vadim Babenko on Unsplash. Paintbrush by Clker-Free-Vector-Images from Pixabay

Got curious about learning Go or concurrency in Go? Check out Master Go and Concurrency Deep Dive. Made with 🩵 in Munich.


Categories: Releases

© 2024 Applied Go Courses