Learn Go Series (#19) - The Standard Library Tour: io, bufio, os, time, fmt

What will I learn

The two interfaces the whole standard library is built on: io.Reader and io.Writer, plus io.Copy, io.MultiWriter, and the io.ReadAll trap;
bufio.Scanner for reading input line by line, and bufio.Writer for batching output;
os essentials: command-line arguments, environment variables, and the standard streams;
time: durations, time.Since for timing, and the reference-layout formatting from episode 2, in full;
The corners of fmt worth knowing: the %v/%+v/%#v/%T verbs, width and precision, and Sscanf;
Why Go's batteries-included standard library means you reach for third-party packages far less than you would expect.

Requirements

A working modern computer running macOS, Windows or Ubuntu;
An installed Go distribution (1.27 or newer, from go.dev/dl) -- tested against Go 1.27;
Episodes 1-18, especially interfaces (episode 9) for io.Reader/io.Writer;
The ambition to learn Go programming.

Difficulty

Intermediate

Curriculum (of the `Learn Go Series`):

Learn Go Series (#19) - The Standard Library Tour: io, bufio, os, time, fmt

Go ships with an unusually capable standard library, and knowing it well is what separates people who fight the language from people who fly with it. Today is a guided tour of the packages you touch daily: io and its two foundational interfaces, bufio for buffered reading and writing, os for the world outside your program, time for durations and clocks, and the parts of fmt beyond Println. None of it is hard -- the value is knowing it is there, so you reach for the standard tool in stead of reinventing it (or worse, pulling a third-party dependency for something the library already does in ten lines). I have watched people import a whole logging framework to do what io.MultiWriter does for free, and it makes me a little sad every time. ;-) First, though, last episode's exercises.

Solutions to Episode 18 Exercises

Exercise 1 -- benchmark two search strategies:

package search

import (
	"sort"
	"testing"
)

func containsLinear(s []int, target int) bool {
	for _, v := range s {
		if v == target {
			return true
		}
	}
	return false
}

func containsBinary(s []int, target int) bool { // s must be sorted
	i := sort.SearchInts(s, target)
	return i < len(s) && s[i] == target
}

var big = func() []int {
	s := make([]int, 10000)
	for i := range s {
		s[i] = i * 2 // sorted even numbers
	}
	return s
}()

func BenchmarkLinear(b *testing.B) {
	for i := 0; i < b.N; i++ {
		_ = containsLinear(big, 99999) // missing (odd)
	}
}

func BenchmarkBinary(b *testing.B) {
	for i := 0; i < b.N; i++ {
		_ = containsBinary(big, 99999)
	}
}

Binary search wins enormously on a large sorted slice -- O(log n) versus O(n) -- which the ns/op figures make obvious. The key insight: the algorithm dominates, not the micro-tuning. No amount of clever looping makes the linear scan competitive once the slice is big enough.

Exercise 2 -- catch a race (fixed version):

package main

import (
	"fmt"
	"sync"
)

func main() {
	var mu sync.Mutex
	var shared []int
	var wg sync.WaitGroup
	for i := 0; i < 100; i++ {
		wg.Add(1)
		go func(n int) {
			defer wg.Done()
			mu.Lock()
			shared = append(shared, n) // guarded: no race
			mu.Unlock()
		}(i)
	}
	wg.Wait()
	fmt.Println("appended", len(shared), "items") // 100, and -race is clean
	// Without the lock, -race reports a race on the slice header at the append line.
}

The mutex serialises the append, so the slice header is only ever mutated by one goroutine at a time. Run it under go run -race and the tool is silent; remove the lock and it screams with the exact file and line of both racing accesses.

Exercise 3 -- profile a hot function. The benchmark to profile with -cpuprofile:

package strutil

import "testing"

func buildWithPlus(n int) string {
	s := ""
	for i := 0; i < n; i++ {
		s += "x" // the wasteful, reallocating line the profiler highlights
	}
	return s
}

func BenchmarkBuildWithPlus(b *testing.B) {
	for i := 0; i < b.N; i++ {
		_ = buildWithPlus(1000)
	}
}

Run go test -bench=. -cpuprofile=cpu.out then go tool pprof cpu.out, and list buildWithPlus points straight at the s += "x" line. The fix (a strings.Builder) is a story for later -- the point here was letting the profiler tell you where the time went in stead of guessing. Now, the tour.

io.Reader and io.Writer, the two great interfaces

Nearly all input and output in Go flows through two one-method interfaces: io.Reader (Read bytes into a buffer) and io.Writer (Write bytes out). Files, network connections, in-memory buffers, HTTP request and response bodies, gzip streams, and the standard streams all implement them, which is why the same code works across every source and sink you will ever meet. io.Copy streams from any reader to any writer without loading the whole thing into memory:

package main

import (
	"fmt"
	"io"
	"os"
	"strings"
)

func main() {
	r := strings.NewReader("streamed through io.Copy\n") // an io.Reader
	n, err := io.Copy(os.Stdout, r)                       // to an io.Writer
	if err != nil {
		fmt.Println("error:", err)
		return
	}
	fmt.Printf("(copied %d bytes)\n", n)
}

strings.NewReader gives you an io.Reader over a string, os.Stdout is an io.Writer, and io.Copy shuttles bytes between them in fixed-size chunks. Because these are interfaces (remember episode 9 -- satisfaction is implicit, no implements keyword), that exact same io.Copy copies a 10-gigabyte file to a network socket with constant memory. Learn to think in readers and writers and huge amounts of the standard library falls into place at once.

The io package has a few more helpers worth having in your head. io.MultiWriter fans a single write out to several destinations -- print to the screen and capture into a buffer with one call, which is the entire trick behind most "tee" and logging setups:

package main

import (
	"bytes"
	"fmt"
	"io"
	"os"
	"strings"
)

func main() {
	var buf bytes.Buffer
	w := io.MultiWriter(os.Stdout, &buf) // one write, two destinations

	io.Copy(w, strings.NewReader("logged and printed\n"))
	fmt.Printf("(buffer also captured %d bytes)\n", buf.Len())
}

Nota bene: there is also io.ReadAll(r), which slurps an entire reader into a []byte. It is convenient, and for a small config file it is perfectly fine -- but it is the exact opposite of the streaming philosophy above. Call it on a multi-gigabyte body and you have loaded a multi-gigabyte body into RAM. Having said that, when the input is genuinely small and you need it all in memory anyway, do not feel guilty about reaching for it. Know both, and pick on purpose.

bufio: buffered reading and writing

Reading input a line at a time is so common it has a dedicated tool: bufio.Scanner. Wrap any io.Reader, then loop while Scan() returns true, calling Text() for each line (with the newline stripped). It handles the buffering and the splitting for you:

package main

import (
	"bufio"
	"fmt"
	"strings"
)

func main() {
	input := "line one\nline two\nline three"
	scanner := bufio.NewScanner(strings.NewReader(input))

	n := 0
	for scanner.Scan() { // one line per iteration
		n++
		fmt.Printf("%d: %s\n", n, scanner.Text())
	}
	if err := scanner.Err(); err != nil { // check for a read error at the end
		fmt.Println("scan error:", err)
	}
}

Scan returns false at end of input or on error, so you check scanner.Err() afterward to tell the two apart. Point the scanner at os.Stdin in stead of a string and you have read standard input line by line -- the backbone of countless CLI tools. You can also switch its split function to words or runes with scanner.Split(bufio.ScanWords). One gotcha to file away: the default line buffer maxes out around 64KB, so for pathologically long lines you call scanner.Buffer(...) to raise the ceiling.

The mirror image is bufio.Writer. Every raw Write to a file or socket is a system call, and system calls are expensive; if you write a million tiny lines one at a time you make a million syscalls. A buffered writer batches those small writes in memory and flushes them in big chunks. The one rule you must never forget: you have to Flush, or the last bufferful never reaches the destination.

package main

import (
	"bufio"
	"fmt"
	"os"
)

func main() {
	w := bufio.NewWriter(os.Stdout)
	defer w.Flush() // MUST flush, or buffered bytes are silently lost

	for i := 1; i <= 3; i++ {
		fmt.Fprintf(w, "line %d\n", i) // batched in memory, not one syscall each
	}
}

The defer w.Flush() on the line right after you create the writer is the idiom -- it guarantees the flush happens no matter how the function returns. Forget it and you get the classic beginner bug: a program that runs cleanly, exits zero, and produces an empty file. I have debugged that one for someone at least three times, and it is always the missing Flush. ;-)

os: arguments, environment, and streams

The os package is your program's window on the outside world. os.Args holds the command-line arguments (index 0 is the program itself), os.Getenv/os.LookupEnv read environment variables (the LookupEnv form distinguishes "empty" from "unset"), and os.Stdin/Stdout/Stderr are the standard streams -- which, not by accident, are *os.File values that implement the reader and writer interfaces we just met:

package main

import (
	"fmt"
	"os"
)

func main() {
	fmt.Println("program:", os.Args[0])
	fmt.Println("arg count:", len(os.Args))

	os.Setenv("GREETING", "hello")
	if v, ok := os.LookupEnv("GREETING"); ok {
		fmt.Println("GREETING =", v)
	}
	if _, ok := os.LookupEnv("DEFINITELY_UNSET_12345"); !ok {
		fmt.Println("that variable is not set")
	}
}

os.LookupEnv returns a value and a bool -- the same comma-ok shape as a map read back in episode 6 -- so you can react to a missing variable rather than treating unset and empty as the same thing. That distinction matters more than it looks: LOG_LEVEL="" (set but empty) and no LOG_LEVEL at all are genuinely different situations, and Getenv alone cannot tell them apart. For real command-line flags you graduate to the flag package (or a third-party library), which we will use when we build CLI tools later in the series; os.Args is the raw layer underneath it all. And os.Exit(code) ends the program immediately with a status code -- though prefer returning from main where you can, because os.Exit skips every pending defer, including that Flush we just made such a fuss about.

time: durations and the reference layout

The time package models two things: an instant (time.Time) and a span (time.Duration). Durations are typed integers of nanoseconds, so you write them readably as 90 * time.Minute and do honest arithmetic on them. time.Since(start) is the idiomatic way to measure elapsed time, and formatting uses the reference-layout trick from episode 2 -- you spell out the date Mon Jan 2 15:04:05 2006 in the shape you want:

package main

import (
	"fmt"
	"time"
)

func main() {
	d := 90 * time.Minute
	fmt.Println("duration:", d)            // 1h30m0s
	fmt.Printf("in seconds: %.0f\n", d.Seconds())

	start := time.Now()
	time.Sleep(5 * time.Millisecond)
	fmt.Println("elapsed >= 5ms:", time.Since(start) >= 5*time.Millisecond)

	t := time.Date(2026, time.September, 11, 14, 30, 0, 0, time.UTC)
	fmt.Println(t.Format("Mon 2006-01-02 15:04")) // Fri 2026-09-11 14:30
}

A Duration prints itself in a human form (1h30m0s), and d.Seconds()/d.Milliseconds() convert it to a plain number when you need one. time.Since(start) returns the duration since an earlier instant -- the standard timing idiom, and exactly what a benchmark harness does under the hood. The layout string "Mon 2006-01-02 15:04" reads oddly until you remember it is an example of the format using that one fixed reference date (01/02 03:04:05PM '06 -7:00, if you line the numbers up), not a set of cryptic codes like most languages use. Time zones matter too: time.UTC versus time.Local changes the printed result, so be explicit about which one you mean, especially for anything you store or send over the wire.

fmt: the verbs beyond Println

You have used fmt all series; here are the parts worth knowing deliberately. The formatting verbs %v (default), %+v (with field names), %#v (Go syntax), and %T (the concrete type) are your debugging toolkit. Width and precision (%05.2f) control layout. And Sscanf parses formatted text back into values -- the inverse of Printf:

package main

import "fmt"

type Point struct{ X, Y int }

func main() {
	p := Point{3, 4}
	fmt.Printf("%v | %+v | %#v | %T\n", p, p, p, p)
	// {3 4} | {X:3 Y:4} | main.Point{X:3, Y:4} | main.Point

	s := fmt.Sprintf("%05.2f", 3.14159) // width 5, 2 decimals, zero-padded
	fmt.Println(s)                       // 03.14

	var a, b int
	fmt.Sscanf("12-34", "%d-%d", &a, &b) // parse into a and b
	fmt.Println(a, b, a+b)               // 12 34 46
}

%+v and %#v are the two you will lean on most when debugging a struct -- the first when you just want to see the field values, the second when you want something you could paste back into code. And %T answers "what type is this actually", which is invaluable the moment you are holding an interface value and are not sure what is inside it. Sprintf builds a string in stead of printing (use it, not manual concatenation, for formatted strings), and Sscanf/Fscanf parse simple structured text. For anything beyond simple parsing you reach for strconv, strings, or a proper parser -- but for quick-and-dirty structured input like "12-34", Sscanf is genuinely handy and saves you a fiddly split-and-convert dance.

The same idea in Python

Python's standard library is also large, and the mappings are direct: file objects are Python's readers/writers, sys.argv is os.Args, os.environ is the environment, datetime/time cover time, and f-strings plus str.format cover fmt. The philosophies rhyme -- both languages pride themselves on batteries included:

import os, sys, datetime

for i, line in enumerate(sys.stdin, 1):  # like bufio.Scanner over os.Stdin
    print(f"{i}: {line.rstrip()}")

print("args:", sys.argv)
print("home:", os.environ.get("HOME", "unset"))  # like os.LookupEnv with a default
print(datetime.datetime(2026, 9, 11, 14, 30).strftime("%a %Y-%m-%d %H:%M"))

Python uses strftime codes (%Y-%m-%d) where Go uses the reference layout, and Python's readers are duck-typed file objects rather than an explicit io.Reader interface you can name in a signature -- but the shape of "read lines, read args, read env, format time" is the same in both. Go's version leans harder on those two named interfaces (io.Reader/io.Writer everywhere), which is precisely what makes its I/O compose so uniformly: any function that takes an io.Reader will happily accept a file, a socket, a string, or a gzip stream without knowing or caring which.

Exercises

A line counter. Read from a strings.Reader (or os.Stdin) with a bufio.Scanner and print both the total number of lines and the number of non-empty lines. Use strings.TrimSpace to decide what counts as "non-empty".
An env-var default. Write func envOr(key, fallback string) string that returns the environment variable's value if it is set (via os.LookupEnv) and the fallback otherwise. Show it returning both a set value and a fallback in main.
A tiny stopwatch. Write a function that runs a piece of work (say, a loop that sums to a million), measures how long it took with time.Now and time.Since, and prints the elapsed duration with %v. Then format the current time with a layout of your own choosing.

What we learned

io.Reader and io.Writer are the two one-method interfaces the whole library is built on; io.Copy streams from any reader to any writer with constant memory, io.MultiWriter fans one write out to several destinations, and io.ReadAll is the convenient-but-greedy option to use only on small inputs;
bufio.Scanner reads input line by line (Scan/Text), stripping newlines, and you check scanner.Err() afterward to distinguish end-of-input from a read error; bufio.Writer batches many small writes -- just never forget to Flush;
os gives you os.Args, os.Getenv/LookupEnv (comma-ok for set-vs-unset), and the Stdin/Stdout/Stderr streams, which are themselves readers and writers;
time models durations (90 * time.Minute, d.Seconds()) and instants; time.Since(start) times things, and formatting uses the reference layout Mon Jan 2 15:04:05 2006;
fmt's %v/%+v/%#v/%T verbs are your debugging toolkit, width/precision control layout, and Sscanf parses simple formatted text back into values;
Go's standard library is broad enough that you reach for third-party packages far less than in many ecosystems -- knowing it well is a real productivity multiplier, and quit a satisfying one at that.

Next episode we work with structured data: JSON encoding and decoding, the struct tags that control field names and options, and a look at how the xml and gob encoders share the very same design. See you there.

Thanks for reading, and happy hacking! ;-)

@scipio

Learn Go Series (#19) - The Standard Library Tour: io, bufio, os, time, fmt

Learn Go Series (#19) - The Standard Library Tour: io, bufio, os, time, fmt

What will I learn

Requirements

Difficulty

Curriculum (of the `Learn Go Series`):

Learn Go Series (#19) - The Standard Library Tour: io, bufio, os, time, fmt

Solutions to Episode 18 Exercises

io.Reader and io.Writer, the two great interfaces

bufio: buffered reading and writing

os: arguments, environment, and streams

time: durations and the reference layout

fmt: the verbs beyond Println

The same idea in Python

Exercises

What we learned

Comments · 2

More from @scipio

	Feedback from the October Hive Power Up Day
	Hive Power Up Month Challenge - September 2026 Winners List
	Be ready for the October edition of the Hive Power Up Month!

Learn Go Series (#19) - The Standard Library Tour: io, bufio, os, time, fmt

What will I learn

Requirements

Difficulty

Curriculum (of the Learn Go Series):

Learn Go Series (#19) - The Standard Library Tour: io, bufio, os, time, fmt

Solutions to Episode 18 Exercises

io.Reader and io.Writer, the two great interfaces

bufio: buffered reading and writing

os: arguments, environment, and streams

time: durations and the reference layout

fmt: the verbs beyond Println

The same idea in Python

Exercises

What we learned

Comments · 2

More from @scipio

Curriculum (of the `Learn Go Series`):