Build Production-Ready Operators with client-go, controller-runtime, CRDs, Webhooks, and envtest

Read before you buy: the PDF here is a free sample — the first 20 pages of the 482-page book (payments, sales and reviews on this demo site are sample data). Get the full e-book at thenexbooks on Gumroad.

Build Kubernetes operators that behave correctly when real clusters misbehave — tested against a real API server, observable, secure, and safe to upgrade.

Every Deployment, Service, and Job in Kubernetes is handled by a controller: a small program that watches desired state and keeps nudging the real world until it matches. You can write your own. You can teach the API server new types of objects with Custom Resource Definitions and build controllers that understand them. That is what an operator is.

Writing a controller that works in a demo is easy. Writing one that stays correct when the cache is stale, an update conflicts, an external API is down, two replicas start at once, or an object is deleted mid-rollout takes a real understanding of how the Kubernetes API, client-go, and controller-runtime behave. This book teaches that understanding from the ground up and turns it into habits you can apply to any operator.

What's inside

• How the Kubernetes API really works: groups, versions, kinds, resources, resourceVersion, generation, subresources, and optimistic concurrency
• API machinery in Go: runtime.Object, schemes, ObjectMeta, unstructured objects, and serialization
• client-go: clientsets, the dynamic client, informers, listers, indexers, and rate-limited workqueues — including a complete controller written with client-go alone
• Custom Resource Definitions and API design: spec vs status, conditions, optional fields, references, and immutable fields
• Go types, Kubebuilder markers, and controller-gen for deepcopy code, CRDs, and RBAC
• Schema validation, defaults, and CEL validation rules, including transition rules
• controller-runtime in depth: the manager, cached client, reconcile loops, CreateOrUpdate, owner references, status patching, server-side apply, and finalizers for external resources
• Watches, predicates, map functions, and field indexes
• Defaulting and validating admission webhooks, multiple API versions, deprecation, and conversion webhooks with lossless round trips
• Unit tests with the fake client and integration tests with envtest against a real kube-apiserver and etcd
• Structured logging, Kubernetes events, and Prometheus metrics
• Least-privilege RBAC, hardened pods, leader election, graceful shutdown, and performance at scale
• Packaging with manifests, Helm charts, and OLM bundles, plus safe upgrades of operators and CRDs

One running example, start to finish. You join the platform team at Cobblestack Commerce, a fictional company that hosts online storefronts for independent shops. Together you design a Storefront custom resource and build the storefront operator: it creates each shop's configuration, Deployment, and Service, registers a route on the company's edge router, reports honest status with conditions, cleans up with a finalizer, validates and defaults new objects, converts an older v1alpha1 API to v1, and ships with tests, metrics, and an upgrade plan.

Current and honest. The reference project was built and tested with Go 1.27, the Kubernetes 1.36 client libraries, controller-runtime v0.24, and controller-gen v0.21, and its tests run against real Kubernetes 1.36 control-plane binaries with envtest. Output that needs a full cluster with running pods is clearly marked. There are no invented benchmarks or statistics. Every chapter includes runnable code, Tips, Notes, Warnings, Try It Yourself exercises with selected solutions, and Key Takeaways.

Format: PDF, 482 pages (A4), linked table of contents and bookmarks. English. First Edition, 2026. Published by thenexbooks. Includes the storefront operator starter project (Go module) that the book builds on. This book is independent and is not affiliated with or endorsed by The Linux Foundation, the Cloud Native Computing Foundation, the Kubernetes project, or Google.

Table of Contents

Part I — The Kubernetes API and client-go

• Ch 1 · Operators and Controllers: The Reconciliation Pattern
• Ch 2 · The Kubernetes API Model: Groups, Versions, Kinds, Resources, and Objects
• Ch 3 · API Machinery in Go: runtime.Object, Schemes, ObjectMeta, and Serialization
• Ch 4 · client-go Essentials: rest.Config, Clientsets, Dynamic Clients, and Typed Calls
• Ch 5 · Informers, Listers, Indexers, and Workqueues: A Controller with client-go Alone

Part II — Designing Custom Resources

• Ch 6 · Custom Resource Definitions: Schemas, Scope, Subresources, and Printer Columns
• Ch 7 · Designing a Good Kubernetes API: Spec, Status, and the Storefront Resource
• Ch 8 · Go Types, Markers, and controller-gen: Deepcopy, CRDs, and RBAC from Code
• Ch 9 · Schema Validation and Defaulting: OpenAPI Constraints and CEL Rules
• Ch 10 · The Operator Project: A Kubebuilder-Style Layout Built by Hand

Part III — Building Controllers with controller-runtime

• Ch 11 · Inside controller-runtime: Manager, Client, Cache, and Scheme
• Ch 12 · Writing the Reconcile Loop: Level-Triggered, Idempotent, and Retry-Safe
• Ch 13 · Owning Child Objects: CreateOrUpdate, Owner References, and Garbage Collection
• Ch 14 · Status, Conditions, and Patching: observedGeneration, Update, Patch, and Server-Side Apply
• Ch 15 · Finalizers and External Resources: Cleaning Up Safely

Part IV — Watches, Webhooks, and API Evolution

• Ch 16 · Watches, Predicates, Event Handlers, and Field Indexes
• Ch 17 · Admission Webhooks: Defaulting and Validation in Go
• Ch 18 · Versioning a Custom Resource: Multiple Versions, Storage Versions, and Deprecation
• Ch 19 · Conversion Webhooks: Hub and Spoke and Lossless Round Trips

Part V — Testing and Observability

• Ch 20 · Unit Testing Reconcilers with the Fake Client
• Ch 21 · Integration Testing with envtest
• Ch 22 · Testing Webhooks, Conversion, and Failure Paths
• Ch 23 · Observability: Structured Logging, Events, and Prometheus Metrics

Part VI — Running Operators in Production

• Ch 24 · RBAC and Security: Least Privilege, Service Accounts, and Hardened Pods
• Ch 25 · Leader Election, High Availability, and Graceful Shutdown
• Ch 26 · Performance at Scale: Caches, Concurrency, Rate Limits, and API Server Load
• Ch 27 · Packaging and Distribution: Manifests, Helm Charts, and OLM Bundles
• Ch 28 · Upgrades and Day-2 Operations: Releasing New Versions Safely

Appendices
A · Cheat Sheet: client-go, controller-runtime, Markers, and kubectl
B · Templates and Reference: Manifests, Markers, Tests, and Build Targets
C · Production Readiness Checklist for Operators
D · Glossary

New to containers or Kubernetes itself? The companion book Docker and Kubernetes for Developers covers images, Deployments, Services, Helm, and cluster operations; this book starts where it ends.

Book details