Long read · Tetrate
Tetrate
Three and a half years explaining a thing nobody can see, at a company that grew from 30 people to more than 100.
The thing nobody can see
Tetrate sells something you will never see. Explaining it was my job.
A big app today isn’t one program. It’s dozens or hundreds of small ones, calling each other over the network all day. Any of those calls can fail, crawl, or come from someone who shouldn’t be calling at all.
A service mesh is the layer that answers those questions for every program at once: who is allowed to talk to whom, what to do when a call fails, and a record of all of it. Envoy is the proxy that sits beside each program and handles its traffic. Istio tells all those proxies what to do. Tetrate’s product, Tetrate Service Bridge (TSB), sat on top and ran Istio across many clusters, many clouds and many teams.
The people in the room had built most of this. The 2020 team page lists one of Istio’s co-creators, an Envoy senior maintainer and the founder of Apache SkyWalking. Matt Klein, who wrote Envoy, was an advisor.
The titles on that page were mostly jokes. One co-founder was “Punching Bag”. Another was “El Loco”. Mine was “Divergent Psyche, Visual Designer”. Accurate, for a mechanical engineer from Odisha in a room with the people who wrote the internet’s plumbing.
My part was making the plumbing explanations people could follow: people deciding whether to buy TSB, engineers learning Istio from zero, and anyone who had been told “we’re adopting a service mesh” and nodded.
A toilet paper company, April 2020
In April 2020 the whole world was buying toilet paper. I was sketching a game about a company that makes it.
The sketches have two bars at the top, demand and capacity, and a wheel of tools at the bottom: load balancer, sidecar proxies, Istio, TSB. Most of the wheel starts locked. You split a monolith into small services, wire them up, and watch capacity try to keep up with demand. Each tool unlocks when the old way stops working. The last frame of the wireframes says “CONGRATULATIONS!!! YOU HAVE COMPLETED THE GAME.” with three exclamation marks, because it was 2020 and we all needed them.
By the end of September 2020 it was live at play.tetrate.io as Make it Mesh, and Tetrate’s homepage carried it under “Interactive Play”: Can you tame the complexity of a large, distributed system?
You arrive at TP, Inc. by email. “We’re a small toilet paper manufacturer, but we’ve seen such a surge in demand that’s resulted in an increased load on our services.” Your guide is T, from Platform Ops.
T has two monoliths, one active and one passive, and they’re drowning. You carve the monolith into microservices. You put Envoy beside them. When updating every Envoy by hand gets silly, you bring in Istio. Then somebody changes a config on the active cluster and traffic breaks, and the game asks whether you’d like to fail over.
You can say no. Every choice in Make it Mesh has a “No, continue” button, and the service health bar tells you how that’s going. Somewhere along the way, you get a post-mortem:
Hard down outage post-mortem report
Outage Duration: 8 hours
Root cause: No process in place to manage configuration changes across clusters, which allowed configurations to become out of date.
Next Steps: Review possible implementations of a management plane to exercise greater control over our infrastructure.
The next step is a management plane. By a remarkable coincidence, that’s what Tetrate sold. Install TSB and you earn the third badge, after Envoy and Istio, and the screen says MISSION ACCOMPLISHED.
Every choice had a “No, continue” button. The health bar handled the rest.
The game is still online. Six years later, T is still waiting for someone to fix the monolith.
Thirteen clips, one feature each
A game gets people curious. Then they want to see the product, and the product is a dashboard.
In April 2021 I made a set of short clips of TSB, each showing exactly one thing. Roles. Policies on a single resource. Authentication and authorisation. Workspaces, so one team can’t trample another’s settings. Configs spread over several clusters. The inventory of every service. The topology map with its traffic, its health and its numbers. Thirteen in all, from 13 to 55 seconds.
They’re silent, and they’re short on purpose. A feature clip has one job: show the thing working before the reader scrolls past it. If it needs a voiceover, it’s two features.
The app inside them is Bookinfo, Istio’s sample: a product page, reviews, ratings and details. Every Istio demo uses it. By the end of that month I knew Bookinfo’s reviews service better than most actual book reviews.
The classroom
Some people don’t want to be sold a mesh. They want to learn one. Tetrate Academy was for them: free courses and a certification, “from founders and maintainers of Istio and Envoy”.
Istio Fundamentals went up free in February 2021. In May came the exam, the Certified Istio Administrator by Tetrate. Two hours, in a real command line, on Istio 1.9.1, for $299. More than 600 people had taken the free course by then. By October the Academy listed more than five hours of video, over 3,000 people enrolled and a hundred-odd lessons.
I made the certification’s film. It puts two goals on screen, Overview of Istio and How to use Istio in practice, each with a padlock beside it. That’s the whole pitch, and it’s the right one.
The coda happened after I left. In late 2023 Tetrate gave the exam to the Linux Foundation and the CNCF, and it became the Istio Certified Associate. By then the Academy had delivered more than 16,000 courses to 13,000 people. I’m not taking credit for that. I’m just glad the lessons outlived the job.
Launch films
Releases got films. They’re the loud cousin of the silent clips: sound, motion, and a minute and a half to say what changed.
The one for TSB 1.4 draws the product as a building. TSB across the top, then a row of clusters, each running Istio with SkyWalking and Zipkin, then the workloads underneath, containers and virtual machines side by side. Then the integrations line up in labelled boxes: Venafi and AWS for security, Splunk for logs, ServiceNow for workflow, PagerDuty, xMatters and IBM Netcool for alerts, Datadog for monitoring, VMware vCenter for servers.
One of the last things I made there was the demo for Tetrate Service Express in April 2023: the fastest way to put Istio on Amazon EKS, says the product page. A login, a four-step getting-started guide, a topology, one request traced through Bookinfo. Bookinfo again. It never left.
The parts that don’t screenshot
Most of the job wasn’t films. It was documentation, and documentation doesn’t make a good figure.
I built a React navigation library for the docs platform. I worked with product and engineering on how the docs were structured and on the product’s own words, the UX writing, across TSB and TSE. I made interactive explainers, SVG and canvas animations, and small browser games, whenever a diagram wasn’t enough.
Docs are where a confused engineer lands late at night. Navigation decides whether they find the answer or close the tab. Nobody thanks the navigation. That’s how you know it works.
I joined in February 2020, remote from India. A month later, everyone else was remote too. I’d like to say I saw it coming.
Past a hundred
Tetrate grew from about 30 people to more than 100 while I was there. A company that size needs different things from one with 30, and by 2023 I wasn’t contributing what it needed. I was let go.
It stung. It was also fair, and it was clarifying. What I was good at had fit a company of 30. I was slow to notice the company had changed shape.
I left with a lot of respect for the people there, who were, and are, some of the best engineers I’ve worked near.
Three things I still use: a good explanation has a villain (in Make it Mesh, it’s an eight-hour outage). Show the thing working before you explain it. And keep asking what the place needs next, not only what you’re good at.
What I took with me
Everything I make now descends from Tetrate. DataFolks explains packets and payments to people who never asked to learn about either. serverBound is a game where the infrastructure is the world, and the problems in it play by rules you can learn.
Make it Mesh had a guide called T, a health bar and an eight-hour outage. serverBound has a Hydra that likes Spam. The systems got stranger. The job is the same one: make the invisible part something you can see, poke, and break safely.