Istio Architecture for Kubernetes: the Ultimate Guide

In a Kubernetes environment of any scale, there’s a lot going on. A typical cluster could easily be home to dozens of individual microservices, each of which is constantly exchanging data with others.

How can you ensure that data moves between all of those microservices in the way it needs to? And how can you secure communications between microservices, monitor microservices health, balance load between services and perform the myriad other tasks required to operate a microservices app effectively at scale?

The answer – conventionally, at least – is a platform like Istio. Istio is a service mesh designed to help manage microservices monitoring, authentication, security, load balancing and more. Although you don't need to use Istio – or any service mesh, for that matter – to use Kubernetes, Istio makes Kubernetes much easier to manage.

Keep reading for a breakdown of what Istio is, how the Istio architecture works, which benefits Istio offers to Kubernetes and what to watch out for when getting started with Istio virtual services in a Kubernetes cluster.

What is Istio?

Istio is an open source service mesh that helps to monitor and manage microservices. It's designed to integrate easily with microservices-based applications, collect data for them for monitoring and observability purposes and enforce various performance and security policies that admins configure.

To understand what Istio does and why it's important, it's helpful to think about the term "service mesh" and the analogy behind it. In the software world, the concept of using a mesh to unify applications services is not that old; it originated in the 2010s. However, the use of meshes to strengthen and enhance complex systems has a much longer history. For example, in the construction industry, meshes (which are sheets of wires welded together) have long been used as backings or underlayments for materials (like plaster or concrete) that would otherwise be prone to crumbling without the reinforcement that a mesh provides.

Istio does something similar for distributed, microservices-based applications: It provides a backing that helps to unify the microservices and keep them operating smoothly. You don't strictly need Istio to run a microservices app, but Istio makes microservices simpler to monitor and manage. By extension, it helps ensure that microservices apps meet performance and security requirements.

Why Istio service mesh matters for Kubernetes

At this point, you may be wondering: "If Kubernetes is an orchestration platform for containers and microservices, shouldn't it monitor and manage microservices for me?"

The answer is that, although Kubernetes does indeed orchestrate some layers of microservices apps, there are a lot of things that Kubernetes does not do. Kubernetes deals primarily with managing the deployment of microservices instances by deciding which nodes will host which Pods (each of which typically hosts one microservice). Kubernetes also provides some functionality for monitoring the health of Pods and Services (note the capital S because we're referring to a Service as Kubernetes defines it, not application services in the generic sense), and you can use tools like Kubernetes RBAC to enforce some security rules over your applications.

However, Kubernetes doesn't handle things like managing the flow of traffic between microservices, enforcing access policies to manage microservices authentication or collecting telemetry data from individual microservices. You need separate tooling to do those things – tooling that Istio provides.

Beyond addressing important monitoring and management needs for microservices apps, an additional selling point for Istio is that it can be deployed without requiring any changes to application code. In other words, you don't need to modify your applications in any way to use Istio; you just need to deploy Istio into your Kubernetes environment using sidecar containers (as we'll explain in a bit more detail below).

Istio's ability to run in virtually any type of distributed environment makes the tool even sweeter. You can use Istio whether you run applications in the cloud or on-premises. It supports every type of modern Kubernetes distribution – including managed services like EKS and self-managed clusters that admins create on their own – and it also works with other types of application orchestration platforms and all types of microservices application types, including serverless architectures.

Advantages of Istio

By choosing to leverage Istio inside Kubernetes (or any Istio-compatible platform) you gain several critical benefits:

• Security: Although Istio certainly doesn't address every aspect of application security, it can be used to enforce strong authentication and authorization requirements between microservices.
• Application performance: Istio helps to keep applications performing optimally by routing traffic efficiently to and between microservices, as well as handling retries and failovers.
• Istio observability: Istio provides a means of collecting telemetry data from individual microservices. This leads to granular visibility into the health of each microservice, as well as how the performance of one microservice impacts others. Without Istio, it would be much harder to get tracing, logging and monitoring data from a distributed application in many cases.
• Problem troubleshooting: Istio doesn't automatically resolve complex performance or security problems, but because it can monitor each microservice individually, the data it provides may play a pivotal role in your ability to identify root causes and remediate issues quickly.

In these ways and more, Istio makes life simpler for Kubernetes admins – or, again, almost anyone administering a modern, microservices-based application.

Understanding the Istio architecture

Internally, Istio's architecture includes two main components:

• The data plane, which consists of a set of proxies (which are deployed using Envoy, an open source proxy for distributed applications) that run alongside microservices as sidecar containers.
• The control plane, which manages the proxies and tells them which actions to take.

Let's look at each component in a little more detail.

The Istio data plane

If microservices were bars, the Istio control plane would be bouncers responsible for controlling what happens in each bar. That's because the control plane provides a set of proxies (again, using Envoy) that operate as sidecar containers inside each Kubernetes Pod. Because the proxies run inside the same Pods as the microservices they support, they can easily collect telemetry data from the microservices, as well as enforce security and performance policies over them. They can also intercept and manage network communication between microservices.

The Istio control plane

If Istio proxies are bouncers in a bar, then the Istio control plane is the bar manager who sets the policies that the bouncers enforce. In other words, the control plane manages the proxies and tells them which policies to enforce. It also oversees the network of microservices and provides a centralized location for exposing telemetry data.

‍

The Istio control plane includes three main components:

• Pilot: Manages the lifecycle of proxy instances deployed within the service mesh.
• Citadel: Oversees security for proxies, including authentication and encryption.
• Mixer: Governs policy checks and telemetry reporting.

On the backend, each of these components runs as its own microservice within the Istio control plane software. However, starting with the introduction of Istiod in 2020, the control plane components are now consolidated into a single binary, istiod, which operates the entire control plane.

Downsides of Istio

Istio is a powerful tool. But like many powerful tools, it comes with some caveats and potential downsides.

The main disadvantages of Istio include:

• More complexity: Kubernetes is complicated enough on its own, and adding Istio virtual to the mix makes things even more complicated. Istio becomes one more layer within your stack that you have to deploy and manage. Fortunately, Istio's sidecar deployment pattern makes the proxies relatively easy to run and operate. But at the end of the day, Istio adds complexity to your microservices application stack and infrastructure.
• Performance reduction: By creating an additional layer of abstraction between microservices, Istio can reduce application performance – especially if you enforce a large number of policies that must be processed every time a request takes place.
• Resource consumption: The Istio data plane and control plane components require compute and memory resources to operate. As a result, they increase the overall resource consumption of your Kubernetes environment, and they may increase your infrastructure costs as well.
• Learning curve: Although Istio features fairly good out-of-the-box configurations, getting the most out of the Istio gateway and service mesh requires advanced understanding of the Istio architecture and Istio tooling.
• Reduced visibility: One of the greatest potential downsides of using Istio is that Istio proxies essentially become middlemen within application traffic flows, but the proxies are not visible by monitoring application requests. This means that if service A is talking to B, you can't "see" the traces passing through Istio because you can't instrument and track Istio code easily, which in turn makes root cause analysis more difficult. If you have an application performance problem, in other words, it may be hard to know if the root cause is the application itself, or the way Istio is managing application traffic.

For a long time, Kubernetes admins tolerated these Istio disadvantages because the benefits that Istio provided – such as enhanced microservices visibility and security – outweighed the downsides, and service meshes like Istio were the only way to achieve those benefits. The downsides of Istio, in other words, were a necessary evil if you wanted to leverage the important functionality that Istio delivers.

But that's no longer the case, which brings us to our final point…

The future of service meshes

Istio can help to monitor and manage microservices applications. But the fact that it's relatively complex to use, may negatively impact application performance and can complicate performance monitoring and troubleshooting mean that Istio has clear downsides, too.

Again, in the past, admins used Istio despite its downsides because it was the only way to achieve deep visibility into microservices apps. But that's no longer true today, thanks to the emergence of eBPF.

eBPF is a tool for running sandboxed programs directly in Linux kernel space. From there, the programs have deep visibility into anything hosted on a Linux system – including individual microservices. The programs also run extremely efficiently, and because they are sandboxed, they present very low security risks.

This means that eBPF can do many of the same things as Istio, except more easily and efficiently. Rather than running Istio proxies in sidecars to monitor microservices, you could use eBPF programs.

This is why we tend to think that the future of microservices monitoring and observability will involve less Istio sidecars and more eBPF. Service meshes may continue to have their selling points – for instance, they are currently easier to deploy and manage than eBPF, thanks to more mature service mesh tooling – but going forward, Kubernetes admins won't have to choose between microservices visibility on the one hand and performance on the other. With eBPF, unlike traditional service meshes, they get the best of both worlds.