Istio Architecture for Kubernetes: The Ultimate Guide

In a Kubernetes environment of any scale, there’s a lot going on. A typical cluster could easily be home to dozens of individual microservices, each of which is constantly exchanging data with others.

How can you ensure that data moves between all of those microservices in the way it needs to? And how can you secure communications between microservices, monitor microservices health, balance load between services, and perform the myriad other tasks required to operate a microservices app effectively at scale?

The answer – conventionally, at least – is a platform like Istio. Istio is a service mesh designed to help manage microservices monitoring, authentication, security, load balancing, and more. Although you don't need to use Istio – or any service mesh, for that matter – to use Kubernetes, Istio makes Kubernetes much easier to manage.

Keep reading for a breakdown of what Istio is, how the Istio architecture works, which benefits Istio offers to Kubernetes, and what to watch out for when getting started with Istio virtual services in a Kubernetes cluster.

What is Istio?

Istio is an open source service mesh that helps to monitor and manage microservices. It's designed to integrate easily with microservices-based applications, collect data for them for monitoring and observability purposes, and enforce various performance and security policies that admins configure.

To understand what Istio does and why it's important, it's helpful to think about the term "service mesh" and the analogy behind it. In the software world, the concept of using a mesh to unify applications services is not that old; it originated in the 2010s. However, the use of meshes to strengthen and enhance complex systems has a much longer history. For example, in the construction industry, meshes (which are sheets of wires welded together) have long been used as backings or underlayments for materials (like plaster or concrete) that would otherwise be prone to crumbling without the reinforcement that a mesh provides.

Istio does something similar for distributed, microservices-based applications: It provides a backing that helps to unify the microservices and keep them operating smoothly. You don't strictly need Istio to run a microservices app, but Istio makes microservices simpler to monitor and manage. By extension, it helps ensure that microservices apps meet performance and security requirements.

Why Istio service mesh matters for Kubernetes

At this point, you may be wondering: "If Kubernetes is an orchestration platform for containers and microservices, shouldn't it monitor and manage microservices for me?"

The answer is that, although Kubernetes does indeed orchestrate some layers of microservices apps, there are many things that Kubernetes does not do. Kubernetes deals primarily with managing the deployment of microservices instances by deciding which nodes will host which Pods (each of which typically hosts one microservice). Kubernetes also provides some functionality for monitoring the health of Pods and Services (note the capital S because we're referring to a Service as Kubernetes defines it, not application services in the generic sense), and you can use tools like Kubernetes RBAC to enforce some security rules over your applications.

However, Kubernetes doesn't handle things like managing the flow of inbound and outbound traffic between microservices, enabling rich routing rules to provide granular control over traffic management or collecting network telemetry data from individual microservices. You need separate tooling to do those things – tooling that Istio provides.

Beyond addressing important monitoring and management needs for microservices apps, an additional selling point for Istio is that it can be deployed without requiring any changes to application code. In other words, you don't need to modify your applications in any way to use Istio; you just need to deploy Istio into your Kubernetes environment using sidecar containers (as we'll explain in a bit more detail below).

Istio's ability to run in virtually any type of distributed environment makes the tool even sweeter. You can use Istio whether you run applications in the cloud or on-premises. It supports every type of modern Kubernetes distribution – including managed services like EKS and self-managed clusters that admins create on their own – and it also works with other types of application orchestration platforms and all types of microservices application types, including serverless architectures.

Understanding the Istio architecture and components

Internally, Istio's architecture includes two main components: A data plane and a control plane. Each of these parts of the Istio architecture can be further broken into sub-components.

Let’s take a closer look by examining the Istio data and control planes in detail.

1. The Istio data plane (Envoy)

In Istio, the data plane is responsible for traffic management between different workload instances.

The main component within the data plane is Envoy, a proxy service that is responsible for service discovery, load balancing, and traffic management within the Istio service mesh. By operating a proxy instance alongside each microservice, Envoy provides a variety of capabilities for data plane traffic management and telemetry, including:

• Service discovery, which allows microservices to identify each other on the network.
• Load balancing, to distribute network requests strategically between microservices.
• TLS termination, which decrypts network traffic to make it readable to the appropriate microservices.
• Health checks, for monitoring the availability of microservices and ensuring they are reachable on the network.
• The collection of telemetry data to provide visibility into the health and responsiveness of microservices.

The Envoy proxies are deployed via the sidecar model. This means that an Envoy instance runs in its own container within each Pod that exists within the service mesh. Because the proxies run alongside the containers that host application microservices, they can enforce network and security policies for the microservices, while also collecting telemetry data from them.

Note that it is possible to run Istio without Envoy. Doing so means you won’t be able to enforce networking or security policies that you configure within Istio, although your microservices will still be able to communicate (provided they can reach each other over the network). This is not a common approach, but it may make sense in situations where you don’t need to enforce network or security policies and want to avoid the overhead of having to deploy Envoy sidecar containers (which may reduce the overall resource efficiency of your cluster because they require CPU and memory to run).

In most situations, though, you’d set up a data plane using Envoy.

2. The Istio control plane (Istiod)

The control plane is responsible for determining which networking and security policies to enforce within the service mesh. Put another way, it’s the “boss” that tells the data plane what to do.

The main component of the control plane is Istiod, which provides three key categories of capabilities:

1. Proxy lifecycle management: This is powered by a part of Istiod called Pilot. Pilot monitors the high-level routing rules and security policies configured within Istio, and then translates them into instructions for each proxy sidecar.  
2. Security management: Using a sub-component called Citadel, Istiod oversees security policies for Envoy proxies. More specifically, Citadel handles tasks like user authentication, certificate management, and ensuring the encryption of network data.
3. Infrastructure isolation: Mixer is the control plane component that isolates applications from backend infrastructure. It does this by operating as an intermediary between applications and the infrastructure hosting them, controlling their behavior in ways that align with the policies configured in Istio. Mixer is also responsible for collecting telemetry data associated with service calls, and it makes it possible to enforce access control.

Historically, Pilot, Citadel, and Mixer were distinct parts of Istio. But with the introduction of Istiod in 2020, they were consolidated into a single binary – Istiod – which is a single service that operates the control plane.

How Istio cluster configurations work

One of the cool things about Istio is that it’s capable of supporting multi-cluster Kubernetes setups. This means it can provide a single service mesh that connects microservices hosted in different Kubernetes clusters.

Let’s explore what this means by comparing single-cluster to multi-cluster meshes.

Single-cluster mesh

A single-cluster Istio mesh is the simpler and more common approach. It means that all of the data plane and control plane components operate in the same Kubernetes cluster. You can assume in this case that there will be minimal network latency, and that tasks like load balancing and service discovery will be relatively straightforward, since everything runs in the same cluster.

Multi-cluster mesh

In a multi-cluster mesh, one Istio service mesh connects services running in two or more Kubernetes 

clusters. You can implement a multi-cluster mesh by deploying a single control plane within one cluster and configuring that control plane to manage Envoy proxies running across all of the clusters you want to control. You can also deploy a different control plane instance within each cluster.

The main reason why you’d create a multi-cluster mesh is that you want to enforce a central set of networking and security policies across microservices that exist in different clusters. The ability to do this via a multi-cluster mesh is easier than having to create a different mesh for each cluster, and then manage networking and security configurations within each mesh separately.

While a multi-cluster mesh can simplify service management by centralizing it, it also presents some challenges, such as:

• Network latency: Network latency between clusters can be high, especially if the clusters are located in geographically disparate places. High latency may result in slow communication between the control plane and the data plane if the control plane is located in a different cluster from the proxies it’s managing. Data plane traffic could also be slow in cases where one service is trying to communicate with another service in a different cluster.
• Service discovery issues: Network topology complexities may make it challenging or impossible for services in different clusters to perform service discovery for each other. For example, one service may be located on a subnet that is not reachable from another cluster due to firewall rules. Or, tracking service identity in a continuous way could be problematic if a service is only intermittently reachable.
• Increased overhead: If you deploy a separate control plane instance in each cluster, you end up with more control planes, leading to higher overhead.
• Monitoring and troubleshooting complexity: Having multiple clusters translates to more variables to consider when monitoring and troubleshooting performance issues. For example, if a microservice is not adhering to a policy you’ve configured, it could be because of a problem with Istio, but it could also be due to networking problems that are preventing the control plane from communicating reliably with the service.

The best way to mitigate these challenges is to simplify your multi-cluster architecture and network topology as much as possible. When setting up each Kubernetes cluster and the backend infrastructure behind it, avoid unnecessary firewall rules or inconsistent networking settings. Sticking with a single control plane can also simplify multi-cluster meshes in some respects because it means you only have one control plane to manage, although the tradeoff is that you may experience higher latency rates if the connection between the control plane and remote clusters is poor.

Istio architecture for virtual machines

Another excellent Istio capability is support for connecting virtual machines to a service mesh. You can do this even if the virtual machines are located outside of the Kubernetes cluster (or clusters) that your mesh manages.

This feature is useful for use cases where you have applications that you can’t or don’t want to run using Kubernetes for some reason, but that you still want to be able to connect with microservices hosted inside Istio. For example, you might have a legacy app that can’t be containerized, but that needs to exchange data with other, containerized apps. Note that another solution in this case could be to use a component like KubeVirt, which lets you run VMs directly inside a Kubernetes cluster and which is compatible with Istio; however, you can also simply use Istio to connect to VMs that are external to your cluster.

There are two ways to implement an Istio network architecture for connecting virtual machines: Single-network and multi-network setups.

Single-network

In a single-network configuration, pods inside your Kubernetes cluster and virtual machines hosted outside of it can communicate with each other directly for the most part. Certain types of traffic – namely, requests associated with Istio’s extensible discovery service (xDS) protocol and certificate signing requests – flow through an Istio gateway (which is a load balancer that sits at the “edge” of the mesh. Otherwise, however, VMs and pods can communicate using a single, shared network.

A single-network setup is the simplest to implement. It works well when your pods and VMs are already on the same network, and when you don’t need to isolate workloads at the network level for security reasons.

Multi-network

Under a multi-network configuration, you operate one network for your pods and another one for your VMs, with a gateway serving as an intermediary between them. All network requests between pods and VMs need to flow through the gateway.

A multi-network configuration is more complex, and it may reduce performance because requests will typically take a bit longer to route through the gateway than they would if they flowed between pods and VMs directly. However, multi-network architectures make it possible to isolate pods and VMs at the network level, which can provide security benefits. It’s also the only way to use Istio for managing VMs when the VMs are on an independent, external network – which is likely to be the case if, for example, they are in a different data center or cloud than your Kubernetes cluster.

Benefits of Istio

By choosing to leverage Istio inside Kubernetes (or any Istio-compatible platform) you gain several critical benefits:

• Security: Although Istio certainly doesn't address every aspect of application security, it can be used to enforce strong authentication and authorization requirements between microservices.
• Application performance and traffic management: Istio helps to keep applications performing optimally by routing packets efficiently to and between microservices based on traffic behavior, as well as handling retries and failovers.
• Istio observability: Istio provides a means of collecting telemetry data from individual microservices. This leads to granular visibility into the health of each microservice, as well as how the performance of one microservice impacts others. Without Istio, it would be much harder to get tracing, logging, and monitoring data from a distributed application in many cases.
• Problem troubleshooting: Istio doesn't automatically resolve complex performance or security problems, but because it can monitor each microservice individually, the data it provides may play a pivotal role in your ability to identify root causes and remediate issues quickly.

In these ways and more, Istio makes life simpler for Kubernetes admins – or, again, almost anyone administering a modern, microservices-based application.

Limitations of Istio

Istio is a powerful tool. But like many powerful tools, it comes with some caveats and potential downsides.

The main disadvantages of Istio include:

• More complexity: Kubernetes is complicated enough on its own, and adding Istio to the mix makes things even more complicated. Istio becomes one more layer within your stack that you have to deploy and manage. Fortunately, Istio's sidecar deployment pattern makes the proxies relatively easy to run and operate. But at the end of the day, Istio adds complexity to your microservices application stack and infrastructure.
• Performance reduction: By creating an additional layer of abstraction between microservices, Istio can reduce application performance – especially if you enforce a large number of policies that must be processed every time a request takes place.
• Resource consumption: The Istio data plane and control plane components require compute and memory resources to operate. As a result, they increase the overall resource consumption of your Kubernetes environment, and they may increase your infrastructure costs as well.
• Learning curve: Although Istio features fairly good out-of-the-box configurations, getting the most out of the Istio gateway and service mesh requires an advanced understanding of the Istio architecture and Istio tooling.
• Reduced visibility: One of the greatest potential downsides of using Istio is that Istio proxies essentially become middlemen within application traffic flows, but the proxies are not visible by monitoring application requests. This means that if service A is talking to B, you can't "see" the traces passing through Istio because you can't instrument and track Istio code easily, which in turn makes root cause analysis more difficult. If you have an application performance problem, in other words, it may be hard to know if the root cause is the application itself, or the nature of Istio traffic management.

For a long time, Kubernetes admins tolerated these Istio disadvantages because the benefits that Istio provided – such as enhanced microservices visibility and security – outweighed the downsides, and service meshes like Istio were the only way to achieve those benefits. The downsides of Istio, in other words, were a necessary evil if you wanted to leverage the important functionality that Istio delivers.

But that's no longer the case, which brings us to our final point…

The future of service meshes

Istio can help to monitor and manage microservices applications. But the fact that it's relatively complex to use, may negatively impact application performance, and can complicate performance monitoring and troubleshooting mean that Istio has clear downsides, too.

Again, in the past, admins used Istio despite its downsides because it was the only way to achieve deep visibility into microservices apps. But that's no longer true today, thanks to the emergence of eBPF.

eBPF is a tool for running sandboxed programs directly in Linux kernel space. From there, the programs have deep visibility into anything hosted on a Linux system – including individual microservices. The programs also run extremely efficiently, and because they are sandboxed, they present very low security risks. This means that eBPF can do many of the same things as Istio, except more easily and efficiently. Rather than running Istio proxies in sidecars to monitor microservices, you could use eBPF programs.

This is why we tend to think that the future of microservices monitoring and observability will involve less Istio sidecars and more eBPF. Service meshes may continue to have their selling points – for instance, they are currently easier to deploy and manage than eBPF, thanks to more mature service mesh tooling – but going forward, Kubernetes admins won't have to choose between microservices visibility on the one hand and performance on the other. With eBPF, unlike traditional service meshes, they get the best of both worlds.