How Similarweb increases in-org Kubernetes adoption & improves its K8s observability with groundcover

‍Background on Similarweb‍

Similarweb Ltd. (SMWB (NYSE)) is a web analytics company with over 1,000 employees that develops tools that enable the analysis of the traffic and behavior of users on websites and apps. The service and datasets are provided in a limited free edition, but the paid platform is addressed to SMBs and large companies which require access to accurate comprehensive data at larger scales for marketing, sales and market research. The data is collected from a number of different sources that provide information about the internet usage of users, including various information partners, and anonymous data from users of the various dedicated browser addons that the company distributes.

Similarweb provides insights for over:

• 100M websites • 4M apps • 190 countries • 1B Search Terms

Challenges

Similarweb is a big data company with aggregation from many different sources flowing in large volumes of data that needs to be analyzed and queried at high scale. Similarweb is internally built from many different services, and a complex tech stack that spans across many code languages and platforms, posing a challenge for any monitoring and observability solution. Similarweb used containerized Nomad over the past couple years and recently began migrating its workload to Kubernetes.

Similarweb hosts many kinds of applications in their Kubernetes environment, some of them being open-source tools - each with its own  different way of sending logs & metrics. Besides its main applications Similarweb have many infra services which run as daemonSets and other infra components in need of monitoring and till now, the team had to handle each one of them manually.
With its growing migration to AWS’s EKS managed K8s solution, Similarweb faces new challenges that require monitoring the K8s infrastructure layer and providing the right K8s context to development teams that are not proficient within this new deployment method.

The Similarweb team sought out a fast, efficient, safe way to provide observability to their Kubernetes stack  with zero integration and seamless ease of use for both the production engineering teams as well as the developer teams.  

Solutions & Benefits

Similarweb have been running groundcover on all production and staging K8s clusters since January 2023.

{{quote-block}}

Eli, a senior production engineer, explains that groundcover provided his teams with out-of-the-box visibility to all Kubernetes components, without integration, code changes or anything else so.  “It was super easy to view Similarweb’s K8s cluster health with a nice UI, in one place”, Eli adds.

Similarweb’s R&D teams use groundcover to monitor and visualize their applications in K8s, but Eli says that even for Production engineers like himself, it provides a full picture of what is happening in the cluster, including cross-applications issues, K8s infrastructure monitoring and APM metrics, amongst many other useful features.

The Similarweb team heard about groundcover when it was first founded, but were skeptical of its production readiness. Months later, when the team was asked to review different tools the name came up and Eli and his team were really pleasantly shocked to see how far it came. The production engineers were weary of the solution but the developers really championed the solution and upon further inquiry, they realized it’s definitely production ready. “We have daily deployments and growth in K8s and groundcover really allows us for automation and scalability in this sometimes rocky-road of a process, especially thanks to the inCloud enterprise model which allows for ultimate stability,” Eli explains.

Amongst other tools that were tried and utilized by the Similarweb team, logs were covered through Logz.io and metrics through Datadog.  

“It was important for us to keep our logs and metrics stack in place to not rock the boat too much for the developers’ services, but as we look into the future, we don’t see a reason not to migrate more and more into groundcover which now covers all of our clusters -  dev, staging and production.  Datadog requires a lot of specific integrations which involves too much overhead and external deployments that are not managed directly by Similarweb. eBPF solves this problem for us and allows us to rely more heavily on groundcover for things like alerts, logs, etc.”

After having tested Datadog integrations with K8s, and Komodor for infrastructure - Eli explains that beyond better pricing, they opted for groundcover for many reasons, amongst which are:

Context -  what’s happening in the pod, events going on in the cluster, all became clearer with groundcover, which made for easier and more focused troubleshooting. The ‘Killer feature’ - groundcover’s network dependency map really showed the team what they were thirsty for - a visualization that really pinpointed issues where they were. Since adopting the groundcover solution, and finally coming to understand which services are talking to which -

“We had a handful of production failures incidents in Kubernetes, where all the services, including everything, crashed. As soon as we solved the original problem, we had to go through each service, to redeploy the service to run again -- otherwise it’d get stuck in an infinite loop, even though the bug was resolved. With several hundred services in production, you’ve got to know the dependencies between the services, to know who is first in the chain to pick them up in order for everything else to take care of itself. groundcover’s network map helped us identify it in a second flat. Having everything laid out in front of my eyes is priceless”
_{-Eli says}

After having tested different solutions, Similarweb felt that groundcover was the ultimate out-of-the box solution for visibility to logs, metrics and application performance. Eli explains that It saved all of the integration overhead and developer time on endless configurations and allowed us all to speak the same language. Now it’s far easier to write new services for Similarweb since he can see everything on the infra side without requiring new demands from the dev team.

Why groundcover

groundcover provides great monitoring and K8s visibility which covers the entire monitoring needed for any K8s cluster - infrastructure resource usage metrics, K8s events, APM, logs & custom metrics and all of this - with no code changes.

“We looked for a tool like that and researched a few SaaS and open source projects that use eBPF technology to collect this data, but found that groundcover provides it faster, more seamlessly, with easy scale up options and with a very friendly UI.  It’s not just metrics and logs themselves, it’s how pods are behaving, how events are coming up over time - with groundcover, teams don’t need to fully understand K8s in order to get an understanding of what’s going on. This visibility was really hard to make accessible to developers and groundcover made our lives easier. The support and feedback we received from groundcover was outstanding and same-day releases for our requests and issues became prime features even before we were ready to become paying customers.”

By using groundcover’s observability solution, SimilarWeb increased operational efficiency, improved system reliability, resolved issues faster, and gained a better understanding of their system’s performance. These benefits contribute to a more seamless and reliable user experience, further strengthening SimilarWeb’s competitive position in the market.