Arlo delivers world’s leading Video Analytics platform on Amazon EKS and saves 50% on usage for GPU servers

Arlo Technologies, Inc., an industry leader in connected cameras and smart home security solutions, uses machine learning and computer vision (Arlo SMART) to process millions of video streams each day. The platform intelligently identifies objects such as people, animals, packages, vehicles, and more, to notify customers of these AI-Detected objects occurring in and around their homes and businesses. The platform supports 180 M video uploads each day equivalent to 850 Hr of video uploaded every minute. The Platform generates 22 M plus rich Smart notification each day using Arlo SMART enabling instant action. In order to reduce time to market, speed build times, and cut costs, Arlo containerized individual workloads within its computer vision platform by migrating from a virtual machine on Amazon Elastic Cloud Compute (Amazon EC2) to a microservices based architecture on Amazon Elastic Kubernetes Service (Amazon EKS). The result: one of the largest video analytics deployments on Amazon EKS.

Imagine trying to properly scale compute for a video analytics platform that analyzes millions of video streams daily. That was the challenge facing the Engineering team at Arlo. Within the monolithic application there were three major workloads: video pre-processing, video metadata, and GPU-based object classifiers. Because Arlo’s application usage follows a day-night curve—during the day there’s more video activity that tapers off at night—auto-scaling enabled optimal infrastructure usage for the pre-processing and image metadata workloads. The GPU-based workloads, however, fluctuated between 20 and 90 percent with no real pattern. Usage would spike to very high points very quickly, then come back down fast.

Through some reverse engineering in Amazon CloudWatch, the Arlo team was able to track usage spikes in the GPU-based workloads at a rate of six times per second. Working with AWS, the team then wrote scripts that determined utilization. All told, the move to Amazon EKS, saved Arlo up to 50 percent on expensive GPU instances. “What we were able to accomplish with auto-scaling has never been done before in Amazon EKS with GPU-based workloads on AWS,” explained Jishnu Kinwar, vice president of cloud platform engineering at Arlo.

Deploying new updates to any of the workloads used to take up to one month, significantly affecting the team’s productivity. “Because any small change required touching everything else in the software, we wanted to push to microservices on Amazon EKS to improve our time to market, as well as to reduce our testing cycles,” Kinwar said. With each service broken out, Arlo can optimize its ML-based inferencing services running on GPU servers. Now the architecture supports each classifier to scale independently based on the load. “Now, when we want to make a change to an package classifier to improve the model, it can go to production within a few days or even a few hours, instead of a month,” Kinwar said.

With containerized workloads, Arlo was able to incorporate continuous integration and continuous delivery (CI/CD) to ship code much faster. Automating the creation of a container image, as well as basic acceptance testing has helped speed pipelines. “We moved from deployments through scripts and putting code on EC2 machines, to knowing exactly what’s running in each of the containerized environments,” Kinwar said. “Containers allow us to be more consistent in what we build, and CI/CD has become a big part of it. Deployment and taking to production have been faster.” In addition, integrations with Arlo’s log analytics and tools ensure that everything generates signatures and response times before going into production.

Amazon EKS has enabled Arlo to have deeper visibility into how different portions of its video analytics platform behave. As a result, it’s easier and faster for the team to look into issues, enhancing customer support. “Having this new architecture allows faster go to market and definitely creates an overall cost savings for the customer,” Kinwar said.