Overview
Scale-Out Computing on AWS helps customers deploy and operate a multiuser environment for computationally intensive workflows, such as computer-aided engineering (CAE). This AWS Solution features a large selection of compute resources, a fast network backbone, unlimited storage, and budget and cost management directly integrated within AWS.
Benefits
Deploy and set up an example user interface (UI) with a common set of APIs that allow the administrator and users to interact with their Amazon Elastic Compute Cloud (Amazon EC2) cluster.
Help users easily access the EC2 cluster to perform any pre- and post-processing visualization actions through Desktop Cloud Visualization (DCV) graphical sessions.
Ingest schedulers and application logs in near real-time, and then store them in the data lake for further processing.
Customize and extend a collection of scripts deployed with the solution to help users collect data and run common cluster tasks.
Technical details
You can automatically deploy this architecture using the implementation guide.
Step 1
Amazon EC2 Auto Scaling automatically provisions the resources necessary to run cluster user tasks such as scal-out compute jobs.
Step 2
This solution also deploys Amazon Elastic File System (Amazon EFS) for persistent storage, Amazon Simple Storage Service (Amazon S3) for persistent logs, and optional parallel file system Amazon FSx for Lustre.
Step 3
At its core, the Amazon Elastic Compute Cloud (Amazon EC2) instance implements a scheduler, which dynamically provisions AWS resources required for jobs submitted by users. The scheduler instance also hosts a web interface which allows users and administrators to interact with the environment.
Step 4
Launch a 2D or 3D Workstation that uses Amazon DCV, which can be used to submit batch jobs and run GUI tools.
Step 5
Security services and resources that are used include AWS Secrets Manager, AWS Certificate Manager, Security Groups, and AWS Identity and Access Management (IAM).
Step 6
AWS Lambda functions verify the required prerequisites and create a default signed certificate for an Application Load Balancer (ALB) to manage access to DCV workstation sessions.
Step 7
An Amazon OpenSearch Service cluster stores job and host information.
Step 8
Elastic Load Balancing is used to ensure accessibility across Availability Zones, and Cost Allocation Tags are used with AWS Cost Explorer.
Step 1
Amazon EC2 Auto Scaling automatically provisions the resources necessary to run cluster user tasks such as scale-out compute jobs.
Step 2
This solution also deploys Amazon Elastic File System (Amazon EFS) for persistent storage, Amazon Simple Storage Service (Amazon S3) for persistent logs, and optional parallel file system Amazon FSx for Lustre.
Step 3
At its core, the Amazon Elastic Compute Cloud (Amazon EC2) instance implements a scheduler, which dynamically provisions AWS resources required for jobs submitted by users. The scheduler instance also hosts a web interface which allows users and administrators to interact with the environment.
Step 4
Launch a 2D or 3D Workstation that uses Amazon DCV, which can be used to submit batch jobs and run GUI tools.
Step 5
Security services and resources that are used include AWS Secrets Manager, AWS Certificate Manager, Security Groups, and AWS Identity and Access Management (IAM).
Step 6
AWS Lambda functions verify the required prerequisites and create a default signed certificate for an Application Load Balancer (ALB) to manage access to DCV workstation sessions.
Step 7
An Amazon OpenSearch Service cluster stores job and host information.
Step 8
Elastic Load Balancing is used to ensure accessibility across Availability Zones, and Cost Allocation Tags are used with AWS Cost Explorer.
- Publish Date
Related content
This video shows a re:Invent 2020 presentation detailing how how Rivian overcame obstacles and improved the engineering experience while encouraging self-service with cloud-based services and automation.
This video demonstrates how to implement Scale-Out Computing on AWS to deploy and operate a multiuser environment for computationally intensive workflows.