Overview
Generative AI Application Builder on AWS facilitates the development, rapid experimentation, and deployment of generative artificial intelligence (AI) applications without requiring deep experience in AI. This AWS Solution accelerates development and streamlines experimentation by helping you ingest your business-specific data and documents, evaluate and compare the performance of large language models (LLMs), rapidly build extensible applications, and deploy those applications with an enterprise-grade architecture.
Generative AI Application Builder on AWS includes integrations with Amazon Bedrock and its included LLMs, and LLMs deployed on Amazon SageMaker. Additionally, this solution enables connections to your choice of model using LangChain or AWS Lambda. Start with the no-code deployment wizard to build generative AI applications for conversational search, AI-generated chatbots, text generation, and text summarization.
Benefits
This solution allows users to experiment quickly by removing the heavy lifting required to deploy multiple instances with different configurations and compare outputs and performance. Experiment with multiple configurations of various LLMs, prompt engineering, enterprise knowledge bases, and other parameters.
With pre-built connectors to a variety of LLMs, such as models available through Amazon Bedrock, this solution gives you the flexibility to deploy the model of your choice, as well as the AWS and leading FM services you prefer.
Built with AWS Well-Architected design principles, this solution offers enterprise-grade security and scalability with high availability and low latency, ensuring seamless integration into your applications with high performance standards.
Extend this solution’s functionality by integrating your existing projects or natively connecting additional AWS services. Because this is an open-source application, you can use the included LangChain orchestration layer or Lambda functions to connect with the services of your choice.
Technical details
You can automatically deploy this architecture using the implementation guide and the accompanying AWS CloudFormation template that deploys two separate architectures:
- Deployment Dashboard – The Deployment Dashboard is a web user interface (UI) that serves as a management console for admin users to view, manage, and create their use cases. This dashboard enables customers to rapidly experiment, iterate, and deploy generative AI applications using multiple configurations of LLMs and data.
- Text Use Case – The Text Use Case enables users to experience a natural language interface using generative AI. This use case can be integrated into new or existing applications, and is deployable through the Deployment Dashboard or independently through a provided URL.
-
Deployment Dashboard
-
Text Use Case
-
Deployment Dashboard
-
Click to enlarge
Step 1
Admin users log in to the Deployment Dashboard user interface (UI).Step 2
Amazon CloudFront delivers the web UI which is hosted in an Amazon Simple Storage Service (Amazon S3) bucket.Step 3
AWS WAF protects the APIs from attacks. This solution configures a set of rules called a web access control list (web ACL) that allows, blocks, or counts web requests based on configurable, user defined web security rules and conditions.
Step 4
The web UI leverages a set of REST APIs that are exposed using Amazon API Gateway.
Step 5
Amazon Cognito authenticates users and backs both the CloudFront web UI and API Gateway. An Amazon DynamoDB table stores the AWS Identity and Access Management (IAM) policy of authorized users.Step 6
AWS Lambda provides the business logic for the REST endpoints. This Backing Lambda function manages and creates the necessary resources to perform use case deployments using AWS CloudFormation.Step 7
DynamoDB stores the list of deployments.Step 8
When a new use case is created by the admin user, the Backing Lambda function initiates a CloudFormation stack creation event for the requested use case.Step 9
All of the LLM configuration options provided by the admin user in the deployment wizard are saved in DynamoDB. The deployment uses this DynamoDB table to configure the LLM at runtime.Step 10
Using Amazon CloudWatch, this solution collects operational metrics from various services to generate custom dashboards that allow you to monitor the solution's performance and operational health.
Step 1
Admin users log in to the Deployment Dashboard user interface (UI).Step 2
Amazon CloudFront delivers the web UI which is hosted in an Amazon Simple Storage Service (Amazon S3) bucket.Step 3
AWS WAF protects the APIs from attacks. This solution configures a set of rules called a web access control list (web ACL) that allows, blocks, or counts web requests based on configurable, user defined web security rules and conditions.
Step 4
The web UI leverages a set of REST APIs that are exposed using Amazon API Gateway.
Step 5
Amazon Cognito authenticates users and backs both the CloudFront web UI and API Gateway. An Amazon DynamoDB table stores the AWS Identity and Access Management (IAM) policy of authorized users.Step 6
AWS Lambda provides the business logic for the REST endpoints. This Backing Lambda function manages and creates the necessary resources to perform use case deployments using AWS CloudFormation.Step 7
DynamoDB stores the list of deployments.Step 8
When a new use case is created by the admin user, the Backing Lambda function initiates a CloudFormation stack creation event for the requested use case.Step 9
All of the LLM configuration options provided by the admin user in the deployment wizard are saved in DynamoDB. The deployment uses this DynamoDB table to configure the LLM at runtime.
Step 10
Using Amazon CloudWatch, this solution collects operational metrics from various services to generate custom dashboards that allow you to monitor the solution's performance and operational health.
-
Text Use Case
-
Click to enlarge
Step 1
Admin users deploy the use case using the Deployment Dashboard. Business users log in to the use case UI.
Step 2
CloudFront delivers the web UI which is hosted in an S3 bucket.Step 3
The web UI leverages a WebSocket integration built using API Gateway. API Gateway is backed by a custom Lambda Authorizer function, which returns the appropriate IAM policy based on the Amazon Cognito group to which the authenticating user belongs. The policy is stored in DynamoDB.Step 4
Amazon Cognito authenticates users and backs both the CloudFront web UI and API Gateway.
Step 5
Incoming requests from the business user are passed from API Gateway to an Amazon Simple Queue Service (Amazon SQS) queue and then to the LangChain Orchestrator. The LangChain Orchestrator is a collection of Lambda functions and layers that provide the business logic for fulfilling requests from the business user.The queue enables the asynchronous operation of the API Gateway to Lambda integration. The queue passes connection information to the Lambda functions, which will then post results directly back to the API Gateway WebSocket connection to support long running inference calls.
Step 6
The LangChain Orchestrator uses DynamoDB to get the configured LLM options and necessary session information (such as the chat history).Step 7
If the deployment has a knowledge base configured, then the LangChain Orchestrator leverages Amazon Kendra or Knowledge Bases for Amazon Bedrock to run a search query to retrieve document excerpts.Step 8
Using the chat history, query, and context from the knowledge base, the LangChain Orchestrator creates the final prompt and sends the request to the LLM hosted on Amazon Bedrock or Amazon SageMaker.Step 9
When the response comes back from the LLM, the LangChain Orchestrator streams the response back through the API Gateway WebSocket to be consumed by the client application.Step 10
Using CloudWatch, this solution collects operational metrics from various services to generate custom dashboards that allow you to monitor the deployment’s performance and operational health.Step 1
Admin users deploy the use case using the Deployment Dashboard. Business users log in to the use case UI.
Step 2
CloudFront delivers the web UI which is hosted in an S3 bucket.Step 3
The web UI leverages a WebSocket integration built using API Gateway. API Gateway is backed by a custom Lambda Authorizer function, which returns the appropriate IAM policy based on the Amazon Cognito group to which the authenticating user belongs. The policy is stored in DynamoDB.Step 4
Amazon Cognito authenticates users and backs both the CloudFront web UI and API Gateway.
Step 5
Incoming requests from the business user are passed from API Gateway to an Amazon Simple Queue Service (Amazon SQS) queue and then to the LangChain Orchestrator. The LangChain Orchestrator is a collection of Lambda functions and layers that provide the business logic for fulfilling requests from the business user.The queue enables the asynchronous operation of the API Gateway to Lambda integration. The queue passes connection information to the Lambda functions, which will then post results directly back to the API Gateway WebSocket connection to support long running inference calls.
Step 6
The LangChain Orchestrator uses DynamoDB to get the configured LLM options and necessary session information (such as the chat history).Step 7
If the deployment has a knowledge base configured, then the LangChain Orchestrator leverages Amazon Kendra or Knowledge Bases for Amazon Bedrock to run a search query to retrieve document excerpts.Step 8
Using the chat history, query, and context from the knowledge base, the LangChain Orchestrator creates the final prompt and sends the request to the LLM hosted on Amazon Bedrock or Amazon SageMaker.Step 9
When the response comes back from the LLM, the LangChain Orchestrator streams the response back through the API Gateway WebSocket to be consumed by the client application.Step 10
Using CloudWatch, this solution collects operational metrics from various services to generate custom dashboards that allow you to monitor the deployment’s performance and operational health.
- Publish Date
Related content
The easiest way to build and scale generative AI applications with foundation models.
This Guidance demonstrates how to deploy a generative artificial intelligence (AI) model provided by Amazon SageMaker JumpStart to create an asynchronous SageMaker endpoint with the ease of the AWS Cloud Development Kit (AWS CDK).
This Guidance demonstrates how to build an application enabling users to ask questions directly of relational databases using natural language queries (NLQ).
Boost productivity, build differentiated experiences, and innovate faster with AWS.
This case study demonstrates how broadcast company Megamedia created a generative AI–powered chatbot to simplify access to important public information using AWS.