使用Amazon Nova Sonic部署全栈语音AI代理 |

语言 Chinese, Simplified

SEO Title

Deploy a full stack voice AI agent with Amazon Nova Sonic

解决方案概述

下图提供了可部署解决方案的概述。

The solution is composed of the following layers:

Frontend layer – The frontend layer of this system is built with scalability and performance in mind:
- Amazon CloudFront distribution serves as the content delivery network for the web application.
- Amazon Simple Storage Service (Amazon S3) hosts static assets.
- The UI handles audio streaming and user interaction.
Communication layer – The communication layer facilitates seamless real-time interactions:
- Network Load Balancer manages WebSocket connections. WebSockets enable two-way interactive communication sessions between a user’s browser and the server, which is essential for real-time audio streaming applications.
- Amazon Cognito provides user authentication and JSON web token (JWT) validation. Amazon Cognito provides user authentication, authorization, and user management for web and mobile applications, alleviating the need to build and maintain your own identity systems.
Processing layer – The processing layer forms the computational backbone of the system:
- Amazon Elastic Container Service (Amazon ECS) runs the containerized backend service.
- AWS Fargate provides the serverless compute backend. Orchestration is provided by the Amazon ECS engine.
- The Python backend processes audio streams and manages Amazon Nova Sonic interactions.
Intelligence layer – The intelligence layer uses AI and data technologies to power the core functionalities:
- The Amazon Nova Sonic model in Amazon Bedrock handles speech processing.
- Amazon DynamoDB stores customer information.
- Amazon Bedrock Knowledge Bases connects foundation models (FMs) with your organization’s data sources, allowing AI applications to reference accurate, up-to-date information specific to your business.

The following sequence diagram highlights the flow when a user initiates conversation. The user only signs in one time, but authentication Steps 3 and 4 happen every time the user starts a new session. The conversational loop in Steps 6–12 is repeated throughout the conversational interaction. Steps a–c only happen when the Amazon Nova Sonic agent decides to use a tool. In scenarios without tool use, the flow goes directly from Step 9 to Step 10.

Prerequisites

Before getting started, verify that you have the following:

Python 3.12
Node.js v20
npm v10.8
An AWS account
The AWS CDK set up (for prerequisites and installation instructions, see Getting started with the AWS CDK)
Amazon Nova Sonic enabled in Amazon Bedrock (for more information, see Add or remove access to Amazon Bedrock foundation models)
Chrome or Safari browser environment (Firefox is not supported at the time of writing)
A working microphone and speakers

Deploy the solution

You can find the solution and full deployment instructions on the GitHub repository. The solution uses the AWS CDK to automate infrastructure deployment. Use the following code terminal commands to get started in your AWS Command Line Interface (AWS CLI) environment:

git clone https://github.com/aws-samples/sample-sonic-cdk-agent.git 
cd nova-s2s-call-center 

# Configure environment variables
cp template.env .env

# Edit .env with your settings

# Deploy the solution 
./deploy.sh

The deployment creates two AWS CloudFormation stacks:

Network stack for virtual private cloud (VPC) and networking components
Stack for application resources

The output of the second stack gives you a CloudFront distribution link, which takes you to the login page.

You can create an Amazon Cognito admin user with the following AWS CLI command:

aws cognito-idp admin-create-user \
  --user-pool-id YOUR_USER_POOL_ID \
  --username USERNAME \
  --user-attributes Name=email,Value=USER_EMAIL \
  --temporary-password TEMPORARY_PASSWORD \
  --region YOUR_AWS_REGION

The preceding command uses the following parameters:

YOUR_USER_POOL_ID: The ID of your Amazon Cognito user pool
USERNAME: The desired user name for the user
USER_EMAIL: The email address of the user
TEMPORARY_PASSWORD: A temporary password for the user
YOUR_AWS_REGION: Your AWS Region (for example, us-east-1)

You can choose Start Session to start a conversation with your assistant. Experiment with prompts and different tools for your use case.

Customizing the application

A key feature of this solution is its flexibility—you can tailor the AI agent’s capabilities to your specific use case. The sample implementation demonstrates this extensibility through custom tools and knowledge integration:

Customer information lookup – Retrieves customer profile data from DynamoDB using phone numbers as keys
Knowledge base search – Queries an Amazon Bedrock knowledge base for company information, plan details, and pricing

These features showcase how to enhance the functionality of Amazon Nova Sonic with external data sources and domain-specific knowledge. The architecture is designed for seamless customization in several key areas.

Modifying the system prompt

The solution includes a UI in which you can adjust the AI agent’s behavior by modifying its system prompt. This enables rapid iteration on the agent’s personality, knowledge base, and conversation style without redeploying the entire application.

Adding new tools

You can also extend the AI agent’s capabilities by implementing additional tools using the MCP framework. The process involves:

Implementing the tool logic, typically as a new Python module
Registering the tool with the MCP server by using the @mcp_server.tool custom decorator and defining the tool specification, including its name, description, and input schema in /backend/tools/mcp_tool_registry.py

For example, the following code illustrates how to add a knowledge base lookup tool:

@mcp_server.tool(
    name="lookup",
    description="Runs query against a knowledge base to retrieve information."
)
async def lookup_tool(
    query: Annotated[str, Field(description="the query to search")]
) -> dict:
    """Look up information in the knowledge base"""
    results = knowledge_base_lookup.main(query)
    return results

The decorator handles registration with the MCP server, and the function body contains your tool’s implementation logic.

Expanding the knowledge base

The solution uses Amazon Bedrock Knowledge Bases to provide the AI agent with company-specific information. You can update this knowledge base with:

Frequently asked questions and their answers
Product catalogs and specifications
Company policies and procedures

Clean up

You can remove the stacks with the following command:

# move to the cdk folder, assuming you are in the project root folder
cd cdk
# Removes both stacks sequentially
npx cdk destroy --all

结论

人工智能代理正在改变组织提供客户服务的方式，其解决方案能够同时处理多个对话，全天候提供一致的服务，并在保持质量和降低运营成本的同时即时扩展。该解决方案通过为AWS上的Amazon Nova Sonic应用程序提供可部署的基础，使这些好处变得触手可及。该解决方案展示了人工智能代理如何有效地处理客户查询、访问实时数据并提供个性化服务，同时保持客户期望的自然对话流。

通过将Amazon Nova Sonic模型与强大的云架构、安全身份验证和灵活的工具集成相结合，组织可以快速从概念转变为概念验证。该解决方案不仅有助于构建语音人工智能应用程序，还可以帮助公司在多个行业提高客户满意度和生产力。

要了解更多信息，请参阅以下资源：

本文地址

https://architect.pub