【多租户】在Amazon OpenSearch Service中构建多租户无服务器架构

Environment: PoC or pilot

Technologies: Modernization; SaaS; Serverless

Workload: Open-source

AWS services: Amazon OpenSearch Service; AWS Lambda; Amazon S3; Amazon API Gateway

Create an S3 bucket.

Create an S3 bucket in your AWS Region. This bucket will hold the non-indexed tenant data for the sample application. Make sure that the S3 bucket's name is globally unique, because the namespace is shared by all AWS accounts.

To create an S3 bucket, you can use the AWS CLI create-bucket command as follows:

aws s3api create-bucket \   --bucket tenantrawdata \   --region <your-AWS-Region>

where tenantrawdata is the S3 bucket name. (You can use any unique name that follows the bucket naming guidelines.)

Create an Amazon OpenSearch Service domain.

Run the AWS CLI create-elasticsearch-domain command to create an Amazon OpenSearch Service domain:

aws es create-elasticsearch-domain \   --domain-name vpc-cli-example \   --elasticsearch-version 7.10 \   --elasticsearch-cluster-config InstanceType=t3.medium.elasticsearch,InstanceCount=1 \   --ebs-options EBSEnabled=true,VolumeType=gp2,VolumeSize=10 \   --domain-endpoint-options "{\"EnforceHTTPS\": true}" \   --encryption-at-rest-options "{\"Enabled\": true}" \   --node-to-node-encryption-options "{\"Enabled\": true}" \   --advanced-security-options "{\"Enabled\": true, \"InternalUserDatabaseEnabled\": true, \ \"MasterUserOptions\": {\"MasterUserName\": \"KibanaUser\", \ \"MasterUserPassword\": \"NewKibanaPassword@123\"}}" \   --vpc-options "{\"SubnetIds\": [\"<subnet-id>\"], \"SecurityGroupIds\": [\"<sg-id>\"]}" \   --access-policies "{\"Version\": \"2012-10-17\", \"Statement\": [ { \"Effect\": \"Allow\", \ \"Principal\": {\"AWS\": \"*\" }, \"Action\":\"es:*\", \ \"Resource\": \"arn:aws:es:region:account-id:domain\/vpc-cli-example\/*\" } ] }"

The instance count is set to 1 because the domain is for testing purposes. You need to enable fine-grained access control by using the advanced-security-options parameter, because the details cannot be changed after the domain has been created. 

This command creates a master user name (KibanaUser) and a password that you can use to log in to the Kibana console.

Because the domain is part of a virtual private cloud (VPC), you have to make sure that you can reach the Elasticsearch instance by specifying the access policy to use.

For more information, see Launching your Amazon OpenSearch Service domains using a VPC in the AWS documentation.

Set up a bastion host.

Set up a Amazon Elastic Compute Cloud (Amazon EC2) Windows instance as a bastion host to access the Kibana console. The Elasticsearch security group must allow traffic from the Amazon EC2 security group. For instructions, see the blog post Controlling Network Access to EC2 Instances Using a Bastion Server.

When the bastion host has been set up, and you have the security group that is associated with the instance available, use the AWS CLI authorize-security-group-ingress command to add permission to the Elasticsearch security group to allow port 443 from the Amazon EC2 (bastion host) security group.

aws ec2 authorize-security-group-ingress \ --group-id <SecurityGroupIdfElasticSearch> \ --protocol tcp \ --port 443 \ --source-group <SecurityGroupIdfBashionHostEC2>

Create the Lambda execution role.

Run the AWS CLI create-role command to grant the Lambda index function access to AWS services and resources:

aws iam create-role \ --role-name index-lambda-role \ --assume-role-policy-document file://lambda_assume_role.json

where lambda_assume_role.json is a JSON document in the current folder that grants AssumeRole permissions to the Lambda function, as follows:

{      "Version": "2012-10-17",      "Statement": [          {              "Effect": "Allow",              "Principal": {                  "Service": "lambda.amazonaws.com"                },              "Action": "sts:AssumeRole"          }      ]  }

Attach managed policies to the Lambda role.

Run the AWS CLI attach-role-policy command to attach managed policies to the role created in the previous step. These two policies give the role permissions to create an elastic network interface and to write logs to CloudWatch Logs.  

aws iam attach-role-policy \ --role-name index-lambda-role \ --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole aws iam attach-role-policy \ --role-name index-lambda-role \ --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole 

Create a policy to give the Lambda index function permission to read the S3 objects.

Run the AWS CLI create-policy command to to give the Lambda index function s3:GetObject permission to read the objects in the S3 bucket:

aws iam create-policy \   --policy-name s3-permission-policy \   --policy-document file://s3-policy.json

The file s3-policy.json is a JSON document in the current folder that grants s3:GetObject permissions to allow read access to S3 objects. If you used a different name when you created the S3 bucket, provide the correct bucket name in the Resource section in the following:  

{     "Version": "2012-10-17",     "Statement": [         {            "Effect": "Allow",            "Action": "s3:GetObject",            "Resource": "arn:aws:s3:::tenantrawdata/*"         }     ] }

Attach the Amazon S3 permission policy to the Lambda execution role.

Run the AWS CLI attach-role-policy command to attach the Amazon S3 permission policy you created in the previous step to the Lambda execution role:

aws iam attach-role-policy \   --role-name index-lambda-role \   --policy-arn <PolicyARN>

where PolicyARN is the Amazon Resource Name (ARN) of the Amazon S3 permission policy. You can get this value from the output of the previous command.

Create the Lambda index function.

Run the AWS CLI create-function command to create the Lambda index function, which will access Amazon OpenSearch Service:

aws lambda create-function \   --function-name index-lambda-function \   --zip-file fileb://index_lambda_package.zip \   --handler lambda_index.lambda_handler \   --runtime python3.7 \   --role "arn:aws:iam::account-id:role/index-lambda-role" \   --timeout 30 \   --vpc-config "{\"SubnetIds\": [\"<subnet-id1\>", \"<subnet-id2>\"], \     \"SecurityGroupIds\": [\"<sg-1>\"]}"

Allow Amazon S3 to call the Lambda index function.

Run the AWS CLI add-permission command to give Amazon S3 the permission to call the Lambda index function:  

aws lambda add-permission \ --function-name index-lambda-function \ --statement-id s3-permissions \ --action lambda:InvokeFunction \ --principal s3.amazonaws.com \ --source-arn "arn:aws:s3:::tenantrawdata" \ --source-account "<account-id>"

Add a Lambda trigger for the Amazon S3 event.

Run the AWS CLI put-bucket-notification-configuration command to send  notifications to the Lambda index function when the Amazon S3 ObjectCreated event is detected. The index function runs whenever an object is uploaded to the S3 bucket. 

aws s3api put-bucket-notification-configuration \ --bucket tenantrawdata \ --notification-configuration file://s3-trigger.json

The file s3-trigger.json is a JSON document in the current folder that adds the resource policy to the Lambda function when the Amazon S3 ObjectCreated event occurs.

Create the Lambda execution role.

Run the AWS CLI create-role command to grant the Lambda search function access to AWS services and resources:

aws iam create-role \ --role-name search-lambda-role \ --assume-role-policy-document file://lambda_assume_role.json

where lambda_assume_role.json is a JSON document in the current folder that grants AssumeRole permissions to the Lambda function, as follows:

{      "Version": "2012-10-17",      "Statement": [          {              "Effect": "Allow",              "Principal": {                  "Service": "lambda.amazonaws.com"                },              "Action": "sts:AssumeRole"          }      ]  }

Attach managed policies to the Lambda role.

Run the AWS CLI attach-role-policy command to attach managed policies to the role created in the previous step. These two policies give the role permissions to create an elastic network interface and to write logs to CloudWatch Logs.

aws iam attach-role-policy \ --role-name search-lambda-role \ --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole aws iam attach-role-policy \ --role-name search-lambda-role \ --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole 

Create the Lambda search function.

Run the AWS CLI create-function command to create the Lambda search function, which will access Amazon OpenSearch Service:

aws lambda create-function \   --function-name search-lambda-function \   --zip-file fileb://search_lambda_package.zip \   --handler lambda_search.lambda_handler \   --runtime python3.7 \   --role "arn:aws:iam::account-id:role/search-lambda-role" \   --timeout 30 \   --vpc-config "{\"SubnetIds\": [\"<subnet-id1\>", \"<subnet-id2>\"], \     \"SecurityGroupIds\": [\"<sg-1>\"]}"

Create tenant IAM roles.

Run the AWS CLI create-role command to create two tenant roles that will be used to test the search functionality:  

aws iam create-role \   --role-name Tenant-1-role \   --assume-role-policy-document file://assume-role-policy.json

aws iam create-role \   --role-name Tenant-2-role \   --assume-role-policy-document file://assume-role-policy.json

The file assume-role-policy.json is a JSON document in the current folder that grants AssumeRole permissions to the Lambda execution role:

{     "Version": "2012-10-17",     "Statement": [         {             "Effect": "Allow",             "Principal": {                  "AWS": "<Lambda execution role for index function>",                  "AWS": "<Lambda execution role for search function>"              },             "Action": "sts:AssumeRole"         }     ] }

Create a tenant IAM policy.

Run the AWS CLI create-policy command to create a tenant policy that grants access to Elasticsearch operations:  

aws iam create-policy \   --policy-name tenant-policy \   --policy-document file://policy.json

The file policy.json is a JSON document in the current folder that grants permissions on Elasticsearch:

{     "Version": "2012-10-17",     "Statement": [         {             "Effect": "Allow",             "Action": [                 "es:ESHttpDelete",                 "es:ESHttpGet",                 "es:ESHttpHead",                 "es:ESHttpPost",                 "es:ESHttpPut",                 "es:ESHttpPatch"             ],             "Resource": [                 "<ARN of Elasticsearch domain created earlier>"             ]         }     ] }

Attach the tenant IAM policy to the tenant roles.

Run the AWS CLI attach-role-policy command to attach the tenant IAM policy to the two tenant roles you created in the earlier step:

aws iam attach-role-policy \   --policy-arn arn:aws:iam::account-id:policy/tenant-policy \   --role-name Tenant-1-role aws iam attach-role-policy \   --policy-arn arn:aws:iam::account-id:policy/tenant-policy \   --role-name Tenant-2-role

The policy ARN is from the output of the previous step.

Create an IAM policy to give Lambda permissions to assume role.

Run the AWS CLI create-policy command to create a policy for Lambda to assume the tenant role:

aws iam create-policy \   --policy-name assume-tenant-role-policy \ --policy-document file://lambda_policy.json

The file lambda_policy.json is a JSON document in the current folder that grants permissions to AssumeRole:  

{     "Version": "2012-10-17",     "Statement": [        {             "Effect": "Allow",             "Action":  "sts:AssumeRole",             "Resource": "<ARN of tenant role created earlier>"        }     ] }

For Resource, you can use a wildcard character to avoid creating a new policy for each tenant.

Create an IAM policy to give the Lambda index role permission to access Amazon S3.

Run the AWS CLI create-policy command to give the Lambda index role permission to access the objects in the S3 bucket:  

aws iam create-policy \   --policy-name s3-permission-policy \   --policy-document file://s3_lambda_policy.json

The file s3_lambda_policy.json is the following JSON policy document in the current folder:

{     "Version": "2012-10-17",     "Statement": [         {             "Effect": "Allow",             "Action": "s3:GetObject",             "Resource": "arn:aws:s3:::tenantrawdata/*"         }     ] }

Attach the policy to the Lambda execution role.

Run the AWS CLI attach-role-policy command to attach the policy created in the previous step to the Lambda index and search execution roles you created earlier:

aws iam attach-role-policy \   --policy-arn arn:aws:iam::account-id:policy/assume-tenant-role-policy \   --role-name index-lambda-role aws iam attach-role-policy \   --policy-arn arn:aws:iam::account-id:policy/assume-tenant-role-policy \   --role-name search-lambda-role aws iam attach-role-policy \   --policy-arn arn:aws:iam::account-id:policy/s3-permission-policy \   --role-name index-lambda-role

The policy ARN is from the output of the previous step.

Create a REST API in API Gateway.

Run the CLI create-rest-api command to create a REST API resource:  

aws apigateway create-rest-api \   --name Test-Api \   --endpoint-configuration "{ \"types\": [\"REGIONAL\"] }"

For the endpoint configuration type, you can specify EDGE instead of REGIONAL to use edge locations instead of a particular AWS Region.

Note the value of the id field from the command output. This is the API ID that you will use in subsequent commands.

Create a resource for the search API.

The search API resource starts the Lambda search function with the resource name search. (You don’t have to create an API for the Lambda index function, because it runs automatically when objects are uploaded to the S3 bucket.)

1. Run the AWS CLI get-resources command to get the parent ID for the root path:  

   aws apigateway get-resources \ --rest-api-id <API-ID>

   Note the value of the ID field. You will use this parent ID in the next command.

   { "items": [ { "id": "zpsri964ck", "path": "/" } ] }

2. Run the AWS CLI create-resource command to create a resource for the search API. For parent-id, specify the ID from the previous command.  

   aws apigateway create-resource \   --rest-api-id <API-ID> \   --parent-id <Parent-ID> \   --path-part search

Create a GET method for the search API.

Run the AWS CLI put-method command to create a GET method for the search API:  

aws apigateway put-method \   --rest-api-id <API-ID> \   --resource-id <ID from the previous command output> \   --http-method GET \   --authorization-type "NONE" \ --no-api-key-required

For resource-id, specify the ID from the output of the create-resource command.

Create a method response for the search API.

Run the AWS CLI put-method-response command to add a method response for the search API:

aws apigateway put-method-response \   --rest-api-id <API-ID> \   --resource-id  <ID from the create-resource command output> \   --http-method GET \   --status-code 200 \ --response-models "{\"application/json\": \"Empty\"}"

For resource-id, specify the ID from the output of the earlier create-resource command.

Set up a proxy Lambda integration for the search API.

Run the AWS CLI command put-integration command to set up an integration with the Lambda search function:

aws apigateway put-integration \   --rest-api-id <API-ID> \   --resource-id  <ID from the create-resource command output> \   --http-method GET \   --type AWS_PROXY \   --integration-http-method GET \   --uri arn:aws:apigateway:region:lambda:path/2015-03-31/functions/arn:aws:lambda:<region>:<account-id>:function:<function-name>/invocations

For resource-id, specify the ID from the earlier create-resource command.

Grant API Gateway permission to call the Lambda search function.

Run the AWS CLI add-permission command to give API Gateway permission to use the search function:  

aws lambda add-permission \   --function-name <function-name> \   --statement-id apigateway-get \   --action lambda:InvokeFunction \   --principal apigateway.amazonaws.com \ --source-arn "arn:aws:execute-api:<region>:<account-id>:api-id/*/GET/search

Change the source-arn path if you used a different API resource name instead of search.

Deploy the search API.

Run the AWS CLI create-deployment command to create a stage resource named dev:  

aws apigateway create-deployment \   --rest-api-id <API-ID> \ --stage-name dev

If you update the API, you can use the same CLI command to redeploy it to the same stage.

Log in to the Kibana console.

1. Find the link to Kibana on your domain dashboard on the Amazon OpenSearch Service console. The URL is in the form: <domain-endpoint>/_plugin/kibana/.

2. Use the bastion host you configured in the first epic to access the Kibana console.

3. Log in to the Kibana console by using the master user name and password from the earlier step, when you created the Amazon OpenSearch Service domain.

4. When prompted to select a tenant, choose Private.

Create and configure Kibana roles.

To provide data isolation and to make sure that one tenant cannot retrieve the data of another tenant, you need to use document security, which allows tenants to access only documents that contain their tenant ID.

1. On the Kibana console, in the navigation pane, choose Security, Role.

2. Create a new tenant role.

3. Set cluster permissions to indices_all, which gives create, read, update, and delete (CRUD) permissions on the Amazon OpenSearch Service index. 

4. Restrict index permissions to the tenant-data index. (The index name should match the name in the Lambda search and index functions.) 

5. Set index permissions to indices_all, to enable users to perform all index-related operations. (You can restrict operations for more granular access, depending on your requirements.)

6. For document-level security, use the following policy to filter documents by tenant ID, to provide data isolation for tenants in a shared index:

   {   "bool": {     "must": {       "match": {         "TenantId": "Tenant-1"       }     }   } }

   The index name, properties, and values are case-sensitive.

Map users to roles.

1. Choose the Mapped users tab for the role, and then choose Map users.

2. In the Backend roles section, specify the ARN of the IAM tenant role that you created earlier, and then choose Map. This maps the IAM tenant role to the Kibana role so that tenant-specific search returns data for that tenant only. For example, if the IAM role name for Tenant-1 is Tenant-1-Role, specify the ARN for Tenant-1-Role (from the Create and configure tenant roles epic) in the Backend roles box for the Tenant-1 Kibana role.

3. Repeat steps 1 and 2 for Tenant-2.

We recommend that you automate the creation of the tenant and Kibana roles at the time of tenant onboarding.

Create the tenant-data index.

In the navigation pane, under Management, choose Dev Tools, and then run the following command. This command creates the tenant-data index to define the mapping for the TenantId property.

PUT /tenant-data {   "mappings": {     "properties": {       "TenantId": { "type": "keyword"}     }   } }

Create a VPC endpoint for Amazon S3.

Run the AWS CLI create-vpc-endpoint command to create a VPC endpoint for Amazon S3. The endpoint enables the Lambda index function in the VPC to access the Amazon S3 service.  

aws ec2 create-vpc-endpoint \   --vpc-id <VPC-ID> \   --service-name com.amazonaws.us-east-1.s3 \   --route-table-ids <route-table-ID>

For vpc-id, specify the VPC that you’re using for the Lambda index function. For service-name, use the correct URL for the Amazon S3 endpoint. For route-table-ids, specify the route table that’s associated with the VPC endpoint.

Create a VPC endpoint for AWS STS.

Run the AWS CLI create-vpc-endpoint command to create a VPC endpoint for AWS Security Token Service (AWS STS). The endpoint enables the Lambda index and search functions in the VPC to access the AWS STS service. The functions use AWS STS when they assume the IAM role.

aws ec2 create-vpc-endpoint \   --vpc-id <VPC-ID> \   --vpc-endpoint-type Interface \   --service-name com.amazonaws.us-east-1.sts \   --subnet-id <subnet-ID> \ --security-group-id <security-group-ID>

For vpc-id, specify the VPC that you’re using for the Lambda index and search functions. For subnet-id, provide the subnet in which this endpoint should be created. For security-group-id, specify the security group to associate this endpoint with. (It could be the same as the security group Lambda uses.)

Update the Python files for the index and search functions.

1. In the index_lambda_package.zip file, edit the lamba_index.py file to update the AWS account ID, AWS Region, and Elasticsearch endpoint information.

2. In the search_lambda_package.zip file, edit the lambda_search.py file to update the AWS account ID, AWS Region, and Elasticsearch endpoint information.

You can get the Elasticsearch endpoint from the Overview tab of the Amazon OpenSearch Service console. It has the format <AWS-Region>.es.amazonaws.com.

Update the Lambda code.

Use the AWS CLI update-function-code command to update the Lambda code with the changes you made to the Python files:  

aws lambda update-function-code \   --function-name index-lambda-function \   --zip-file fileb://index_lambda_package.zip aws lambda update-function-code \   --function-name search-lambda-function \   --zip-file fileb://search_lambda_package.zip

Upload raw data to the S3 bucket.

Use the AWS CLI cp command to upload data for the Tenant-1 and Tenant-2 objects to the tenantrawdata bucket (specify the name of the S3 bucket you created for this purpose):  

aws s3 cp tenant-1-data s3://tenantrawdata aws s3 cp tenant-2-data s3://tenantrawdata

The S3 bucket is set up to run the Lambda index function whenever data is uploaded so that the document is indexed in Elasticsearch.

Search data from the Kibana console.

On the Kibana console, run the following query:  

GET tenant-data/_search

This query displays all the documents indexed in Elasticsearch. In this case, you should see two, separate documents for Tenant-1 and Tenant-2.

Test the search API from API Gateway.

1. In the API Gateway console, open the search API, choose the GET method inside the search resource, and then choose Test.

2. In the test window, provide the following query string (case-sensitive) for the tenant ID, and then choose Test.  

   TenantId=Tenant-1

   The Lambda function sends a query to Amazon OpenSearch Service that filters the tenant document based on the document-level security. The method returns the document that belongs to Tenant-1.

3. Change the query string to:  

   TenantId=Tenant-2

   This query returns the document that belongs to Tenant-2.

For screen illustrations, see the Additional information section.