category
本文将指导您完成选择Azure容器服务的过程。它概述了对于某些工作负载来说常见且关键的功能级别考虑因素。它可以帮助您做出决策,以确保您的工作负载满足可靠性、安全性、成本优化、卓越运营和性能效率的要求。
注:
本文是从选择Azure容器服务开始的系列文章的第二部分。我们强烈建议您先阅读这篇概述文章,以了解这些架构考虑的背景。
概述
本文中的考虑因素分为四类:
- 架构和网络考虑因素
- 操作系统支持
- 网络地址空间
- 了解交通流量
- 规划子网
- 可用入口IP数
- 用户自定义路由和NAT网关支持
- 专用网络集成
- 协议覆盖范围
- 负载平衡
- 服务发现
- 自定义域和托管TLS
- 双向TLS
- 特定Azure服务的网络概念
- 安全考虑
- 通过使用网络策略为集群内流量提供安全保障
- 网络安全组
- Azure密钥库集成
- 管理身份支持
- 使用Defender for Containers进行威胁防护和漏洞评估
- 安全基线
- Azure架构良好的安全框架
- 操作注意事项
- 更新和补丁
- 容器图像更新
- 垂直基础设施可扩展性
- 横向基础设施可扩展性
- 应用程序可扩展性
- 可观察性
- 构建良好的卓越运营框架
- 可靠性考虑
- 服务水平协议
- 通过可用区实现冗余
- 健康检查和自我修复
- 零停机应用程序部署
- 资源限制
- 结构良好的可靠性框架
请注意,本文重点介绍Azure容器服务的一个子集,这些服务为web应用程序和API、网络、可观察性、开发人员工具和操作提供了成熟的功能集:Azure Kubernetes Service(AKS)、Azure容器应用程序和容器web应用程序。有关所有Azure容器服务的完整列表,请参阅容器服务产品类别页面。
注:
在本指南中,术语工作负载是指支持业务目标或业务流程执行的应用程序资源的集合。工作负载使用多个组件,如API和数据存储,它们协同工作以提供特定的端到端功能。
架构考虑因素
本节描述了在不需要大量停机或重新部署的情况下难以逆转或纠正的架构决策。对于网络和安全等基本组件,特别有必要牢记这一点。
这些考虑因素并不特定于架构良好的框架支柱。但是,当您选择Azure容器服务时,它们应该根据业务需求进行额外的审查和评估。
操作系统支持
大多数容器化应用程序在Linux容器中运行,所有Azure容器服务都支持Linux容器。对于需要Windows容器的工作负载组件,您的选择更加有限。
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Linux support | ✅ | ✅ | ✅ |
Windows support | ❌ | ✅ | ✅ |
Mixed OS support | ❌ | ✅ | ❌* |
*Mixed OS support for Web App for Containers requires separate Azure App Service plans for Windows and Linux.
Networking considerations
It's important to understand networking design early in your planning processes due to security and compliance constraints and imposed guidelines. In general, the major differences among the Azure services covered in this guide depend on preference:
- Container Apps is a PaaS offering that provides many Azure-managed networking features, like service discovery and internal managed domains. Workload teams that need a bit more configurability can use workload/dedicated profiles before considering alternatives to maximize their networking options.
- AKS is the most configurable of the three services and provides the most control over network flow. For example, it provides custom ingress controllers and the control of intra-cluster traffic via Kubernetes network policies. Workload teams can take advantage of various Azure managed networking add-ons, as well as install and operate any add-ons from the broader Kubernetes ecosystem.
- Web App for Containers is feature of App Service. Thus, the networking concepts, especially private networking integration, are very specific to App Service. This service will be familiar to workload teams that already use App Service. Teams that don't have experience with App Service and that want a more familiar Azure virtual network integration are encouraged to consider Container Apps.
Keep in mind that networking is a foundational infrastructure layer. It's often difficult to make changes in design without re-deploying the workload, which can lead to downtime. Therefore, if your workload has specific networking requirements, review this section carefully before you narrow down your Azure container service selection.
Network address spaces
When you integrate applications into virtual networks, you need to do some IP address planning to ensure that enough IP addresses are available for container instances. During this process, plan for additional addresses for updates, blue/green deployments, and similar situations in which extra instances are deployed, which consumes additional IP addresses.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Dedicated subnets | Consumption plan: optional Dedicated plan: required |
Required | Optional |
IP address requirements | Consumption plan: See Consumption-only environment. Dedicated plan: See Workload profiles environment. |
See Azure virtual networks for AKS. | See App Service subnet requirements. |
Note that AKS requirements depend on the chosen network plug-in. Some network plug-ins for AKS require broader IP reservations. Details are outside the scope of this article. For more information, see Networking concepts for AKS.
Understanding traffic flow
The types of traffic flow required for a solution can affect the network design.
The following sections provide information about various networking constraints. These constraints affect your need to deploy additional subnets, depending on whether you require:
- Multiple colocated workloads.
- Private and/or public ingress.
- An access-controlled flow of east-west traffic in a cluster (for Container Apps and AKS) or within a virtual network (for all Azure container services).
Subnet planning
Ensuring that you have a subnet that's large enough to include instances of your application for your workload isn't the only factor that dictates the network footprint where these applications are deployed.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Support for colocated workloads within a subnet* | ❌* | ✅ | N/A* |
*This describes a best practice, not a technical limitation.
For Container Apps, subnet integration applies only to a single Container Apps environment. Each Container Apps environment is constrained to a single ingress IP, public or private.
Each Container Apps environment is meant only for a single workload in which dependent applications are colocated. Therefore, you need to introduce additional Azure networking appliances for ingress load balancing if you need both public and private ingress. Examples include Azure Application Gateway and Azure Front Door. Also, if you have multiple workloads that need to be segregated, additional Container Apps environments are required, so an additional subnet must be allocated for each environment.
AKS provides granular east-west network flow control within the cluster in the form of Kubernetes network policy. This flow control enables you to segment multiple workloads with different network security boundaries within the same cluster.
For Web App for Containers, there are no constraints on how many apps you can integrate with a single subnet, as long as the subnet is large enough. There are no best practices for access control between web apps in the same virtual network. Each web app independently manages access control for east-west or north-south traffic from the virtual network or internet, respectively.
Note
You can't resize subnets that have resources deployed in them. Take extra care when you plan your network to avoid needing to redeploy entire workload components, which can lead to downtime.
Number of ingress IPs available
The following table takes the previous subnet planning section into consideration to define how many IPs can be exposed for an arbitrary number of applications that are hosted in a single deployment of an Azure container service.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Number of ingress IPs | One | Many | App Service Environment: One No App Service Environment: Many |
Container Apps allows one IP per environment, public or private. AKS allows any number of IPs, public or private. Web App for Containers, outside of an App Service Environment, allows one public IP for all apps within an App Service plan and multiple, different private IPs that use Azure private endpoints.
It's important to note that web apps that are integrated into an App Service Environment only receive traffic through the single ingress IP that's associated with the App Service Environment, whether it's public or private.
User-defined routes and NAT gateway support
If a workload requires user-defined routes (UDRs) and NAT gateway capabilities for granular networking control, Container Apps requires the use of workload profiles. UDR and NAT gateway compatibility is not available in the consumption-only plan for ACA.
AKS and Web App for Containers implement these two networking features through standard virtual network functionality or virtual network integration, respectively. To elaborate, AKS node pools and Web App for Containers in an App Service Environment are already direct virtual network resources. Web App for Containers that aren't in an App Service Environment support UDRs and NAT gateway via virtual network integration. With virtual network integration, the resource technically doesn't reside directly in the virtual network, but all of its outbound access flows through the virtual network, and the network's associated rules affect traffic as expected.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
UDR support | Consumption-only plan: ❌ Workload profile plan: ✅ |
✅ | ✅ |
NAT gateway support | Consumption-only plan: ❌ Workload profile plan: ✅ |
✅ | ✅ |
Private networking integration
For workloads that require strict Layer 4 private networking for both ingress and egress, you should consider Container Apps, AKS, and the single-tenant App Service Environment SKU, where workloads are deployed into a self-managed virtual network, providing the customary granular private networking controls.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Private ingress into a virtual network | ✅ | ✅ | Via private endpoint |
Private egress from a virtual network | ✅ | ✅ | Via virtual network integration |
Fully suppressed public endpoint | ✅ | ✅ | App Service Environment only |
Private networking with Web App for Containers
Web App for Containers provides additional networking features that aren't surfaced in the same way by the other Azure services described in this article. To implement strict private networking requirements, workload teams need to familiarize themselves with these networking concepts. Carefully review these networking features:
If you want a PaaS solution and prefer networking concepts that are shared across multiple Azure solutions, you should consider Container Apps.
Protocol coverage
An important consideration for the hosting platform is the networking protocols that are supported for incoming application requests (ingress). Web App for Containers is the strictest option, supporting only HTTP and HTTPS. Container Apps additionally allows incoming TCP connections. AKS is the most flexible, supporting unconstrained use of TCP and UDP on self-selected ports.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Protocol and port support | HTTP (port 80)* HTTPS (port 443)* TCP (ports 1-65535, except 80 and 443) |
TCP (any port) UDP (any port) |
HTTP (port 80) HTTPS (port 443) |
WebSocket support | ✅ | ✅ | ✅ |
HTTP/2 support | ✅ | ✅ | ✅ |
*In the Container Apps environment, HTTP/S can be exposed on any port for intra-cluster communication. In that scenario, built-in Container Apps HTTP features like CORS and session affinity don't apply.
Both Container Apps and Web App for Containers support TLS 1.2 for their built-in HTTPS ingress.
Load balancing
With Container Apps and Web App for Containers, Azure fully abstracts away the Layer 4 and Layer 7 load balancers.
In contrast AKS uses a shared responsibility model in which Azure manages the underlying Azure infrastructure that the workload team configures by interfacing with the Kubernetes API. For Layer 7 load balancing in AKS, you can choose an Azure-managed options, for example the AKS managed application routing add-on or the Application Gateway for Containers, or deploy and self-manage an ingress controller of your choice.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Layer 4 load balancer | Azure-managed | Shared responsibility | Azure-managed |
Layer 7 load balancer | Azure-managed | Shared or self-managed | Azure-managed |
Service discovery
In cloud architectures, runtimes can be removed and re-created at any time to rebalance resources, so instance IP addresses regularly change. These architectures use fully qualified domain names (FQDNs) for reliable and consistent communication.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Service discovery | Azure-managed FQDN | Kubernetes configurable | Azure-managed FQDN |
Web Apps for Containers provides public ingress (north-south communication) FQDNs out of the box. No additional DNS configuration is required. However, there's no built-in mechanism to facilitate or restrict traffic between other apps (east-west communication).
Container Apps also provides public ingress FQDNs. However, Container Apps goes further by allowing the app FQDN to be exposed and restricting traffic only within the environment. This functionality makes it easier to manage east-west communication and enable components like Dapr.
Kubernetes deployments are not initially discoverable within or from outside the cluster. You must create Kubernetes services as defined by the Kubernetes API, which then expose applications to the network in an addressable way.
Important
Only Container Apps and AKS provide service discovery through internal DNS schemes within their respective environments. This functionality can simplify DNS configurations across dev/test and production environments. For example, you can create these environments with arbitrary service names that have to be unique only within the environment or cluster, so they can be the same across dev/test and production. With Web App for Containers, service names must be unique across different environments to avoid conflicts with Azure DNS.
Custom domains and managed TLS
Both Container Apps and Web App for Containers provide out-of-the-box solutions for custom domains and certificate management.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Configure custom domains | Out of the box | BYO | Out of the box |
Managed TLS for Azure FQDNs | Out of the box | N/A | Out of the box |
Managed TLS for custom domains | In preview | BYO | Out of the box or BYO |
AKS users are responsible for managing DNS, cluster configurations and TLS certificates for their custom domains. Although AKS does not offer managed TLS, customers can leverage software from the Kubernetes ecosystem, for example the popular cert-manager to manage TLS certificates.
Mutual TLS
Another alternative for restricting incoming traffic is mutual TLS (mTLS). Mutual TLS is a security protocol that ensures that both the client and server in communication are authenticated. To accomplish authentication, both parties exchange and verify certificates before any data is transmitted.
Web App for Containers has built-in mTLS support for incoming client connections. However, the application needs to validate the certificate by accessing the X-ARR-ClientCert
HTTP header that the App Service platform forwards.
Container Apps also has built-in support for mTLS. It forwards the client certificate to the application in the HTTP header X-Forwarded-Client-Cert. You can also easily enable automatic mTLS for internal communication between apps in a single environment.
Mutual TLS in AKS can be implemented through the Istio-based service mesh as a managed add-on, which includes mTLS capabilities for incoming client connections and intra cluster communication between services. Workload teams could also choose to install and manage another service mesh offering from the Kubernetes ecosystem. These options make mTLS implementation in Kubernetes the most flexible.
Service-specific networking concepts
The preceding sections describe some of the most common considerations to take into account. For more details and to learn more about networking features that are specific to individual Azure container services, see these articles:
The preceding sections focus on network design. Continue to the next section to learn more about networking security and securing network traffic.
Security considerations
Failure to address security risks can lead to unauthorized access, data breaches, or leaks of sensitive information. Containers offer an encapsulated environment for your application. The hosting systems and underlying network overlays, however, require additional guardrails. Your choice of Azure container service needs to support your specific requirements for securing each application individually and provide proper security measures to prevent unauthorized access and mitigate the risk of attacks.
Security comparison overview
Most Azure services, including Container Apps, AKS, and Web App for Containers, integrate with key security offerings, including Key Vault and managed identities.
Of the services in this guide, AKS offers the most configurability and extensibility in part by surfacing underlying components, which often can be secured via configuration options. For example, customers can disable local accounts to the Kubernetes API server or turn on automatic updates to underlying nodes via configuration options.
For a detailed comparison, carefully review the following considerations to ensure that your workload security requirements can be met.
Kubernetes control plane security
AKS offers the most flexibility of the three options considered in this article, providing full access to the Kubernetes API so that you can customize container orchestration. This access to the Kubernetes API, however, also represents a significant attack surface, and you need to secure it.
Important
Note that this section isn't relevant for Web App for Containers, which uses the Azure Resource Manager API as its control plane.
Identity-based security
Customers are responsible for securing identity-based access to the API. Out of the box, Kubernetes provides its own authentication and authorization management system, which also needs to be secured with access controls.
To take advantage of a single plane of glass for identity and access management on Azure, it's a best practice to disable Kubernetes-specific local accounts and instead implement AKS-managed Microsoft Entra integration together with Azure RBAC for Kubernetes. If you implement this best practice, administrators don't need to perform identity and access management on multiple platforms.
Container Apps | AKS | |
---|---|---|
Kubernetes API access controls | No access | Full access |
Customers who use Container Apps don't have access to the Kubernetes API. Microsoft provides security for this API.
Network-based security
If you want to restrict network access to the Kubernetes control plane, you need to use AKS, which provides two options. The first option is to use private AKS clusters, which use Azure Private Link between the API server's private network and the AKS cluster's private network. The second option is API Server VNet integration (preview), where the API server is integrated into a delegated subnet. See the documentation to learn more.
There are consequences to implementing network-restricted access to the Kubernetes API. Most notably, management can be performed only from within the private network. Typically this means you need to deploy self-hosted agents for Azure DevOps or GitHub Actions. To learn about other limitations, see the product-specific documentation.
Container Apps | AKS | |
---|---|---|
Kubernetes API network security | Not configurable in PaaS | Configurable: public IP or private IP |
These considerations don't apply to Container Apps. Because it's PaaS, Microsoft abstracts away the underlying infrastructure.
Data plane network security
The following networking features can be used to control access to, from, and within a workload.
Using network policies to provide security for intra-cluster traffic
Some security postures require network traffic segregation within an environment, for example when you use multitenant environments to host multiple or multi-tiered applications. In these scenarios, you should choose AKS and implement network policies, a cloud-native technology that enables granular configuration of Layer 4 networking within Kubernetes clusters.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Network policies | Consumption plan: ❌ Dedicated plan: ❌ |
✅ | ❌ |
Of the three Azure services described in this article, AKS is the only one that supports further workload isolation within the cluster. Network policies aren't supported in Container Apps or Web App for Containers.
Network security groups
In all scenarios, you can regulate networking communication within the wider virtual network by using network security groups, which enables you to use Layer 4 traffic rules that regulate ingress and egress at the virtual network level.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Network security groups | Consumption plan: ✅ Dedicated plan: ✅ |
✅ | ✅ VNet-integrated apps: egress only |
IP restrictions for ingress
Typically network traffic restrictions are applied via Layer 4 rules described above. However, in PaaS scenarios of applications without virtual network integration, it can be useful to restrict traffic on the application layer.
Container Apps and Web App for Containers provide built-in source IP restrictions for ingress traffic on individual applications.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Ingress IP restrictions on application layer | Out of the box | Self-managed or via managed add-on | Out of the box |
Resource consumption | - | Consumes cluster resources | - |
For AKS workloads, implementation depends on the chosen ingress controller. Both self-managed and the Azure managed application routing add-on consume cluster resources.
Application-level security
You need to secure workloads not just at the network and infrastructure level, but also at the workload and application level. Azure container solutions integrate with Azure security offerings to help standardize security implementation and controls for your applications.
Key Vault integration
It's a best practice to store and manage secrets, keys, and certificates in a key management solution like Key Vault, which provides enhanced security for these components. Instead of storing and configuring secrets in code or in an Azure compute service, all applications should integrate with Key Vault.
Key Vault integration enables application developers to focus on their application code. All three of the Azure container services described in this article can automatically sync secrets from the Key Vault service and provide them to the application, typically as environment variables or mounted files.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Key Vault integration | ✅ | ✅ | ✅ |
For more information, see:
- Manage secrets in Azure Container Apps
- Integrate Key Vault with AKS
- Use Key Vault references as app settings in Azure App Service
Managed identity support
Applications with assigned managed identities can access Azure resources without passwords. All container services mentioned in this guide support managed identities.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Managed identity support | ✅ | ✅ | ✅ |
For more information, see:
- Use a managed identity in AKS
- Managed identities in Azure Container Apps
- How to use managed identities for App Service
Threat protection and vulnerability assessments with Defender for Containers
Threat protection against vulnerabilities is also important. It's a best practice to use Defender for Containers. Vulnerability assessments are supported in Azure container registries, so they can be used by any Azure container service, not just the ones described in this article. Defender for Containers runtime protection, however, is available only for AKS.
As AKS exposes the native Kubernetes API, cluster security also can be evaluated with Kubernetes specific security tooling from the Kubernetes ecosystem.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Runtime threat protection | ❌ | ✅ | ❌ |
For more information, see Containers support matrix in Defender for Cloud.
Note that container image vulnerability assessments aren't real-time scans. The Azure container registry is scanned at regular intervals.
Security baselines
In general, most Azure container services integrate Azure security offerings. Overall, keep in mind that a security feature set is just a small part of implementing cloud security. For more information about implementing security for container services, see the following service-specific security baselines:
- Azure security baseline for Container Apps
- Azure security baseline for Azure Kubernetes Service
- Azure security baseline for App Service
The security baselines cover other Azure integrations, including hardware encryption and logging, that are out of scope for this article.
Well-Architected Framework for Security
This article focuses on the main differences among the container services features described here.
For more complete security guidance about AKS, see Well-Architected Framework review - AKS.
Operational considerations
To successfully run a workload in production, teams need to implement operational excellence practices, including centralized logging, monitoring, scalability, regular updates and patching, and image management.
Updates and patches
It is important that an application's underlying OS is updated and regularly patched. Keep in mind, however, that with every update there's a risk of failure. This section and the next one describe the main considerations for the three container services with regard to the shared responsibility between the customer and the platform.
As a managed Kubernetes service, AKS will provide the updated images for the node OS and control plane components. But workload teams are responsible for applying updates to their clusters. You can manually trigger updates or leverage the cluster auto-upgrade channels feature to ensure your clusters are up to date. See the AKS day-2 operations guide to learn about patching and upgrading AKS clusters.
Container Apps and Web App for Containers are PaaS solutions. Azure is responsible for managing updates and patches, so customers can avoid the complexity of AKS upgrade management.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Control plane updates | Platform | Customer | Platform |
Host updates and patches | Platform | Customer | Platform |
Container image updates and patches | Customer | Customer | Customer |
Container image updates
Regardless of the Azure container solution, customers are always responsible for their own container images. If there are security patches for container base images, it's your responsibility to rebuild your images. To get alerts about these vulnerabilities, use Defender for Containers for containers that are hosted in Container Registry.
Scalability
Scaling is used to adjust resource capacity to meet demands, adding more capacity to ensure performance and removing unused capacity to save money. When you choose a container solution, you need to consider infrastructure constraints and scaling strategies.
Vertical infrastructure scalability
Vertical scaling refers to the ability to increase or decrease existing infrastructure, that is, compute CPU and memory. Different workloads require different amounts of compute resources. When you choose an Azure container solution, you need to be aware of the hardware SKU offerings that are available for a particular Azure service. They vary and can impose additional constraints.
For AKS, review the sizes for virtual machines in Azure documentation and the per-region AKS restrictions.
These articles provide details about SKU offerings for the other two services:
Horizontal infrastructure scalability
Horizontal scaling refers to the ability to increase or decrease capacity via new infrastructure, like VM nodes. During scaling increases or decreases, the Container Apps consumption tier abstracts the underlying virtual machines. For the remaining Azure container services, you manage the horizontal scaling strategy by using the standard Azure Resource Manager API.
Note that scaling out and in includes re-balancing of instances, so it also creates a risk of downtime. The risk is smaller than the corresponding risk with vertical scaling. Nevertheless, workload teams are always responsible for ensuring that their applications can handle failure and for implementing graceful startups and shutdowns of their applications to avoid downtime.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Infrastructure scale in and out | Consumption plan: N/A Dedicated plan: configurable |
Configurable | Configurable |
Flexible hardware provisioning | Consumption plan: N/A Dedicated plan: abstracted with workload profiles |
Any VM SKU | Abstracted. See App Service plan. |
Important
The hardware provisioning options available through the Container Apps Dedicated plan (workload profiles) and Web App for Containers (App Service plans) aren't as flexible as AKS. You need to familiarize yourself with the SKUs available in each service to ensure that your needs are met.
Application scalability
The typical measure on which to trigger scaling of infrastructure and applications is resource consumption: CPU and memory. Some container solutions can scale container instance count on metrics with application-specific context, like HTTP requests. For example, AKS and Container Apps can scale container instances based on message queues via KEDA and many other metrics via its scalers. These capabilities provide flexibility when you're choosing the scalability strategy for your application. Web App for Containers relies on the scalability options provided by Azure. (See the following table.) Web App for Containers doesn't support custom scaler configurations like KEDA.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Container scale out | HTTP, TCP, or metrics-based (CPU, memory, event-driven). | Metrics-based (CPU, memory, or custom). | Manual, metrics-based, or automatic (preview). |
Event-driven scalability | Yes. Cloud-native. | Yes. Cloud-native. Additional configuration required. | Yes. Azure-resource specific. |
Observability
Workload instrumentation
Gathering metrics for complex or multi-tiered applications can be challenging. To get metrics, you can integrate containerized workloads with Azure Monitor in two ways:
- Automatic instrumentation. No code changes required.
- Manual instrumentation. Minimal code changes required to integrate and configure the SDK and/or client.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Automatic instrumentation via platform | ❌ | ❌ | Partial support* |
Automatic instrumentation via agent | ❌ | Partial support* | N/A |
Manual instrumentation | Via SDK or OpenTelemetry | Via SDK or OpenTelemetry | Via SDK or OpenTelemetry |
*AKS and Web App for Containers support automatic instrumentation for certain configurations of Linux and Windows workloads, depending on the application language. For more information, see these articles:
- Automatic instrumentation supported environments, languages, and resource providers
- Zero instrumentation application monitoring for Kubernetes
Instrumentation within application code is the responsibility of application developers, so it's independent of any Azure container solution. Your workload can use solutions like:
Logs
All Azure container services provide application and platform log functionality. Application logs are console logs that are generated by your workload. Platform logs capture events that occur at the platform level, outside the scope of your application, like scaling and deployments.
The main differences among the logging functionality for container services pertain to platform logging: what gets logged and how logs are organized internally. Azure Monitor is the main logging service in Azure that integrates with these services. Monitor uses resource logs to separate logs that come from different sources into categories. One way to determine which logs are available from each Azure service is to look at the resource log categories that are available for each of the services.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Support for log streaming (real-time streaming) | ✅ | ✅ | ✅ |
Support for Azure Monitor | ✅ | ✅ | ✅ |
Azure Monitor resource logs | Console and system | Kubernetes API server, Audit, Scheduler, Cluster Autoscaler, and more | ConsoleLogs, HTTPLogs, EnvironmentPlatformLogs, and more |
For a detailed description of each of the resource logs in the table, select the links in the table.
Here's a short summary of the logging capabilities of the container services:
- Container Apps abstracts all of its internal Kubernetes logs into two categories. One, called Console logs, contains the workload container logs. A second System category contains all platform-related logs.
- AKS provides all Kubernetes-related logs and granular control over what can be logged. It also retains full compatibility with Kubernetes client tools for log streaming, like kubectl.
- Web App for Containers provides many categories of resource logs because its platform (App Service) isn't exclusively for container workloads. For container-specific operations that manage its internal Docker platform, it provides the AppServicePlatformLogs log category. Another important category is AppServiceEnvironmentPlatformLogs, which logs events like scaling and configuration changes.
Well-Architected Framework for Operational Excellence
This article focuses on the main differences among the container services features described here. See these articles to review the complete Operational Excellence guidance for the following services:
Reliability
Reliability refers to the ability of a system to react to failures and remain fully functional. At the application-software level, workloads should implement best practices like caching, retry, circuit breaker patterns, and health checks. At the infrastructure level, Azure is responsible for handling physical failures, like hardware failures and power outages, in datacenters. Failures can still happen. Workload teams should select the appropriate Azure service tier and apply necessary minimum-instance configurations to implement automatic failovers between availability zones.
To choose the appropriate service tier, you need to understand how service-level agreements (SLAs) and availability zones work.
Service-level agreements
Reliability is commonly measured by business-driven metrics like SLAs or recovery metrics like recovery time objectives (RTOs).
Azure has many SLAs for specific services. There's no such thing as a 100% service level, because failures can always occur in software and hardware, and in nature, for example, storms and earthquakes. An SLA isn't a guarantee but rather a financially backed agreement of service availability.
For the latest SLAs and details, download the latest SLA for Microsoft Online Services document from the Microsoft licensing website.
Free vs. paid tiers
Generally, free tiers of Azure services don't offer an SLA, which makes them cost-effective choices for non-production environments. For production environments, however, it's a best practice to choose a paid tier that has an SLA.
Additional factors for AKS
AKS has different SLAs for different components and configurations:
- Control plane. The Kubernetes API server has a separate SLA.
- Data plane. Node pools use the underlying VM SKU SLAs.
- Availability zones. There are different SLAs for the two planes, depending on whether the AKS cluster has availability zones enabled and running multiple instances across availability zones.
Note that when you use multiple Azure services, composite SLOs might differ from and be lower than individual service SLAs.
Redundancy with availability zones
Availability zones are distinct datacenters that have independent electric power, cooling, and so on, within a single region. The resulting redundancy increases the tolerance of failures without requiring you to implement multi-region architectures.
Azure has availability zones in every country/region in which Azure operates a datacenter region. To allow multiple instances of containers to cross availability zones, be sure to select SKUs, service tiers, and regions that provide availability zone support.
Feature | Container Apps | AKS | Web App for Containers |
---|---|---|---|
Availability zone support | Full | Full | Full |
For example, an application or infrastructure that's configured to run a single instance becomes unavailable if a problem occurs in the availability zone where the hardware is hosted. To fully use availability zone support, you should deploy workloads with a minimum configuration of three instances of the container, spread across zones.
Health checks and self-healing
Health check endpoints are crucial to a reliable workload. But building those endpoints is only half of the solution. The other half is controlling what the hosting platform does, and how, when there are failures.
To better distinguish among types of health probes, take a look at the built-in types of probes from Kubernetes:
- Startup. Checks whether the application successfully started.
- Readiness. Checks whether the application is ready to handle incoming requests.
- Liveness. Checks whether the application is still running and responsive.
Another important consideration is how often those health checks are requested from the application (internal granularity). If you have a long interval between these requests, you might continue to serve traffic until the instance is deemed unhealthy.
Most applications support health checks via the HTTP(S) protocol. However, some might need other protocols, like TCP or gRPC, to perform those checks. Keep this in mind when you design your health check system.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Startup probes | ✅ | ✅ | Partial support |
Readiness probes | ✅ | ✅ | ❌ |
Liveness probes | ✅ | ✅ | ✅ |
Interval granularity | Seconds | Seconds | 1 minute |
Protocol support | HTTP(S) TCP |
HTTP(S) TCP gRPC |
HTTP(S) |
Health checks are easiest to implement in Web App for Containers. There are some important considerations:
- Its startup probes are built in and can't be changed. It sends an HTTP request to the starting port of your container. Any response from your application is considered a successful start.
- It doesn't support readiness probes. If the startup probe is successful, the container instance is added to the pool of healthy instances.
- It sends the health check at one-minute intervals. You can't change the interval.
- The minimum threshold that you can set for an unhealthy instance to be removed from the internal load balancing mechanism is two minutes. The unhealthy instance gets traffic for at least two minutes after it fails a health check. The default value for this setting is 10 minutes.
Container Apps and AKS, on the other hand, are much more flexible and offer similar options. In terms of specific differences, AKS provides the following options for performing health checks, which are not available in Container Apps:
Auto-healing
To identify a bad container instance and stop sending traffic to it is a start. The next step is to implement auto-healing. Auto-healing is the process of restarting the application in an attempt to recover from an unhealthy state. Here's how the three container services compare:
- In Web App for Containers, there's no option to restart a container instance immediately after a health check fails. If the instance keeps failing for one hour, it's replaced by a new instance. There's another feature, called auto-healing, that monitors and restarts instances. It's not directly related to health checks. It uses various application metrics, like memory limits, HTTP request duration, and status codes.
- Container Apps and AKS automatically try to restart a container instance if the liveness probe reaches the defined failure threshold.
Zero-downtime application deployments
The ability to deploy and replace applications without causing any downtime for users is crucial for a reliable workload. All three of the container services that are described in this article support zero-downtime deployments, but in different ways.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Zero-downtime strategy | Rolling update | Rolling update, plus all other Kubernetes strategies | Deployment slots |
Please note that the application architectures must also support zero-downtime deployment. See Azure Well-Architected Framework for guidance.
Resource limits
Another important component of a reliable shared environment is your control over the resource usage (like CPU or memory) of your containers. You need to avoid scenarios in which a single application takes up all the resources and leaves other applications in a bad state.
Container Apps | AKS | Web App for Containers | |
---|---|---|---|
Resource limits (CPU or memory) | Per app/container | Per app/container Per namespace |
Per App Service plan |
- Web App for Containers: You can host multiple applications (containers) in a single App Service plan. For example, you might allocate a plan with two CPU cores and 4 GiB of RAM in which you can run multiple web apps in containers. You can't, however, restrict one of the apps to a certain amount of CPU or memory. They all compete for the same App Service plan resources. If you want to isolate your application resources, you need to create additional App Service plans.
- Container Apps: You can set CPU and memory limits per application in your environment. You're restricted, however, to a set of allowed combinations of CPU and memory. For example, you can't configure one vCPU and 1 GiB of memory, but you can configure one vCPU and 2 GiB of memory. A Container Apps environment is analogous to a Kubernetes namespace.
- AKS: You can choose any combination of vCPU and memory, as long as your nodes have the hardware to support it. You can also limit resources at the namespace level if you want to segment your cluster that way.
Well-Architected Framework for Reliability
This article focuses on the main differences among the container services features in Azure. If you want to review the complete reliability guidance for a specific service, see these articles:
- Well-Architected Framework review for AKS
- Reliability in Container Apps
- Azure App Service and reliability
Conclusion
Well-architected solutions set the foundations for successful workloads. Although architectures can be adjusted as a workload grows and teams progress on their cloud journeys, some decisions, especially around networking, are difficult to reverse without significant downtime or re-deployment.
In general, when you compare Azure container services, a theme emerges: AKS surfaces the most underlying infrastructure, thus offering the greatest configurability and extensibility. The amount of operational overhead and complexity is highly variable for AKS workloads. Some teams can greatly reduce the operational overhead by using Microsoft managed add-ons and extensions, as well as auto-upgrade features. Other customers may prefer full control of the cluster in order to leverage full extensibility of Kubernetes and the CNCF ecosystem. For example, although Microsoft offers Flux as a managed GitOps extension, many teams choose instead to setup and operate ArgoCD on their own.
Workload teams that, for example, do not require CNCF applications, have less operations experience or prefer to focus on application features might prefer a PaaS offering. We recommend that they first consider Container Apps.
Although Container Apps and Web App for Containers are both PaaS offerings that provide similar levels of Microsoft-managed infrastructure, a key difference is that Container Apps is closer to Kubernetes and provides additional cloud-native capabilities for service discovery, event-driven autoscaling, Dapr integration, and more. However, teams that don't need these capabilities and are familiar with App Service networking and deployment models might prefer Web App for Containers.
Generalizations can help you narrow down the list of Azure container services to consider. But keep in mind that you also need to verify your choice by examining individual requirements in detail and matching them to service-specific feature sets.
Contributors
This article is maintained by Microsoft. It was originally written by the following contributors.
Principal authors:
- Andre Dewes | Senior Customer Engineer
- Xuhong Liu | Senior Service Engineer
- Marcos Martinez | Senior Service Engineer
- Julie Ng | Senior Engineer
Other contributors:
- Mick Alberts | Technical Writer
- Martin Gjoshevski | Senior Customer Engineer
- Don High | Principal Customer Engineer
- Nelly Kiboi | Service Engineer
- Faisal Mustafa | Senior Customer Engineer
- Walter Myers | Principal Customer Engineering Manager
- Sonalika Roy | Senior Customer Engineer
- Paolo Salvatori | Principal Customer Engineer
- Victor Santana | Principal Customer Engineer
To see nonpublic LinkedIn profiles, sign in to LinkedIn.
Next steps
Related resources
- 登录 发表评论
- 2 次浏览
最新内容
- 6 hours 34 minutes ago
- 8 hours ago
- 9 hours ago
- 3 days ago
- 3 days 7 hours ago
- 3 days 7 hours ago
- 3 days 8 hours ago
- 3 days 8 hours ago
- 1 week ago
- 1 week ago