Azure Container Apps vs AKS: the choice of control

When choosing a container-based application platform on Azure, there are several key thought points that are appropriate to ponder on.

The key decisions come down to the level of control the developer, team or enterprise wishes to retain for their container based applications and based on their software cadence In this blog, we take a quick look at the key pillar issues that each platform addresses, how each one compromises and where they share an offering so we can help ourselves make better decisions.

Access to Networking Controls

With Azure Kubernetes Service, there are a number of networking capabilities that we are given, but it comes with the knowledge that we must use that access to fine tune our application networking. For example when we talk about the need to access Service meshes like Istio on our containerised workloads for a more robust observability posture, AKS is our cleanest choice to make this a possibility because ACA would simply not have tooling to make that work. If we require even finer needs from Azure CNI networking such as needing to plan around pod level assignments to VNet IP addresses when dealing with private access to other Azure resources. At that point you begin thinking about wider possibilities to how the individual pods themselves (not just VM nodes) communicate in the architecture with points of network traffic scrutiny such as Azure Firewall and other VNets.

Finer Pod Management and Scheduling

Azure Container Apps will assume that you don't really wish to look at the pod instances themselves. If you wanted to distribute the different kinds of workloads within the same cluster by having multiple node pools across memory intense work, GPU enabled pools, and general CPU pools, then Azure Container Apps would not be the right choice in a case like this. There is simply no way to be looking at node pools in ACA. It even misses out on newer initiatives such KAITO and Runway that is currently available to AKS for when attempting to schedule for GPU workloads (more in this in the future).

Azure Container apps does give you an opinionated control in terms of being able to natively scale pods to zero unlike Azure Kubernetes Service, but once again, there is built in assumption that you as a developer will want that as a given.

Scaling Requirements

Now here, they are largely the same in the sense that they give capacity to scaling where the workload asks for it. Scaling in the context of any cloud service is a management we use to save on costs, but particularly with AKS it does require extra configuration we have to make in order to fully realise the cost savings. While Azure Kubernetes Service does not have automatic scaling of scaling pods all the way to zero as discussed before, it does have the ability to have the KEDA addon added onto the cluster to through a quick CLI command:

az aks update --resource-group <rg_name> --name <cluster_name> --enable-keda

The command is applicable for AKS Standard clusters where they do not ship with addons (not applicable to (AKS Automatic clusters as this is prebuilt).

Then we have to deploy a CRD onto the AKS cluster like the following to be able to scale KEDA to scale to zero pods:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-worker-scaler
spec:
  scaleTargetRef:
    name: order-worker          # name of Deployment
  minReplicaCount: 0            # KEDA controlled pod min
  maxReplicaCount: 10           # KEDA controlled pod max
  cooldownPeriod: 300      # how many seconds of idle time before scaling back to 0
  pollingInterval: 30           # how often KEDA checks the trigger
  triggers:
    - type: azure-queue
      metadata:
        queueName: orders
        queueLength: "5"        
        accountName: mystorageacct
      authenticationRef:
        name: keda-trigger-auth-azure-queue

Next, we do require to deprovision the VMs that are allocated in the node pool because while the pods (work to carry out) may have scaled to zero, we are still being billed for the VMs or nodes that sit in the nodepool that are meant to carry out that work. For this on AKS we would want to use a command such as the following where we want the amount of nodes in the pool to be a minimum of 0 and a maximum of 5:

az aks nodepool update --resource-group <rg_name> --cluster-name <cluster_name> --name workerpool --enable-cluster-autoscaler --min-count 0 --max-count 5

I wanted to show the above in order to further illustrate that at least on ACA, the above configurations are not necessary to do (no real access to CRD tooling), scaling is built in and simplified for us.

Conclusions to consider

Overall, the decision framework for leaning towards Azure Kubernetes Service or Azure Container Service revolves around control and abstraction of build and design concerns as we have seen in this blog, more so compared to the need to make the application better designed or more architecturally sound over the other choice.

Azure Container Apps effectively wants to have more of the control onto the Azure platform in the hope of giving and edge to developer velocity; being able to quickly iterate in a containerised app environment without thinking much around networking and compute resource allocation. Azure Kubernetes Service does appear to be geared towards enterprise mechanics, architects, system and network admins who wish to control how the application platform communicates with the rest of the systems in the Azure Landing Zone. That would be a simplified, codified way of concluding that.

ACA = less initial decision making , more developer velocity and application development.

AKS = finer controls of cluster behaviour, more responsibility on architects, more awareness of architecture required by team the more complex it becomes.