Specialised Services: HPC, Quantum, Satellite and Custom Chips

High performance computing (ParallelCluster, EFA, placement groups, Batch), Amazon Braket, AWS Ground Station, and AWS-designed chips: Graviton, Inferentia and Trainium.

What is it?

Most AWS lessons cover everyday building blocks. This one covers the specialised services the exam may name in a single question: supercomputing, quantum computing, satellites and AWS's own processors. You only need to recognise what each one is for.

High performance computing (HPC)

HPC means hundreds or thousands of servers with very fast network links working together on one problem that is too big or too slow for an ordinary computer - weather and fluid simulations, genomics, financial risk models, scientific research and large-scale deep learning.

  • AWS ParallelCluster: an AWS-supported, open-source tool that builds and manages HPC clusters for you. You describe the cluster in a config file and run the pcluster command-line tool; behind the scenes it uses CloudFormation. A cluster has a head node plus compute nodes that scale between a minimum and maximum, and a job scheduler queues the work - current versions use Slurm or AWS Batch (older schedulers such as SGE were removed).
  • Elastic Fabric Adapter (EFA): a network interface for EC2 that gives very low-latency, high-throughput communication between instances, which tightly coupled HPC and ML training jobs need.
  • Cluster placement groups: ask EC2 to place instances physically close together in one AZ for the lowest latency between them. (Spread placement groups do the opposite, separating instances to reduce correlated failures.)
  • AWS Batch: fully managed batch job scheduling - you submit jobs and Batch provisions the right amount of compute (EC2, Spot or Fargate) and runs them. Good for many independent jobs rather than one tightly linked simulation.
  • HPC-optimised instances: compute-optimised families with enhanced networking and the dedicated Hpc instance family are typical choices.
  • AWS CloudShell is a handy place to run tools like pcluster: a browser-based shell in the console with the AWS CLI preinstalled and your console credentials.

Amazon Braket: a managed quantum computing service for research and experiments. It gives you access to quantum hardware from several third-party providers plus simulators and managed notebooks, so you can try quantum algorithms without owning a quantum computer. You pay per task and per shot (one run of a quantum circuit), with rates that depend on the device.

AWS Ground Station: a fully managed service to communicate with satellites and process their data without building your own antennas. You schedule a contact (which satellite, start and end time, which ground station location), then either use EC2 instances launched from a Ground Station AMI to uplink and downlink, or deliver downlinked data straight to S3. Use cases: weather forecasting, earth imaging, communications and video broadcasting.

AWS-designed chips

- AWS Graviton: Arm-based general-purpose processors for EC2 (and many managed services). They usually give better price-performance and use less energy than comparable x86 instances - a Sustainability and Cost Optimization win. - AWS Inferentia: an accelerator built for machine learning inference (running trained models) at low cost. It replaces the discontinued Amazon Elastic Inference. - AWS Trainium: an accelerator built for training machine learning models. Exam trap: Inferentia and Trainium are ML accelerators, not GPUs. GPU instances on AWS use NVIDIA or AMD chips.

Explain like I'm 10

Think of AWS as a city. Most people use ordinary shops and buses (EC2, S3). This lesson covers the specialist facilities: the research supercomputer centre with its own fast internal roads (HPC with EFA and placement groups), the experimental physics lab you can book by the hour (Braket), the satellite dish farm you rent instead of building one (Ground Station), and the city's own factory that makes engines tuned for its buses (Graviton, Inferentia, Trainium).

Examples

Create an HPC cluster with ParallelCluster

# Install the CLI (or use CloudShell), then:
pcluster configure --config cluster.yaml   # interactive wizard
pcluster create-cluster \
  --cluster-name cfd-sim \
  --cluster-configuration cluster.yaml
pcluster describe-cluster --cluster-name cfd-sim

# On the head node, submit work to Slurm:
sbatch run-simulation.sh
# Delete the cluster when finished to stop charges
pcluster delete-cluster --cluster-name cfd-sim

ParallelCluster turns the YAML file into a CloudFormation stack with a head node, a Slurm scheduler and auto-scaling compute nodes.

Which specialised service?

Need                                        Service
------------------------------------------  ------------------
Build and manage an HPC cluster             ParallelCluster
Low-latency networking between instances    EFA
Put instances physically close together     Cluster placement
Run thousands of independent batch jobs     AWS Batch
Experiment with quantum algorithms          Amazon Braket
Download data from a satellite              Ground Station
Cheaper, efficient general-purpose CPU      Graviton
Low-cost ML inference                       Inferentia
Train large ML models                       Trainium

How it works

HPC on AWS: ParallelCluster reads your configuration, creates a CloudFormation stack, launches a head node and registers compute queues with the scheduler. When jobs arrive, compute nodes scale out (often in a cluster placement group with EFA for fast node-to-node traffic) and scale back in when the queue empties, so you pay only while jobs run. Shared storage such as Amazon FSx for Lustre or EFS is commonly attached.

Braket: you write a quantum circuit (for example with the Braket SDK in a managed notebook), choose a simulator or a provider's quantum device, and submit a task; results land in S3.

Ground Station: you onboard your satellite, reserve a contact at an antenna location, and during the pass AWS streams the radio data to your EC2 instances or S3 for processing - no antennas or data center of your own.

Custom chips: you pick them through instance types - Graviton-based families, Inf (Inferentia) and Trn (Trainium) instances - so adopting them is mostly a matter of choosing the right instance and recompiling or using a supported framework.

  HPC cluster (ParallelCluster)
  +-----------+     Slurm / Batch
  | Head node |---> job queue
  +-----------+         |
        |               v
  +-----------------------------------+
  | Compute nodes (scale min..max)    |
  | cluster placement group + EFA     |
  +-----------------------------------+
        |
   Shared storage (FSx for Lustre/EFS)

  Satellite --radio--> Ground Station --> EC2 / S3
  Quantum circuit --> Braket --> device/simulator --> S3

Why does it exist?

Supercomputers, quantum hardware and satellite antennas are hugely expensive and slow to build. Renting them on demand lets researchers and companies run occasional large jobs, experiment with new technology, or use satellite data without a capital investment. AWS-designed chips exist to give better price-performance and lower energy use than off-the-shelf processors for common workloads.

When to use it

Use ParallelCluster with EFA and cluster placement groups for tightly coupled simulations; AWS Batch for large numbers of independent jobs; Braket to learn and prototype quantum algorithms; Ground Station when you operate or rent satellite capacity; Graviton for most general-purpose workloads that run on Arm; Inferentia for cost-efficient inference and Trainium for training at scale.

When not to use it

Do not build an HPC cluster for a simple web app or a job one large instance can handle. Do not expect Braket to speed up production workloads today - quantum computing is still experimental. Do not move software to Graviton without checking it supports Arm. Do not use Inferentia or Trainium if your framework or model is not supported; GPU instances may be simpler.

Common mistakes

  • Calling Inferentia or Trainium GPUs - they are purpose-built ML accelerators (inference and training respectively).

  • Mixing up Inferentia (inference) and Trainium (training).

  • Using a spread placement group when the goal is the lowest latency - that needs a cluster placement group.

  • Leaving an HPC cluster's head node running after the work is done.

  • Thinking Braket is AWS's own quantum computer - it gives access to third-party quantum hardware plus simulators.

  • Assuming Ground Station requires you to build antennas - AWS provides and operates them.

  • Following old ParallelCluster tutorials that use the SGE scheduler - current versions use Slurm or AWS Batch.

Practice exercises

  1. Easy:

    Match each to its job: ParallelCluster, EFA, Braket, Ground Station, Graviton, Inferentia, Trainium.

  2. Medium:

    Explain when you would choose AWS Batch over ParallelCluster, with one example workload for each.

  3. Medium:

    Use the AWS Pricing Calculator to compare a Graviton-based instance with a similar x86 instance and note the difference.

  4. Hard:

    Design an HPC environment for a weather simulation: cluster tool, scheduler, networking, placement, storage, and how you keep cost low when idle.

Interview questions

Exam-style: Which service provides access to quantum computers and simulators?

Amazon Braket.

Exam-style: Which service lets you control satellites and process their data without building ground antennas?

AWS Ground Station.

Exam-style: Which open-source AWS-supported tool deploys and manages HPC clusters?

AWS ParallelCluster.

Exam-style: Which EC2 network interface gives low-latency, high-throughput communication for tightly coupled HPC jobs?

Elastic Fabric Adapter (EFA).

Exam-style: Which placement group strategy packs instances close together for the lowest network latency?

Cluster placement group.

Exam-style: Which AWS-designed chip is built for low-cost machine learning inference?

AWS Inferentia.

Exam-style: Which AWS-designed chip is built for training ML models?

AWS Trainium.

Exam-style: Which Arm-based AWS processor offers better price-performance for general workloads?

AWS Graviton.

What is a 'shot' in Amazon Braket?

One execution of a quantum circuit; tasks are made of many shots and pricing is per task and per shot.

What does ParallelCluster use to build its infrastructure?

AWS CloudFormation, driven by the pcluster CLI and a cluster configuration file.