I have watched more than one “enterprise” ML project fall over because the architect treated cloud compute like a web server that happens to have a GPU in it. “Throw it at the cloud” became the default advice somewhere along the way, and it wrecks performance and budgets at the same time. AWS SageMaker and Azure ML both do scalable model training, but they hand you the machines in very different ways, and that difference shapes how your team works day to day.
This is part 2 of the series, and it moves past storage and permissions into compute: what the training job actually runs on, and how the runtime environment gets built. The differences show up whether you are training a custom WooCommerce recommendation engine or a full-scale LLM.
Compute architecture: persistent or on demand
Azure ML is workspace-centric, and compute is a persistent asset inside that workspace. Someone holding the “AzureML Compute Operator” role creates a cluster once and the data science team reuses it, a bit like a shared office. The catch is idle time. Nobody notices a rack of high-end GPUs sitting there doing nothing until the bill arrives.
SageMaker treats compute as a parameter of the job instead. You rarely log into a cluster. You name the instance type, say ml.m5.xlarge, in the job config, and SageMaker starts it, runs your training script, and kills it the moment the script exits. That is excellent for cost control and harder on developers who do not know much about infrastructure.
# Azure ML: Referencing a persistent cluster
# This cluster exists in the workspace until deleted
job = command(
code='./src',
command='python train.py',
compute='cpu-cluster', # The name of the existing asset
experiment_name='bbioon_ml_run'
)
# AWS SageMaker: Defining compute on-the-fly
# The instance is created for this job only
estimator = Estimator(
image_uri=image_uri,
role=role,
instance_type="ml.m5.xlarge",
instance_count=1
)Runtime environments for scalable model training
Running your code locally is easy. Running it across a distributed training cluster without falling into dependency hell is the hard part. Azure ML ships “Curated Environments”, pre-built Docker images for PyTorch, TensorFlow and Scikit-learn. They hold up well, mostly because the naming conventions are strict enough that versioning stays predictable.
SageMaker gives you three levels of customization:
- Built-in algorithms are the black box option. Fast to start, very little control.
- Script mode means you bring the Python script and AWS supplies the managed container.
- Bring your own container (BYOC) hands you the whole Docker setup, which you need for specialized libraries.
If you read the earlier infrastructure critique, none of this will surprise you: SageMaker’s flexibility costs you time up front. It scales well, but you have to understand Elastic Container Registry and IAM roles before you can ship even a basic model.
Spot instances and what they cost you
A dev team I worked with once burned $4,000 in a single weekend on on-demand GPU instances. If a training run is not time-sensitive, use spot instances. AWS makes that a one-line change with the use_spot_instances flag. The part people skip is model checkpointing, and without it a capacity interruption throws away everything the job has done so far. You save around 70% and pay for it in architectural complexity.
# AWS Spot Instance Configuration
estimator = Estimator(
instance_type="ml.p3.2xlarge",
use_spot_instances=True,
max_run=3600,
max_wait=7200, # Willing to wait for capacity
checkpoint_s3_uri="s3://bbioon-checkpoints/model/"
)If this kind of pipeline work is eating your dev hours, I take it on. I have been wrestling with WordPress since the 4.x days and have wired up more cloud ML pipelines than I can count.
Which one to pick
Azure ML is the easier place to start, because everything is a modular resource inside a workspace. That suits teams where the data scientists would rather not write YAML. SageMaker is built for people who already do MLOps: it asks for more AWS knowledge and gives back much finer control over distributed training. Read the AWS SageMaker docs and the Azure ML resources against your team’s current stack before you sign up for either.
