Scaling Python with Ray on one machine gets you further than most people expect. Then the laptop fan spins up like a jet engine, the script still takes twenty minutes, and it turns out you are out of cores. Another tuning pass will not fix that. More machines will, along with something to coordinate them.
Fourteen years of chasing performance bottlenecks has taught me that the local-to-cloud jump is where projects stall. The code is rarely the hard part. What gets people is the environment, the networking, and the bill for a c7g.8xlarge that ran all weekend because nobody wrote the teardown script. If you want the single-machine basics first, they are in my guide to distributed computing with Ray.
From one node to a cluster
A Ray cluster is a head node that hands out the work and a set of worker nodes that do it. On AWS, both are EC2 instances sitting on a private network.
You never SSH into each machine to sync your code. The Ray Autoscaler starts instances based on what your tasks ask for, and that is the part that saves real time. Get the YAML wrong, though, and you hit race conditions where workers come up before the head node has finished initializing.
Defining the cluster in YAML
Ray reads a YAML file. I find it easier to live with than JSON, and it covers instance types and setup commands in the same place. This is the AWS config I keep reaching for, with Spot instances on the workers to hold the cost down:
cluster_name: ray_cluster_prod
provider:
type: aws
region: us-east-1
max_workers: 10
available_node_types:
head_node:
node_config:
InstanceType: c7g.8xlarge
ImageId: ami-06687e45b21b1fca9
worker_node:
min_workers: 2
max_workers: 5
node_config:
InstanceType: c7g.8xlarge
InstanceMarketOptions:
MarketType: spotCheck your AWS quotas before you blame Ray. I lost three hours to a cluster that refused to scale past four workers, and the cause was a vCPU limit on the client’s account. The official EC2 docs are worth ten minutes when you are matching instance types to a workload.
Changing the code for a cluster
Less changes than you would think. Locally you called ray.init(). On a cluster the workers have to find the head node, and if you submit through the Jobs CLI you never hardcode an IP, because Ray builds the connection string for you.
import ray
# The 'auto' address tells Ray to find the existing cluster
ray.init(address='auto')
@ray.remote
def bbioon_process_data(chunk_id):
# Your heavy logic here
return f"Processed {chunk_id}"
# Submitting tasks across the cluster
results = ray.get([bbioon_process_data.remote(i) for i in range(100)])If the job is still sluggish on a cluster, the problem is usually in your loops rather than your node count. I have written about how to fix slow Python code first, before you pay for hardware to hide it.
Cost against performance
You can scale almost anything now. That does not mean you should. I have audited projects paying $2,000 a month to AWS that would have run on $50 after one refactor. Run ray down the moment a job finishes, because forgotten instances bill at full rate for doing nothing at all, and nobody notices until the invoice lands.
If Ray cluster work is eating your dev hours, I can take it over. I have been wrestling with WordPress since the 4.x days.
What the numbers looked like
One test moved a prime number search off a high-end desktop and onto a 6-node EC2 cluster. Runtime dropped from 18 seconds to 5.7 seconds, roughly 3x, and the code barely changed. The Ray official documentation goes deeper into the tuning details.
