A client came to me recently with a data processing script that would not finish. They had just spent a small fortune on a 64-core Threadripper workstation, and the script still took over an hour. Task manager told the whole story: one core pinned at 100%, the other 63 having a nap. All that silicon going to waste. What they needed was Distributed Computing with Ray, and they had no idea where to start.
My first move was the obvious one: the standard Python multiprocessing library. I spent an afternoon refactoring their code into pools and workers, and it worked, more or less. The boilerplate was awful, and one crashed process turned the whole run into a zombie. Moving it to a cloud cluster later would have meant writing the thing again from scratch. It is a common trap for devs who want to fix slow Python code fast without thinking about what it costs to maintain.
Distributed computing with Ray
So I reached for Ray. If you have not used it, it is a framework for scaling Python from a single laptop up to a large cluster with almost no friction. It takes care of scheduling, data serialization and process management. The Ray Core official documentation has the full picture, but here is what I did with the client’s code.
Instead of managing processes by hand, you use decorators. Mark a function as “remote” and Ray handles the rest. Below is a simplified version of what we ran for a heavy CPU-bound job, counting primes across a large range.
import math
import time
import ray
# Initializing the local cluster
ray.init()
@ray.remote
def bbioon_is_prime_task(n: int) -> bool:
if n < 2: return False
for i in range(2, int(math.isqrt(n)) + 1):
if n % i == 0:
return False
return True
@ray.remote
def bbioon_process_range(start: int, end: int) -> int:
count = 0
for i in range(start, end):
# We're calling the function locally here for simplicity in the demo
if i > 1:
is_p = True
for j in range(2, int(math.isqrt(i)) + 1):
if i % j == 0:
is_p = False
break
if is_p:
count += 1
return count
if __name__ == "__main__":
A, B = 10_000_000, 15_000_000
num_cpus = int(ray.cluster_resources().get("CPU", 1))
step = (B - A) // num_cpus
# Launching remote tasks
futures = [bbioon_process_range.remote(A + i * step, A + (i + 1) * step) for i in range(num_cpus)]
# Gathering results
results = ray.get(futures)
print(f"Total primes: {sum(results)}")With Distributed Computing with Ray, the run went from 45 minutes to roughly 2 minutes, and the code came out cleaner than the original single-threaded version. Coming from a data science background, you will find it far sturdier than basic threading, much like the way we master reliable agents by letting the framework carry the load.
Using actors for shared state
Shared state is where this gets interesting. Global variables are useless in a distributed system because every process has its own memory space. Ray answers that with actors, which are classes decorated with @ray.remote. I used one to build a global progress tracker so the client could watch all 64 cores as the job ran. It is a better approach than the one you will find in most of the distributed processing guides online.
The dashboard is the other thing I like. Run your script and Ray brings up a web interface at localhost:8265 showing which cores are busy, how much memory is in use, and whether any task has hung. That saves a lot of time when you are building your first distributed application and something starts behaving oddly.
Hardware will not fix bad code
I have been doing this for 14 years and I have lost count of the times someone threw expensive hardware at bad code. Ray buys you speed, but the bigger win is code that can grow. Laptop or 100-node cluster, the API does not change. That is the sort of pragmatism I care about.
Data pipelines get complicated fast. If you are tired of debugging someone else’s mess and you want your site or your pipeline to just work, send me a message. I have probably seen it before and can get it sorted without the fluff.
If you are still pinning one core and wondering why the bill is so high, it is probably time to refactor. Part 2 takes this to the cloud.
