Meta’s SAM3 has been out long enough for the takes to arrive, and one of them says foundation models have made custom-trained detectors pointless. In production that idea gets expensive fast, and computer vision performance is what pays for it. I have watched this cycle before. Everyone moves to the newest general purpose tool, then works out six months later that infrastructure costs tripled while latency went backwards.
SAM3 is impressive, and I am not going to pretend otherwise. Promptable Concept Segmentation is real work on the vision-language side. It is also the wrong tool for a real-time system on a modest budget. The benchmarks below are where the specialist keeps its 30x speed advantage.
What a foundation model costs you
SAM3 carries roughly 840 million parameters. On an NVIDIA P100 that comes out to about 1100 ms of inference per image. YOLOv11 and ISNet were built small on purpose. If your pipeline has to answer in under 50 ms, accuracy stops being the deciding factor, because the large model cannot answer at all inside that budget.
I wrote recently about why your training metrics might be lying to you, and the same thing happens here. General capability does not carry over into production. A task-specific model trained on a 6-hour compute budget has beaten these giants for me, as long as the environment stays narrow and runs unattended.
Benchmarking computer vision performance
Put SAM3 against a specialist like YOLOv11 on object detection, instance segmentation and saliency, and the answer comes out the same way each time. The specialist is faster, it costs less to run, and it breaks less often.
- On the Global Wheat object detection set, YOLOv11-Large beat SAM3 by a wide margin. SAM3 draws tight boxes, but the annotations want the “awns” (hair bristles) included, and YOLO picks those up.
- For weapon detection on CCTV footage, a specialized YOLO model trained on just 131 images beat SAM3 by 20.5%. Surveillance frames are low resolution and highly correlated, which is exactly where foundation models struggle.
- Blood cell images look like the generalist’s chance to shine. The specialist came out 23% better overall, because it handles the overlapping cells that general models tend to miss.
A lean specialist pipeline
Shipping a specialist is less work than people expect. Here is a plain YOLOv11 inference script, the kind of code that scales sideways across cheap T4 instances.
from ultralytics import YOLO
# Load a specialized model (e.g., custom-trained on your dataset)
# Unlike SAM3, this footprint is tiny and runs on standard CPUs if needed
model = YOLO("bbioon_specialist_detector.pt")
def bbioon_run_inference(image_path):
# Perform detection with lean parameters
results = model.predict(source=image_path, conf=0.25, imgsz=640)
for result in results:
# High-speed processing without the 1000ms latency hit
print(f"Detected {len(result.boxes)} objects in {result.speed['inference']:.2f}ms")
return results
# Ship it. No H100 required.What owning the model buys you
Better computer vision performance buys you hardware independence. When the model is yours, you can prune it, quantize it, and deal with a specific edge case such as a hallucination yourself, rather than waiting for Meta to publish the next paper.
A specialist still needs watching, because its accuracy drifts as the incoming data changes. My guide to drift detection for ML systems covers how to catch that before it reaches your users.
If computer vision performance work is eating your dev hours, I can take it over. I have been wrestling with WordPress since the 4.x days.
Where SAM3 does belong
Keep SAM3 in development, where it earns its keep: interactive image editing, open-vocabulary search, and labeling your data for you. Once you have around 500 clean labeled frames, train the specialist and deploy that instead. A short training run buys you 30x faster inference and a model that behaves the same way tomorrow as it did today.
