ModalModal
Python in, GPUs out, billed by the second
Modal is the serverless GPU cloud you write as Python: decorate a function, name an accelerator, run. They build the container, place the machine across clouds, and scale it to zero when it is idle. Starter is $0 with $30 of compute credits a month. After that you pay per second of CPU, RAM and GPU — an H100 is about $3.95/hr, which is dear next to a reserved box and cheap next to a cluster you keep warm for a spiky job.
[ 01 ] The verdict
Buy this when the work is bursty and the team already writes Python. Do not buy it as a cheaper GPU: there is no self-host, no bring-your-own-cloud, and a box you keep busy all month still wins on the hourly rate. The decorator is the product. The bill is the catch if you pretend it is a reserved cluster.
Best for
Python teams doing batch inference, fine-tunes, sandboxes for generated code, or jobs that sit idle most of the day and then want a pile of GPUs.
Watch out
Time a cold start with your actual weights, not the hello-world. Starter is three seats and a $30 credit, not a free production plan. Team is $250 a month before a second of compute. Do not read $0 as the invoice.
Strengths
- The decorator is the product: local Python becomes a GPU job without YAML or a cluster
- Per-second billing and scale-to-zero, so idle time is actually free
- GPU ladder from T4 through B200; they place the job across clouds
- Sandboxes for untrusted code, plus notebooks that boot on a GPU in seconds
- $30/month compute credits on Starter, no platform fee
Trade-offs
- No self-host, no BYOC. If the card or the region is a procurement problem, you are stuck
- Python is the native runtime; JS and Go can call functions, they do not replace it
- Sustained utilisation is where reserved GPUs win, often by a wide margin on the same H100
- Team is $250/month on top of compute; Starter keeps logs for a day
[ 02 ] What it actually does
What Modal actually ships.
Functions in code
An @app.function with the image, GPU and concurrency written next to the Python. modal run is the deploy. No cluster file, no YAML.
GPU pool
T4 through B200, including H100 at about $3.95/hr. Modal picks capacity across clouds so you ask for an accelerator, not an account in three providers.
Sandboxes
Isolated machines for code a model just wrote. The point is to let an agent run untrusted programs without handing it your laptop.
Snapshots
Memory and CUDA checkpoints so a fat inference server does not spend half an hour pulling weights on every scale-from-zero. Still measure yours.