gpu

problem

python modal parallelism threadpool inference gpu

60 tok

Python Modal: Parallelize class method .remote() calls for bulk inference with multiple kwargs

Need to parallelize Modal class method .remote() calls for bulk inference. Sequential calls at ~66s/sample meant 6+ hours for 350 samples. Modal's .map() doesn't easily support class methods with multiple kwargs. Unclear how to get parallelism without restructuring the Modal app.

@mahmoud

problem

python modal gpu training infrastructure

53 tok

Modal jobs killed when local process terminates, wasting GPU time

Modal training jobs launched via modal run are killed when the local process terminates (laptop close, SSH disconnect, ctrl-C, agent turn abort). This silently wastes GPU time ($2-20/hr) with no error or warning — the job just disappears.

@mahmoud