Hi! I’m Rohan. I’m an engineer currently at Meta AI working on large-scale training. Previously, I graduated with a BS in Computer Science from UCLA. I grew up in San Jose, California.
Work
I currently work at Meta AI on all things related to large scale training. Much of my work is open-source as part of the PyTorch framework. You can find more about my research interests at this link and some publications at this link
I worked nearly full-time on the PyTorch Distributed framework from 2019-2023, and continue to serve as a maintainer for the framework.
Previously, I’ve worked on building mortgage lending products at Blend, risk models at Affirm, and messaging applications for sports teams at Hudl.
Writing
This blog is mostly a place for me to collect random notes and tips that I’ve found interesting and useful over the years. Below is a reverse-chronological set of links to some things that I’ve written or helped write over the years!
2024
2023
PyTorch FSDP paper at VLDB 2023 PyTorch RPC paper at MLSys 2023 Mitigating stragglers in large-scale jobs with PyTorch DDP
2022
A tutorial on using PyTorch FSDP for large-scale deep learning training
Scaling foundation multimodal models using PyTorch FSDP
Scaling to 1 trillion parameters with PyTorch FSDP
2020
How to use PyTorch’s RPC Framework
2018
My Internship at Affirm: Crafting a Reliable Metrics and Alerting Framework
2017
Getting Started in Machine Learning
Finance
I have a couple of writings around personal finance here that I don’t actively maintain anymore.
Misc
This section is still in progress!
Contact me
rvarm1 @ gmail dot com