Luma AI Machine Learning Onsite Interview Experience and Coding Challenges

Here's an update on a less common interview experience with Luma AI, they've been quite popular lately. Please give me some points! The screening round was with the hiring manager (HM). They asked abo

Here's an update on a less common interview experience with Luma AI, they've been quite popular lately. Please give me some points! The screening round was with the hiring manager (HM). They asked about my past experience, the content of my papers, and some questions about model architecture, pre-training, and post-training. It felt like the questions in this round were very relevant to the interviewer's background. Coding Round 1: Transformer Optimization. I was given a Colab notebook, which was bug-free (not transformer debugging). They asked how to make the transformer training notebook faster. They provided step time printouts. There were some basic questions, such as training on a GPU (device), vectorizing some operations, and some less obvious ones, like fading things in the torch layer (QKV). The follow-up questions included methods to accelerate training on distributed training systems. Coding Round 2: Another Colab notebook. Given a transformer model, I was asked how to perform various load/save checkpoint operations (3-4 questions). I was required to verify as I wrote the code. The last question was about how to perform distributed save/load checkpoints (simulation). Personally, I felt the coding round was easier than the previous one. The follow-up questions were about RoPE. The final round was with the CEO, who also asked about my previous experience, but nothing particularly technical, and about culture fit. I felt their work was very interesting. Due to personal reasons, I didn't accept the offer; we can discuss the package details privately. Thank you everyone for your support! Please give me lots of points!