Process
The technical round hit hard with a question to implement multi-head self-attention from scratch in PyTorch.
I felt the pressure as I explained the time and memory complexity, but the real surprise came when I recalled a similar problem I had practiced on PracHub just days before.
Following that, I had a casual chat about the role and my experience, and then wrapped up with a behavioral interview.
Once everything was done, I received an offer, which I gladly accepted.
Overall, a challenging yet rewarding experience.
Questions
Implement multi-head self-attention from scratch in PyTorch and explain the time/memory complexity, then describe how FlashAttention reduces memory usage.
Was this helpful?
Interview Statistics
The following metrics were computed from 2 interview experiences for the xAI Machine Learning Engineer role.
Success Rate
xAI's interview process for their Machine Learning Engineer roles is fairly selective, failing a large portion of engineers who go through it.
Experience Rating
Candidates reported having very good feelings for xAI's Machine Learning Engineer interview process.
xAI Work Experiences
- 5.0Huge learning opportunitySoftware Engineer(Internship) • Former EmployeePalo Alto, California, United StatesOctober 22, 2025Pros: Full autonomy as an intern, and you work on important things. No pet projects.Cons: Be ready to work exactly like a full-time employee; you most likely won't have much free time during the week.
- 2.0Good pay but poor management and cultureSoftware Engineer • Current EmployeePalo Alto, CAMay 18, 2026Pros: Good money and a big platform.Cons: Very poor management; people only mind their own business and don't care about others at all.
- 1.0Only IPO AI company worth it, but poor cultureSenior Software Engineer • Current EmployeePalo Alto, CAAugust 6, 2026Pros: Working on tech that lands in spaceCons: - The work culture is as bad as you would expect.
