3

As a MLE, how should I communicate with a difficult backend engineering lead?

Profile picture
Mid-Level Software Engineer at Startup2 months ago

I’m a machine learning engineer (MLE) working on an e-commerce recommendation system, while our backend engineers manage the serving server. I’m facing communication challenges with the backend lead:

  1. We're building an e-commerce search system, but he refuses to use a search engine due to maintenance overhead, insisting a regular database is sufficient for vector search. However, a search engine offers better algorithms, scalability, and faster improvements in click-through rates.

    • (a) Should I just acknowledge his opinion and say I'll bring it back to my team instead of debating in meetings? Because we are discussing the business requirement with PM in that meeting, not focusing on the tech stack.
    • (b) If he still refuses, should I find other collaborators or have MLEs maintain the search engine ourselves?
  2. He often shares strong opinions on ML algorithms in meetings, but while 80% is correct, 20% contradicts my understanding. Our discussions are usually focused on system-level topics, if I challenge him, the meeting may go off track; if I don’t, others (PM and other engineers) might assume we agree, leading to misconceptions. How should I handle this?

41
3

Discussion

(3 comments)
  • 0
    Profile picture
    Tech Lead @ Robinhood, Meta, Course Hero
    2 months ago

    This is a classic communication conundrum, and the particular stacks you're on don't super matter here as the solutions are the same across all backgrounds.

    Before I go into more details, I first recommend going through this lesson in my communication course: Resolving Disagreements

    With that out of the way, there are many, many ways to tackle this problem:

    1. Lay out your case more pragmatically - In particular, I recommend creating a system design document that lays out the pros and cons in detail of your approach vs. theirs. Make sure that you work backwards from first principles by identifying the engineering cultural values your org cherishes most and using that to anchor the discussion. On top of that, attach as much cold, hard, data as possible. Your approach leads to higher CTR? - Prove it. From what I can see here, the database approach is simpler but less performant/harder to scale while your search engine approach has more complexity but more general power underneath. This isn't a clear case of right or wrong - Different orgs would make different decisions here.
    2. Introspect and be a Devil's Advocate against yourself - As I talk about in that "Resolving Disagreements" lesson, this is a critical step. Stepping back and looking for flaws in your own approach objectively makes you have a lot more empathy for the other side. Assuming you didn't change your mind on your approach, that empathy can help you convince them better.
    3. Assemble backers - It's much harder for this tech lead to turn you down if there are 10 other engineers on your team who support your strategy vs. if you were just vouching for it on your own. Talk to others on your team and see what they think, ideally after you made that super detailed system design doc.
    4. Disagree and commit - I'm stealing from an Amazon LP here, because it's a good one. You can't win every possible battle. I've done detailed proposals for engineering solutions I deeply believed in countless times and gotten shot down. It's a part of life. When this happens, it's important to be graceful in loss so that you can earn some social capital that hopefully lets you pass future decisions.
    5. Escalate - This is a nuclear option, but if the tech lead continues giving you a hard time, talk to your manager. It's their job to resolve internal politics like these.

    It's important in situations like these to assume good intent because it's frankly easier to work with a tech lead who likes you vs. one who doesn't.

    Your goal isn't to prove that their database idea is wrong, it's to make better engineering decisions as a team.

    Your goal isn't to call out their gaps with machine learning knowledge, it's to deepen the team's collective understanding of machine learning.

    • 1
      Profile picture
      Mid-Level Software Engineer [OP]
      Startup
      2 months ago

      Hi Alex, thanks for your suggestion—it was super helpful. I played Devil’s Advocate against myself today, and the meeting went much more smoothly than previous ones.

      I've completed the communication course, but applying it in real-world situations is still challenging. I’ll keep this in mind—thank you!

    • 0
      Profile picture
      Tech Lead @ Robinhood, Meta, Course Hero
      2 months ago

      I played Devil’s Advocate against myself today, and the meeting went much more smoothly than previous ones.

      That's awesome! Well done 😊