Taro Logo

How to build depth as a machine learning engineer?

Profile picture
Senior Machine Learning Engineer at Taro Community3 months ago

With a background in hardware, firmware development and validation , I was able to get a master in MLE and transition to machine learning position, but my current machine learning role doesn't focus on building and deploying model but mainly on building infrastructure for machine learning which require a tech stack different from my previous experiences and background. Due to that, my code velocity is pretty slow and I'm also worrying that if I continue with that role, my machine learning skill will fade away or I won't able to build technical depth and be very proficient at machine learning. What do you recommend?



  • 17
    Profile picture
    ML Engineer
    3 months ago

    Congrats on getting your masters! From my understanding, ML infra is generally a SWE for the backend SWE that happens to be for ML but the work is basically SWE.

    In terms of skills for MLE, the core things that matter and keep you relevant are

    • Are you able to build data pipelines to feed the model the data
    • Are you able to train a model that works well
    • Are you able to take a model and deploy to production
    • Are you able to evaluate, benchmark, and run experiments with the model
    • Are you able to monitor the model when it is in production?

    You can be an MLE without ever training a model. In my opinion MLE is basically a full stack data scientist

    Based on what you said, it seems like you're basically doing SWE work. I would look into transitioning into a different team that does one of what I mentioned above. Is that something possible for you?

    My guess is that if your company is building infra, then they should also have a team responsible for training/testing/deploying (unless the company is a pure infra team)

  • 8
    Profile picture
    Senior Machine Learning Engineer @ Roku
    3 months ago

    Pick a cloud provider, AWS or google cloud, spend few days on building a model , data pipelines and deploying e2e, there are plenty of tutorials online to do this, this will give the required confidence. All the best

  • 3
    Profile picture
    Senior Machine Learning Engineer [OP]
    Taro Community
    3 months ago

    Thanks Sai, that makes sense , most of my team mates are pure SWE with no ML background, I already started looking for a transition for one of the team that are focusing on ML core skills that your mentioned above, except for model training which are exclusively for DS role. In the meantime any recommendations on resources that I can use to keep sharpening my ML skillset.

    • 4
      Profile picture
      Founding ML Engineer @ Lancey (YC S22)
      3 months ago

      To be honest what I've learned is that there's so so so many tools and technologies and every company does things a different way -- so it's just best to learn as you need to. I'm not sure how much background you already have in ML. If you've already done some projects then I would not worry too much about keeping skills sharp

      Project Framework

      If you havent had any project experience, I would do an end to end AWS project

      Pick a dataset of your choice (ideally that's big data - over 10-20ish GB, something that wont fit in RAM)

      Put into S3

      Write some ETL scripts to create features

      Train a model on it like XGBoost and use some logging tool like wandb

      Deploy it using sagemaker endpoint and docker and create a simple flask API/website

      setup some continuous testing/monitoring to monitor the quality of predictions in the cloud

      Bonus points:

      • If someone uses your model and you have users
      • You are able to support retraining

      I would avoid deep learning and stick to tabular data. Most companies apply ML to tabular data


      Summary of Designing Machine Learning Systems

      A fantastic summary of Chip Huyen's book - Designing ML systems -- The gold standard of MLE


      An ML course that focuses on the engineering side of ML (deploying ML, feature engineering, how to tackle ML problems). It's a fantastic course that is super industry oriented and teaches how to use sagemaker and other fundamental ML concepts. I've taken it and it's brilliant and is the best i've seen. Though if you've gotten a master's focusing on MLE it might not be super useful. Check the syllabus and see if it applies for you. If you havent done any cloud work with sagemaker this might be helpful

  • 2
    Profile picture
    Senior Machine Learning Engineer [OP]
    Taro Community
    3 months ago

    Thanks Kapil , I was looking into AWS sagemaker , any other recommendation on tutorial for a e2e solution