My team took over a system that we didn't write about a year ago, and it is the only system my team owns. It is a highly critical service in my org.
I was happily making changes to it all of last year, but I broke production twice in a few months, and then almost a third time, and now I have lost faith in my coding ability.
The benefits are that it prevented other teams from experiencing these issues and I took full ownership of resolving them, however, the outages seem to be something that is hanging over my EM's head in spite of it.
How do you get out of this rut?
I'm really sorry this happened to you and it has to be so difficult for you like this. Can you tell us anything about what happened? I've seen some things happen like this where some was out for vacation, forgot to reset the password on a critical build system, and a whole org couldn't ship product for a week while they were out. I think set backs like this are difficult, but it would help you to understand what gaps in knowledge you have and break through those brick walls hard. I did that in my career multiple times and I never regret learning too much as it made me such a better engineer and teammate. Right now, it's up to you to prove to everyone it won't happen again, but I would go overboard and really lay it on thick so they truly realize and see that commitment. Might be a bit overboard, but it certainly can't hurt.
Sorry to hear you're in this rut.
Is your lack of confidence impacting your ability to perform? As in, you're too afraid to make code changes right now?
Either way, if you need external validation, I suggest talking to your EM. Go into it prepared. I suggest doing a brief post-mortem of what went wrong and how you've taken steps to safeguard it from happening again. This will show a reasonable manager that you're taking responsibility and action for your mistakes.
The next time they bring it up or 'hang it over your head', I would ask them if they are confident in the system now. And if not, how can you improve upon that.
With the confidence issue, it could help to work on some smaller tickets to build your confidence back up and look back on the work you've accomplished. Talking to a trusted peer could help as well. I keep a log of kudos that I look through when I feel down.
But the best path forward is to move on. This won't be the last mistake you make. The best you can do is to minimize risk moving forward and avoiding the same mistakes.
If you want to dig deeper, I feel you are experiencing shame over your mistake and you're trying to avoid that shame again. That's a natural response. But maybe there's some internal work that needs to be done there.
Hoping you feel better, good luck!
@Xue Ha That is such a great response. We'll always make mistakes but it is important to treat this as a learning process and move on. Lots of different variables will come into play to make unique situations and we always want to act in healthiest manner possible. I've also been in situations where I was forced to make a decision on mine own since I wasn't getting and would never get the face time needed to help resolve the issue because of how peers were spending their time. It really stinks, but I'm glad to not be around those folks anymore and that I can focus on problems that are just more interesting overall.
Sorry to hear that you're feeling like this: Every engineer has been there.
Fun fact: A few weeks after I started at Meta, I broke a logging endpoint during bootcamp. Around 50 million events were dropped before I fixed it. 😅
What was nice about that instance and Meta as a whole is something absolutely crucial to any healthy engineering organization: Blameless culture. I wasn't scolded at all for breaking that endpoint - The owner of the endpoint just told me to be more careful next time and gave me some resources around how I can better ensure the quality of my code later on. My bootcamp mentor even smiled and laughed when I told him about it.
The core idea here is that when it comes to mistakes, the focus should always be on how to improve.
In your case, it seems like the blameless aspect is missing to some extent, but you can still apply the growth mentality. The best way to get out of this rut both mentally and with your manager is to show a laser-focus on learning from your mistakes and improving the overall engineering landscape because of them. Here's how you can do it:
It's totally okay to feel bad about breaking stuff: What separates the great engineers is that they're able to convert that negative feeling into motivation to keep getting better. It's clear to me that you care a lot - I'm sure you have the skills to come out of this a better engineer than you were before. 😊
Here's some other resources around blameless culture and learning from mistakes: