0

Balancing user freedom and version control: Should operational spreadsheet rules be freely editable or controlled through Git for better tracking and review?

Profile picture
Senior Data Engineer at Taro Community10 months ago

In our operational workflows, business users aim to update rules in a SharePoint-stored spreadsheet. This sheet influences diverse data processing workflows. I'm contemplating to do it on a version-controlled repository like Git.

The dilemma is whether granting users editing freedom enhances usability or if control via Git ensures thorough tracking and review of changes.

How do I delineate between business processes open to user modification and those mandating Git control for optimal balance?

63
4

Discussion

(4 comments)
  • 1
    Profile picture
    Tech Lead @ Robinhood, Meta, Course Hero
    10 months ago

    I need more context in particular around the downsides of the current spreadsheet-based approach, but my general philosophy is "Don't fix what ain't broke".

    On the surface, Git is clearly cleaner and more fully-featured than a spreadsheet for the reasons you mentioned. But the real question is: Is this improvement worth the time it takes to migrate this system?

    Spreadsheets are janky, but they work. For example, levels.fyi scaled to millions of users with a spreadsheet back-end. If you feel like the spreadsheet approach is a 7/10 experience (primitive but it's solid and gets the job done), it doesn't feel worth it to spend a few weeks (or maybe even a few months) switching to Git for a 9/10 experience.

    My advice to you and anybody else considering some sort of migration effort is to follow these steps:

    1. Detail the problems - This is the most important step. Concretely describe the pain behind the current approach. Is it slow? Unstable? Prone to error? Hurting end-users in some way? And after you figure out the pain, what's the impact? Are we shipping features slower than we could? Are broken rules leading to data loss or something?
    2. Get a rough time estimate of solutions - Everything in software has an opportunity cost. If you're working on something, you are not working on an infinite pool of alternative projects. It doesn't matter if something would contribute decent impact if it would take forever to build. Migrations generally take more effort than people expect as there's inevitably bugs that pop up during the cut-over, you have to teach people how to use the new system, etc.
    3. Socialize the proposal - If the first 2 steps pass (i.e. you have good problem analysis and the required time for a solution is reasonable), run it by the team to get feedback. Migrations are painful as humans naturally hate change. If I was somebody who was currently using the spreadsheet, and I had gotten really good at it (i.e. it's like muscle memory to me), I would be quite irked if it changed into a Git-based system overnight. Get everyone on your page (or maybe you get on their page as people think the current setup is fine).

    I highly recommend these other resources as well:

  • 1
    Profile picture
    Senior Data Engineer [OP]
    Taro Community
    10 months ago

    Thank you, Alex, for your thorough response. I forgot to mention that the workflow is going to be fully migrated to a new tech stack. That's the reason I'm contemplating using Git for this spreadsheet in the new workflow.

    Reflecting on your response, I might need to discuss the proposal and address potential pain points. Human-prone errors could be minimized; errors in a non-version control system could lead to the emergence of "bugs" for my team. Sometimes, for someone new to the process, it might take time to identify that the problem was caused by an incorrect value on the spreadsheet.

    Considering there are multiple hardcore fans of spreadsheets on the business side, presenting this argument won't be easy.

  • 1
    Profile picture
    Senior Data Engineer [OP]
    Taro Community
    10 months ago


    I like the way how you define social capital. Definitely, I should start choosing my battles wisely to avoid wasting social capital.

    Thank you, Alex, for your input!

  • 0
    Profile picture
    Tech Lead @ Robinhood, Meta, Course Hero
    10 months ago

    Considering there are multiple hardcore fans of spreadsheets on the business side, presenting this argument won't be easy.

    Hehe, that's what I thought. It's up to you to figure out whether this struggle is worth it.

    When pushing through a contentious change like this, you always spend some amount of social capital. That's why I really like the term of "social capital" - The strength of one's relationships are indeed sort of like a currency. If you keep asking your teammates to go to bat for you for tricky proposals, they will get tired of you (especially if you don't do the same for them).

    So something you need to figure out is if the social "spend" here is worth it or if there are other bold, exciting projects on your plate that are higher impact and more worth the spend.

    On a final note: I have consistently been surprised at how much people can struggle understanding Git, haha. Something for you to keep in mind, especially if there are non-technical (or weakly technical) stakeholders who care about this spreadsheet system.