How to workout the economics surrounding AI enhanced functionality?
Adding AI powered responses and searching via embeddings is becoming increasingly popular.
But unless you have the infra to self host or deploy an open source model (plus the time to train) you’ll inevitably be paying the almighty OpenAi/Anthropic.
How do you workout these numbers so you’re providing your users with enough value but on the other hand maintaining reasonable profitability?
Discussion
(3 comments)- 23 months agoTech Lead @ Robinhood, Meta, Course Hero
There's only so much you can theorize and calculate as I have very much learned doing my own startup and across my 10 years of experience in Silicon Valley. Just ship something and see how it goes. To make things faster, I would actually just pay for APIs instead of doing something like self-hosting:
- Release some AI-powered app and see how much compute cost each user takes up.
- Limit the API budget to $100, so it's a quick experiment and you don't go bankrupt.
- After you figure out the cost per user, introduce a paywall to offset that cost.
- If the uptake on the paywall isn't good enough, then you know this project isn't viable in the market so you can shut it down.
- Repeat the process from Step #1 with a new AI-oriented idea if you're feeling feisty.
In general, biasing towards action is the best path forward. To learn how you can come up with an idea that's both valuable to users and can be shipped quickly, check this out: [Masterclass] How To Come Up With 100k+ Users App Ideas You Can Build For Free
- 13 months agoTech Lead/Manager at Meta, Pinterest, Kosei
This is a privileged problem to have, and I wouldn't worry about it until you have a meaningful ($100+ monthly) bill.
If you unexpectedly get a bill above $100, ping me and I'll cover the cost for a month. (up to $1K)
- 2
3 months agoML EngineerI would look into using an API for open source models.
Here are some open source LLM vendors:
mistral
There's a ton more
If you need chatGPT3.5 level stuff it is effectively free in this day and age. GPT4 level stuff is where it gets pricey and mistral medium seems to be the best value LLM rn
If you need to fine tune I've seen https://monsterapi.ai/ Never used it but might be worth looking into
