Adding AI powered responses and searching via embeddings is becoming increasingly popular.
But unless you have the infra to self host or deploy an open source model (plus the time to train) you’ll inevitably be paying the almighty OpenAi/Anthropic.
How do you workout these numbers so you’re providing your users with enough value but on the other hand maintaining reasonable profitability?
There's only so much you can theorize and calculate as I have very much learned doing my own startup and across my 10 years of experience in Silicon Valley. Just ship something and see how it goes. To make things faster, I would actually just pay for APIs instead of doing something like self-hosting:
In general, biasing towards action is the best path forward. To learn how you can come up with an idea that's both valuable to users and can be shipped quickly, check this out: [Masterclass] How To Come Up With 100k+ Users App Ideas You Can Build For Free
This is a privileged problem to have, and I wouldn't worry about it until you have a meaningful ($100+ monthly) bill.
If you unexpectedly get a bill above $100, ping me and I'll cover the cost for a month. (up to $1K)
I would look into using an API for open source models.
Here are some open source LLM vendors:
There's a ton more
If you need chatGPT3.5 level stuff it is effectively free in this day and age. GPT4 level stuff is where it gets pricey and mistral medium seems to be the best value LLM rn
If you need to fine tune I've seen https://monsterapi.ai/ Never used it but might be worth looking into