How I Reduced Our Startup's LLM Costs by Almost 90%

[taraellen27]

@fav54con I believe that highly depends on your usecase and what you are trying to build. In his case he was clever to use his customer's requests and he's existing system's(OpenAI API) responses to create rows of data. If you already have established users you can find a similar way. If not then try other means. If your application is too general you don't need to fine-tune. depends

 

[shaebaybee]

@redeemed2000 Interesting, thanks for sharing

 

[jonathan_ckidz_ministry]

@redeemed2000 I'm building an LLM cost management solution and would love to partner with folks who want to track and optimize their LLM usage and cost. Please email hello@greenscale.ai.

 

[melindapurcell]

@redeemed2000 Good stuff, the thing about ai is that by yourself you can’t do much and you really need a PhD in ai to build anything meaningful yourself so the alternative is building your business based on api calls. Back in the days, Twitter, Facebook open their apis and people build third party apps on top.. but you can’t scale because it’s too costly and they could kill your business any day.. so the aim is build a small app, hope it’s make some good money but not an actual business out of it?

 

[redeemed2000]

@melindapurcell I partially agree but I don't think a PhD is necessary. Or even a bachelors honestly (I am a huge pro-college person though).

Engineers always build on layers of abstraction. If you needed to know everything to build a piece of software, no one would. That would be like saying you need to know x86 assembly to program in C/C++. All you need to know is the interfect provided.

AI is the same. You either build on REST API calls to an AI hosted in the cloud, an API call to a locally running model, etc. No need to rebuild something that others do way better than you could individually

 

[arisemydarling]

@redeemed2000 Strongly agree. I'm a dumb BCom grad with 0 coding background until last year and I'm building useful stuff with AI.

If you can read documentation and have the patience to iterate through errors, you'd be amazed at what you can build with 0 technical background.