Is anyone building LLM apps on top of structured data e.g. SQL databases or MongoDB?

[japharish]

Every other company is now building AI chatbots and related apps on top of their Notion, documentation, PDFs, etc. But I'm curious if anyone is doing something similar on top of their structured database, e.g. Postgres, MongoDB etc? If you are, I'd love to know more about your usecase and how you're doing it currently.

 

[joana27]

@japharish I’m working on it now. But I have a dev doing it for me since that’s not my area of expertise

 

[japharish]

@joana27 Can you share more details on what you're building and how you are using the data from your database?

 

[joana27]

@japharish Using postgre for the database. I wish I could give more details but I literally had to find someone else to do it since I don’t know enough about implementing LLM’s in this capacity.

But the end result should be natural language input > sql search > graph output

 

[japharish]

@joana27 I see. So visualizing SQL data using natural language, nice. Can I DM you to talk more about this?

 

[joana27]

@japharish Sure can

 

[wemchud]

@japharish Yes, this is a current project of mine. It's an AI chatbot but meant to also assist with data parsing and visualization, it's a piece of a platform I'm building.

 

[japharish]

@wemchud Interesting. Are you using the SQL database as a knowledge base for doing RAG? Or is it more like you're generating SQL queries using an LLM and then running that against the database?

Would love if you can share more about your use-case. If you don't wanna share it here let me know if I can DM you.

 

[wemchud]

@japharish I'm actually look for folks like yourself to play with it to help me expand what it does. It was intended to be an information capture tool, surveys, to push that data in a more interactive way to those data sets and it turned into a giant "personalized" chatbot system. If you DM, I can get you set up with the registration code to test if you're interested/the link to look around. I'm starting to make it public but it'd be easier to keep communications that way for now versus comments if you want to check it out and offer some feedback.

 

[japharish]

@wemchud Hi, yes would love to check it out. You can DM me about the details.

 

[wemchud]

@japharish And to answer that a bit more if case you wanted more context, it's actually not generating the SQL queries or doing a ton with them yet, it's using a relationship management database system on top of the code to manage the data structure, the AI right now is just helping formulate trends, responses, and generate content from the said data. Once I have more data, I can run large analytics on it and test some more with those types of models but for right now, it was more about getting it and capturing it - and understanding how to best to do that.

 

[tikvaisaiah43]

@japharish The question is why you would want to do this?

What problems is this going to solve?

 

[japharish]

@tikvaisaiah43 So far most of the examples of SQL + vector embeddings that I've seen are around semantic search, e.g searching for similar products, creating personalized user feed from things the user already likes, etc.

Some resources I've found so far

https://rockset.com/blog/5-use-cases-for-vector-search/
https://aws.amazon.com/blogs/databa...stgresql-using-amazon-sagemaker-and-pgvector/

I'm yet to see a lot of usecases where an SQL database is being used for RAG though. That's why I was curious.

 

[alina11]

@japharish If you're using for personalized search purposes, I'd just implement an AI-enabled search platform like Algolia rather than try to create the whole model yourself. The amount of time/resources you're spending to implement would probably be a similar amount to what you would pay anyways.

 

[613jono]

@japharish Yes. Without getting into the weeds, SQL db as kb for RAG, vector storage for some other AI capabilities, a couple other small things, and with a long AI-heavy roadmap in general.

 

[japharish]

@613jono Interesting, yes this is sort of the architecture I had in mind. Can I DM you? Would love to talk to you more about this.

 

[613jono]

@japharish Sure.

 

[rocotheepic]

@japharish Checkout StackAI

 

[whereheis]

@japharish I don't think it makes sense to do it directly against a SQL DB. Its not a challenge, but it makes more sense to denormalize then dump data to S3 or similair and read it from there.

 

[japharish]

@whereheis I see, denormalizing is definitely a challenge, so is filtering. As in indexing only the rows you're interested in and not all. Given these challenges dumping to S3 first and then indexing from there can be a good approach.

Are you building something around this right now? If so I'd love to talk with you in the DMs!