@maeve Thank god I didn't have to scrape. I looked a bit into scraping and it sounds like a nightmare nowadays. Probably need to use some kind of paid service.
@rdavis0720 Most useful thing.
I agree that Google results are crappy.
These days I'm using the keywords reddit in the end for the Google search to get good answers. Previously it was quora. Now it turned into shit.
Reddit based Google search engine.
I hope if you can do it then it's a big win for many people.
@christianguy03 Exactly! Reddit is a special place. Although I’m also hoping that Reddit will have a way to make sure it doesn’t end up in with the same fate as Quora. If it does then what I’m doing won’t fly well too
@rdavis0720 I think I want to learn programming in the same way you did, anyway you can help further?
I do understand that you don't know much, and AI did most of the job for you, but that's the place I wish to be right now. I just want to build something but I find myself unable to for some reason even though I have several ideas.
Nowadays I don’t use Grimoire on chatGPT anymore because i didn’t want to pay extra for ChatGPT pro when I’m already paying for Replit AI. But the general flow is the same.
In fact it’s usually better because Replit AI has more context of your code. Although Grimoire is sometimes a bit better at guiding you through step by step
How do you pick out products from the reddit posts? How do you generate the affilate links?
I'm guessing your backend workflow goes something like this:
Code:
connect/authenticate via praw
select posts based on criteria
for post in posts:
if post contains product name:
perform sentiment analysis
generate affilate link
store info in database
Then your frontend can just pull from the database and generate the webpage -- am I close? The problem to me seems that the data, especially product name, would be messy (incomplete, spelling errors, etc.). Do you analyze replies or just the original post?
@sweetpea55 It’s more like:
1. Search for posts , sort by relevance
2. For post in posts, use LLM to identify any opinions about product (prompt engineering required)
3. Do further extraction of opinion details with LLM
4. Store in database
5. Generate json from database to display on static site
LLM does a lot of heavy lifting. Probably wasn’t so easy few years ago but it’s doable even for a noob like me now.
Tricky thing is to get the prompt right. To get the prompt right you need to know precisely what kind of data you are looking for and do quite a bit of testing.