I got the workers up and running!! I got a free tier ec2 instance hosting a bunch of node scripts. They’re my little robots that do a bunch of data gathering that I would normally do manually for myself. Each script doing a super specific task. That way if a script fails the other parts of the data I need continues to be pulled until I fix it.
The current set up is a small db on Amazon RDS which I’ll call my production-db and and ec2 instance which will be used solely for altering and adding to the database. This includes updating rankings, pulling new info and the like. This is called production-worker instance. Later I’ll add a third instance where web accessible stuff will live. That will be solely responsible for serving up the front end of the site to the user.
Reading up on NLP
There’s no way I could possibly do what I do for others if I were to continue manually process recommendations from different sources. So I’ll be using a bit of NLP to help find recommendations and match those recommendations to books in the database.