Notice: this project is abandoned
MyGirlMel is a project I started in early 2016. It was designed to help sales consultants automate and better manage their facebook sales. This is focused primarily on LuLaRoe, but could be generalized to other audiences.
Consultants will post pictures of inventory in a facebook group photo album. Customers would comment "sold" on pictures to buy the item.
- Capitulum - the API portion, serving up data for the front-end
- Collector - A facebook scraper - used to grab all kinds of data from facebook
- Controller - A Scheduler for facebook scraping to ensure that we stay within the call limits of the facebook API
- Data Services - There would be many data services, but these are responsible for storing data. Redis is currently used.
- UI-Frontend - The UI for consultants
- A consultant logs into the service with facebook, granting access to group albums.
- The user selects which albums to scan for sales
- The controller places these albums on the schedule for periodic ripping
- "sold" items are recorded
- The consultant then views their sales in the UI
I wanted all communications to run on NATS - a lightweight messaging system. I chose this because the components above are just the beginning of many other services which will require integration and I wanted to keep communication light and fast. Capitulum would be the boundary where the HTTP API meets internal communication.
I chose Golang as the primary language for performance reasons. I also chose it because I believe it is faster to write in than other systems languages. So, you get performance and somewhat reduced development time. For services that needed to be created quickly, I would opt for writing them in Ruby and then later rewriting in Go.
The Facebook API has limitations, such as 200 Requests per app user per hour. This means if I have 10 users, I can make 2000 calls per hour. If this is exceeded, then Facebook begins limiting traffic and ultimately will stop responding. Controller is designed to prevent this. It will keep track of how many request have been made and prioritize the requests that need to be made.
Controller is in it's very infancy. Right now, it just sends a single scraping request to Collector.