In my previous post, I talked about a proof of concept on developing a self-adapting web scraper. As I was adding onto the project, I was having difficulty adding constraints for improving structure accuracy. After some time, I came to one conclusion: My Initial Design Was Flawed!
by Joseph Woolf
Last year, I created the IssueHunt-Statistics website project on tracking repository, issues, and funding for open source projects. Shortly after, however, the website changed and my project breaks down. I did change the scraping code to bring back functionality, only for it to break down again a little while later.
I now have a problem. I don't want to always spend time constantly reworking the scraping code to make it functional. I wonder if I could automate this task?