The system right now is highly reliable, I have no fear of doing a live demo of it, but live demos come off as strange because my feed is a strange mix of arXiv abstracts, Guardian articles about association football, etc. so it comes off as idiosyncratic and personal. (Oddly when I started this project I loved the NFL and hated the Premier League, when I started doing feature engineering as to "Why does it perform so well for arXiv papers and so poorly for sports" I started studying football articles in detail and started thinking "How would I feel if my team got relegated?" and "Wow, that game went 1-0 and it was an own goal" and next thing I knew I was hanging on every goal in every game Arsenal and Man City play -- it changed me.)
It's not even that hard for me to swap algorithms in and out but it should be easier, for instance I like the scikit-learn system for model selection mostly but there are some cases like SVC-P where I want to bypass it and I am not so sure how to comfortably fit fine-tuned transformer models into the system.
Another problem with it is that it depends on AWS Lambda and Suprfeeder for ingestion, it costs me less than $5 a month to run and about 10 cents per feed but (1) that's not cost-effective if I want to add a few hundred blogs like
and (2) I know many people hate AWS and other cloud services.
If somebody were interested in contributing some elbow grease that would help the case for open source, alternately a hosted demo of some kind would also be possible but I'm not ready to put my time and money into it. Contact me if you're interested in finding out more.
The system right now is highly reliable, I have no fear of doing a live demo of it, but live demos come off as strange because my feed is a strange mix of arXiv abstracts, Guardian articles about association football, etc. so it comes off as idiosyncratic and personal. (Oddly when I started this project I loved the NFL and hated the Premier League, when I started doing feature engineering as to "Why does it perform so well for arXiv papers and so poorly for sports" I started studying football articles in detail and started thinking "How would I feel if my team got relegated?" and "Wow, that game went 1-0 and it was an own goal" and next thing I knew I was hanging on every goal in every game Arsenal and Man City play -- it changed me.)
It's not even that hard for me to swap algorithms in and out but it should be easier, for instance I like the scikit-learn system for model selection mostly but there are some cases like SVC-P where I want to bypass it and I am not so sure how to comfortably fit fine-tuned transformer models into the system.
Another problem with it is that it depends on AWS Lambda and Suprfeeder for ingestion, it costs me less than $5 a month to run and about 10 cents per feed but (1) that's not cost-effective if I want to add a few hundred blogs like
https://www.righto.com/
and (2) I know many people hate AWS and other cloud services.
If somebody were interested in contributing some elbow grease that would help the case for open source, alternately a hosted demo of some kind would also be possible but I'm not ready to put my time and money into it. Contact me if you're interested in finding out more.