My e-commerce company started tracking all page visits in the database about a year ago, it's been one of the best decisions we've made.
It's amazing to have all of your data in one database, you can do so much with that data. I can answer virtually any question about how people interact with our sites without needing to closely integrate with 3rd party services.
Sure, if you have thousands of visits a second, you need to put some thought into handling that load, but it's definitely doable and not too hard. Once we get to that point, we'll probably have the page views going to another dedicated databases and use postgres_fdw to handle the queries across the two databases.
This looks great! I was just thinking I needed something like this a couple days ago, and then this popped up on HN today. I will definitely check it out.
I haven't had a chance to dig too deeply into it yet, but is the info stored in a session cookie and then you can decide if/when to write it to the db? For example, if I didn't want to write every unique visit to the db, but only when a user signs up, or signs in, or takes some action x, do I have the flexibility of when I write to the db?
Redis, yes. Perfect for this kind of thing. delayed_job might make less sense, since delayed_job still INSERTs to the database.
I haven't benchmarked it specifically but I'd have to guess your time savings to store the information in a postgresql job queue vs the visits table would be negligible at best.
nice work. one of the biggest problems with external tracking services (mixpanel/kissmetrics/google analytics) is reconciling the data with what's present in your database. this solves that problem really well. great start
Most of these analytics providers also allow you send data from your server that you can then correlate with your visit data. While I think solutions like this works initially, in my experience you will spend a lot more time building the capabilities that an analytics package could provide out of the box with little work. For eg., being able to visualize tracking live as it happens is something I have really gotten used to.
I'm not familiar with the other analytics providers, but can't you track certain events (like orders and charges) and bind properties to them using Mixpanel? This could help you reconcile the data in your database.
It's amazing to have all of your data in one database, you can do so much with that data. I can answer virtually any question about how people interact with our sites without needing to closely integrate with 3rd party services.
Sure, if you have thousands of visits a second, you need to put some thought into handling that load, but it's definitely doable and not too hard. Once we get to that point, we'll probably have the page views going to another dedicated databases and use postgres_fdw to handle the queries across the two databases.