Hacker News new | past | comments | ask | show | jobs | submit login

‘’’ It runs on a schedule at 6, 26 and 46 minutes past the hour—I like to offset my cron times like this since I assume that the majority of crons run exactly on the hour, so running not-on-the-hour feels polite. ‘’’

Not sure how much of a difference it makes to the underlying service but I will also do this with my scraping.

Thank you for point that out




Further to this, I also put the following snippet in front of my web-scraping cron jobs so they start at a random time between the minute boundaries:

    perl -le 'sleep rand 60';
It's the most compact code I could find to do the job.


If you're using a somewhat modern shell there is $RANDOM which gives you a 15 bit random number. So e.g.

    sleep $((RANDOM / 546))
but I guess most cron jobs run with an extremely conservative shell that might not have it.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: