Hacker News new | past | comments | ask | show | jobs | submit login

I worked for one company that enforced on-call for the entire team. For a reason that's out of scope for this comment, it didn't apply to me, though it should have by the normal rules.

I loved working for the company, but at the same time I would have totally despised the on-call system as implemented.

The main problem was that I was working on client apps. My focus was an Android app, to be specific. There was a completely separate web team that developed the backend; we not only didn't work on the backend, we were prohibited from working on the backend. I never even saw the backend code. We even asked to develop part of the backend we needed at one point, and they refused to let us, telling us that we didn't understand their security requirements and therefore couldn't contribute.

And from what I heard from people who were in the on-call rotation, every single 2AM call was from some badly designed alarm. Designed by their team.

Yes, every one of those alarms was fixed. Some may have been only "fixed" the first time, but I got the idea that each did eventually get adjusted to not be completely spurious.

But what really offended my sensibilities was that the backend team was pushing for the client team to be on-call to cover for what seemed to me to be profoundly poor alarm definition. They were eager to get others on board to cover the on-call rotation because being on-call was such a nightmare.

Another comment [1] points out that the median number of alerts in a week should have been zero. It wasn't close, from what I could tell. And the whole "make them eat their own dog food" approach of putting the people on-call who are actually responsible for writing the code was broken by the practice of including people only loosely associated with the code (as in, we used it) in suffering the consequences of writing the code (or designing the alarms) badly.

In general, if I've broken something that's affected a site or product, I'm happy to fix it, even if it's after hours, though I prefer it to be on a "best efforts" basis rather than a "drop everything and work on it now" basis. What I don't want to sign up for is being roused out of bed at 2AM to fix problems caused by someone who isn't even on my team, where I wouldn't have even seen the PR that caused the problem or any of the related code, and there's absolutely no way I could have prevented it.

/rant

[1] https://news.ycombinator.com/item?id=32163155




Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: