You're right. When I think about it now, it's clear I'm really comparing these two states:

1. uBO defaults + uMatrix which is mostly in "allow all mode"

2. uBO in medium mode (as defined by the above video) w/o uMatrix installed

I think (2) is safer than (1). I've been in (1) mode pretty much since I started using uMatrix (2 years?).