I mean if you use this logic the whole Alexa thing immediately becomes useless anyway. It's all about extreme convenience and reducing the friction between "I want X" to "Amazon please chove X into my face" to a minimum.
"Waaa it's inconvenient waaa why can't it obey any voice it hears waaa my neighbour bought a lot of crap using my Alexa then got the packages off my porch waaa why didn't they do something?!"