I haven't seen any research that says passive listening without instruction has any effect at all.

You may have some success in a language related to your native language. Good luck with something truly foreign...