Change the networking config from NAT to bridged. The VM will now appear as a truly independent device on your LAN, as if it were plugged into a switch.

But then it can talk to the lan. I don't want that. The whole reason I have some apps running in a VM is to be shielded from them.

Then you need to add a port forward to get through the NAT. However, NAT isn't security and you might be better off using a bridge and a firewall to shield your VM.

virtualbox adds a network interface to connect host and VM to each other, you can reach them via private IPs. the exact IPs used depend on the type of network mapping used (NAT, host-only, bridged)

That depends on how you configure the networking for the VM.

Yes, I stated that in my original question already.

Not clearly IMHO. If the VM being able talk to the host is not acceptable to you, even if you only activate that during runtime when you need it, then scp obviously isn't a solution.