The proper analogy would be refusing to give a combination for a safe, which you can probably do, its not such an issue because they can break the safe open.
The 5th amendment right to avoid self-incrimination only extends to testimony. A search warrant can compel you to furnish the contents of a safe (as property, it is governed by the 4th amendment -- due process is all that is required, and in this case, the process of getting a search warrant is all that the Constitution requires).
(Yes, that's my point.)