Until we solve "the password problem", what I'd really like is a small dedicated hardware password manager. Like Trezor (http://www.bitcointrezor.com/) but for passwords.
But there are a number of problems:
1. How do you authenticate yourself with it? If you lose it you don't want the thief to be able to extract your passwords. You need to reintroduce the "something you know" factor (hard to enter passwords in keychain sized devices), or maybe "something you are" factor (fingerprint? RFID implant? only half joking, I'd consider it)
2. How do you perform backups without exposing the whole database to your hosts?
3. How do you interface with mobile devices? Public computers?
I'd like it to actually be hardware with tamper evidence (or response, even better), unlike trezor. That makes it a lot easier to use a weaker password or biometric to authenticate with it, safely.
The unknown thing is whether it should communicate directly to the computer, or have all communications mediated by the user. I'd be more comfortable if it only had one-way communications capability (user enters something on a device-local keypad, it sends data transmit-cable-only back to the computer), but that's not going to work with mobile, probably.
You could emulate a keyboard, and have the Bluetooth/USB stacks implemented in dedicated chips, with a 1-way serial connection from the main MCU.
But it's pretty nice to be able to hit a keyboard shortcut and have it figure out which password to fill rather than scrolling through a list. It would be pain to enter all the site names without management software too.
I am actively developing a Bitcoin hardware wallet (like the Trezor), which has a nice color touch screen. This allows it to have an on-screen keyboard and such. It's about half the overall size of a phone, and could easily be made smaller.
Assuming a device similar to that:
1. You can protect the device with a simple pin, and have it self-destruct or similar after a number of failed attempts.
2. There are a lot of great ways to handle backups. On first thought, I would lean towards the following model. When initializing the device, you enter a master password. This password is fed into a ridiculously expensive KDF to create an entropy pool. By expensive, I mean the device will spend an hour or longer deriving the entropy pool. Since you only need to do this when first using the device, or restoring from backup, the inconvenience is minor. The entropy pool is used to derive any and all site-specific passwords. It is also used to derive your backup key. From now on, as you use the device, creating new logins, etc, it can frequently and automatically create encrypted backups of this metadata and shove it to your PC/cloud. The entropy pool is stored securely inside the device, and possibly encrypted with a pin number (see answer to #1 above).
Now, if you lose/damage this device, you just grab a new one, initialize it with your master password to re-create the entropy pool, and then it can sync to a backup.
Thanks to the ridiculously expensive KDF, a malicious attacker would need to spend one CPU hour (or more) per brute-force attempt. Good luck!
Problems with this model: The master password must _still_ be a good password. None of this dog's name nonsense. Otherwise, anyone who gets hold of your backup(s) could chew through the top 100 passwords or something like that. One way to help mitigate this is to mix personal information into the master KDF's input. e.g. ask the user for their driver's license number. Doing this, however, exposes the user to privacy issues; should a hacker successfully crack their backup they have strong evidence who owns the now exposed logins.
Also, backups are still mandatory, or else you'll be unable to re-derive your passwords and other metadata. Since the backups are quite secure, one might feel confident storing them in the cloud. I would prefer a fully deterministic solution, where one can just enter a website name into the device and get their password out ... but due to the varying password requirements of each site this isn't feasible.
Finally, since the master password is rarely ever used, it may be difficult for the user to memorize it. This could be mitigated by also using the master password as the device's pin (a weak KDF can be used here), but then the user has to enter a password instead of a pin, which takes longer, every time they wish to use the device.
3. It can act as a bluetooth and USB keyboard.
The thought of such a device certainly tickles _my_ fancy. Heck, I could build one right now, since I already have the hardware platform to do it. But there is one fatal flaw ... you _have_ to use the device. Suppose you're travelling, forgot to bring it with you, and have no way to get another one. Now you're screwed. The only alternative at that point is to use a software emulation of the device on a PC/phone, at which point you've exposed your master password to the PC and thus potential theft. You get to look forward to coming home and cycling all your passwords later.
Addendum: What I would ultimately like to see is a secondary, secure processor in cellphones; one with direct (muxed) access to the phone's screen and inputs. This processor runs its own OS, and is completely isolated from everything else except for a tiny communication channel with the main processor. When triggered by the main processor, it can take control of the screen and inputs to fulfill its functionality.
Now, to use it, you whip out your phone, launch the related app, and this causes the main processor to pass control to the secure processor where you are greeted by your password manager.
Better yet, the phone's browser could trigger it, and even tell the secure processor what website we wish to log into. Now the user just has to hit confirm/enter a pin, and the rest is handled automatically.
Not amazing enough yet? When plugged into your PC by USB/Bluetooth, if you've got the right software installed, even the PC's browser could trigger this.
Now you have all the benefits of a hardware based password manager, but you don't need a "second device."
But there are a number of problems:
1. How do you authenticate yourself with it? If you lose it you don't want the thief to be able to extract your passwords. You need to reintroduce the "something you know" factor (hard to enter passwords in keychain sized devices), or maybe "something you are" factor (fingerprint? RFID implant? only half joking, I'd consider it)
2. How do you perform backups without exposing the whole database to your hosts?
3. How do you interface with mobile devices? Public computers?