The problem with optimization is that you always have to compromise, either speed, features and performance for size or features and size for speed etc.. I think a probable solution would be to create a dynamically generated framework which allows the user to specify the needed features (jQuery-ui style).
Actually there's a builder that allows you to pick the functions you need and compiles custom versions: http://minifiedjs.com/builder/
I don't think that this is a practical approach for most people though. After a while it becomes quite annoying to create a new version everytime you need a function. It would be great, but unfortunately also quite difficult, to automate this.
