I love the idea of coding on a giant virtual wraparound screen but I'm not sure even a DK2 is going to be up to it. Think about it: The text in your windows can't look "fuzzy" (antialiased or rendered) at all. It really has to be "pixel perfect". We go nuts around here over fonts that get this or that pixel in just the right place on a given serif.
If you doubt it, try setting your monitor to something other than its native resolution and coding for a while using the interpolated resolution. I'm afraid text in floating 3d windows on vr sets is going to look like that for a long time to come.
This is really cool. I ordered a DK2 to try out some similar things in a similar domain. The idea of a 120° workspace which can be expanded indefinitely seems like a reasonable alternative to getting bunches of physical screens on a desk.
I've spent quite a deal of time thinking about how you make a 3D interface actually work for usability, and its interesting to see some of those ideas implemented in this example (allowing 3D views to either pop out in the workspace, or be pushed back into their own world).
The beauty of this approach is that it frees up window managers from managing desktop spaces and instead allows you to manage windows and the relation of the windows relative to other windows. Being freed of borders and boundaries presents amazing opportunities. Defining things in absolute space is very confining. Defining things in relativistic space, gives you the opportunity to move into multi-view/multi-screen modes with multiple participants much more easily. It's not longer about desktops, but about taskspaces.
I don't want to say "put window A on the lefthand side of the desktop and put window B on the righthand side of the desktop." Instead I want to be able to say "put A to the left of B in when mode X" or "put C on top of D when performing task Y."
It's simply much more natural and more intuitive to tell the computer where your focus should be (or if the focus should be split equally between 2 or more windows), and then defining all the peripheral windows that should appear around the focused window. Ideally these wouldn't even separate windows, but simply a rectangular stream of pixels from the buffers of your choice, so that the same application could be multiplexed to layout modes. This would basically be tmux for windows/apps in 3D space.
A prolog-like declarative language with relations and predicates for describing windows and how they are organized relative to one another under variable conditions would be awesome. Type one command to switch to the window you want to have focus in the center of your screen. Once that window is in place, bring into focus the last used "mode" for that window, with all the peripheral windows coming in from behind you or zooming in from the back (time machine style, but with your primary focused window held constant). Then you can activate switching between modes with another key stroke combination.
This is pretty cool as far as tech goes, but also seems rather ridiculous practically. Productivity with a HMD will likely amount to a lot more than the same 2D dynamic images we already have(screens) being placed arbitrarily in 3D space. What will it look like instead? one may ask, to which I have no definitive response.
What we have here seems like an extrapolation of what we have now, which may prove fruitful if we have nothing else to go off of (and is the nature of incremental progress). If I had many more monitors I'd place a class (or any other abstraction) on every screen, so maybe every class will have its own 2D plane of text and they'll be organized semantically & dynamically in 3D space. Also resolution here needs to be really really high. Idk, just interesting food for thought.
As you say, it's an interesting tech demo (which looks like it would be even more fun if you coupled it with a physics engine btw), but not particularly practical.
The way I could see it working though (without a significant change in OS UI) is by arranging your 2D desktop across multiple virtual 2D screens in 3D space, ie a bank of monitors angled towards you. You could use a conventional keyboard and mouse to navigate the 2D space (with perhaps mouse movement assisted or replaced by hand/head/eye tracking), and drag windows between screens, set programs to open on certain ones, with full screen clipping to screen edges - all basically as many OSes do already.
But because they're virtual, you could resize, split and merge screens, curve large ones, have ones which float in front of your vision to use for toolbar windows and HUDs. You could even use existing technology like VMs or VNC to use multiple OSes at the same time.
Mount your keyboard and mouse to a swivel chair which tilts, and you could comfortably have a 360 degree working area (assuming you can touch type) 4 or so screens deep, which would put traders' setups to shame (and at a fraction of the price). Throw in some panoramic backgrounds and you can move your office to a forest or a beach. Take it further you could even set up some sort of virtual office, being able to virtually get up from your desk and visit the workspaces of remote-working colleagues and friends to collaborate.
The exciting thing is that none of this is a massive leap from where we are now, and it would map pretty seamlessly to the software and UI we already use. The main thing holding it back would seem to be the resolution of 3D goggles, but with 4k rising, even that can't be too far off.
It'd be fantastic to have an Oculus-based desktop environment! Imagine not having to drag giant monitors around. Combine it with xmonad and you're golden. I hope this becomes reality soon, at least for development purposes.
Xmonad is a way to optimize on 2D spatial scarcity. I don't think Xmonad will exist in any similar implementation in 3D viewing space. It might end up being similar, but I think that depth, being what humans with camera vision are used to, will become a much more dominating organizational pattern.
Drop xmonad, what you want is no-longer the virtual desktops, but the virtual rooms! (As a bonus it has been proved humans memory interact with entering and leaving rooms.) We will be hanging applications on the walls like pictures.
I always thought what you need is instead a kind of body model.
So you have things which are attached to your head, things which follow position relative to your body, and things which follow your absolute position in the world.
Most people would work in body-space, and maybe have several different overlays they switch between for that.
On top of that, you probably need the concept of the private, shared and public objects in all these spaces - you can interact or allow people into a limited view of your 3D environment.
I feel like jumping straight into floating resizable screens is too great a leap.
I think we have to start off by simulating a desk and two fixed size monitors so the user understands the metaphor, nail that experience and then evolve from there.
The one effect I really really want is when you switch from 2D -> 3D mode with the Goggles I kind of want the 2D desktop to do the laser activation effect from Tron Legacy.
If you doubt it, try setting your monitor to something other than its native resolution and coding for a while using the interpolated resolution. I'm afraid text in floating 3d windows on vr sets is going to look like that for a long time to come.
I'm still going to try, dammit.