Reminds me of the cool stuff that Survival Research Labs[1] used to do in the Bay Area in the 1990s and early 2000s. They'd have very unpermitted and somewhat dangerous shows in abandoned lots in industrial zones and under freeway overpasses. Fun times. Updating it for 2020s robotics, by doing the same thing with semi-autonomous control would be even more wild, and potentially dangerous.
And Compressorhead was a hobbyist-ish project created by some German robotics enthusiasts years ago, yet it was way ahead of what this billion dollar company wants to present as cutting edge robotics. What a joke, I couldn't help laughing watching that Xiaomi video.
It really doesn’t though. Limited by zero upper body flexibility, all arm motions displayed here are not in any way organic and don’t surpass the capabilities of say, an orchestrion
It's hard to know if they've really solved the problem. It looks suspiciously like they're using a very basic MIDI -> gesture look up table with a very limited range of context-dependent extensions. And no expression or dynamics.
That would make it a toy project, about forty years behind the state of the art.
It is just using MIDI, and it doesn't look like it has velocity support, that drum kit definitely has velocity clamped to 127 to make it sound more consistent. They claim it's solving whole-body trajectories with self-collision avoidance, which is pretty cool IMO. But I don't really see any sequences in the demo that had any risk of collision and couldn't be done with simpler motion mapping. Show us some rudiments on the toms!
I would be surprised if it wasn't scripted, the movements are so angular and linear.
I can only see ML being involved if the ML's interface was an over simplified model with all of the individual movements pre-scripted, i.e only allowing it to decide when to position limbs and onto which drums, but not how, which isn't much of an achievement. It smells of lies and hype. I could be wrong but either way the result hardly looks groundbreaking.
So it's not really doing anything except anticipate how long it takes to hit a drum to hit it on time. From this point of view a music box is a "robot".
If you Google for it you'll get the gist of it. Wikipedia redirects "kitchen robot" to "food processor", and companies like Thermomix or Dúvida markets their products as "cooking robots" and "kitchen robots".
wow, I was expecting to be blown away considering everything else robot /AI lately and this is....pretty bad? did they have any actual drummers show them how to hit a cymbal at least ?
I mean, the current bar for robotics is at least the stuff Boston Dynamics has put out which is way ahead of this.
>What was the most challenging part of this research?
>Ren: The most challenging part of this research was that when receiving the long sequences of drum beats, CyberOne needs to assign sequences to each arm and leg and generate continuous collision-free whole-body trajectories within the hardware constraints. So, we extract the basic beats and build our drum beat motion trajectory library offline by optimization. Then, CyberOne can generate continuous trajectories consistent with any drum score. This approach gives more freedom to CyberOne playing drums, and is only limited by the robotics capability.
Nice research, but I'd be more impressed with an online optimizer that only looks ahead about a dozen moves. The point of a humanoid robot is to put them in environments with uncertainty where a limited library of optimized moves won't be enough. Still, AFAICT, it's a fully closed loop from the cameras to the motors so that's nice.
Props to the engineers for making it, but it's definitely not optimized for drumming as it fails the very basic technique to correctly use sticks to play effectively and minimize fatigue: you don't play using arms and shoulders, unless you want to become exhausted or even hurt yourself really soon, but wrists and optimizing sticks grip for elasticity rather than force.
This short video will explain it much better than a thousand words:
For as stunningly and embarrassingly terrible as this is, I'm not even sure that the video isn't completely fake. During the flourish just before the end of the video the physical contacts don't seem to line up with the beats at all.
There's also some really interesting stuff going on in music informatics, audio and signal processing, especially where everything comes together - accompanying human players:
I can say I'm much impressed. At the same time I haven't watched Boston Dynamics videos for last couple of years. When I checked latest Atlas demo from 2021 it look like sci-fi to me:
However, I'm spoiled:
https://www.youtube.com/watch?v=b7nMZvueKNQ
https://robocross-machines.com/robot-punk-band/