I’m (only) half joking as I say this... if you’re particularly interested in robotics (be it the mechanical side, or the algorithmic side), try not to get too caught up with “deep RL”. It’s a minefield that generates a LOT of buzz for as yet dubious performance, and IMHO there’s much simpler low-hanging fruit to get started with.
I've been trying to understand why robotics doesn't work for quite a while. It seems to me like the next logical field of radical innovation. I understand there are funding issues for startups, the expertise issues, the utility issues of the current robots themselves and the vast number of theoretical issues of the sub-fields themselves.
What I don't understand is why not every single hobbyist is not focusing on robotics. It seems to me to be the pinnacle of generalism and technical entertainment.
What I'm trying to ask semi-jokingly is: It is winter, where is your miniature, autonomous snow plow that you built yourself from scratch and why aren't you working on one?
It’s hard and cross disciplinary. A snow plow needs real horse power, a snow blower might be better but still has a whole kettle of problems. Those problems exist in the real world where you can break things that you need to pay for. There’s lots of cool projects that would be fun to do but without a source of free money or parts they aren’t going to get done. Software is easy because all you need to do to get started it type mkdir and open a text editor. The solution for robotics is physical simulation with accurate modeling of real parts and chips. Make it as easy as opening an integrated game engine like Unity, drag and drop parts that you can either download or model yourself. Write some code and run. Print a part list when you’re done.
Also, laws and rules you have to follow, and generally other people. I'm pretty sure if I started clearing snow in my neighbourhood with a DIY snowplow, I'd get in trouble with the housing cooperative, as they're already paying someone else to do that (however shitty job they do). Police may eventually get involved too, as I'm pretty sure that operating a car-sized robot on a road is illegal in more than one way.
Now if I really cared about doing this, I could probably follow some legal procedures to get my snowplow registered, and I could probably convince the neighbours/co-op to let me play with it. But it would all fare better if I formed a company around it and enter the discussion as a legal entity. At which point I may as well start a real snowplow business - which is way beyond my hobby interests.
And this applies to a lot of meatspace innovation - unless you own (and not rent) a home with a large backyard, and confine the scope of your experiments to just your backyard, pretty much all interesting ideas I can come up with require enough red tape that it's not even worth it, unless you're doing it for money.
This is why I’m hoping remote work continues as a permanent option. We need to get creative people out of the cities while still maintaining good connections among them.
The connection part is tough. I'm moving back to a major city because I found living in a small town for the past couple of years to be mind-numbingly boring.
Yeah it’s not great depending on who your neighbours are. I think better transportation and better VR will be necessary if you want the option of living anywhere. Until then finding a town that has a thriving scene that you’re into is the only option.
I wonder how far you tried robotics yourself, as in what is your frame of reference.
Here’s my framing: Setting aside sheer knowledge, building anything will require power, motors and parts.
Power is doable up to some point, buying batteries and plugin them is a low barrier with low risks (still need to care about them not burning though)
Motors are a matter of costs and learning what’s available and how to use them.
Then come the parts. Will you machine them ? That would be a CNC machine and it’s a rabbit hole needing space, money, time, material, learning and safety measures. The mild version is a 3D printer, but it’s still a significant cost, a learning curve, a dedicated place available for long hours with effective ventilation.
We’re pretty far away from a simple hobby anyone could try, compared to fishing or papercraft, or guitare, or even wood working. Basically that seems miles away from anything you would start out of boredom.
Instead of CNC or 3D printing, couldn't you just use something simple as Lego or Fischertechnik? Or other kits? (Maybe depends on how big it is supposed to be, and what it is supposed to do.)
But also, I thought that you can get some pretty cheap 3D printers nowadays?
Instead of batteries, couldn't you just use a cable? (Depends how freely the robot is supposed to move around.)
I have seen some robotic kits you can buy, varying widely in what they contain, and the target audience (ranging from kids to semi professionalists), and the price (starting at 100€ or so, up to 10k€ or so).
Lego bricks are fine, but you’ll still need to make custom parts to build anything mildly complex. In particular to make your motorization system hook to them (the native Lego system is pretty limited)
It depends on where you live, but the main cost of a 3D printer for me would be less the machine itself than the space and ventilation needed.
Friends who made the jump basically dedicated a room to making stuff so they can run prints overnight while ventilating (setting up a fume extractor is another option, it’s just a bit more investment in the “hobby”)
I see all of it as totally doable, but needs a serious level of dedication, time, space and money.
Because it requires either a team or an experienced multidisciplionary person - you need to do mechanic design and part fabrication and electronics and software.
Focusing on just one of these areas would be suitable for a hobbyist, but that's not really a practical option until/if you get a solid ecosystem of cheap and interchangeable components, which really isn't there; sensors and mechanical parts and tools cost a lot, and complete platforms even more, so if you want to work with anything interesting you generally have to do it as part of a lab or team, not as a lone hobbyist.
Also, there is a very long path to any gratification - it takes a lot of work on all the segments until anything starts working and you get some positive feedback; e.g. on the autonomous snowplow example, it would be a loooong road until the 'hello world' autonomous snowplow that's e.g. able to drive forward while the camera is showing all white, as that requires all your engine work and driving parts and electronic driving controls and camera integration to be in place, and if any of these things fails because you're not skilled enough, it just won't work at all. Contrast that with many hobbies where you can get some visible (even if sloppy) stuff done relatively early when starting out; robotics has a very high barrier of entry compared to other hobbies.
There is a place in the world where this is a lot easier. And that is Shenzhen China. You need to be in a place with hundreds of manufacturers of every little component who can then customize your order within days.
For this reason this place is likely to become a sort of Silicon Valley of robotics.
The problem with robotics, is the difference between a world of atoms and a world of bits.
When the plague hit, I was sucked into an effort to build a ventilator, I still have a pile of parts that were purchased for it, but the chassis fabrication never happened, so I had nothing to write code for, thankfully the need evaporated. I learned that it takes MONTHS to get motors and gear trains supplied if they don't happen to be in stock.
If you thought the 1960s world of waiting a day for your output deck to see if your program compiled and ran was bad... it's got nothing on the world of supply chains.
You can buy off the shelf components from Motion Industries, but they are out of the price range of hobbyists.
What do you mean by miniature? I guess you need to have a certain physical size to make a snow plow work. Maybe the size of a lawn mower. Instead of starting from scratch you could get one of the robotic lawn mowers and try to modify it to plow snow on your street.
I think it will be hard enough to get it to navigate freely instead of relying on cables in the ground, which is what the lawn mowers use as boundaries of their world, as far as I know.
Here are some formulas to get started calculating the snow resistance force and the required vehicle weight :)
Significant robotics projects usually need a team of specialists and possibly expensive hardware. Hobbyist robotics is fun, but unlikely to achieve state of the art in anything. It is easier for a single person to make a meaningful contribution in fields like programming language design, graphics, etc.
It is very time consuming, and requires a lot of materials that need to be purchased and shipped, a workshop or at least a decent sized room with good tools (and good neighbours, I guess), and ideally a good location because getting all that stuff when you live in a less developed and less populated city is a major pain.
Basically, it's a serious time and money investment.
Although the article begins with a photo of a quadruped robot, it focuses mostly on the software - path planning side of robotics. While still a very active research field, BD and any other places doing legs are much more concerned with mechanical - dynamics, actuators, battery, locomotion problems.
If you want to do some real work on those, pursuing grad school in Electrical (actuators, battery) / Mechanical Engineering (legs, locomotion) is the best, and for the legged robots industry probably the only way (same goes for biomimetics - robot hands). The related research area encapsulating the above is called passive dynamics and primarily is a control theory-based field.
Yup, BD is doing amazing things in the hardware space, though obviously still loads of planning & SW involved in their robot. One of their engineers gave a talk at my campus last semester, crazy what they are doing now with pushing the limits on topological optimisation, printing components including metal etc to push weight down and strength up.
In the article I focus mostly on the software aspects b/c it's more accessible when getting started if you aren't in grad school (+ what I have most experience with).
Hi, I didn't know you were the OP of the article. If you're interested in adding a Books section in the future, you may want to include Robotics: Modelling, Planning and Control by Bruno Siciliano, an all-time classic reading in most introductory robotics university courses.
Thanks for sharing this. As a Masters' student currently studying robotics, I felt pretty overwhelmed with all the sub-fields when coming from a computer science degree in undergrad. I only got a clear understanding of what I was interested in after a year of courses. I think joining a lab and learning as you go is sound advice, but if you really want to do some novel research, you would need a semester or two of courses just to be able to contribute (by which time you would have almost finished your masters). It also seems like a lot of research directions gravitate towards deep RL when classical approaches could work just as well, if not better.
Also, unfortunately due to COVID, most of the work I've done with robotics has only been in simulation and I've found myself frustrated by the limitations of the simulator or the work required to simulate a real world scenario. Working with real world data might be slower, but it's a lot more satisfying when it works.
I'm starting to see some undergraduate programs offering Robotics Engineering degrees. What are your thoughts on pursuing one of those rather than a more traditional engineering discipline (Mechanical, Electrical, CS, etc.)?
I’m currently an undergrad student considering studying robotics eng. at my university. (Olin College of Engineering)
It depends on how the university structures their program, but for us it’s extremely open-ended, besides a few basic fundamental courses. They essentially want you to choose a concentration and pursue that concentration in your robotics projects throughout the 4 years.
It’s a risky choice for us, as unless you have some cool projects to show at the end you could end up ‘not concentrating in anything’ in particular, as robo is such a general field. But if you know what you want to do the flexibility can be nice.
Thanks for your perspective. My son is looking at Miami(OH)'s new RE program. Theirs is a blend of required ME, EE, and CS classes with students adding an additional 2-3 courses in one of the 3 areas.
My biggest concern, as a parent, would be having him be a jack of all trades, master of none at the end of his BsE and having a hard time getting hired or into a Masters program.
I once worked at a robotics company doing Automated Guided Vehicles.
I was stunned that they handled everything with a tiny team. Mechanical design, welding, electricity, electronic design, programming, etc. Crazy. That company started with 2 brothers, and they did everything from top to bottom with only the both of them.
It was really inspiring to see how some people can have knowledge and skills (eg welding) in so many fields, and are able to pull it off commercially.
What stunned me in 2020 is that it takes 20 people to program a simple web app. Too much money in this industry now, people became hyper-specialized plumbers, with no siloes and yet everyone has their own little island. Very depressing.
In the world of the web in the late 90s, 5 guys designed, built and shipped an entire product without any frameworks.
That's because every new hip protein drink and organic chocolate brand that comes on the market wants its own webshop and app for branding, marketing, sales and payment which creates huge demand for web and app devs and the barrier to entry is relatively low, just buy a Macbook and sink a few months into following tutorials and building projects.
Not that many people need robotics experts and the industries that do, already buy from established players like Fanuc, Kuka, ABB, Siemens and the barrier of entry is very high, often a MSc being the lowest bar to clear with many workers in that industry having PhDs or postdocs so it's a lot of money and years of your life you have to dedicate before you can even enter and once you do the work may not be as exciting as one would expect, often times just updating simulink or labview models or days of just shoveling through ISO and regulatory specs.
I'm not saying there are no self taught robotics devs who earn well and play with cool stuff for a living but that's the exception in this industry, not the norm.
Yeah that's mostly on point. I have an MScA in mechanical engineering and my thesis was in robotics. I worked in industry (not robotics) for a few years but my current job is as an RA in academia.
I play with the the cool stuff all day, but pay is below industry average (for ME, don't even think about software engineer type pay).
This a great observation. Robotics requires a strong foundation in mathematics, control, embedded and similar, something that is unheard in typical software engineering roles. Plus, there is a "real-world effect" - if something works in simulation there is no guarantee it will work in the real-world (especially if the simulation is not properly designed). The software has to written in multiple programming languages, often minimum is C, C++ and Python. And everything stated here are software challenges, there are also mechanics, electronics and expensive equipment.
In summary, it requires more than a few tutorials to get started, it is multidisciplinary, and you have to deal with the unpredictable real-world.
I always thought that companies in that industry had specialized teams (software team, hardware team, mechanics team etc etc.) and that it's impossible for a person to possess all skills. But Stuff Made Here has shown me that I was wrong. It's awe-inspiring to say the least.
They have all those specialized teams, because in the industry customers have very high expectations and low tolerance for failure.
Stuff Made Here is a great channel and produces a lot of fun projects, but that only works in a context of prototypes where you need a wide range of skills (but not necessarily a deep one for each of them). E.g. the hair-cutting robot is fun to watch, but barely works (and needs human intervention) and is years away from being a viable end-user product.
As the author says "... it’s tough to hack together a demo in a few weeks, get users, and bootstrap / apply to some accelerator like in many software projects". So the financial rewards don't seem to be there in robotics startups. And as a result it seems that engineers in the field aren't paid that well compared to software developers, despite it being highly technical and requiring a rather vast skillset. Will that ever change (the financial part)?
In 30-40 years when supply chain logistics, manufacturing and distribution are fully automatable it will likely be possible for a small team to quickly convert an idea to a prototype to a product in the hands of the consumer.
By then the software/service delivery process is going to be so encumbered by regulatory response to the likely hundreds of thousands if not millions of deaths and hundreds of trillions in economic losses due to bugs and hacks that the playing field will be largely leveled (through convergence more than anything, but still).
(That or the first company to ship a reasonable facsimile of Ava from Ex Machina. Go the Tesla route and start off selling them for $5-10M each and work your way down. People will be mortgaging their homes to buy one.)
If you can hack something together easily that can earn a lot of money, others can too. So that value will drop. Once upon a time you could slap together websites with html and make six figures. Now you'd probably be lucky to make 5. Same with phone apps.
Also from the perspective of the same person over their career, engineering skills definitely hold their value better than software skills.
Anyone know of any resources for advanced computer vision? This seems like a critical starting point.
And I don't mean using OpenCV or something to draw boxes around objects. I mean understanding the 3d scene in detail. And ideally even understanding the dynamics of the world.
Also, anyone have any experience with HASEL artificial muscles or something similar? Anyone know a reason I shouldn't pursue that over servos?
I never heard of HASEL artificial muscles and just looked them up. Very interesting! But the main downside seems to be the high voltage requirement. Hopefully there are more innovations in the space of robotic actuators. I hate that servos usually are not compliant and rather are controlled by position and not by torque / force. I believe active compliance is not a real replacement for passive compliance in most use cases
I took this graduate course, which I think did a good job covering more advanced / modern CV approaches: https://web.eecs.umich.edu/~ahowens/eecs504/w20/
The python notebooks are locked, but the slides & recommended reading materials are freely available.
Simulation is definitely an important and useful tool.
That said, to me the real beauty of robotics is seeing a physical machine that you put together and wrote the code for move around on its own accord.
It's definitely harder to get started this way than with simulators, but it's now easier than ever. And you'll learn all sorts of things that never come up in simulated environments. Something like the nvidia jetbot (https://www.nvidia.com/en-us/autonomous-machines/embedded-sy...) is easy enough to get started without much fuss and thanks to the onboard camera and GPU, can get you pretty far in implementing advanced deep learning-based algorithms.
(That said, the default camera isn't that great, which you can consider either part of the fun, or just making your life unnecessarily harder. I would consider adding something like a realsense D435i to get IMU+depth for state estimation).
I don't have direct knowledge of how the SpaceX flight control software works, but it's almost certainly classical. A descending rocket is essentially an inverted pendulum [1] with two-axis control via gimbals in the engines, which is a pretty well-studied and solved problem in robotics (just search for "inverted pendulum robot"). Hover-slam timing is probably empirically derived and then computed based on velocity, position, and mass (again classically). High-accuracy rocket positioning systems date back to the cold war (or maybe earlier) and are definitely not ML, either. As for why it wasn't done before, the answer is mostly lack of incentive/competition. There really wasn't any reason for the launch companies to make reusable rockets when the government and commercial customers were willing to pay for non-reusable and no one else in the market showed any interest at all in developing their own.
I wouldn't say that. There were many projects with reusable stages, e.g. Space Shuttle boosters were partially reusable and the Energia rocket had a variant in development with first stage boosters landing like airplanes.
And there were number of practical problems at the time as well:
- Computers were relatively big and heavy.
- Rocket development was mostly focused on improving efficiency via higher chamber pressure, meaning that engines worked on the edge of their capabilities, which is not a great fit for reusability. It's one of a good reasons why Falcon 9's Merlin engines have the simplest open-cycle design with a relatively low chamber pressure.
- Number of launches was too small to make reusable designs economically viable (i.e. your production line would be idle for too long). Even today this number is not high enough, this is why Musk focuses on projects like Starlink to create an additional "artificial" demand for launches.
I guess the control is pretty wall tuned classical control loops but I've seen a paper on the trajectory optimisation to get down to earth while experiencing perturbation and land using minimal fuel and achieving zero velocity in 6DoF etc. Can't find the paper now unfortunately.
They understandably don't publish much. The best clues I have found are in their lead landing engineer's pre-spacex publications. This one a good one for example: http://www.larsblackmore.com/BlackmoreEtAlJGCD10.pdf
The tldr is that they find a convex approximation to the problem and use standard convex solvers to find the right inputs.
This paper[1] says that they use cvxgen[2] to generate the on-board solver code.
Read up on metal fabrication, general manufacturing and assembly processes, and then build a bunch of stuff using a minimum of custom components and a maximum of third party modules with an eye on the true costs including BOM, design iterations, and assembly. MIT2.008 (Fundamentals of Manufacturing) is not bad. Search Library Genesis for manufacturing / mechanical and go for the textbooks/references. If starting from scratch you can get a long way for personal experience with a home-built wood/polymer router, a drill, low end MCUs and scavenged parts. Even if you want to focus on design, you can gain from either outsourcing all your parts fabrication (this will also force you to learn how to communicate designs effectively) or getting a membership of a shared workshop/makerspace and learning the machines and processes yourself, which will help you to do better designs.
IMHO it's sort of a schizophrenic field. You could start by dissecting the parts lists from factory automation supplier catalogs, but then you'd likely miss custom mechanical component design, be stuck with fat form factors and have no idea about custom control systems. You could go oldschool machinist like timepiece design and repair stuff, but then you'd miss modern mechatronics entirely. You could go kit/hobby projects but then you'd likely miss industrial standard pneumatics, while you'd gain enough electronics insights for custom mechatronic control. Each area's boundaries have their commercial and historical reasons but there's no clear map of the whole landscape that I've seen, and I'm not an authority.
Can anyone give some examples how a "proper" robot control application looks like? I once was working on an DIY delta robot and the software layout is something I never understood how to do right, I fear.
I had no idea how to go about it, and if I remember correctly, I wrote the firmware to do the following every 10ms: (I was bare-metal C/C++ on a Cortex-m3.)
* Calculate new desired position of effector.
* Calculate new desired angle of every stepper motor.
* For every stepper motor, calculate deviation between current and desired angle.
* Program the timers to generate a PWM signal to make deviation zero. (Note: This whole thing was open loop, so I fed the PWM signal back into a counter to track the motor angles.)
Each of these bullet points was implemented as individual function, taking input parameters as function parameters and returning the result as return value.
At one point, I was wondering if I shouldn't just call the last function for the final value, and have it call the previous functions (and thereby calculate the input values) on demand. This would look a bit more functional-like, but I didn't see any benefits besides stylistic ones. Is functional programming a thing in embedded robot control software?
While building that demonstrator I realized that my attempt at doing so was a very shabby one. It's not enough to have in mind the acceleration/deceleration of the effector for smooth looks. If you would do things right, you would have to watch the torque and motion parameters of each individual joint. You would probably have to calculate the whole motion profile in advance. But I have no idea how I would go about doing that. Do you simply run a loop and iterate over time? What if you realize you violate some constraints at some point? Adjust some initial values by some fixed offset and run anew? I suppose there are smarter attempts.
edit: Not a HN heavy user, wow do I make lists right?
I think the usual way of handling stuff like that in the embedded world is with an RTOS. Something like FreeRTOS or RTEMS (both have ARM Cortex ports). Also, depending on complexity of the system (more actuators) you might have the individual steppers/servo motors etc run by their own microcontroller and being fed commands centrally.
Before rushing out to buy a CNC and building your own fabrication workshop, there are a number of inexpensive robotic arm kits that can be used to develop/build skills. Also given popular interest in drones, there are a lot of cheap parts that can be used to build your own and add vision systems etc. Interested to hear from others about their best recommendations for inexpensive development of mechanical skills.
Thanks for sharing!
I'd agree with your point that getting started in robotics is challenging on your own, and that finding a research group is a great place to start.
