Getting started in Robotics

Jan 01, 2021

People sometimes ask me for robotics resources. There aren’t many high quality internet-first lists available on this topic, so I made one I can link to. I'll continually update this list so feel free to give suggestions if things are missing or you have better links.

There's some more interesting comments on this topic in the Hacker News discussion.

General Advice

Robotics can be overwhelming from the sheer number of fields involved (everything from mathematics to physics to computer science to mechanical engineering enters in). Remember: you won’t start as an expert in anything; you can’t end up as an expert in everything. Also learning will not be a linear path; outside of courses, acquire knowledge as you need it in projects.

Try as much as possible to avoid this antipattern (I still struggle)… {{< tweet 1343701831210307586 >}}

Communities

Instead of attempting to pre-load your head with maximum theory, try to get as much practical experience as possible (see below), whether that be joining a club, lab, or if you’re lucky in an internship or job.

It is incredibly hard to do something on your own without some kind of environmental forcing function. If at all possible, embed yourself in a community of people working on problems of interest to you.

FIRST robotics competition - Simply put, FRC was the awesomest thing ever for me. If you can find a team, beg them to let you participate…
If you are a student at a university - try joining a lab!

Unfortunately outside of these robotics is currently not an easy field to break into. The sheer number of ‘moving parts’ to a (useful) robotic system - from hardware to drivers to algorithms to user interfaces - means it’s tough to hack together a demo in a few weeks, get users, and bootstrap / apply to some accelerator like in many software projects. Making the jump from “hacker” to “career”, even with all the knowledge in the world is hard. If you know solutions to this, let me know.

Prerequisites

Computer programming - this is essential if you want to do basically anything in robotics. Python and C++ are the most important languages to know - learn both. Familiarity with basic data-structures and algorithms concepts is helpful for problem solving and in some fields like motion planning but initially less vital than you may think - it’s not necessary to be a master leetcoder.

Maths

Linear algebra - vital in most topics beyond basic basic control. If you want to become an expert, Axler’s book is good (though I didn’t make it through the whole thing 😳).
Calculus - fundamental differential calculus and partial derivatives is required to understand optimisation algorithms which are everywhere in robo.
The book A Programmer’s Introduction to Mathematics is actually a solid foundation if you want a quick overview of both of the above topics.

Important Concepts

Controllers

Wiki's PID controller isn’t bad if you want to learn the basics (95% of controllers used in industry are PID). Ben Recht's blog post is also decent.
These course slides are good for basics and more advanced concepts like MPC .
Control challenges is a set of (very fun) challenges that will allow you to test your ability to write basic controllers.
This talk from FIRST Robotics team 254 is great for going over the basics of robot control and showing how we can improve on basic PID using simple interpolation.

Mathematics and Mechanics of Robotics & Manipulation - This set of notes by Nathan Ratliff provides a great overview of many topics. These notes on robot dynamics from ETH are also great.

Filtering - Kalman and Bayesian Filters in Python is a great read with interactive notebook examples throughout. Good for building intuition on both Bayesian statistics as well as how filters for linear and nonlinear dynamical systems work. The wiki article on the Kalman Filter and this blog post is also well written if you want a shorter overview.

Machine Learning & Neural Networks - Focus less on the basics of machine learning than the application of neural networks as function approximators, which is their primary utility as perception and decision making modules in robotics. Michael Nielsen’s book is my favourite resource in this regard. The canonical deep learning book is good reference but you don’t need to read the whole thing - pick out specific topics as required.

Path and motion planning - Wiki has a decent overview and details some basic algorithms. I haven't read it but this book is apparently a good comprehensive guide, and a course based on it is available here.

Reinforcement Learning & more on controllers - Ben Recht’s “outsider’s tour of RL” provides a nice concise overview of many relevant concepts and ties them to classical control. Spinning up in Deep Reinforcement Learning is another great reference on the topic and exhibits many of the most important algorithms for research in the field.

Software

The space of software is so vast I couldn’t fit it all here... this list provides a much more comprehensive overview in this regard. Some utilities I’ve found useful are:

ROS is very widely used many projects, worth picking up the basics.
RLLib has implementations of many reinforcement learning algorithms and support for distributed training.

Simulators

Robosuite is great for robot learning and has many richly-designed built-in environments.
Isaac Gym is recently released and provides extremely fast samples for reinforcement learning. It does this by running thousands of environments in parallel on the GPU (in almost all other sims physics is run on the CPU meaning you can only run one environment at once). You can train tasks in minutes that previously took hours.

Interesting Lines of Work

Domain Randomisation is highly underrated, and will likely get more important as we get faster simulators which are able to generalize better. Learning Dexterity from OpenAI is a very interesting paper, and the line of work around improving simulator parameter show two different and interesting takes on this powerful approach.
Hierarchy in reinforcement learning has shown promising results in getting behaviour that generalises, the DeepGait and UniCon paper being two interesting examples.
DexPilot is a really cool low-cost system for teleoperation (video). Partially inspired my piece on teleoperation.

Hardware

Probably my weakest area… up for recommendations about specific great resources. OnShape is a great resource for designing things and is great even with minimal CAD experience (even I am able to use it!)

Open Dynamic Robot initiative has many robots that you can build with off-the-shelf components.

Others' recommendations -

In the HN thread, contingencies had some tips on how to get started yourself on the hardware side.

I'm told cadathons such as those organised by Solidworks are a good way to get started. They also have a series of lessons on CADding fundamentals.

GrabCAD has big libries of pre-existing models you can use.

Business & Higher-Level Thinking

Very interesting side of things, however there is somewhat of a paucity of great resources (I’m pretty sure many in the space refrain from publishing their theses.) Please send if you have others I have missed!

Michael Dempsey has some great writing on the business of autonomy.
Trucks VC has an interesting newsletter about autonomy.
Jessy Lin wrote some essays on topic surrounding the economics and future of autonomy and HRI.
Rodney Brooks makes predictions in the robotics space on a yearly basis and has some good rigorous thinking on there.
EE Times has news and frequent opinion pieces in this space.
Though inexperienced, I sometimes try my best to write on this topic :)

Double A Rebooted

Discussion about this post