Autonomous Vehicle Architecture

Introduction to System Integration, Sensor Subsystem, Perception Subsystem, Planning Subsystem and Control Subsystem

Prateek Sawhney
3 min readSep 13, 2021

Welcome to this medium article around System Integration. In this medium article we’ll be discussing the system architecture used in self-driving cars so that we can fit all of the pieces together. A self driving car has four major subsystems.

Photo by Gabe Pierce on Unsplash

The first subsystem is the sensor subsystem, which consists of the hardware components that gather data about the environment like lidar, radar, cameras and GPS sensors. The second subsystem, the perception subsystem, consists of software to process sensor data. Meaningful information is processed from the sensor data in the perception subsystem. The planning subsystem is the third subsystem on the vehicle. It uses the output from perception for behavior planning and for both short and long range path plan. The subsystem which ensures that the vehicle follows the path provided by the planning subsystem is the control subsystem. It also sends control commands to the vehicle.

Perception Subsystem

The perception subsystem, processes data from sensors into structured information that can eventually be used for path planning or control. This is where environment analysis takes place by the vehicle. So we can think of the perception subsystem as the vehicle center of understanding about its environment. Broadly speaking, we can further divide the perception subsystem itself into two underlying subsystems;

  1. Detection
  2. Localization

Using sensor and map data to determine the vehicle’s precise location is done by the localization subsystem. The detection subsystem is responsible for understanding the surrounding environment. This subsystem includes software components such as;

  1. Lane detection
  2. Traffic sign and traffic light detection classification
  3. Object detection and tracking
  4. Free space detection.

Different components in the detection and localization subsystems draw information from different sensors. Additionally, the localization subsystem requires map data which it uses to identify the vehicle’s location. The perception subsystem passes the data from localization and detection to the planning subsystem. The planning subsystem determines what maneuver the vehicle should undertake next.

Planning Subsystem

The vehicle uses the information processed by the perception subsystem to plan its path. There are several components of the planning system like route planning, which is the high level path of the vehicle between two points on a map; prediction, which is what identifies which maneuver other objects on the road might take; behavior planning, which decides what maneuver our vehicle should take; and trajectory generation, which plots the precise path we’d like our vehicle to follow.

The planning subsystem takes in data from all of the components of the perception subsystem. Additionally, map data is used for long range planning. Once the vehicle has a planned trajectory the next step is to execute that trajectory. This is the responsibility of the control subsystem.

Control Subsystem

The last subsystem in the vehicle is the control subsystem. It contains the software components which makes sure that the exact path is followed by the vehicle which is specified by the Planning Subsystem. The control subsystem may include components such as PID controllers, model predictive controllers, or other controllers. As we know that the control subsystem sends acceleration, braking, and steering commands to the vehicle, this completes the chain of information from sensors to actuation and allows the vehicle to drive.

--

--

Prateek Sawhney

AI Engineer at DPS, Germany | 1 Day Intern @Lenovo | Explore ML Facilitator at Google | HackWithInfy Finalist’19 at Infosys | GCI Mentor @TensorFlow | MAIT, IPU