1 M. Anderson Asked: October 21, 2020In: Basics of self-driving cars Will self-driving technology from Wayve revolutionize autonomous vehicles? 1 They use end-to-end machine learning approach to learn to drive and this makes them special aiautonomous vehicleswayve Share Facebook 3 Answers Voted Oldest Recent Buzzcar Added an answer on November 2, 2020 at 9:09 am I think real world robotics in all facets is VERY hard I know that Nvidia tried end-to-end learning for self-driving using 3D sensors, but then decided to switch to Waymo-like approach. Interesting papers: 1) Nvidia End to End Learning for Self-Driving Cars https://arxiv.org/abs/1604.07316 (Submitted on 25 Apr 2016) We trained a convolutional neural network (CNN) to map raw pixels from a single front-facing camera directly to steering commands. This end-to-end approach proved surprisingly powerful. With minimum training data from humans the system learns to drive in traffic on local roads with or without lane markings and on highways. It also operates in areas with unclear visual guidance such as in parking lots and on unpaved roads. The system automatically learns internal representations of the necessary processing steps such as detecting useful road features with only the human steering angle as the training signal. We never explicitly trained it to detect, for example, the outline of roads. Compared to explicit decomposition of the problem, such as lane marking detection, path planning, and control, our end-to-end system optimizes all processing steps simultaneously. We argue that this will eventually lead to better performance and smaller systems. Better performance will result because the internal components self-optimize to maximize overall system performance, instead of optimizing human-selected intermediate criteria, e.g., lane detection. Such criteria understandably are selected for ease of human interpretation which doesn’t automatically guarantee maximum system performance. Smaller networks are possible because the system learns to solve the problem with the minimal number of processing steps. We used an NVIDIA DevBox and Torch 7 for training and an NVIDIA DRIVE(TM) PX self-driving car computer also running Torch 7 for determining where to drive. The system operates at 30 frames per second (FPS) 2) ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst https://arxiv.org/abs/1812.03079 Our goal is to train a policy for autonomous driving via imitation learning that is robust enough to drive a real vehicle. We find that standard behavior cloning is insufficient for handling complex driving scenarios, even when we leverage a perception system for preprocessing the input and a controller for executing the output on the car: 30 million examples are still not enough. We propose exposing the learner to synthesized data in the form of perturbations to the expert’s driving, which creates interesting situations such as collisions and/or going off the road. Rather than purely imitating all data, we augment the imitation loss with additional losses that penalize undesirable events and encourage progress — the perturbations then provide an important signal for these losses and lead to robustness of the learned model. We show that the ChauffeurNet model can handle complex situations in simulation, and present ablation experiments that emphasize the importance of each of our proposed changes and show that the model is responding to the appropriate causal factors. Finally, we demonstrate the model driving a car in the real world. 1 Reply Share Share Share on Facebook Share on Twitter Share on LinkedIn Share on WhatsApp Nick Added an answer on October 29, 2020 at 9:15 am The main problem – “corner cases”, you face them every day as a human driver and there are thousands of them! And its very very difficult to train the NN to handle all of them, without risking passengers’ lives 0 Reply Share Share Share on Facebook Share on Twitter Share on LinkedIn Share on WhatsApp Liam Carter 0 Questions 1 Answer 0 Best Answers 22 Points View Profile Liam Carter Added an answer on November 11, 2020 at 8:53 am The simple answer is “No, not at the moment” They claim that self-driving is possible with: No HD-Maps, No expensive sensor/compute suite, No hand-coded rules, Driving on roads never-seen during training. Replacing the entire self-driving pipeline with one end-to-end trained network seems to be impossible at this moment, but their case successfully demonstrates the potential of computer vision and deep learning approach under some contitions. Getting a robotic system to work in the field is a very complicated game, which may take years and years to achieve stable results in city and urban driving, under numerous conditions (day, night, rain, snow etc), in different countries with their own local laws and traffic/driving patterns. Anyway they have built a very strong team of professionals and are doing very well so far, check some of their videos on urban driving: Link to their blog post: https://wayve.ai/blog/driving-like-human 0 Reply Share Share Share on Facebook Share on Twitter Share on LinkedIn Share on WhatsApp Leave an answerLeave an answerCancel reply Featured image Select file Browse Click on image to update the captcha. Save my name, email, and website in this browser for the next time I comment. This site uses Akismet to reduce spam. Learn how your comment data is processed.