Inside Tesla as Elon Musk pushed an unflinching vision for self-driving cars

Families are suing Tesla over fatal crashes, and Tesla customers are suing the company for misrepresenting Autopilot and a set of sister services called Full Self Driving, or FSD.

As the guiding force behind Autopilot, Musk pushed it in directions other automakers were unwilling to take this kind of technology, interviews with 19 people who worked on the project over the past decade show. Musk repeatedly misled buyers about the services’ abilities, many of those people say. All spoke on the condition of anonymity, fearing retaliation from Musk and Tesla.

Musk and a top Tesla lawyer did not respond to multiple email requests for comment for this article over several weeks, including a detailed list of questions. But the company has consistently said that the onus is on drivers to stay alert and take control of their cars should Autopilot malfunction.

Regulators have warned that Tesla and Musk have exaggerated the sophistication of Autopilot, encouraging some people to misuse it.

“Where I get concerned is the language that’s used to describe the capabilities of the vehicle,” said Jennifer Homendy, chair of the National Transportation Safety Board, which has investigated accidents involving Autopilot and criticized the system’s design. “It can be very dangerous.”

Within Tesla, some argued for pairing cameras with radar and other sensors that worked better in heavy rain and snow, bright sunshine and other difficult conditions. For several years, Autopilot incorporated radar, and for a time Tesla worked on developing its own radar technology. But three people who worked on the project said Musk had repeatedly told members of the Autopilot team that humans could drive with only two eyes and that this meant cars should be able to drive with cameras alone.

In May, Musk said on Twitter that Tesla was no longer putting radar on new cars. He said the company had tested the safety implications of not using radar but provided no details.

Some people have applauded Musk, saying a certain amount of compromise and risk was justified as he strove to reach mass production and ultimately change the automobile industry.

But recently, even Musk has expressed some doubts about Tesla’s technology. After repeatedly describing FSD in speeches, in interviews and on social media as a system on the verge of full autonomy, Musk in August called it “not great.” The team working on it, he said on Twitter, “is rallying to improve as fast as possible.”

Cameras as eyes

Tesla began developing Autopilot more than seven years ago as an effort to meet new safety standards in Europe, which required technology such as automatic braking, according to three people familiar with the origins of the project.

The company originally called this an “advanced driver assistance” project, but was soon exploring a new name. Executives led by Musk decided on “Autopilot,” although some Tesla engineers objected to the name as misleading, favoring “Copilot” and other options, these three people said.

The name was borrowed from the aviation systems that allow planes to fly themselves in ideal conditions with limited pilot input.

At Autopilot’s official announcement in October 2014, Tesla said that the system would brake automatically and keep the car in a lane but added that “the driver is still responsible for, and ultimately in control of, the car.” It said that self-driving cars were “still years away from becoming a reality.”

At the beginning, Autopilot used cameras, radar and sound-wave sensors. But Musk told engineers that the system should eventually be able to drive autonomously from door to door — and it should do so solely with cameras, according to three people who worked on the project.

They said the Autopilot team continued to develop the system using radar and even planned to expand the number of radar sensors on each car, as well as exploring lidar — “light detection and ranging” devices that measure distances using laser pulses.

But Musk insisted that his two-eyes metaphor was the way forward and questioned whether radar was ultimately worth the headache and expense of buying and integrating radar technology from third parties, four people who worked on the Autopilot team said.

Over time, the company and the team moved closer to his way of thinking, placing more emphasis on camera technology, these people said.

In late 2014, Tesla began installing radar on its Model S sedans as it prepared to roll out the first version of Autopilot. But Musk did not like the way the radar looked inside an open hole in the front of the cars and told his engineers to install a rubber seal, according to two people who worked on the project at the time, even though some employees warned that the seal could trap snow and ice and prevent the system from working properly.

These people said the company went ahead with Musk’s instructions without testing the design in winter weather — but resolved the situation after customers complained that the radar stopped working in winter.

In mid-2015, Musk met with a group of Tesla engineering managers to discuss their plans for the second version of Autopilot. One manager, an auto industry veteran named Hal Ockerse, told Musk he wanted to include a computer chip and other hardware that could monitor the physical components of Autopilot and provide backup if parts of the system suddenly stopped working, according to two people with knowledge of the meeting.

But Musk slapped down the idea, they said, arguing it would slow the progress of the project as Tesla worked to build a system that could drive cars by themselves. Ockerse soon left the company.

In May 2016, a Model S owner, Joshua Brown, was killed in Florida when Autopilot failed to recognize a tractor-trailer crossing in front of him. His car had radar and a camera.

The quest for fully autonomous cars

As Tesla approached the introduction of Autopilot 2.0, most of the Autopilot team dropped their normal duties to work on a video meant to show just how autonomous the system could be. But the final video did not provide a full picture of how the car operated during the filming.

The route taken by the car had been charted ahead of time by software that created a 3D digital map, a feature unavailable to drivers using the commercial version of Autopilot, according to two former members of the Autopilot team. At one point during the filming of the video, the car hit a roadside barrier on Tesla property while using Autopilot and had to be repaired, three people who worked on the video said.

The video was later used to promote Autopilot’s capabilities, and it is still on Tesla’s website.

When Musk unveiled Autopilot 2.0 in October 2016, he said at the news conference that all new Tesla cars now included the cameras, computing power and all other hardware they would need for “full self driving” — not a technical term, but one that suggested truly autonomous operation.

By 2017, Tesla began selling a set of services that the company has described as a more advanced version of Autopilot, calling the package Full Self Driving. Its features include responding to traffic lights and stop signs — and changing lanes without being prompted to by the driver.

Engineers who have worked on the technology acknowledge that these services have yet to reach the full autonomy implied in its name and promised by Musk in public statements.

In early November, Tesla recalled nearly 12,000 vehicles that were part of the beta test of new FSD features, after deploying a software update that the company said might cause crashes because of unexpected activation of the cars’ emergency braking system.

Amnon Shashua, CEO of Mobileye, a former Tesla supplier that has been testing technology that is similar to the electric-car maker’s, said Musk’s idea of using only cameras in a self-driving system could ultimately work, although other sensors may be needed in the short term. He added that Musk might exaggerate the capabilities of the company’s technology, but that those statements shouldn’t be taken too seriously.

“One should not be hung up on what Tesla says,” Shashua said. “Truth is not necessarily their end goal. The end goal is to build a business.”