Ready, Set, Algorithms! Teams Learn AI by Racing Cars

By Sara Castellanos 

Employees at companies including Morningstar Inc. and Liberty Mutual Group Inc. are learning about advanced artificial-intelligence techniques by programming and racing miniature self-driving cars.

The DeepRacer league, developed by Amazon.com Inc.'s Amazon Web Services cloud business, is designed to teach a branch of AI known as reinforcement learning. In this type of machine learning, algorithms learn the correct way to perform an action based on trial-and-error and observations. The technique is different from a type of AI commonly used in business called supervised learning, in which algorithms have to be fed with labeled training data to learn to recognize images or make predictions. Here, the cars supply their own data: images collected with cameras.

Anyone with an Amazon Web Services account can participate in the league. Teams or individuals can compete online in "virtual" races or in person at events world-wide.

Teams build and train AI algorithms using Amazon SageMaker software, deploy them to self-driving cars measuring about 10 inches, then race them around a track of roughly 17 feet by 26 feet. The fastest car wins.

"It's actually having practical applications," said James Rhodes, chief technology officer of investment research firm Morningstar. Thanks to the training, the company expects to have dozens of projects based on reinforcement learning and other machine-learning techniques in deployment by the end of 2020, he said.

Besides training autonomous vehicles, reinforcement learning can be used to help robots walk faster or to develop security systems that can automatically adapt to different environments, experts say. "[It's] a pretty complicated technology and there's a pretty steep learning curve, " said Mike Miller, general manager of AI devices at Amazon Web Services.

AWS developed the DeepRacer program as a way to teach software developers about machine learning in a more engaging way than reading scientific articles, Mr. Miller said.

The algorithms are complex because they gather data on their own, instead of being fed millions of images to learn from, said Peter Stone, professor of computer science at the University of Texas at Austin, who isn't involved with DeepRacer. Programmers write code to "reward" the algorithms when they do something right, such as winning a race or avoiding an obstacle. In the case of algorithm-powered cars, that could include such tasks as staying close to the center line on the track, minimizing wide steering angles and turning to avoid barriers and crashes.

At Morningstar, more than 450 software developers, equity analysts and quantitative researchers have formed nearly 100 racing teams in 10 countries since January, when Mr. Rhodes began allowing employees to use the technology.

Morningstar has invested "north of tens of thousands of dollars" on the miniature cars and training software to date, Mr. Rhodes said. "It's bringing the virtual [world] into the physical space, especially for individuals who are not necessarily computer scientists," he said.

Earlier this year, one of the Morningstar teams came up with an idea for a tool based on reinforcement learning that looks for patterns in regulatory filings to more accurately identify various information. The tool was deployed in June. Another team came up with an idea for a tool that uses reinforcement learning to automatically find and fix broken links to financial institutions' websites, Mr. Rhodes said. That tool is still in development.

Insurer Liberty Mutual, meanwhile, has about 270 employees, including software engineers and data scientists, participating in DeepRacer.

"It's a fun way for people to get practical exposure for what are very important algorithms in a safe environment where they're not going to mess up any core business applications," said James McGlennon, the company's chief information officer.

The company is already using other machine-learning techniques to tweak prices for auto insurance based on risk factors and to look for anomalies in operations. The goal of the DeepRacer program is for employees to think about ways the company can use reinforcement learning to help the business, Mr. McGlennon said.

Reinforcement learning is one of many machine-learning methods that will ultimately be used by companies, said Dario Gil, director of research at International Business Machines Corp. It is challenging to rely on that technique alone when training a real-world autonomous vehicle, he said, because so much of the method relies on trial and error. "There's a reason why reinforcement learning gets stuck in the world of games," he added.

Write to Sara Castellanos at [e-mail]

(END) Dow Jones Newswires

October 01, 2019 05:44 ET (09:44 GMT)

Copyright (c) 2019 Dow Jones & Company, Inc.