By Lynn Greiner

AWS RE:INVENT — What do one-eighteenth scale replicas of race cars have to do with machine learning and artificial intelligence (AI)?

A whole lot, it turns out, if you’re at AWS re:Invent and want to learn about reinforcement learning as a training method for AI models.

The DeepRacer car (you can find full specs, and a pre-ordering link, on the AWS blog) is an autonomous vehicle that is trained by reinforcement learning, a new form of AI model training that has so far been too complex for companies to comprehend without experts on board. While it requires large amounts of data, reinforcement learning is a very effective technique to use for problems requiring sequential decision making.

Basically, DeepRacer performs actions and receives a reward (or lack of one) depending on the result. Just as you train a puppy by giving it a treat when it does the right thing, the AI receives a virtual reward when it acts toward a specified goal. In the case of DeepRacer, it is rewarded when it stays on the track and drives speedy laps.

You don’t, however, send the car out to learn by itself. Instead, the training takes place in an online simulator, with the developer writing Python routines to define the goal and reward. The simulator, which uses Amazon SageMaker and newly announced AWS RoboMaker, then cycles through, tweaking the model to gain the best rewards. Once satisfied with the model, the developer loads it into the car and quickly finds out whether the code delivers desired results in the real world.

At re:Invent, AWS also announced a global DeepRacer racing league so developers can pit their cars against others to see who produced the best model; there will also be a wholly online league for developers who can’t compete in the physical world. The first championship race featuring the top three developers, who have had access to the system since the end of CEO Andy Jassy’s Wednesday morning keynote, was held before CTO Werner Vogels’ keynote Thursday morning. It was won by Rick Fish of London, co-founder of Jigsaw XYZ.

Vogels then carried on the theme, announcing a collection of developer-focused goodies.

To start with, AWS Cloud9 IDE received integrated support for serverless development, and AWS Toolkits for PyCharm, IntelliJ and VS Code were announced. Ruby was added to the list of supported languages for Lambda, bringing the number up to ten.

Two new programming models joined the mix to help eliminate code duplication. Lambda Layers and Nested Applications Using Serverless Application Repository, both available today, allow for partner contributions, and Step Functions Service integration lets customers or MSPs build workflows.

The upcoming Web Socket Support for API Gateways provides real-time, stateful, two-way communication between apps, ideal for the IoT applications touted on Day 1 of re:Invent, and Amazon Load Balancer for Lambda, available now, lets developers integrate Lambda functions in existing applications.

Customers complained that Apache Kafka was difficult to set up, manage and scale. The AWS response is Amazon Managed Streaming for Kafka, now in preview.

Finally, for customers in need of assistance in architecting workloads according to best practices, there are several options: Have AWS or a certified partner come in and review the workloads with the customer team (70 percent of partners who did these reviews ended up performing the work to bring the applications up to scratch, Vogels said), or do the review themselves. To assist in the latter case, Vogels announced the AWS Well Architected Toolkit, available now. The Toolkit not only evaluates workloads, it provides educational material including videos, resources and best practices to help customers create improvement plans.

Vogels also announced a video series, Now Go Build, with Dr. Werner Vogels (see below), in which he travels the world highlighting builders (AWS’ term for its developers) worldwide.