‘It’s fast beyond human capabilities’: How Sophy, the artificial intelligence that will change driving games, works

Anyone who has played a driving game for an extended period of time feels at one point or another that the console was cheating in some way to win. And it certainly is Known as the so-called elastic effect: When they are in front, the computer controlled cars stretch the gap until it looks out of reach, but when they are in the back they stick to the player and are difficult to get rid of. Just as if there was a rubber band regulating its movements.

It’s a gimmick that programmers often use to keep up with the feeling of being challenged in virtual races, but we’re not talking about that here. We are talking about skill here Incredible, beyond human limits“.

“To beat her you must never be distracted.”

words from Valerius GalloUndoubtedly the best Italian player in the world Gran Turismo It’s likely Among the best sim drivers in the worldand they’re referring to GT Sophy, the artificial intelligence Sony AI and Polyphony Digital is developing for the hugely popular PlayStation video game.

Make it accessible to the public on a limited number of circuits inside Gran Turismo 7 between February and MarchSophie owes its name to the Greek word Sophia, which means wisdom: try to be an AI that is wiser than other AIs seen so far in video games, and that (simplistically) knows what the optimal travel speeds are in various different sectors. Tracks and aims to keep them as long as possible by following the exact tracks.

Sophie does not“I’ve seen them take paths unimaginable to anyone,” Gallo told us again, with whom we met in Amsterdam during the movie show Gran Turismo (video). Unthinkable twice, meaning “I didn’t think it was possible” and also “I never thought I’d try it.” Precisely because a human pilot wouldn’t think they were practical. Again: “On the GT7 circuits that I know by heart and where I think I drive very well, he has all his braking points farther forward than mine”, because he clearly learned that he can delay braking by a few meters to enter the corners more quickly. with a smile, Gallo tells us Sophie is ‘illegally fast’, an exaggeration to indicate that we are on the verge of what has hitherto been understood as fraud. But it’s not that Sophie cheats, she knows: “She’s become very good, she adapts very well to situations, even to the usual struggle in the first laps of a race, It’s aggressive rightAlmost on a human level.” This is an important detail, because (again) anyone with a bit of driving game experience knows that, no matter how good they are, computer opponents are rarely as aggressive and cunning as people. They rarely drive dirty cars, as they say.

However, Sophy’s true strength is not (only) this, but a common characteristic of almost all AI systems: it’s performance stability. They are machines, artificial minds, and computers They never get tired, they don’t get distracted, they don’t feel pressured And they always perform at their best: “To overcome it, you must always be at the limit and never be distracted – Gallo told us – or you will fail.”

youtube: Valerio Gallo vs. GT Sophy

How GT Sophy was trained

Sophie that too magazine Nature devoted a rich and interesting study (this)it is trained in what is called reinforcement learning or reinforcement learning: in English it is called reinforcement learning, which is a form of machine learning and It is basically about learning from your mistakesSo we don’t need them again.

It’s a process that has gone on for Sophy Two years, even if the actual training part is much shorter: about twenty months are those needed to create the training program, determining how and on what data and information to make it learn, but two weeks. on 45,000 hours of continuous testingAnd two days after taking to the track for the first time, Sophie was already faster than 95% of the human pilots taken as a reference. arrive to Super powers Also note Gallo, particularly in 3 circles of Gran Turismo 7: For those who know the game, they are the ones at Lake Maggiore, the Dragon Trail and the Circuit de la Sarthe (where the 24 Hours of Le Mans race takes place in the real world).

In detail, the training program started in April 2020, with more than 10,000 hours of single player gameplayand came to life in the second half of 2021: between July and October of that year, Sophie was challenged to a series of 3 recurring matches by 4 of the best players in the world. Gran Turismo Sport. Each had a different circuit and car combination to use, and points were awarded at the end based on how well the different cars were, Some of them are run by Sophie and others are run by players: During the first race, the human team reached 86, while the AI ​​stopped at 70. Since the second race, the tips have reversed: 104 points for the Sophy team and only 52 for the players.

From there it was a crescendo, fueled by it as well innate advantages Which Sophie, being an artificial intelligence, compares it to a human: it knows the track map perfectly, with the exact coordinates of the borders and the boundaries that must not be crossed, and has in real time all the “exact information about the load of each tire and so on.” Seal limits for each rubberas well as other data on the condition of the vehicle you are driving, ”as the developers themselves admit.

He is actually looking for it To reduce it, slow it down and somehow make it more human. In other words, less legitimate. They do this by working on two parameters in particular: the refresh rate of its actions (initially blocked at 10 Hz, against a theoretical maximum of 60) and above all reaction times. According to the results, Sophie is able to react to what is happening around her Measured velocity between 23 and 30 ms, versus 200-250 ms for a human pilot. For a good human pilot, of course. To make it more manageable, its creators added an artificial delay, locking it between 100 and 250ms. Which did not prevent her from hitting everyone who more or less encountered her, as well as from turning this situation in her favor.

Twitter: Sophie learned to drift

Zero shot, the things Sophie taught herself

In Amsterdam, Sony explained to us that “while we were trying to make it slower and less perfect” something strange happened. It just so happened that Sophie learned to drift. And of course you do it better now than anyone has done it before.

Drifting is a type of controlled skidding, and knowing how to master it is essential to managing and for any excessive skidding Increase speed outside the corner. In addition to putting on a show, as can be seen in the video above, it is being shown for the first time on stage at the Amsterdam Theatre.

There is no scientific explanation here for how Sophie learns this skill: like other AI systems, she exhibits so-called emergent behaviours, i.e. skills the developers didn’t think she had and on top of that. They won’t teach her (it’s called Learning from scratch). What is known is that even here, again, he’s doing things that humans didn’t think they could do, didn’t think useful to do and probably wouldn’t be able to do in the same way: “He learned that passing suddenly D a R (from the front To the rear, ed.) manages to stabilize the car while skidding, and manages to do so at a speed unimaginable to us.” An important point to understand is that whatever he does Possible and “depends on the physics of the game”: We could do that too, if only we could.

It is still not clear When the GT Sophy finally becomes available to everyone players Gran Turismo 7, after tests at the beginning of the year: What is certain is that it “does things that would have been unimaginable 5-6 years ago for an AI of this type”, Sony explained. Or even if you tell us To rework the Nexus 6’s Blade Runner“You humans have done things like that You don’t think it can be done. “

@employee