Eureka: With GPT-4 overseeing training, robots can learn much faster

GPU-based physics simulator speeds up reality by “1,000x” while GPT-4 calls the shots.

In this still captured from a video provided by Nvidia, a simulated robot hand learns pen tricks, trained by Eureka, using simultaneous trials.

Enlarge / In this still captured from a video provided by Nvidia, a simulated robot hand learns pen tricks, trained by Eureka, using simultaneous trials. (credit: Nvidia)

On Friday, researchers from Nvidia, UPenn, Caltech, and the University of Texas at Austin announced Eureka, an algorithm that uses OpenAI's GPT-4 language model for designing training goals (called "reward functions") to enhance robot dexterity. The work aims to bridge the gap between high-level reasoning and low-level motor control, allowing robots to learn complex tasks rapidly using massively parallel simulations that run through trials simultaneously. According to the team, Eureka outperforms human-written reward functions by a substantial margin.

Before robots can interact with the real world successfully, they need to learn how to move their robot bodies to achieve goals—like picking up objects or moving. Instead of making a physical robot try and fail one task at a time to learn in a lab, researchers at Nvidia have been experimenting with using video game-like computer worlds (thanks to platforms called Isaac Sim and Isaac Gym) that simulate three-dimensional physics. These allow for massively parallel training sessions to take place in many virtual worlds at once, dramatically speeding up training time.

"Leveraging state-of-the-art GPU-accelerated simulation in Nvidia Isaac Gym," writes Nvidia on its demonstration page, "Eureka is able to quickly evaluate the quality of a large batch of reward candidates, enabling scalable search in the reward function space." They call it "rapid reward evaluation via massively parallel reinforcement learning."

Read 6 remaining paragraphs | Comments

The Daily Telescope: A look at a young star cluster in a nearby galaxy

In today’s image, we get an infrared view of NGC 346.

A new infrared image of NGC 346 from the Mid-Infrared Instrument on NASA’s James Webb Space Telescope.

Enlarge / A new infrared image of NGC 346 from the Mid-Infrared Instrument on NASA’s James Webb Space Telescope. (credit: NASA, ESA, CSA, STScI, Nolan Habel (NASA-JPL))

Welcome to the Daily Telescope. There is a little too much darkness in this world and not enough light; a little too much pseudoscience and not enough science. We'll let other publications offer you a daily horoscope. At Ars Technica, we're going to take a different route, finding inspiration from very real images of a universe that is filled with stars and wonder.

Good morning. It is October 23, and today's image features a new view of a star cluster within the Small Magellanic Cloud, one of the nearest galaxies to Earth. This galaxy has an estimated 3 billion stars, which sounds like a lot. However it is tiny compared to the nearest galaxy that is of a similar size to our own Milky Way. That would be the Andromeda Galaxy, which has an estimated 1 trillion stars. That's ... a lot.

Anyway, one of the neatest features in the Small Magellanic Cloud is a particularly bright cluster of stars known as NGC 346, discovered about 200 years ago by a Scottish astronomer. Some of these stars may be as young as 2 million years old.

Read 3 remaining paragraphs | Comments

The Daily Telescope: A look at a young star cluster in a nearby galaxy

In today’s image, we get an infrared view of NGC 346.

A new infrared image of NGC 346 from the Mid-Infrared Instrument on NASA’s James Webb Space Telescope.

Enlarge / A new infrared image of NGC 346 from the Mid-Infrared Instrument on NASA’s James Webb Space Telescope. (credit: NASA, ESA, CSA, STScI, Nolan Habel (NASA-JPL))

Welcome to the Daily Telescope. There is a little too much darkness in this world and not enough light; a little too much pseudoscience and not enough science. We'll let other publications offer you a daily horoscope. At Ars Technica, we're going to take a different route, finding inspiration from very real images of a universe that is filled with stars and wonder.

Good morning. It is October 23, and today's image features a new view of a star cluster within the Small Magellanic Cloud, one of the nearest galaxies to Earth. This galaxy has an estimated 3 billion stars, which sounds like a lot. However it is tiny compared to the nearest galaxy that is of a similar size to our own Milky Way. That would be the Andromeda Galaxy, which has an estimated 1 trillion stars. That's ... a lot.

Anyway, one of the neatest features in the Small Magellanic Cloud is a particularly bright cluster of stars known as NGC 346, discovered about 200 years ago by a Scottish astronomer. Some of these stars may be as young as 2 million years old.

Read 3 remaining paragraphs | Comments