Robot jazz musician, Shimon, to play Aspen Ideas Fest

Shimon plays a mean marimba. This much-buzzed-about bandleader knows just about everything about music theory and can play any style. With his jazz band, Shimon listens attentively, watches the guitarist and drummer’s body language for cues, and improvises with them in the moment.

Shimon also has four arms and no brain or heart. He’s a robot, a marvel of machine learning and artificial intelligence, invented by Georgia Tech Center for Music Technology founding director Gil Weinberg.

Weinberg is bringing Shimon to the Aspen Ideas Festival for a concert Friday night, where the groundbreaking robot — and perhaps, the future of music — will take the stage alongside human musicians.

“I’m looking to inspire,” Weinberg explained. “I’m looking for something that can play music that I’ve never heard before, that will surprise me. I’d go so far as to hope that it will do what good music does to us — shivers down our spine, make us cry, blow our minds.”

The one-hour set will showcase jazz compositions made collaboratively between Shimon and human musicians, along with some written solely by the robot. As an improviser, Shimon responds to elements like tempo, rhythm and pitch, along with visual cues (Shimon’s moving “head” has a camera programmed to recognize directions from bandmates or a conductor).

Shimon’s band at Ideas Fest will include drummer Jason Barnes, who is missing the lower part of his right arm. Weinberg used his technology to build Barnes an enhanced robot arm — it wields two drumsticks that respond to the muscles in Barnes’ bicep. It enabled Barnes to play faster and with more accuracy than an ordinary human drummer. They’ll also play Sunday’s Ideas Fest closing session, alongside talks by David Brooks, Thomas Friedman and Katie Couric.

Weinberg began working on robot musicians in 2004. His first version of Shimon came online in 2008. Through trial and error and meticulous adjustments on the tech inside Shimon, Weinberg has gradually improved Shimon’s skill. He’s listened closely to Shimon’s style. In early renditions, Shimon’s compositions didn’t sound like music and Weinberg struggled to understand why. Over the course of years, the inventor tweaked the robot’s cognitive abilities to the point where it could improvise with other musicians and come up with inspiring, exciting musical ideas.

At this point, the spark of Shimon’s creative inspirations are often as mysterious as the human imagination.

“It’s beautiful to me and I’m trying to understand how he came up with what he came up with — and I can’t,” Weinberg explained.

These days, Weinberg is working on adding more physical gestures to Shimon’s repertoire. The robot now might bob his head with the beat or sway to a melody. But Weinberg wants more. Going to see the Rolling Stones, he pointed out, is as much about Mick Jagger’s strutting as it is about the songs. Weinberg is trying to teach Shimon to move with more personality and to gesture with more precise emotional reactions to the music — to develop an onstage persona and panache.

Making good music, of course, requires more than great technical skill. One of the open questions for Shimon is, as Weinberg put it, “Will it ever have soul?” Writing an algorithm for soul is elusive, Weinberg said, but the ideas that Shimon brings to jam sessions are already inspiring soulful collaborations.

“The idea is that a robot brings mechanical abilities and processing power, and humans can contribute emotion and what we call ‘soul’ and together they can start something new,” Weinberg said.

The international music community has been fascinated by Weinberg’s progress with Shimon. Some feel threatened. But the inventor does not imagine that robots are going to replace human musicians any more than cameras replaced painters. Weinberg argues that photography actually improved and inspired new movements in painting that may not have happened without the invention.

“Without the camera, I don’t know if you would have had expressionism and surrealism. I don’t think painting would be the same without the camera,” he said. “Not only did it help with painting, but it created the whole new art form of photography.”

Rather than using automation and machine learning to replace musicians, Weinberg sees Shimon as the beginning of a new age for music and augmented human creativity.

“It’s all about the humans,” he said. “I don’t see a scenario where robots will replace humans.”

atravers@aspentimes.com