AI reinvents classic song 'Pearl of the East' with unimagined

Featuring lyrics "the meandering stream flows southward, running to Hong Kong to have a look," the song "Pearl of the East" has been chanted across generations for over three decades, bringing a sense of belonging to many Hong Kongers.

At a recent concert in the city, the song was once again played to celebrate the 25th anniversary of Hong Kong's return to the motherland. The performance was exceptional this time: it was sung by a choir of artificial intelligence (AI) musicians.

Accompanied by an orchestra from the Hong Kong Baptist University (HKBU), the AI choir consisting of 320 virtual singers followed the gestures of the conductor in presenting the classic, with AI-generated 3D animation as background.

"This should be the world's very first human-machine collaborative performance," said Guo Yike, vice president of HKBU and an AI expert.

The AI-powered performance was part of the latest progress of a research project that Guo led. The project aims to explore new opportunities that AI-based art technology could bring to Hong Kong and has landed government funding of over 52 million Hong Kong dollars (about 6.6 million U.S. dollars) for a period of five years.

For the AI singers, it was quite a learning process. At the very beginning, the virtual musicians had trouble singing high notes or even went out of tune from time to time.

"Sound is about frequency. It is not difficult to teach them to sing as long as the frequency is adjusted constantly," Guo said.

The bigger challenge is that the AI singers could be supposed to understand the gestures of the conductor. The communication was made possible by sensors tied to the waist, shoulders and wrists of the conductor, who repeatedly practiced with the AI singers to better the teamwork.

After almost six months of training, the AI singers were able to understand the language of the conductor well and present the music with a perfect tempo.

But what really amazed Guo and his team was the creation by AI itself. Accompanying the choral piece was the 3D visual narrative generated by the AI based on the lyrics and the music. The machine needs to first understand the meaning of the song before generating nearly 3,000 images and connecting them with natural transition.

For instance, when the lyrics say "flowing to 'fragrant river' to have a look," the machine needs to understand that "fragrant river" stands for Hong Kong and then create relevant images.

What the AI really achieved was beyond Guo and his team's imagination. When presenting the lyrics, the AI first drew a pearl, which then blossomed in layers to become the petals of bauhinia, an emblem of Hong Kong.

"We were all stunned at the moment. The concept of 'fragrant river' was presented in such an unexpected way. This was the creativity of the machine itself." Guo said.

For Guo, what AI can create goes far beyond mere imitation of human behavior. With the help of inputs from human beings, machines can create art that is distinct from human creation, he said.

"Perception is very important to scientific breakthroughs. We changed our perception, and here comes the breakthroughs," he said.

Guo said that his team would continue the study to empower art with technology. In the future, they could present new AI-powered art forms, such as opera.

"There is no end to exploration, and nothing is impossible," he said.