How Japanese rock star Miyavi performs in a world without live music

The only live music I’ve managed to see all year came from an unexpected place. I recently found myself on a Friday night after hours at teamLab Planets, a popular tourist spot in Tokyo, watching rock star Miyavi working on his latest “Miyavi Virtual” project. Miyavi Virtual 3.0 will be available to buy and stream later today — it’s a live performance mixing drone footage with dazzling digital art.

At one point Miyavi, a bouncy, enthusiastic character with blue-green hair and a wiry frame covered in black ink wash-style tattoos, came over to me for a distanced elbow bump and asked if I was feeling sleepy. To be honest, I kind of was — it turns out that recordings for glitzy live-streaming productions can involve a lot of waiting around well past midnight. 

But that’s just how it goes in 2020. As the coronavirus pandemic continues to rob creators of the ability to play in front of live audiences or even record music videos in traditional studios, Miyavi and his creative teams are resorting to technology — and unusual work hours — to keep performing in front of fans.

Born Takamasa Ishihara in Osaka in 1981, Miyavi is best known for his fast-paced, catchy electro-rock music and frenetic guitar playing. He started out in the visual kei band Dué le Quartz before embarking on a solo career that eventually began to encompass modeling and acting, including roles in Hollywood movies like Unbroken and Kong: Skull Island.

In other words, he tends to have a full calendar, and 2020 was supposed to have been no different. Miyavi normally bases himself in LA, and early this year was booked for some movies, TV projects, and a Gucci campaign. Then, his latest solo album Holy Nights was set for an April release with a tour to follow. But by the beginning of March, Miyavi found himself back in Tokyo, and it was clear that plans would have to change.

“For us creators, you know, we can’t do live performances now,” Miyavi told me earlier over Zoom. “So it’s really a crucial time to find a new way, a new normal.” 

To start with, Miyavi’s need to work in both Japan and the US complicates matters. “To be honest, we had lots of paths and lots of ideas, and we also had a lot of arguments because nobody knew what it was gonna be like,” he says. “Each country has a different situation. Usually I have my team in Japan in Tokyo, and a team in America, so we all talk. But the situations [with COVID-19] we’re in are all different. So we heard lots of ‘it’s not the time to shoot a music video.”’ We listened but we didn’t stop, because in Japan, even at that time, record stores were not closed. So we were planning to do a regular campaign and I was even shooting TV programs.”

But it soon became evident that plans to shoot two music videos in the US weren’t going to work out. “In America, the emergency declaration already happened, and as soon as we found out that Miyavi cannot fly out to the States, we’ve got to switch to more virtual creation,” Miyavi says. (He sometimes talks in the third person.) “That’s why we started making the music video for ‘Holy Nights’ with our animators, so that creates a world without having me.”

Miyavi dropped the anime-influenced “Holy Nights” video, developed by his US creative team, on YouTube on May 10th, proclaiming it to be the beginning of “Miyavi Virtual.” But an anime music video doesn’t capture a real performance. For that, a new technological approach would be required.

Director David Cihelna talked with Miyavi’s US-based creative directors Dyan Jong and Annie Stoll about using volumetric capture with the video for “Need for Speed,” the next single. This is a technique that employs several cameras at once to capture a 3D model that can be used in CGI renderings. It allows for “virtual” video that’s based on an actual performance — and of course, it’s easier to do safely than a traditional shoot right now.

“The only physical shooting we did was in a volumetric capture studio in Japan,” Cihelna tells The Verge. “We had a minimal team there, pretty much just Miyavi recording 3D models of his movements. The rest of the team was on Zoom, I directed him remotely.” 

“This technology is between VR, AR, and reality, so I was really fascinated,” Miyavi says. “Especially since it captures, like, my tattoos and my hairstyle! So it was pretty tough to choose a costume — I wasn’t allowed to wear any green stuff, it’s got to be really solid and tight because while shooting I was surrounded by a green background. It’s all imagination. It was a really interesting experience, and I felt the future. I think the key for the post-corona era is how real we can feel in a virtual world.”

“The capture studio has dozens of cameras pointed at him recording video that is then processed into frame-by-frame 3D models played back in the game engine,” Cihelna says. “That took half a day. The rest (99 percent) was all virtual and directly in Unity — camera movements, lighting, set design, VFX, color. I even used augmented reality to record camera shots in my living room.”

“The amount of data is huge,” Miyavi adds. “We could only use a seven minute clip — we shot more, but David and his staff had to go all through the data, and sending that data from Tokyo to LA is a big deal as well. I really appreciated it, all the staff and David and his team did a great job.”

The result, well, looks like a video game starring Miyavi. The resolution on his model is of course not as high as you’d get from a conventional camera or something rendered in 3D by hand, but the production leans into it with otherworldly glitches and particle effects.

“It was cool — I’m using superpowers when playing the guitar!” Miyavi laughs, adding that the effects were all added later based on his performance. “I know it’s really hard work for them, but it’s fun for me just jumping around. We all like manga and anime, Dragon Ball and Akira, we all have the same references so I think we’re all connected as humans without having a conversation.”

The next step for Miyavi was figuring out how he could actually perform live online. At first he turned to streaming on platforms like Twitch, YouTube, Weibo, and Line, setting up a studio in his home and performing a “Virtual Live” concert standing at his desk. Unsurprisingly, the results weren’t quite as slick as the “Need for Speed” video, although it’s still striking to see someone theatrically shred on their Telecaster right into a webcam in a setting that looks more suited to streaming Overwatch. 

“It’s quite challenging — setting up OBS, connecting the sound inside my PC, and I was doing the sound mix and lighting and everything,” Miyavi says. “So I just realized that I’m not a professional at this kind of thing. But I just didn’t want to stop, because there’s something we can do in any situation, and I didn’t want to give up — just [wanted to] find something, find a way to move forward. And I still think that was the right move even if it was not perfect, at least I was able to be connected with my fans and share time with them and with my creation.”

Miyavi then wanted to try combining high-tech experimentation with live broadcasting. “We just did the broadcasting from a studio in Tokyo with drones,” he says. “That was the biggest priority for me to try — I didn’t even know how long drones can fly, you know? And that’s the approach that I’m trying to do in the real world.”

Miyavi Virtual Live Level 2.0 was a performance with a live drummer on a socially distanced stage with no audience except for the fleet of drones buzzing around the band and capturing its performance. “You can do anything with volumetric capture, but it takes time for the production, and I wanted to do broadcasting using my real body. It’s really important to experiment,” Miyavi says.

I haven’t seen the finished version of Miyavi Virtual 3.0, but from what I saw it appears to be a similar production — just in a much more visually impressive setting. Much of the video capture is also by drone, and the mirrors and lights surrounding Miyavi make it look like he’s performing in the middle of a spectacular void. There’s a reason why teamLab Planets has been maybe the most ubiquitous “I finally went there” sight on Tokyo’s Instagram feeds since it opened a couple of years ago.

Unlike 2.0, however, Virtual 3.0 won’t be streamed live — it was pre-recorded, and you’ll need to buy a $30 ticket to watch it over the period of its availability. Outside Japan, you can start viewing from 6PM ET today. While the original plan was for a live performance, Miyavi says that the decision to switch to a pre-recording is “due to the intention of creating and prioritizing artwork that has high video quality, amazing light design etc.” (I’m not sure whether the shoots I was present for will actually make it into the final production, but you can at least take it from me that he was there and performing live.)

I wanted to know where Miyavi thinks he might go next with these virtual productions. Even before the pandemic, virtual concerts were getting increasingly popular, with Fortnite at the vanguard. Is this an area he’d like to explore?

“Yeah I’d love to!” he says. “Because now I’m really into Fortnite — I started playing Fortnite in 2018 while I was shooting the Maleficent sequel in London, because I had lots of time while waiting for my turn and I was playing with my daughters. Now I play with my daughters even before they go to school — we call it ‘Fort-morning,’ we wake them up and play for 30 or 45 minutes. As a musician, I feel the music Miyavi makes has a chemistry with those games. Even “Need for Speed” — [my wife] Melody used to be in the game! She was one of the characters in [EA’s] Need for Speed over 10 years ago. So Need for Speed, Fortnite, we believe that Miyavi’s music really really works with those high-energy concepts. But I’m always trying not to play too much!”

As far as the actual experience of Fortnite in-game concerts goes, however, Miyavi thinks there’s room for improvement. “That was cool as an advertisement, for sure,” he says of Travis Scott’s recent event. “There were lots of people — 10 or 20 million, right? So it’s huge numbers, and even before that Marshmello did it and Steve Aoki and other big players. It’s pretty cool, but I don’t know how much he performed for that performance. It seems like it was programmed, no? It was a creation, which is cool. But to me, it was not the chemistry between real and virtual worlds. It was 90 percent virtual, with the music made by human beings. So I didn’t feel real in that virtual world. Of course it’s great and I really respect it but I still kind of think there [can be] closer contact with people in a virtual world.”

Miyavi is careful to note that this experimentation is the direct result of a desperate period in human history. “I do not say that I am enjoying this time,” he says. “There are lots of crazy, sad things happening all over the world and people losing their jobs. Politics are turning really messed up, especially in the States. And then right after corona, Black Lives Matter [protests] started — sooner or later it was going to happen and it’s good to have it now so that we start facing the real deep issues and problems that America has been having, which actually other countries like Japan [also] have in some way.”

Miyavi believes that the global situation will prompt people to live and work in new ways even after things improve — himself included. “I’m a traveler — I travel a lot, and I love experiencing things with my eyes,” he says. “To feel that atmosphere is really really crucial. But at the same time, the world is going to be different, no matter what. Even if we get back to normal, I think that our lifestyle will be different, and the way we create will be different. Because now we know what we can do. This situation kind of pushed us forward. We had to find a way — what’s essential? What’s a core value? Even the meaning of music we make.”

“The skill I got in this life is to play the guitar with this physical body, so I wanna find a way to mix with virtual technologies. That’s my responsibility as a bridge to the next generation.”