Professor Teams

Publications Projects Private Works

Private Works

Artist Statement

《Perception of War: AI Data Universe》 is a compelling data art that delves into the collective perception of the Ukraine conflict within the contemporary digital landscape. In today's society, social media has undergone a profound transformation, emerging as a vast and dynamic realm of data. This paradigm shift has reshaped human perception, as real-time messages disseminated through social media platforms have replaced traditional modes of information transmission. Focusing on this evolving societal framework, we employ advanced AI technology to analyze an extensive corpus of 60,000 diverse messages encompassing images, texts, and more, all directly related to the Ukraine conflict. Through this meticulous analysis, we aim to unveil the intricate web of perceptions embedded within the data.

Each perception, shaped by individual characteristics and influenced by socio-cultural factors, manifests itself through a unique array of visual elements such as color, pattern, and texture. By representing these perceptions in a visually captivating manner, our data-driven approach generates compelling and, at times, enigmatic images.

The resulting imagery offers more than aesthetic allure; it serves as a critical lens through which to examine the Ukraine conflict beyond the confines of raw data. By exploring the contextual backdrop of capital, power, and politics, we provide a thought-provoking perspective on the war. Furthermore, this artwork invites deep contemplation and reflection, encouraging a broader examination of the emerging collective vision and the construction of a new world image facilitated by our perception within the data-driven environment.

Perception of War: AI Data Universe

Siggraph Asia Art Gallery

Artist Statement

《Deep.PhotoPlay》 is an interactive VR artwork that uses deep learning-based cooperative AI and immersive VR as its main strategies. It reenacts a silent-film, a Korean cultural heritage, disseminates artistic values, and ultimately realizes remediation. The material for this artwork is the silent-film <There is No Poverty at the End of Labor> (12 minutes, estimated to have been produced in the 1920s) discovered by the Korean Film Archive (KOFA) at the Gosfilmofound of Russia in 2019. As the title suggests, it depicts the life and history of people escaping poverty through hard work. In particular, it is an important piece of cultural heritage that fills gaps in knowledge of early Korean film history. It is an action-film in which performances of piano and string instruments and narration are harmonized. Through cooperation with the KOFA, we discovered that the audience's immersion comes primarily from sounds in the film, which convey the scenes' atmosphere and expand the actors' emotions.

The artificial intelligence we designed functions as an actor that creates sounds. It creates Foley, Ambience, and FX sounds by being aware of the narrative through the film's short length, subtitle text, and character movement. We also performed noise and degradation enhancement on the original film to ensure accuracy. The generated sound is selected through the complementary intervention of human composers and film critic experts and combined as background sound. Collaboration between AI and humans adds depth to creation.

《Deep.PhotoPlay》 builds on the work above to provide audio-visual virtual immersion through a real-time engine. It creates a unique experience through interaction with the film's era, narrative, and sounds. The VR this artwork uses creates a personal yet shared world that blends the silent film era (the 1920s) with modern times. Participants enjoy moving and repeatedly interacting as avatars. This approach is valuable not only for creating a new awareness of cultural heritage but also for improving public access. Participants can also tour the restored Dansungsa Theater, a historic building in Korean film history. While a silent movie is shown in the theater, musicians and narrators replaced by virtual humans present an immersive performance based on the above creative sounds. This presentation has a high degree of realism because the artists' live performances were acquired and produced with motion capture technology. Silent films are still screened at film festivals as they used to be. However, this artwork is the first work of silent film creation using artificial intelligence and virtual technology.

Cinematic Screening VR

Unity Unite MWU TOP 24

Siggraph Asia Art Gallery