Professor Teams

Publications Projects Private Works

Publications

ABSTRACT

With digital humans being used extensively in blockbuster movies, triple-A games, and the entertainment industry, the demand for digital humans is growing exponentially. In response, VFX studios are continuously expanding their research into 3D facial appearance digitization pipelines to generate realistic digital human facial appearances. However, existing 3D facial appearance digitization pipelines have limitations in terms of labor-intensive methods in R&D and system improvement through qualitative methods. Therefore, in this study, we propose a pipeline optimization of 3D face digitizing system through digital twins. The proposed pipeline predicts and solves problems in digitizing through extensive simulations in a virtual space that reflects the characteristics of the physical space with a digital twin and applies it to the physical space. It can solve the labor-intensive method, which is a limitation of the existing pipeline, through the interaction of physical and virtual space. In addition, by improving the system through quantitative evaluation of the facial appearance generated in the virtual space, the system can be optimized to generate better digital human facial appearance compared to the existing pipeline. The proposed digital twin pipeline can generate realistic facial appearance based on quantitative evaluation, and can be expected to reduce the cost of system construction. We believe that this research will contribute to the introduction and establishment of digital twin pipelines in the VFX industry.

3D 얼굴 외형 디지타이징 시스템 최적화를 위한 디지털 트윈 파이프라인 연구

한국방송미디어공학회

ABSTRACT

In this study, we proposed a production pipeline to effectively create virtual spaces for use in a highly immersive metaverse using 3D space capture technology. Previous research on virtual space production has focused primarily on the development of core technologies and the use of hardware. We aimed to combine and optimize the latest engineering technologies to make the production process commercially viable, and not merely focused on advancing them. We introduced automation using 3D reconstruction and cross-polarization photography technology based on existing asset creation and scene rendering techniques. The proposed method considers cost-effectiveness, that is, reducing the production time and costs, as well as productivity and software availability resulting from the automation of some of the work processes. In the future, the proposed pipeline will contribute to the infrastructure and technological development of the next-generation metaverse by enhancing the stability of virtual space production and supplementing technological capabilities.

3D 공간캡처 기술 기반의 메타버스 제작 파이프라인 연구

한국디지털콘텐츠학회

ABSTRACT

Immersive content overcomes spatial limitations through convergence with extended reality, artificial intelligence, andphotogrammetry technology along with interest due to the COVID-19 pandemic, presenting a new paradigm in the content marketsuch as entertainment, media, performances, and exhibitions. However, it can be seen that in order for realistic content to havesustained public interest, it is necessary to study storytelling method that can increase immersion in content rather thantechnological freshness. Therefore, in this study, we propose a immersive content storytelling method using artificial intelligenceand photogrammetry technology. The proposed storytelling method is to create a content story through interaction betweeninteractive virtual beings and participants. In this way, participation can increase content immersion. This study is expected to helpcontent creators in the accelerating immersive content market with a storytelling methodology through virtual existence that utilizesartificial intelligence technology proposed to content creators to help in efficient content creation. In addition, I think that it willcontribute to the establishment of a immersive content production pipeline using artificial intelligence and photogrammetrytechnology in content production.

포토그래메트리 및 인공지능 기술을 활용한 실감 콘텐츠 제작과 스토리텔링 방법 연구

한국방송미디어공학회

ABSTRACT

In today's media environment, there is an increasing demand for a method to technically analyze moving image. In the field of film, which consists of vast amounts of data, an efficient method and structure for analyzing moving image information is indispensable. This study presented short segmentation and film analysis methods and results, focusing on sequences through Barycenter algorithms and machine learning methodologies, in Dziga Vertov's "Man with a Movie Camera," a director representing the silent film era. The existing proposed machine learning methodology simultaneously utilizes the Barycenter algorithm method in short segmentation due to its limited accuracy, although the time required for short segmentation is small. The Barycenter algorithm is a method of finding the center of gravity in the image by brightness and saturation, and has high short segmentation accuracy compared to the machine learning methodology through center movement. This proved that the method of using the Barycenter algorithm presented in this study has higher accuracy than the existing method. This study aims to contribute as a criticism tool based on quantitative indicators of film as well as to improve understanding and characterization of film.

무게 중심을 활용한 무성 영화 장면 분기 검출 - 지가 베르토프의 <카메라를 든 사나이>를 중심으로

한국영상학회

ABSTRACT

After the first to present projected moving pictures to audiences, the film industry has been reshaping along with technological advancements. Through the full-scale introduction of visual effects-oriented post-production and digital technologies in the film-making process, the film industry has not only undergone significant changes in the production, but is also embracing the cutting edge technologies broadly and expanding the scope of industry. Not long after the change to digital cinema, the concept of artificial intelligence, first known at the Dartmouth summer research project in 1956, before the digitalization of film, is expected to bring about a big transformation in the film industry once again. Large volume of clear digital data from digital film-making makes easy to apply recent artificial intelligence technologies represented by machine learning and deep learning. The use of artificial intelligence techniques is prominent around major visual effects studios due to automate many laborious, time-consuming tasks currently performed by artists. This study aims to predict how artificial intelligence technology will change the film industry in the future through analysis of visual effects production cases using artificial intelligence technology as a production tool and to discuss the industrial potential of artificial intelligence as visual effects technology.

인공지능을 이용한 영화제작 : 시각효과를 중심으로

한국엔터테인먼트학회

ABSTRACT

The virtual reality industry has an opportunity to take another leap forward with the surge in demand for non-face-to-face content and interest in the metaverse after Covid-19. Therefore, in order to popularize virtual reality content along with this trend, high-quality content production and storytelling research suitable for the characteristics of virtual reality should be continuously conducted. In order for content to which virtual reality characteristics are applied to be effectively produced through user feedback, a quantitative index that can evaluate the content is needed. In this study, the process of viewing virtual reality contents was analyzed and head movement was set as a quantitative indicator. Afterwards, the experimenter watched five animations and analyzed the correlation between recorded head movement information and immersion. As a result of the analysis, high immersion was shown when the head movement speed was relatively slow, and it was found that the head movement speed can be used significantly as an index indicating the degree of content immersion. The result derived in this way can be used as a quantitative indicator that can verify the validity of the storytelling method applied after the prototype is produced when the creator creates virtual reality content. This method can improve the quality of content by quickly identifying the problems of the proposed storytelling method and suggesting a better method. This study aims to contribute to the production of high-quality virtual reality content and the popularization of virtual reality content as a basic research to analyze immersion based on the quantitative indicator of head movement speed.

헤드 무브먼트 데이터와 가상현실 콘텐츠 몰입도 상관관계

한국방송미디어공학회

ABSTRACT

This study starts with the possibility of a quantitative analysis of artworks through the convergent of art history and technology. The purpose of this study is to computationally analyze the distribution of brightness and saturation of Monet's series through the Barycenter Pattern Analysis and to visualize the data. The results are as follows: In the three series, the positions of center point and the distribution of Barycenter Pattern of brightness and saturation differ depending upon the production time and season of the artworks (i.e. the change of light). In the case of ‘Wheatstacks,’ the distributions of brightness are affected by the arrangement of the subject, and the influences of light and weather on the distribution of the central point of brightness and saturation are presented in a certain pattern in the series of ‘Rouen Cathedral.’ In particular, in the case of ‘Water Liles’ series, several irregular patterns are revealed in the distribution of the Barycenter of brightness and saturation due to the the worsening of cataract problems. For example, the distribution of brightness and saturation in the artworks which were produced before the diagnosis of cataract reveals a certain pattern. On the other hands, the distribution of brightness or saturation in the artworks which were produced after the diagnosis of cataract becomes more irregular. The significance of this study is to prove the possibility of discovering the painter's style change or the painter's personal influence, which cannot be discovered by a qualitative method, through the Barycenter Pattern Analysis. As various technological approaches are converged with art research methodologies, we expect that the applications of quantitative methodologies can be expanded to illuminate the relevance between works of art and to identify art forgeries.

융합적 관점에서 접근한 클로드 모네 작품의 하위 무게 중심 유형 분석 - 건초더미 연작, 루앙대성당 연작, 수련 연작을 중심으로 -

한국전시산업융합연구원

ABSTRACT

This paper presents the remediation of a nineteenth century Korean manuscript (the 1809 Uigwe ) as an interactive digital experience using creative algorithms and gesture recognition developed by LG Electronics. The interactive media project we created using sound visualization invites visitors to jointly experience Joseon ’s royal music and the manuscript’s pictorial illustrations. We present the media development process and the results of a survey assessing users’ engagement. Our findings show that technological remediation can be a fruitful way to elicit interest for cultural heritage. The Uigwe media project enhanced the participants’ curiosity for the 1809 Uigwe , and the added sound of the traditional court music (Yeominlak ) and the images of the musical instruments increased users’ understanding of the 1809 Uigwe and of royal court music. Overall, the study provides evidence about the use of multisensory digital media for fostering learning about and cultural appreciation of historical artefacts.

Remediating Tradition with Technology: A Case Study of From Tangible to Intangible: A Media Showcase of Kisa Chinp’yori Chinch’an Uigwe

Digital Creativity

ABSTRACT

Along with the development of computer graphics, real-time rendering, motion capture, and artificial intelligence technology, virtual being that enables two-way communication has emerged in the content industry. Although the commercialization of technologies and platforms is creating a two-way communication virtual being, there is a lack of analysis of what characteristics this virtual being has and how it can be used in each field. Therefore, through technical background survey and case study for the production of virtual being, the two-way communication virtual being is analyzed on the characteristics necessary for emotional exchange. The characteristics needed for emotional exchange were divided into interaction, individuality, and autonomy, and this characteristic is classified as the focus and how two-way communication virtual being will be used in the content field. This study is expected to provide significant implications for the research of content production and utilization using virtual being as a basic study of virtual being, which analyzes the technical background and characteristics for two-way communication required for virtual being production.

콘텐츠 산업에서 나타난 양방향 소통 가상존재 기술 및 특성 분석

한국방송미디어공학회

ABSTRACT

As hardware and software advancements in VR field increase, demand for VR negative contents is gradually increasing, and demands for highly finished contents. However, VR platforms have different characteristics from ordinary video platforms, which limits the application of existing video grammar and story boards that were previously used for producing images. In this study, we developed storyboard production tools, including painting and editing functions with VR platform characteristics, by releasing improvement directions through case studies on storyboard production tools that were previously used, and expanding them from 2D to 360° space. Storyboard production tools that are tailored to VR platform characteristics allow for detailed production during preproduction phase. Therefore, in this study, we expect that this writing tool will help produce highly mature content as it shares the process as a communication tool when producing animatics.

가상현실 플랫폼에서 프리프로덕션을 위한 스토리보드 연구

한국디지털콘텐츠학회

ABSTRACT

In this study, we develop a non-photorealistic rendering expression technique that can be used in a real-time rendering environment based on the out-of-register concept, which is a type of accidental printing error. First, to develop the expression technique, out-of-register causes and cases were analyzed. We considered why the out-of-register concept was developed as a graphic element. Second, each characteristic was analyzed by examining cases of non-photo-realistic rendering expressions. The implementation method was realized through a linear algorithm without the use of machine learning, which is one of many methods. It is implemented through a linear algorithm for constant and predictable result values. Furthermore, we utilized algorithms and interfaces for effective expressions. We constructed and applied simple 3D cube shapes, characters, and virtual environments in a real-time rendering environment using the implemented expression method. Given that various forms of media are being produced in a real-time environment through expression methods, it is expected that this paper will help with the expression of the essential parts of things and applications of expression techniques when creating content.

실시간 렌더링 환경에서의 Out of Register 표현법 연구

한국영상학회

ABSTRACT

In the case of games and animations in the entertainment industry, the importance of directing has been emphasized because the sense of immersion of audience in storytelling depends on the directing methods. Due to the nature of VR contents, however, the directing methods used in existing animations and games are bound to be used in a limited way. In this study, ‘DreamTherapy’, a VR animation, was produced by combining gaming and interaction elements to enhance immersion and efficient storytelling of VR contents. To sum up, these elements successfully convey the sense of accomplishment and immersion to the audience, but also allow storytelling to be faithful to the director’s intention. From practical and academic perspective, we hope that this study provides significant implications for immersion of storytelling methods of animation used in VR space.

VR 애니메이션 연출법 연구 - VR 애니메이션 <DreamTherapy> 을 중심으로

한국디지털콘텐츠학회

ABSTRACT

The virtual reality (VR) cartoon is a format of VR contents that leverage the characteristics of webtoons that provides the simple story line and graphical storytelling tools to strategically surmount limitations of VR contents design. The VR cartoon enables people to experience the imaginary three-dimensional space in the webtoon as a real space by the transformation of webtoon contents through VR prototyping. The VR cartoon successfully presents focused environment where people can readily pay attention to the contents without notable motion sickness. People have been familiar with the storytelling strategy in the context of published cartoons and webtoons, likely we’ve understood the narrative of a movie through the continuous scenes projected in the screen. Indeed, it has been recognized as a popular toolset of communication, where visual images are sequentially delivered by replacing multiple planar spaces to tell a story narrative. While there are discrete panels with the time and space resolution in the graphical cartoons, people can distill a commit closure based on their past experiences. This is a typical "grammar" of the cartoon, which can be extrapolated to the VR cartoon that provides a seminal storytelling strategy. In this article, we review how the storytelling strategy in webtoons has been transformed into that in VR cartoons, and analyze the key components of VR cartoons. We envision that our research can potentially create keystones to produce variety of new VR contents by reflecting various narrative media including cartoon as a "sequential art".

VR 카툰의 스토리텔링 연구

한국방송미디어공학회