Avatar

Seung Hyun Lee

Ph.D. Student

University of Michigan


CV | Google Scholar | |


I am a Ph.D student wokring with professor Stella X. Yu at CSE, the University of Michigan.

Previously, I got master degree in the Department of Aritificial Intelligence at the Korea University. I interned at Naver Papago, Yonsei Severance CCIDS.

I also worked as Google Student Researcher (MTV, CA).

My research interests lie in the intersection field of computer vision, especially on the tasks of multi-modal generative models, artificial general intelligence, sound-guided visual syntheis, multimodal representation learning.

Contact

  • seungle [at] umich [dot] edu

  • easter3163 [at] korea [dot] ac [dot] kr

Education

  • Ph.D. in CSE, 2024~

    University of Michigan

  • MS in Artificial Intelligence, 2022~2024

    Korea University, Korea

  • BS in Computer Science, 2016~2022

    University of Seoul, Korea

News

  • Parrot has been accepted to ECCV 2024 Oral (< 2.5%). Congrats to all authors!
  • One paper has been accepted to Neural Networks 2024 (JCR 10%). Congrats to all authors!
  • Joined Stella's group at University of Michigan for my PhD Program.

Publications

    @ Google Research

  • Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generation

    Seung Hyun Lee, Yinxiao Li, Junjie Ke, Innfarn Yoo, Han Zhang, Jiahui Yu, Qifei Wang, Fei Deng, Glenn Entis, Junfeng He, Gang Li, Sangpil Kim, Irfan Essa, Feng Yang

    ECCV 2024 Oral (Acceptance rate < 2.5%) / Google Student Research Project

    [ paper ]

  • Cropper: Vision-Language Model for Image Cropping through In-Context Learning

    Seung Hyun Lee, Junjie Ke, Yinxiao Li, Junfeng He, Steven Hickson, Katie Datsenko, Sangpil Kim, Ming-Hsuan Yang, Irfan Essa, Feng Yang

    Preprint

    [ paper ]

    • @ Korea University

  • Sound-Guided Semantic Image Manipulation

    Seung Hyun Lee, Wonseok Roh, Wonmin Byeon, Sang Ho Yoon, Chanyoung Kim, Jinkyu Kim*, Sangpil Kim*

    CVPR 2022

    [ project / paper / code ]

  • Sound-Guided Semantic Video Generation

    Seung Hyun Lee, Gyeongrok Oh, Wonmin Byeon, Chanyoung Kim, Won Jeong Ryoo, Sang Ho Yoon, Jihyun Bae, Jinkyu Kim*, Sangpil Kim*.

    ECCV 2022

    [ paper / dataset ]

  • Robust Sound-Guided Image Manipulation

    Seung Hyun Lee*, Hyung-gun Chi*, Gyeongrok Oh, Wonmin Byeon, Sang Ho Yoon, Hyunje Park, Wonjun Cho, Jinkyu Kim*, Sangpil Kim*

    Neural Networks 2024

    [ paper ]

  • The Power of Sound (TPoS): Audio Reactive Video Generation with Stable Diffusion

    Yujin Jeong, Wonjeong Ryoo, Seung Hyun Lee, Dabin Seo, Wonmin Byeon, Jinkyu Kim

    ICCV 2023

    [ project ]

  • Functional Hand Type Prior for 3D Hand Pose Estimation and Action Recognition from Egocentric View Monocular Videos

    Wonseok Roh, Seung Hyun Lee, Wonjeong Ryoo, Gyeongrok Oh, Jakyung Lee, Soo Yeon Hwang, Hyung-gun Chi, Sangpil Kim

    BMVC 2023 Oral

  • Soundini: Sound-Guided Diffusion for Natural Video Editing

    Seung Hyun Lee, Sieun Kim, Innfarn Yoo, Feng Yang, Donghyeon Cho, Youngseo Kim, Huiwen Chang, Jinkyu Kim*, Sangpil Kim*

    Under review

    [ project / paper / code ]

  • LISA: Localized Image Stylization with Audio via Implicit Neural Representation

    Seung Hyun Lee, Chanyoung Kim, Wonmin Byeon, Sang Ho Yoon, Jinkyu Kim*, Sangpil Kim*

    Under review

    [ paper ]

Awards & Honors

  • Pytorch Open Source Contribution, Pytorch Tutorial Translation (lead menti) 2021
  • Google Developer Student Clubs – University of Seoul, Korea 2021
  • Software Maestro 11th – Best Software talent discovery program, Korea

Academic Activities

Teaching Assistant

  • Machine Learning at Korea University (Spring 2023)