Kwang Moo Yi

Assistant Professor

Research Interests

3D Computer Vision
Computer Vision
Machine Learning
Astronomy Applications of Computer VIsion

Relevant Thesis-Based Degree Programs

Affiliations to Research Centres, Institutes & Clusters

Research Options

I am available and interested in collaborations (e.g. clusters, grants).
I am interested in and conduct interdisciplinary research.
I am interested in working with undergraduate students on research projects.
 
 

Biography

Kwang Moo Yi is an assistant professor in the Department of Computer Science at the University of British Columbia (UBC), and a member of the Computer Vision Lab, CAIDA, and ICICS at UBC. Before, he was at the University of Victoria as an assistant professor, where he is currently an adjunct professor. Prior to being a professor, he worked as a post-doctoral researcher at the Computer Vision Lab in École Polytechnique Fédérale de Lausanne (EPFL, Switzerland), working with Prof. Pascal Fua and Prof. Vincent Lepetit. He received his Ph.D. from Seoul National University under the supervision of Prof. Jin Young Choi. He also received his B.Sc. from the same University. He serves as area chair for top Computer Vision conferences (CVPR, ICCV, and ECCV), as well as AAAI. He is part of the organizing committee for CVPR 2023.

Recruitment

Master's students
Doctoral students
Postdoctoral Fellows
2025

Application of Machine Learning methods and Generative Models to 3D Computer Vision

Typically, successful applicants with MSc degrees have prior exposure to 3D Computer Vision and/or Deep Learning, evident from publications at Computer Vision / Graphics conferences (CVPR,ECCV,ICCV,NeurIPS,SIGGRAPH,WACV,BMVC,ICIP). For students directly applying to graduate school with BSc degrees, having a publication record is a plus, and prior exposure to research environments or evidence of research projects is suggested.

 

Note:  For graduate student positions, it is essential that you meet the department deadline, which is December 15th. You will only then be considered as a potential candidate. Also, contacting me in advance will not likely make any difference, as long as you list me as a potential supervisor. Please see the department website before anything if you intend to apply for graduate school.

I support public scholarship, e.g. through the Public Scholars Initiative, and am available to supervise students and Postdocs interested in collaborating with external partners as part of their research.
I support experiential learning experiences, such as internships and work placements, for my graduate students and Postdocs.
I am open to hosting Visiting International Research Students (non-degree, up to 12 months).

Complete these steps before you reach out to a faculty member!

Check requirements
  • Familiarize yourself with program requirements. You want to learn as much as possible from the information available to you before you reach out to a faculty member. Be sure to visit the graduate degree program listing and program-specific websites.
  • Check whether the program requires you to seek commitment from a supervisor prior to submitting an application. For some programs this is an essential step while others match successful applicants with faculty members within the first year of study. This is either indicated in the program profile under "Admission Information & Requirements" - "Prepare Application" - "Supervision" or on the program website.
Focus your search
  • Identify specific faculty members who are conducting research in your specific area of interest.
  • Establish that your research interests align with the faculty member’s research interests.
    • Read up on the faculty members in the program and the research being conducted in the department.
    • Familiarize yourself with their work, read their recent publications and past theses/dissertations that they supervised. Be certain that their research is indeed what you are hoping to study.
Make a good impression
  • Compose an error-free and grammatically correct email addressed to your specifically targeted faculty member, and remember to use their correct titles.
    • Do not send non-specific, mass emails to everyone in the department hoping for a match.
    • Address the faculty members by name. Your contact should be genuine rather than generic.
  • Include a brief outline of your academic background, why you are interested in working with the faculty member, and what experience you could bring to the department. The supervision enquiry form guides you with targeted questions. Ensure to craft compelling answers to these questions.
  • Highlight your achievements and why you are a top student. Faculty members receive dozens of requests from prospective students and you may have less than 30 seconds to pique someone’s interest.
  • Demonstrate that you are familiar with their research:
    • Convey the specific ways you are a good fit for the program.
    • Convey the specific ways the program/lab/faculty member is a good fit for the research you are interested in/already conducting.
  • Be enthusiastic, but don’t overdo it.
Attend an information session

G+PS regularly provides virtual sessions that focus on admission requirements and procedures and tips how to improve your application.

 

ADVICE AND INSIGHTS FROM UBC FACULTY ON REACHING OUT TO SUPERVISORS

These videos contain some general advice from faculty across UBC on finding and reaching out to a potential thesis supervisor.

Graduate Student Supervision

Doctoral Student Supervision

Dissertations completed in 2010 or later are listed below. Please note that there is a 6-12 month delay to add the latest dissertations.

Modularizing deep learning for geometry-aware registration and reconstruction (2023)

In this work, we explore the modularization of deep learning for geometry-aware registration and reconstruction, with a particular focus on cameras registration and human reconstruction from videos. The traditional methods for these tasks have been challenged by deep learning approaches, but end-to-end learning can be limited in terms of generalization, transparency, and controllability. Modularization breaks the task into smaller subtasks and allows each to be addressed individually using traditional methods or deep learning techniques. Through modularization, we are able to embed knowledge from the real world, enabling better generalization, simpler and more effective learning, explainable and transparent models, and geometry-awareness.Specifically, this work consists of four major chapters, each presenting a modularized approach to solve a specific geometric problem. Firstly, a novel linearized multi-sampling method is proposed to enable better image alignment and learning. Secondly, the homography warping is modularized out of the pipeline allowing optimization through the learned error for accurate sports field registration. Thirdly, by modularizing the robust estimation and 3D map from the pose estimation pipeline, the neural network can focus on learning accurate image correspondences. Finally, the modularization of human scene positioning and mesh skinning allows for the reconstruction of animatable human avatar from video.Overall, our work demonstrates the power of modularization, and we hope it will inspire future research on modularization and its potential applications to other areas.

View record

Master's Student Supervision

Theses completed in 2010 or later are listed below. Please note that there is a 6-12 month delay to add the latest theses.

Neural fourier filter bank (2023)

We present a novel method to provide efficient and highly detailed reconstructions. Inspired by wavelets, we learn a neural field that decompose the signal both spatially and frequency-wise. We follow the recent grid-based paradigm for spatial decomposition, but unlike existing work, encourage specific frequencies to be stored in each grid via Fourier features encodings. We then apply a multi-layer perceptron with sine activations, taking these Fourier encoded features in at appropriate layers so that higher-frequency components are accumulated on top of lower-frequency components sequentially, which we sum up to form the final output. We demonstrate that our method outperforms the state of the art regarding model compactness and convergence speed on multiple tasks: 2D image fitting, 3D shape reconstruction, and neural radiance fields. Our code is available at https://github.com/ubc-vision/NFFB.

View record

Bootstrapping human optical flow and pose (2022)

In this work, we propose a bootstrapping framework to enhance human optical flow and 3D human pose. We show that, for videos involving humans in scenes, we can improve both the optical flow and the pose estimation quality of humans by considering the two tasks at the same time. Generic optical flow methods perform better on humans when fine-tuned on human-centric scenes showing that the focus should be on humans when the task is human oriented. On the other hand, an overlooked assumption in recent 3D human pose estimation methods is temporal consistency. As such, we make use of existing human pose estimators and optical flow networks and improve their performance by benefitting from each other. In more detail, we optimize the pose and optical flow networks to, at inference time, agree with each other. We show that this results in state-of-the-art performance on the Human 3.6M and 3D Poses in the Wild datasets, as well as a human-related subset of the Sintel dataset, both in terms of pose estimation accuracy and the optical flow accuracy at human joint locations.

View record

Human pose and stride length estimation (2021)

In this thesis, we develop Computer Vision methods for Human body pose and stride length estimation. We first describe a framework for estimating the stride length of a walking subject from video using a multi-view camera setup. We specifically look into its utility in diagnosing Parkinson's disease. We do this using per frame 3D pose estimates and using an analysis of foot movement, we determine the length of the stride. Parkinson's diagnosis partly relies on stride length information; we claim that our method can be helpful in diagnosis. The current practice in the medical field is to estimate stride with complicated and fundamentally flawed sensors as they tend to affect the gait of the subjects using them. A benefit of our method is that cameras are relatively cheap, easily obtainable, and only need to be set up once. We also describe work done in improving the state of the art in human pose estimation. We first propose a pose refinement method that enhances state-of-the-art methods. Through analysis of our refiner, we show a flaw inherent in the human body model---the inaccuracy in the typical shape-to-pose regressor (joint regressor)---for a standard human pose dataset and show that the results on the top methods are actually being underreported. This flaw results in a situation where the ground truth joints are unsatisfiable with biologically plausible poses. We then address this flaw by modifying a part of the human body model. We reevaluate top state-of-the-art methods and show these models perform better with this modification without retraining.

View record

Publications

  • Layered Controllable Video Generation (2022)
  • Repurposing existing deep networks for caption and aesthetic-guided image cropping (2022)
    Pattern Recognition,
  • Eigendecomposition-Free Training of Deep Networks for Linear Least-Square Problems (2021)
    IEEE Transactions on Pattern Analysis and Machine Intelligence,
  • Eigendecomposition-Free Training of Deep Networks with Zero Eigenvalue-Based Losses (2018)
  • Learning to Find Good Correspondences (2018)
    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, , 2666--2674
  • Learning Lightprobes for Mixed Reality Illumination (2017)
    International Symposium on Mixed and Augmented Reality (ISMAR), (EPFL-)
  • Robust 3D Object Tracking from Monocular Images using Stable Parts (2017)
    IEEE Transactions on Pattern Analysis and Machine Intelligence,
  • Learning to Assign Orientations to Feature Points (2016)
    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, , 107--116
  • LIFT: Learned Invariant Feature Transform (2016)
    European Conference on Computer Vision,
  • Traffic Pattern Analysis and Anomaly Detection via Probabilistic Inference Model (2016)
    Theory and Applications of Smart Cameras, , 215--240
  • A Novel Representation of Parts for Accurate 3D Object Detection and Tracking in Monocular Images (2015)
    IEEE International Conference on Computer Vision (ICCV), (EPFL-)
  • Authoring and Living Next-Generation Location-Based Experiences (2015)
    proceedings of IEEE Virtual Reality 2015,
  • Category Attentional Search for Fast Object Detection by Mimicking Human Visual Perception (2015)
    Applications of Computer Vision (WACV), 2015 IEEE Winter Conference on, , 829--836
  • Development of a Localization System Based on VLC Technique for an Indoor Environment (2015)
  • TILDE: A Temporally Invariant Learned DEtector (2015)
    Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on, , 5279--5288
  • Visual tracking in complex scenes through pixel-wise tri-modeling (2015)
    Machine Vision and Applications, 26 (2-3), 205--217
  • Visual tracking of non-rigid objects with partial occlusion through elastic structure of local patches and hierarchical diffusion (2015)
    Image and Vision Computing, 39, 23--37
  • [DEMO] Tracking texture-less, shiny objects with descriptor fields (2014)
    Mixed and Augmented Reality (ISMAR), 2014 IEEE International Symposium on, , 331--332
  • Motion Interaction Field for Accident Detection in Traffic Surveillance Video (2014)
    Pattern Recognition (ICPR), 2014 22nd International Conference on, , 3062--3067
  • Scale preserving PTZ tracking with size estimation using tilt sensory data (2014)
    Advanced Video and Signal Based Surveillance (AVSS), 2014 11th IEEE International Conference on, , 289--294
  • The Visual Object Tracking VOT2014 Challenge Results (2014)
    ECCV 2014 Workshops,
  • Two-stage online inference model for traffic pattern analysis and anomaly detection (2014)
    Machine vision and applications, 25 (6), 1501--1517
  • Detection of moving objects with a moving camera using non-panoramic background model (2013)
    Machine Vision and Applications, 24 (5), 1015â€
  • Multiple ground plane estimation for 3D scene understanding using a monocular camera (2012)
    Proceedings of the 27th Conference on Image and Vision Computing New Zealand - IVCNZ '12,
  • Visual tracking with dual modeling (2012)
    Proceedings of the 27th Conference on Image and Vision Computing New Zealand - IVCNZ '12,
  • Intelligent visual surveillance — A survey (2010)
    Int. J. Control Autom. Syst., 8 (5), 926â€
  • Orientation and Scale Invariant Kernel-Based Object Tracking with Probabilistic Emphasizing (2010)
    Computer Vision – ACCV 2009, , 130â€
  • PTZ 카메라를 사용한 강인한 물체 추적 (2010)
    대한전자공학회 2010 년 하계종합학술대회, 154̃ 157 쪽 (총 4 쪽),
  • 지능형 영상 감시 알고리즘 개발을 위한 통합 시스템 (2010)
    CICS 2010 정보 및 제어 학술대회 논문집, 109̃ 110 쪽 (총 2 쪽),
  • 탐지 갈라짐과 물체 겹침 상황에 강인한 실시간 다개체 추적 (2010)
    대한전자공학회 2010 년 정기총회 및 추계종합학술대회, 367̃ 368 쪽 (총 2 쪽),
  • 데이터 지연 문제를 해결하기 위한 영상 공유형 감시시스템의 구현 (2009)
    대한전자공학회 2009 년 정기총회 및 추계종합학술대회, 223̃ 224 쪽 (총 2 쪽),
  • 물체 추적 성능 향상을 위한 추적 실패의 검출 방법 (2007)
    2007 CICS 정보 및 제어 학술대회 논문집, 461̃ 462 쪽 (총 2 쪽),
 
 

If this is your researcher profile you can log in to the Faculty & Staff portal to update your details and provide recruitment preferences.

 
 

Planning to do a research degree? Use our expert search to find a potential supervisor!