Dinesh Pai

Professor

Relevant Degree Programs

 

Graduate Student Supervision

Doctoral Student Supervision (Jan 2008 - May 2019)
Measurement and animation of the eye region of the human face in reduced coordinates (2018)

The goal of this dissertation is to develop methods to measure, model, and animate facial tissues of the region around the eyes, referred to as the eye region. First, we measure the subtle movements of the soft tissues of the eye region using a monocular RGB-D camera setup, and second, we model and animate these movements using parameterized motion models. The muscles and skin of the eye region are very thin and sheetlike. By representing these tissues as thin elastic sheets in reduced coordinates, we have shown how we can measure and animate these tissues efficiently.To measure tissue movements, we optically track both eye and skin motions using monocular video sequences. The key idea here is to use a reduced coordinates framework to model thin sheet-like facial skin of the eye region. This framework implicitly constrains skin to conform to the shape of the underlying object when it slides. The skin configuration can then be efficiently reconstructed in 3D by tracking two dimensional skin features in video. This reduced coordinates model allows interactive real-time animation of the eye region in WebGL enabled devices using a small number of animation parameters, including gaze. Additionally, we have shown that the same reduced coordinates framework can also be used for physics-based simulation of the facial tissue movements and to produce tissue deformations that occur in facial expressions.We validated our skin measurement and animation algorithms using skin movement sequences with known skin motions, and we can recover skin sliding motions with low reconstruction errors. We also propose an image-based algorithm that corrects accumulated inaccuracy of standard 3D anatomy registration systems that occurs during motion capture, anatomy transfer, image generation, and animation. After correction, we can overlay the anatomy on input video with low misalignment errors for augmented reality applications, such as anatomy mirroring. Our results show that the proposed image-based corrective registration can effectively reduce these inaccuracies.

View record

Surface based fluid animation using integral equations : simulation and compression (2017)

This dissertation looks at exploiting the mathematics of vorticity dynamics and potential flow using integral equations to reformulate critical parts of fully dynamic fluid animation methods into surface based problems. These reformulations enable more efficient calculation and data-structures due to the reduction of the simulation domain to the two dimensional fluid surface, rather than its volume. We also introduce a surface compression and real-time playback method for continuous time-dependent iso-surfaces. This compression method further increases the impact of our highly efficient surface-based simulation methods.

View record

Simulating water for computer graphics : particle-in-cell, explicit surfaces, and discontinuous Galerkin (2016)

We propose several advances in the simulation of fluids for computer graphics. We concentrate on particle-in-cell methods and related sub-problems. We develop high-order accurate extensions to particle-in-cell methods demonstrated on a variety of equations, including constrained dynamics with implicit-explicit time integration. We track the liquid-air interface with an explicit mesh, which we show how to do in a provably exact fashion. To address the mismatched simulation and surface resolution, we solve the partial differential equations in each time step with with a p-adaptive discontinuous Galerkin discretization. This allows us to use a coarse regular grid for the entire simulation. For solving the resulting linear system, we propose a novel mostly-algebraic domain decomposition preconditioner that automatically creates a coarse discontinuous Galerkin approximation of the problem.

View record

Somputational modeling of neuromusculoskeletal systems: from filaments to behavior (2013)

No abstract available.

Towards dynamic, patient-specific musculoskeletal models (2012)

This thesis focuses on the development of tools to aid in producing dynamic simulations from patient specific volumetric data. Specifically, two new computational methods have been developed, one for image acquisition and one for simulation. Acquiring patient-specific musculoskeletal architectures is a difficult task. Our image acquisition relies on Diffusion Tensor Imaging since it allows the non-invasive study of muscle fibre architecture. However, musculoskeletal Diffusion Tensor Imaging suffers from low signal-to-noise ratio. Noise in the computed tensor fields can lead to poorly reconstructed muscle fibre fields. In this thesis we detail how leveraging a priori knowledge of the structure of skeletal muscle can drastically increase the quality of fibre architecture data extracted from Diffusion Tensor Images. The second section of this thesis describes a simulation technique that allows the direct simulation of volumetric data, such as that produced by the denoising algorithm. The method was developed in response to two key motivations: first, that the medical imaging data we acquire is volumetric and can be difficult to discretize in a Lagrangian fashion, and second that many biological structures (such as muscle) are highly deformable and come into close contact with each other as well as the environment. In response to these observations we have produced an Eulerian simulator that can simulate volumetric objects in close contact. The algorithm intrinsically handles large deformations and potential degeneracies that can result in contacting scenarios. Extending the simulator to produce complex musculoskeletal simulations is also discussed. These two algorithms address concerns in two stages of a proposed pipeline for generating dynamic, patient specific musculoskeletal simulations.

View record

Strand-based musculotendon simulation of the hand (2011)

This dissertation develops a framework for modelling biomechanical systems, with special focus on the muscles, tendons, and bones of the human hand. Two complementary approaches for understanding the functions of the hand are developed: the strand simulator for computer modelling, and an imaging apparatus for acquiring a rich data set from cadaver hands.Previous biomechanical simulation approaches, based on either lines-of-force or solid mechanics models, are not well-suited for the hand, where multiple contact constraints make it difficult to route muscles and tendons effectively. In lines-of-force models, wrapping surfaces are used to approximate the curved paths of tendons and muscles near joints. These surfaces affect only the kinematics, and not the dynamics, of musculotendons. In solid mechanics models, the 3D deformation of muscles can be fully accounted for, but these models are difficult to create and expensive to simulate; moreover, the fibre-like properties of muscles are not directly represented and must be added on as auxiliary functions. Neither of these approaches properly handles both the dynamics of the musculotendons and the complex routing constraints. We present a new, strand-based approach, capable of handling the coupled dynamics of muscles, tendons, and bones through various types of routing constraints.The functions of the hand can also be studied from the analysis of data obtained from a cadaver hand. We present a hardware and software setup for scanning a cadaver hand that is capable of simultaneously obtaining the skeletal trajectory, tendon tension and excursion, and tendon marker motion. We finish with a preliminary qualitative comparison of a simulation model of the index finger with real world data acquired from ex vivo specimen, using the strands framework.

View record

Master's Student Supervision (2010 - 2018)
Perception of motion in virtual reality interception tasks (2017)

Virtual Reality (VR) and related 3D display technologies have recently experienced tremendous growth in promise and popularity, but have significant limitations. Human vision, carefully tuned to integrating multiple cues from the real words can incorrectly perceive the virtual world in these displays. In this thesis, we conduct a series of psychophysics experiments evaluating motion perception in VR, culminating in a user-adapted method to increase interception accuracy of virtual objects by modifying motion-in-depth cues. Using a baseball hitting simulation in VR, we show that our modified motion-in-depth cues result in greater accuracy. Finally, we present implementations of 3D gaze analysis algorithms.

View record

Interactive animation of the eye region (2016)

Humans are extremely sensitive to facial realism and spend a surprisingly amount of time focusing their attention on other people's faces. Thus, believable human character animation requires realistic facial performance. Various techniques have been developed to capture highly detailed actor performance or to help drive facial animation. However, the eye region remains a largely unexplored field and automatic animation of this region is still an open problem. We tackle two different aspects of automatically generating facial features, aiming to recreate the small intricacies of the eye region in real-time. First, we present a system for real-time animation of eyes that can be interactively controlled using a small number of animation parameters, including gaze. These parameters can be obtained using traditional animation curves, measured from an actor’s performance using off-the-shelf eye tracking methods, or estimated from the scene observed by the character using behavioral models of human vision. We present a model of eye movement, that includes not only movement of the globes, but also of the eyelids and other soft tissues in the eye region. To our knowledge this is the first system for real-time animation of soft tissue movement around the eyes based on gaze input. Second, we present a method for real-time generation of distance fields for any mesh in screen space. This method does not depend on object complexity or shape, being only contained by the intended field resolution. We procedurally generate lacrimal lakes on a human character using the generated distance field as input. We present different sampling algorithms for surface exploration and distance estimation, and compare their performance. To our knowledge this is the first method for real-time or screen space generation of distance fields.

View record

A physics-based model for wrinkling skin (2015)

Wrinkling of human skin significantly affects the realism of computer generated characters. Wrinkles convey emotion and expression, provide clues of age and health, and indicate interaction between the skin and external objects. Wrinkling is caused by compression: an elastic material buckles out-of-plane in order to preserve length and volume. Human skin buckles in a distinctive pattern, characterized by sharp valleys with rounded peaks. Many techniques used in visual effects require artists to directly produce wrinkles through sculpting or painted displacement maps, while automated techniques are generally designed for adding detail to coarse, cloth-like simulations which are usually not consistent with human skin. The layered structure of skin, and the properties of each layer are critical to producing the buckling patterns observed in real life.In this work a simulation of wrinkling skin is developed that is physically based, while also simple enough for use in computer graphics. A novel constitutive model suitable for large compressive strain is derived and applied to a three-layered model of skin, with a thin shell outermost layer (stratum corneum), and volumetric dermis and hypodermis layers. Finally, we present a modified Newton scheme and linear finite elements for simulating equilibrium configurations of skin under compressive strain.

View record

Learning periorbital soft tissue motion (2015)

Human observers tend to pay a lot of attention to the eyes and the surrounding soft tissues. These periorbital soft tissues are associated with subtle and fast motions that convey emotions during facial expressions. Modeling the complex movements of these soft tissues is essential for capturing and reproducing realism in facial animations.In this work, we present a data driven model that can efficiently learn and reproduce the complex motion of the periorbital soft tissues. We develop a system to capture the motion of the eye region using a high frame rate monocular camera. We estimate the high resolution texture of the surrounding eye regions using a Bayesian framework. Our learned model performs well in reproducing various animations of the eyes. We further improve realism by introducing methods to model facial wrinkles.

View record

Control of complex biomechanical systems (2014)

Humans show stunning performance on a variety of manipulation tasks. However, little is known about the computation that the human brain performs to accomplish these tasks. Recently, anatomically correct tendon-driven models of the human hand have been developed, but controlling them remains an issue. In this thesis, we present a computationally efficient feedback controller, capable of dealing with the complexity of these models. We demonstrate its abilities by successfully performing tracking and reaching tasks for an elaborated model of the human index finger.The controller, called One-Step-Ahead controller, is designed in a hierarchical fashion, with the high-level controller determining the desired trajectory and the low-level controller transforming it into muscle activations by solving a constrained linear least squares problem. It was proposed to use equilibrium controls as a feedforward command, and learn the controller's parameters online by stabilizing the plant at various configurations. The conducted experiments suggest the feasibility of the proposed learning approach for the index finger model.

View record

Precision manipulations using a low-dimensional haptic interface (2014)

When interacting with physical objects using their own hands, humans display effortless dexterity. It remains a non-intuitive task, however, to specify the motion of a virtual character’s hand or of a robotic manipulator. Creating these motions generally requires animation expertise or extensive periods of offline motion capture. This thesis presents a real-time, adaptive animation interface, specifically designed around haptic (i.e., touch) feedback, for creating precision manipulations of virtual objects. Using this interface, an animator controls an abstract grasper trajectory while the full hand pose is automatically shaped by compliant scene interactions and proactive adaptation. Haptic feedback enables intuitive control by mapping interaction forces from the full animated hand back to the reduced animator feedback space, invoking the same sensorimotor control systems utilized in natural precision manipulations. We provide an approach for online, adaptive shaping of the animated manipulator using our interface based on prior interactions, resulting in more functional and appealing motions.In a user study with nonexpert participants, we tested the effectiveness of haptic feedback and proactive adaptation of grasp shaping. Comparing the quality of motions produced with and without force rendering, haptic feedback was shown to be critical for efficiently communicating contact forces and dynamic events to the user. The effects of proactive shaping, though inarguably beneficial to visual quality, resulted in mixed behavior for our grasp quality metrics.

View record

A parallel active-set method for solving frictional contact problems (2013)

Simulating frictional contact is a challenging computational task and there exist a variety of techniques to do so. One such technique, the staggered projections algorithm, requires the solution of two convex quadratic program (QP) subproblems at each iteration. We introduce a method, SCHURPA, which employs a primal-dual active-set strategy to efficiently solve these QPs based on a Schur-complement method. A single factorization of the initial saddle point system and a smaller dense Schur-complement is maintained to solve subsequent saddle point systems. Exploiting the parallelizability and warm-starting capabilities of the active-set method as well as the problem structure of the QPs yields a novel approach to the problem of frictional contact. Numerical results of a parallel GPU implementation using NVIDIA’s CUDA applied to a physical simulator of highly deformable bodies are presented.

View record

Biomechanical simulation of the hand musculoskeletal system and skin (2013)

This thesis presents a biomechanically based hand simulator. We makecontributions at two di erent levels: hand motion and hand appearance.We rst develop a musculotendon simulator, and apply this simulator toan anatomically based hand model. Anatomically based hand simulation ischallenging because the tendon network of the hand is complicated and itis highly constrained by the skeleton of the hand. Our simulator employsthe elegance of the Eulerian-Lagrangian strand algorithm, and introducesa 2D planar collision approach to e ciently eliminate unnecessary degreesof freedom and constraints. We show that with our method, we obtain thecoupling between joints automatically, and achieve the storage of energy intendons for fast movements. Also, by injuring a tendon, we are able toobtain simulations of common nger deformities.Although the musculotendon based hand simulation produces naturalhand motion, hand animation is usually observed at the skin level. Wepresent a novel approach to simulate thin hyperelastic skin. Real humanskin is a thin tissue which can stretch and slide over underlying body structuressuch as muscles, bones, and tendons, revealing rich details of a movingcharacter. Simulating such skin is challenging because it is in close contactwith the body and shares its geometry. We propose a novel Eulerian representationof skin that avoids all the di culties of constraining the skin to lieon the body surface by working directly on the surface itself. Skin is modeledas a 2D hyperelastic membrane with arbitrary topology, which makes it easyto cover an entire character or object. We use triangular meshes to modelbody and skin geometry. The method is easy to implement, and can use lowresolution meshes to animate high resolution details stored in texture-likemaps. Skin movement is driven by the animation of body shape prescribedby an artist or by another simulation, and so it can be easily added as apost-processing stage to an existing animation pipeline. We demonstraterealistic animations of the skin on the hand using this approach. We alsoextend it to simulate other parts of human and animal skin, and skin-tightclothes.

View record

Eulerian finite volume method for musculoskeletal simulation and data-driven activation (2013)

This thesis describes a solid simulation method and its application to musculoskeletalsimulation. The presented solid simulation method features Eulerian discretizationand avoids mesh tangling during large deformation. Unlike existing Euleriansolid simulation methods, our method applies to elastoplastic material andvolume-preserving material. To further increase the utility of Eulerian simulationsfor solids, we introduce Lagrangian modes to the simulation and present a newsolver that handles close contact while simultaneously distributing motion betweenthe Lagrangian and Eulerian modes. This Eulerian-on-Lagrangian method enablesunbounded simulation domains and reduces the time step restrictions that oftenplague Eulerian simulation.We also introduce a framework for simulating the dynamics of musculoskeletalsystems, with volumetric muscles and a novel muscle activation model. Musclesare simulated using the solid simulator developed and therefore enjoys volumepreservation which is crucial for accurately capturing the dynamics of muscles andother biological tissues. Unlike previous work, in our system muscle deformationis tightly coupled to the dynamics of the skeletal system, and not added as an aftereffect. Our physiologically based muscle activation model utilizes knowledge ofthe active shapes of muscles, which can be manually drawn or easily obtained frommedical imaging data. Finally we demonstrate results with models derived fromMRI data and models designed for artistic effect.

View record

Noisy optimal control strategies for modelling saccades (2012)

Eye movements have for a while provided us a closer view into how the brain commands the body. Particularly interesting are saccades: fast and accurate eye movements that allow us to scan our visual surroundings. One observation is that motor commands issued by the brain are corrupted by a signal-dependent noise. Moreover, the variance of the signal scales linearly with the control signal squared. It is assumed that such uncertainty in the dynamics introduces a probability distribution of the eye that the brain accounts for during motion planning.We propose a framework for computing the optimal control law for arbitrary dynamical systems, subject to noise, and where the cost function depends on a statistical distribution of the eye’s position. A key contribution of this framework is estimating the endpoint distribution of the plant using Monte Carlo sampling, which is done efficiently using commodity graphics hardware in parallel. We then describe a modified form of gradient descent for computing the optimal control law for an objective function prone to stochastic effects. We compare our approach to other methods, such as downhill simplex and Covariance-Matrix-Adaptation, which are considered “gradient-free” approaches to optimization. We finally conclude with several examples that show the framework successfully controlling saccades for different plant models of the oculomotor system: this includes a 3D torque-based model of the eye, and a a nonlinear model of the muscle actuator that drives the eye.

View record

News Releases

This list shows a selection of news releases by UBC Media Relations over the last 5 years.

Current Students & Alumni

This is a small sample of students and/or alumni that have been supervised by this researcher. It is not meant as a comprehensive list.
 

Membership Status

Member of G+PS
View explanation of statuses

Program Affiliations

 

If this is your researcher profile you can log in to the Faculty & Staff portal to update your details and provide recruitment preferences.