Dinesh Pai


Relevant Thesis-Based Degree Programs


Graduate Student Supervision

Doctoral Student Supervision

Dissertations completed in 2010 or later are listed below. Please note that there is a 6-12 month delay to add the latest dissertations.

Constrained dynamics with frictional contact on smooth surfaces (2023)

Friction and contact pose a great challenge to efficient and accuratesimulation of deformable objects for computer graphics and engineeringapplications. In contrast to many engineering applications, simulation softwarefor graphics often permits larger approximation errors in favour of betterpredictability, controllability and efficiency.This dissertation explores modern methods for frictional contact resolution incomputer graphics. In particular, the focus is on offline simulation of smoothelastic objects subject to contact with other elastic solids and cloth. Weexplore traditional non-smooth friction formulations as well as smoothedfrictional contact, which lends itself well to differentiable simulation andanalysis. We then explore a particular application of differentiable simulationto motivate the direction of research.In graphics, even smooth objects are typically approximated using piecewiselinear polyhedra, which exhibit sliding artifacts that can be interpreted asartificial friction making simulations less predictable. We develop a techniquefor improving fidelity of sliding contact between smooth objects.Frictional contacts are traditionally resolved using non-smooth models, whichare complex to analyse and difficult to compute to a desirable error estimate.We propose a unified description of the equations of motion subject tofrictional contacts using a smooth model that converges to an accurate frictionresponse. We further analyse the implications of this formulation and compareour results to state-of-the-art methods.The smooth model uniquely resolves frictional contacts, while also being fullydifferentiable. This allows inverse problems using our formulation to be solvedby gradient-based methods. We begin our exploration of differentiablesimulation applications with a parameter estimation task. Elastic parametersare estimated for a three distinct cloth materials using a novel capture,registration and estimation pipeline. Static equilibrium cloth configurationsare efficiently estimated using a popular compliant constraint dynamics. Inthis work we address a common issue of bifurcation in cloth, which causes finalconfiguration mismatches during estimation. Finally, we postulate an extensionto compliant constraint dynamics using our friction model, to show how ourprevious work can be used in parameter estimation tasks involving contact andfriction.

View record

Integrators for elastodynamic simulation with stiffness and stiffening (2020)

The main goal of this thesis is to develop effective numerical algorithms for stiff elastodynamic simulation, a key procedure in computer graphics applications. To enable such simulations, the governing differential system is discretized in 3D space using a finite element method (FEM) and then integrated forward in discrete time steps.To perform such simulations at a low cost, coarse spatial discretization and large time steps are desirable. However, using a coarse spatial mesh can introduce numerical stiffening that impede visual accuracy. Moreover, to enable large time steps while maintaining stability, the semi-implicit backward Euler method (SI) is often used; but this method causes uncontrolled damping and makes simulation appear less lively.To improve the dynamic consistency and accuracy as the spatial mesh resolution is coarsened, we propose and demonstrate, for both linear and nonlinear force models, a new method called EigenFit. This method applies a partial spectral decomposition, solving a generalized eigenvalue problem in the leading mode subspace and then replacing the first several eigenvalues of the coarse mesh by those of the fine one at rest. We show its efficacy on a number of objects with both homogeneous and heterogeneous material distribution.To develop efficient time integrators, we first demonstrate that an exponential Rosenbrock-Euler (ERE) integrator can avoid excessive numerical damping while being relatively inexpensive to apply for moderately stiff elastic material. This holds even in challenging circumstances involving non-convex elastic energies.Finally, we design a hybrid, semi-implicit exponential integrator, SIERE, that allows SI and ERE to each perform what they are good at. To achieve this we apply ERE in a small subspace constructed from the leading modes in the partial spectral decomposition, and the remaining system is handled (i.e., effectively damped out) by SI. We show that the resulting method maintains stability and produces lively simulations at a low cost, regardless of the stiffness parameter used.

View record

3D Biomechanical simulation and control of the human hand (2019)

The goal of this thesis is to develop novel computational tools and software for detailed modelling of dynamics of biomechanical systems such as the human hand, with potential applications in prosthetics, surgery, robotics, and virtual reality. We study the effect of the finger extensor mechanism, and musculotendon control on the kinematic and dynamic function of the hand.Hand tendons form a complex network of sheaths, pulleys, and branches. A three dimensional model capturing its detailed anatomy would help simulate the coordination and internal dynamics of the musculoskeletal system. Previous approaches include resource-intensive cadaver studies and mathematical force-transmission models, which cannot compute hand motion under muscle action.We developed a modelling and control framework for hand musculotendon dynamics to overcome these limitations. This approach uses Eulerian-on-Lagrangian discretization of tendons with a selective quasistatic assumption, eliminating unnecessary degrees of freedom and the need for generic collision detection. Unlike previous approaches, our approach efficiently and accurately handles constrained musculotendon dynamics. Using this framework, two control approaches were developed for precise fingertip trajectory tracking.To apply these techniques, software tools were developed with goals of interactive design, experimentation, and control of hand biomechanics. They overcome limitations of other available biomechanics software, enabling modelling of complex tendon arrangements, such as the finger extensor assembly. These tools can simulate all musculoskeletal elements of the hand, and allow closed-loop simulation control.With these software tools, we built a detailed anatomical model of the lumbrical muscle of the finger and simulated its role in reshaping finger flexion. The lumbrical plays an important role in determining the flexion order for the interphalangeal and metacarpophalageal joints. Prior cadaver studies have recorded this role, providing an opportunity for model validation. The in vitro experiments were reproduced successfully, establishing its role in increasing the grasp reach of the hand. We also modelled the in vivo function of the activated lumbrical, overcoming the limitations of cadaver experiments. Finally, a preliminary model of the full hand was constructed with the thumb and the wrist, and simulations of tenodesis grasp and simple thumb motions are presented.

View record

Measurement and animation of the eye region of the human face in reduced coordinates (2018)

The goal of this dissertation is to develop methods to measure, model, and animate facial tissues of the region around the eyes, referred to as the eye region. First, we measure the subtle movements of the soft tissues of the eye region using a monocular RGB-D camera setup, and second, we model and animate these movements using parameterized motion models. The muscles and skin of the eye region are very thin and sheetlike. By representing these tissues as thin elastic sheets in reduced coordinates, we have shown how we can measure and animate these tissues efficiently.To measure tissue movements, we optically track both eye and skin motions using monocular video sequences. The key idea here is to use a reduced coordinates framework to model thin sheet-like facial skin of the eye region. This framework implicitly constrains skin to conform to the shape of the underlying object when it slides. The skin configuration can then be efficiently reconstructed in 3D by tracking two dimensional skin features in video. This reduced coordinates model allows interactive real-time animation of the eye region in WebGL enabled devices using a small number of animation parameters, including gaze. Additionally, we have shown that the same reduced coordinates framework can also be used for physics-based simulation of the facial tissue movements and to produce tissue deformations that occur in facial expressions.We validated our skin measurement and animation algorithms using skin movement sequences with known skin motions, and we can recover skin sliding motions with low reconstruction errors. We also propose an image-based algorithm that corrects accumulated inaccuracy of standard 3D anatomy registration systems that occurs during motion capture, anatomy transfer, image generation, and animation. After correction, we can overlay the anatomy on input video with low misalignment errors for augmented reality applications, such as anatomy mirroring. Our results show that the proposed image-based corrective registration can effectively reduce these inaccuracies.

View record

Surface based fluid animation using integral equations: simulation and compression (2017)

This dissertation looks at exploiting the mathematics of vorticity dynamics and potential flow using integral equations to reformulate critical parts of fully dynamic fluid animation methods into surface based problems. These reformulations enable more efficient calculation and data-structures due to the reduction of the simulation domain to the two dimensional fluid surface, rather than its volume. We also introduce a surface compression and real-time playback method for continuous time-dependent iso-surfaces. This compression method further increases the impact of our highly efficient surface-based simulation methods.

View record

Simulating Water for Computer Graphics: Particle-In-Cell, Explicit Surfaces, and Discontinuous Galerkin (2016)

We propose several advances in the simulation of fluids for computer graphics. We concentrate on particle-in-cell methods and related sub-problems. We develop high-order accurate extensions to particle-in-cell methods demonstrated on a variety of equations, including constrained dynamics with implicit-explicit time integration. We track the liquid-air interface with an explicit mesh, which we show how to do in a provably exact fashion. To address the mismatched simulation and surface resolution, we solve the partial differential equations in each time step with with a p-adaptive discontinuous Galerkin discretization. This allows us to use a coarse regular grid for the entire simulation. For solving the resulting linear system, we propose a novel mostly-algebraic domain decomposition preconditioner that automatically creates a coarse discontinuous Galerkin approximation of the problem.

View record

Somputational modeling of neuromusculoskeletal systems: from filaments to behavior (2013)

This thesis describes computational approaches to modeling and simulating aspects of the neuromusculoskeletal system. We make contributions to models at three different levels of detail.We first investigate the mechanics of shortening muscle and evaluate two forms of the traditional Hill-type muscle model, force scaling and f-max scaling, and show that the f-max scaling model is significantly better at predicting experimental results. We hypothesize a new model called the winding filament model that incorporates the role of titin during active force development. Based on the proposed hypothesis, we develop a computational model that is able to simulate residual force enhancement. The suggested model can qualitatively simulate the pattern of the force enhancement observed in previous studies.In order to model the higher levels of the system consisting of muscles and bones, we propose an optimal design framework for estimating parameters of the musculoskeletal model. The method finds a set of morphological and physiological parameters that can optimally simulate the measured force and moment at the point of action. We apply the suggested framework to modeling two rat hindlimb muscles, gracilis posticus and posterior part of biceps femoris, to see if the traditional line segment based muscle geometry model is valid for musculoskeletal system modeling. The result shows that even a complex muscle like biceps femoris can be well modeled as a line segment, but its estimated insertion point is far from that of the traditional model based on anatomy.Finally, this thesis addresses a behavioral aspect of biological movement; in particular, how a high level movement is planned and controlled, in coordination with perception. We present a fully generative model of object interception that can simulate realistic, human-like behavior of ball catching for given arbitrary ball trajectory. The model includes a simplified probabilistic model of vision, a model of eye movements combining saccades and pursuit, and corresponding head, hand and body movements. The movements are constructed from submovements. By combining these components, realistic interception behavior is simulated with minimal user intervention.

View record

Towards dynamic, patient-specific musculoskeletal models (2012)

This thesis focuses on the development of tools to aid in producing dynamic simulations from patient specific volumetric data. Specifically, two new computational methods have been developed, one for image acquisition and one for simulation. Acquiring patient-specific musculoskeletal architectures is a difficult task. Our image acquisition relies on Diffusion Tensor Imaging since it allows the non-invasive study of muscle fibre architecture. However, musculoskeletal Diffusion Tensor Imaging suffers from low signal-to-noise ratio. Noise in the computed tensor fields can lead to poorly reconstructed muscle fibre fields. In this thesis we detail how leveraging a priori knowledge of the structure of skeletal muscle can drastically increase the quality of fibre architecture data extracted from Diffusion Tensor Images. The second section of this thesis describes a simulation technique that allows the direct simulation of volumetric data, such as that produced by the denoising algorithm. The method was developed in response to two key motivations: first, that the medical imaging data we acquire is volumetric and can be difficult to discretize in a Lagrangian fashion, and second that many biological structures (such as muscle) are highly deformable and come into close contact with each other as well as the environment. In response to these observations we have produced an Eulerian simulator that can simulate volumetric objects in close contact. The algorithm intrinsically handles large deformations and potential degeneracies that can result in contacting scenarios. Extending the simulator to produce complex musculoskeletal simulations is also discussed. These two algorithms address concerns in two stages of a proposed pipeline for generating dynamic, patient specific musculoskeletal simulations.

View record

Strand-based musculotendon simulation of the hand (2011)

This dissertation develops a framework for modelling biomechanical systems, with special focus on the muscles, tendons, and bones of the human hand. Two complementary approaches for understanding the functions of the hand are developed: the strand simulator for computer modelling, and an imaging apparatus for acquiring a rich data set from cadaver hands.Previous biomechanical simulation approaches, based on either lines-of-force or solid mechanics models, are not well-suited for the hand, where multiple contact constraints make it difficult to route muscles and tendons effectively. In lines-of-force models, wrapping surfaces are used to approximate the curved paths of tendons and muscles near joints. These surfaces affect only the kinematics, and not the dynamics, of musculotendons. In solid mechanics models, the 3D deformation of muscles can be fully accounted for, but these models are difficult to create and expensive to simulate; moreover, the fibre-like properties of muscles are not directly represented and must be added on as auxiliary functions. Neither of these approaches properly handles both the dynamics of the musculotendons and the complex routing constraints. We present a new, strand-based approach, capable of handling the coupled dynamics of muscles, tendons, and bones through various types of routing constraints.The functions of the hand can also be studied from the analysis of data obtained from a cadaver hand. We present a hardware and software setup for scanning a cadaver hand that is capable of simultaneously obtaining the skeletal trajectory, tendon tension and excursion, and tendon marker motion. We finish with a preliminary qualitative comparison of a simulation model of the index finger with real world data acquired from ex vivo specimen, using the strands framework.

View record

Master's Student Supervision

Theses completed in 2010 or later are listed below. Please note that there is a 6-12 month delay to add the latest theses.

Data-driven models of human body inertia (2024)

Accurate estimation of mass properties of the human musculoskeletal system is of great interest to many tasks, from gait analysis in biomechanics to motion tracking and control in computer animation. Previous work typically simplified the human musculoskeletal structure as a chain of rigid capsules, with muscle mass lumped with body segments. Such simplifications lead to errors in the system’s inertia matrix, and the error propagates to torque and pose estimates. In this study, we show that we can estimate the generalized joint-space inertia matrix of a human in motion, using a deep neural network or with a simple statistical model. The models do not make any assumptions other than that effective inertia matrices must be symmetric and positive definite. The models are trained and tested with real-world human data that includes synchronized motion and ground reaction forces. We show that a joint-space inertia matrix estimated from data can be physically plausible by revealing inertial coupling which a rigid, lumped inertia matrix fails to entail, and that effective inertia estimates are motion-type dependent. Moreover, we show that our neural inertia model SPDNet can predict inertia matrices parametrized by pose, body mass and height, that its predicted matrices are physically plausible, and that it generalizes well to unseen poses and mass distributions when used to reconstruct motion.

View record

Disentangling the latent space of 3D human body meshes (2023)

Deep generative models such as Variational Autoencoders (VAEs), Generative Ad- versarial Networks (GANs), and diffusion models have demonstrated their efficacy in generating 2D images and 3D meshes. However, interpreting the learned latent space poses a significant challenge. The current literature mainly focuses on un- supervised methods, which exhibit two primary limitations: firstly, the inability to control the meaning of each latent variable, and secondly, the occurrence of multiple latent variables possessing overlapping meanings. Moreover, it has been shown that fully disentangling the latent space using only unsupervised methods is theoretically infeasible. In this work, we introduce a method for latent space disen- tanglement on 3D meshes. Our method comprises two components: a feature func- tion for predicting 3D mesh features, and a regular generative model. We employ the derivative of the feature function as part of the loss function to promote dis- entanglement. Experimental results demonstrate that our disentanglement method effectively addresses the limitations mentioned above without compromising the accuracy of the reconstruction. Additionally, given its model-agnostic nature, our method exhibits generality across different generative models and tasks.

View record

FASTR : fast approximation of soft tissue in real time (2022)

Real-time animations are limited by a computational budget and often trade realism for performance. Simulating the shape of a realistic human body requires a tremendous amount of computational resources and may be infeasible in real-time applications. In recent years, deep learning approaches have proven their effectiveness in fields such as computer vision. We present a new method that combines ideas from deep learning and example-based skinning. The method approximates corrections to skin deformation from a skeleton-based animation baseline. A key aspect of the approach is to factor the network into two parts, with part of the network evaluated using shaders in the standard real-time graphics rendering pipeline. Our method adds a minimum overhead to a skeleton-based animation while improving its visual results.

View record

Ufit: interactive attribute driven sewing pattern adjustment (2022)

Fit and sizing of clothing are fundamental problems in the field of garment design, manufacture and retail. Here we propose new computational methods for adjusting the fit of clothing on realistic models of the human body by interactively modifying desired fit attributes. Clothing fit represents the relationship between the body and the garment, and is quantified using fit attributes such as ease and pressure on the body. Such attributes are computed by physically based simulations. We propose a method to learn the relationship between the fit attributes and the space of pattern edits. In contrast to the earlier approaches that use in-the-loop physics simulation, we begin by creating a custom data set capturing all possible edits to the 2D pattern and the corresponding per-vertex fit attributes generated from their 3D drape simulations. With this data we train a model to isolate and predict changes to the 2D pattern caused by edits to the fit attributes. We provide interactive tools to directly edit the fit attributes in 3D and instantaneously predict the corresponding pattern adjustments. Our method introduces a different way to express fit adjustment that it is more intuitive and is capable of cutting short the trial-and-error period.

View record

BodyData: a modular system for the design and implementation of complex multistep experiments (2021)

In this thesis, we address the challenge of acquiring high-quality measurement data from real-world experiments. Experiments with human participants can be expensive, both in terms of scheduling participants as well as equipment requirements. Because of these constraints limiting data collection, we desire software tools for data quality assurances that are active during each measurement session. We propose a modular approach to conducting experiments based on the inputs, outputs, and dependencies between individual data-generating operations that we call measurement services. Formally defining the output of each operation provides clear quality assurance targets to aim for during the experiment session. Our framework of modular components also emphasizes extensibility and reusability in the development of new experiments. We implemented our approach by developing BodyData, a web application-centered system designed to measure, store, and securely access data from experiments with human participants. BodyData was tested in our lab; two case studies are presented to illustrate the utility of the system in practice. We discuss how we provide improved quality assurance through the use of configurable data entry constraints as well as visual feedback during the measurement session. We also discuss how we support queries from authorized clients for use in analysis and visualization of stored data.

View record

Measurement and estimation of material parameters of real garments (2020)

In cloth simulation, the choice of material parameters drive the motion of cloth. A good cloth simulation resembles the real world appearance as best as possible. Functional garments as a whole are inhomogeneous, though every distinct part is homogeneous at a small scale. Here, we measure all individual parts of a sports bra and characterize their material parameters.I build a custom designed cloth tester that is capable of measuring a variety of different cloth samples. In particular, common swathes of cloth, but also thicker and stiffer seams can be assessed. Force-displacement curves for both shear and stretch experiments are estimated. At the same time, visual deformation is tracked with a camera. I then simulate the same piece of cloth and minimize the difference between the simulated and the experimentally observed cloth sample to tune our material parameters. At the heart of our cloth simulation lies a non-linear and anisotropic material model.Results show that our device can handle all respective garment parts of a modern sports bra. From tests with synthetic data we learn that our optimization converges to all ground truth material parameters but the bending stiffness. For the measured sports bra, the estimated material parameters fall within the range of values of comparable materials. Again, the bending stiffness has minimal influence on the objective function and we can not resolve the true bending stiffness.In combination, the new measurement device and a cantilever test can estimate the material parameters shear, bulk and bending stiffness and the non-linear stress-strain curves. Thus, the individual garment pieces can be combined to simulate the whole garment.

View record

Measuring, modelling, simulating, and predicting human tissue properties (2020)

Personalized simulation of human bodies is a long standing goal in many applications, ranging from animation to apparel. Even though personalized geometric models can now be easily acquired, physics-based simulation requires soft tissue properties and their distribution over the person’s body. Here we show that mechanical properties of the human body can be directly measured using a novel hand-held device. We describe a complete pipeline for measurement, modeling, parameter estimation, and simulation. The methods described here can be used to create personalized models of an individual human or of a population. Furthermore, we show how to predict soft tissue properties from widely available 3D geometric models of the human body. To train such a prediction model, we utilize a unique database of registered measurements of body shape and soft tissue properties, acquired from over 70 participants. We use a recently introduced convolutional neural network architecture adapted for 3D surfaces, and train the network to predict the distribution of tissue properties over the 3D human body surface. Once the network is trained, no specialized equipment is required, and soft tissue properties are predicted in minutes. The method can be used with commodity 3D scanners, and even with geometric models downloaded from Internet or created by artists. Our methods make realistic human body simulations available to a wide range of users and applications.

View record

Simulation of incompressible elastic material using zonal volume constraints (2020)

Simulation of human soft tissues in contact with their environment is essential in many fields, including visual effects and apparel design. Biological tissues are nearly incompressible. However, standard methods employ compressible elasticity models and achieve incompressibility indirectly by setting Poisson’s ratio to be close to 0.5. This approach can produce results that are plausible qualitatively but inaccurate quantatively. This approach also causes numerical instabilities and locking in coarse discretizations or otherwise pose a prohibitive restriction on the size of the time step. We propose a novel approach to alleviate these issues by replacing indirect volume preservation using Poisson’s ratios with direct enforcement of zonal volume constraints, while controlling fine-scale volumetric deformation through a cell-wise penalty. To increase realism, we propose an epidermis model to mimic the dramatically higher surface stiffness on real skinned bodies. We demonstrate that our method produces stable realistic deformations with precise volume preservation but without locking artifacts. Due to the volume preservation not being tiedto mesh discretization, our method also allows a resolution consistent simulation of incompressible materials.

View record

Skinprobe 2.0: development of a system for low-cost measurement of human soft tissues (2019)

We present "SkinProbe 2.0," a prototype system for low cost, high volume measurement of the physical properties of human soft tissues through direct contact and perturbation of the skin. Our solution encompasses a handheld device and associated cloud-based AI processing pipeline, and derives physically-representative values of stiffness and thickness directly from video. These input videos include images of the surface under contact, and of a "flexure," our novel apparatus for optical force measurement. Videos are captured using a smartphone embedded in the device.Our system processes these videos, generating dense optical flow fields for selected frames, and passing these frames and flow fields through two bespoke Neural Networks: one providing estimated force readings, and one providing estimates of soft-body material properties in the contact vicinity.We automate the collection of training data for our networks with robotics and a 3D-printed apparatus, along with custom-made silicone tissue phantoms, and a cloud pipeline for data collection, storage, and retrieval. This allows us to scale to thousands of samples in each training dataset, with minimal human involvement in collection, and a highly repeatable collection process.We demonstrate the functionality of our measurement device, cloud pipeline, and force estimation system, and show promising material estimation results on our tissue phantoms. We further consider directions for future research in improving our system, both for handheld data collection, and for eventual usage on human subjects.

View record

Perception of motion in virtual reality interception tasks (2017)

Virtual Reality (VR) and related 3D display technologies have recently experienced tremendous growth in promise and popularity, but have significant limitations. Human vision, carefully tuned to integrating multiple cues from the real words can incorrectly perceive the virtual world in these displays. In this thesis, we conduct a series of psychophysics experiments evaluating motion perception in VR, culminating in a user-adapted method to increase interception accuracy of virtual objects by modifying motion-in-depth cues. Using a baseball hitting simulation in VR, we show that our modified motion-in-depth cues result in greater accuracy. Finally, we present implementations of 3D gaze analysis algorithms.

View record

Interactive Animation of the Eye Region (2016)

Humans are extremely sensitive to facial realism and spend a surprisingly amount of time focusing their attention on other people's faces. Thus, believable human character animation requires realistic facial performance. Various techniques have been developed to capture highly detailed actor performance or to help drive facial animation. However, the eye region remains a largely unexplored field and automatic animation of this region is still an open problem. We tackle two different aspects of automatically generating facial features, aiming to recreate the small intricacies of the eye region in real-time. First, we present a system for real-time animation of eyes that can be interactively controlled using a small number of animation parameters, including gaze. These parameters can be obtained using traditional animation curves, measured from an actor’s performance using off-the-shelf eye tracking methods, or estimated from the scene observed by the character using behavioral models of human vision. We present a model of eye movement, that includes not only movement of the globes, but also of the eyelids and other soft tissues in the eye region. To our knowledge this is the first system for real-time animation of soft tissue movement around the eyes based on gaze input. Second, we present a method for real-time generation of distance fields for any mesh in screen space. This method does not depend on object complexity or shape, being only contained by the intended field resolution. We procedurally generate lacrimal lakes on a human character using the generated distance field as input. We present different sampling algorithms for surface exploration and distance estimation, and compare their performance. To our knowledge this is the first method for real-time or screen space generation of distance fields.

View record

A Physics-Based Model for Wrinkling Skin (2015)

Wrinkling of human skin significantly affects the realism of computer generated characters. Wrinkles convey emotion and expression, provide clues of age and health, and indicate interaction between the skin and external objects. Wrinkling is caused by compression: an elastic material buckles out-of-plane in order to preserve length and volume. Human skin buckles in a distinctive pattern, characterized by sharp valleys with rounded peaks. Many techniques used in visual effects require artists to directly produce wrinkles through sculpting or painted displacement maps, while automated techniques are generally designed for adding detail to coarse, cloth-like simulations which are usually not consistent with human skin. The layered structure of skin, and the properties of each layer are critical to producing the buckling patterns observed in real life.In this work a simulation of wrinkling skin is developed that is physically based, while also simple enough for use in computer graphics. A novel constitutive model suitable for large compressive strain is derived and applied to a three-layered model of skin, with a thin shell outermost layer (stratum corneum), and volumetric dermis and hypodermis layers. Finally, we present a modified Newton scheme and linear finite elements for simulating equilibrium configurations of skin under compressive strain.

View record

Accurate smooth pursuit eye movements lead to more accurate manual interceptions (2015)

In ball sports, athletes are taught to keep their eyes on the ball to catch or hit it successfully. This intuitive field experience has already been studied in the laboratory, indicating that tracking a moving object with smooth pursuit eye movementsenhances our ability to predict the object’s trajectory in time and space. Similarly, intercepting a moving object critically relies on motion prediction. Here we assessed the functional significance of eye movements for manual interceptions.In a novel paradigm, we asked observers (n=32) to track a small moving dot, backprojected onto a translucent screen, and to intercept it with their index finger in a designated ‘hit zone’. Hereby, only the first part (100-300 ms) of the trajectory was shown. Thus, observers had to extrapolate the trajectory and intercept its assumed position anywhere within the hit zone.Results show that better pursuit (low eye position and velocity error, high velocity gain, few catch-up saccades of small amplitude) lead to more accurate interceptions. A Hazard analysis yielded two interception strategies: Early interceptors reliedon tracking quality and memory feedback given at the end of each trial, while late interceptors depended more on tracking smoothness, small initial saccades, and accurate eye latencies. Early interceptions (less time of invisibility) yieldedsmaller 2D interception error, while the interception timing was better for longerperiods of smooth tracking (later interceptions).A regression model tree identified low tracking error and small saccadic eye movements as those eye parameters predicting accurate interceptions best. Not only do observers benefit from smooth pursuit eye movements during manual interception, but the interception accuracy also scales with the quality of the eye movements.

View record

Applications of machine learning in sensorimotor control (2015)

There have been many recent advances in the simulation of biologically realistic systems, but controlling these systems remains a challenge. In this thesis, we focus on methods for learning to control these systems without prior knowledge of the dynamics of the system or its environment.We present two algorithms. The first, designed for quasistatic systems, combines Gaussian process regression and stochastic gradient descent. By testing on a model of the human mid-face, we show that this combined method gives better control accuracy than either regression or gradient descent alone, and improves the efficiency of the optimization routine. The second addresses the trajectory-tracking problem for dynamical systems. Our method automatically learns the relationship between muscle activations and resulting movements. We also incorporate passive dynamics compensation and propose a novel gain-scheduling algorithm. Experiments performed on a model of the human index finger demonstrate that each component we add to the control formulation improves performance of fingertip precision tasks.

View record

Learning Periorbital Soft Tissue Motion (2015)

Human observers tend to pay a lot of attention to the eyes and the surrounding soft tissues. These periorbital soft tissues are associated with subtle and fast motions that convey emotions during facial expressions. Modeling the complex movements of these soft tissues is essential for capturing and reproducing realism in facial animations.In this work, we present a data driven model that can efficiently learn and reproduce the complex motion of the periorbital soft tissues. We develop a system to capture the motion of the eye region using a high frame rate monocular camera. We estimate the high resolution texture of the surrounding eye regions using a Bayesian framework. Our learned model performs well in reproducing various animations of the eyes. We further improve realism by introducing methods to model facial wrinkles.

View record

Control of complex biomechanical systems (2014)

Humans show stunning performance on a variety of manipulation tasks. However, little is known about the computation that the human brain performs to accomplish these tasks. Recently, anatomically correct tendon-driven models of the human hand have been developed, but controlling them remains an issue. In this thesis, we present a computationally efficient feedback controller, capable of dealing with the complexity of these models. We demonstrate its abilities by successfully performing tracking and reaching tasks for an elaborated model of the human index finger.The controller, called One-Step-Ahead controller, is designed in a hierarchical fashion, with the high-level controller determining the desired trajectory and the low-level controller transforming it into muscle activations by solving a constrained linear least squares problem. It was proposed to use equilibrium controls as a feedforward command, and learn the controller's parameters online by stabilizing the plant at various configurations. The conducted experiments suggest the feasibility of the proposed learning approach for the index finger model.

View record

Precision Manipulations Using a Low-Dimensional Haptic Interface (2014)

When interacting with physical objects using their own hands, humans display effortless dexterity. It remains a non-intuitive task, however, to specify the motion of a virtual character’s hand or of a robotic manipulator. Creating these motions generally requires animation expertise or extensive periods of offline motion capture. This thesis presents a real-time, adaptive animation interface, specifically designed around haptic (i.e., touch) feedback, for creating precision manipulations of virtual objects. Using this interface, an animator controls an abstract grasper trajectory while the full hand pose is automatically shaped by compliant scene interactions and proactive adaptation. Haptic feedback enables intuitive control by mapping interaction forces from the full animated hand back to the reduced animator feedback space, invoking the same sensorimotor control systems utilized in natural precision manipulations. We provide an approach for online, adaptive shaping of the animated manipulator using our interface based on prior interactions, resulting in more functional and appealing motions.In a user study with nonexpert participants, we tested the effectiveness of haptic feedback and proactive adaptation of grasp shaping. Comparing the quality of motions produced with and without force rendering, haptic feedback was shown to be critical for efficiently communicating contact forces and dynamic events to the user. The effects of proactive shaping, though inarguably beneficial to visual quality, resulted in mixed behavior for our grasp quality metrics.

View record

Biomechanical simulation of the hand musculoskeletal system and skin (2013)

This thesis presents a biomechanically based hand simulator. We makecontributions at two di erent levels: hand motion and hand appearance.We rst develop a musculotendon simulator, and apply this simulator toan anatomically based hand model. Anatomically based hand simulation ischallenging because the tendon network of the hand is complicated and itis highly constrained by the skeleton of the hand. Our simulator employsthe elegance of the Eulerian-Lagrangian strand algorithm, and introducesa 2D planar collision approach to e ciently eliminate unnecessary degreesof freedom and constraints. We show that with our method, we obtain thecoupling between joints automatically, and achieve the storage of energy intendons for fast movements. Also, by injuring a tendon, we are able toobtain simulations of common nger deformities.Although the musculotendon based hand simulation produces naturalhand motion, hand animation is usually observed at the skin level. Wepresent a novel approach to simulate thin hyperelastic skin. Real humanskin is a thin tissue which can stretch and slide over underlying body structuressuch as muscles, bones, and tendons, revealing rich details of a movingcharacter. Simulating such skin is challenging because it is in close contactwith the body and shares its geometry. We propose a novel Eulerian representationof skin that avoids all the di culties of constraining the skin to lieon the body surface by working directly on the surface itself. Skin is modeledas a 2D hyperelastic membrane with arbitrary topology, which makes it easyto cover an entire character or object. We use triangular meshes to modelbody and skin geometry. The method is easy to implement, and can use lowresolution meshes to animate high resolution details stored in texture-likemaps. Skin movement is driven by the animation of body shape prescribedby an artist or by another simulation, and so it can be easily added as apost-processing stage to an existing animation pipeline. We demonstraterealistic animations of the skin on the hand using this approach. We alsoextend it to simulate other parts of human and animal skin, and skin-tightclothes.

View record

Eulerian finite volume method for musculoskeletal simulation and data-driven activation (2013)

This thesis describes a solid simulation method and its application to musculoskeletalsimulation. The presented solid simulation method features Eulerian discretizationand avoids mesh tangling during large deformation. Unlike existing Euleriansolid simulation methods, our method applies to elastoplastic material andvolume-preserving material. To further increase the utility of Eulerian simulationsfor solids, we introduce Lagrangian modes to the simulation and present a newsolver that handles close contact while simultaneously distributing motion betweenthe Lagrangian and Eulerian modes. This Eulerian-on-Lagrangian method enablesunbounded simulation domains and reduces the time step restrictions that oftenplague Eulerian simulation.We also introduce a framework for simulating the dynamics of musculoskeletalsystems, with volumetric muscles and a novel muscle activation model. Musclesare simulated using the solid simulator developed and therefore enjoys volumepreservation which is crucial for accurately capturing the dynamics of muscles andother biological tissues. Unlike previous work, in our system muscle deformationis tightly coupled to the dynamics of the skeletal system, and not added as an aftereffect. Our physiologically based muscle activation model utilizes knowledge ofthe active shapes of muscles, which can be manually drawn or easily obtained frommedical imaging data. Finally we demonstrate results with models derived fromMRI data and models designed for artistic effect.

View record

Noisy optimal control strategies for modelling saccades (2012)

Eye movements have for a while provided us a closer view into how the brain commands the body. Particularly interesting are saccades: fast and accurate eye movements that allow us to scan our visual surroundings. One observation is that motor commands issued by the brain are corrupted by a signal-dependent noise. Moreover, the variance of the signal scales linearly with the control signal squared. It is assumed that such uncertainty in the dynamics introduces a probability distribution of the eye that the brain accounts for during motion planning.We propose a framework for computing the optimal control law for arbitrary dynamical systems, subject to noise, and where the cost function depends on a statistical distribution of the eye’s position. A key contribution of this framework is estimating the endpoint distribution of the plant using Monte Carlo sampling, which is done efficiently using commodity graphics hardware in parallel. We then describe a modified form of gradient descent for computing the optimal control law for an objective function prone to stochastic effects. We compare our approach to other methods, such as downhill simplex and Covariance-Matrix-Adaptation, which are considered “gradient-free” approaches to optimization. We finally conclude with several examples that show the framework successfully controlling saccades for different plant models of the oculomotor system: this includes a 3D torque-based model of the eye, and a a nonlinear model of the muscle actuator that drives the eye.

View record

Biologically motivated controllers for robotic eyes (2011)

We present the development of computational models of biological motor control used in two different types of eye movements --- gaze shifting and gaze stabilization. They are then implemented and tested on robotic systems. The thesis also investigates the application of these biological motor control strategies in robotics applications.We describe and test a non-linear control algorithm inspired by the behaviour of motor neurons in humans during extremely fast saccadic eye movements involved in gaze shifting. The algorithm is implemented on a robotic eye connected with a stiff camera cable, similar to the optic nerve. This adds a complicated non-linear stiffness to the plant. For high speed movement, our "pulse-step" controller operates in open-loop using an internal model of the eye plant learned from past measurements. We show that the controller approaches the performance seen in the human eye, producing fast movements with little overshoot. Interestingly, the controller reproduces the main sequence relationship observed in animal eye movements. We also model the control of eye movements that serve to stabilize its gaze direction. To test and evaluate this stabilization algorithm, we use a camera mounted on a robotic test platform that can have unknown perturbations in the horizontal plane. We show that using models of the vestibulo-ocular and optokinetic reflexes to control the camera allows the camera to be effectively stabilized using an inertial sensor and a single additional motor, without the need for a joint position sensor. The algorithm uses an adaptive controller based on a model of the vertebrate Cerebellum for velocity stabilization, with additional drift correction. A resolution-adaptive retinal slip algorithm that is robust to motion blur was also developed. We show that the resulting system can reduce camera image motion to about one pixel per frame on average even when the platform is rotated at 200 degrees per second. As a practical robotic application, we also demonstrate how the common task of face detection benefits from active gaze stabilization.

View record

Current Students & Alumni

This is a small sample of students and/or alumni that have been supervised by this researcher. It is not meant as a comprehensive list.

Membership Status

Member of G+PS
View explanation of statuses

Program Affiliations


If this is your researcher profile you can log in to the Faculty & Staff portal to update your details and provide recruitment preferences.


Follow these steps to apply to UBC Graduate School!