Yankai Cao

Model Predictive Control (MPC) is an optimization-based control scheme exploited in various industrial processes. It determines optimal control inputs that achieve the desired outcome by predicting future behavior based on models while satisfying system constraint sets. The consideration of complex system dynamics and multiple constraints enables the control of nonlinear processes with complicated behavior. Furthermore, because of its extensive applicability, MPC has been applied to the design of supply chain management, especially to scheduling problems that are formulated as mixed-integer linear programming (MILP) problems. However, the online implementation of MPC is challenging, especially for large-scale systems, due to the prohibitive computation cost. In recent years, the approximation method of MPC control laws using deep neural networks (DNNs) has been studied to address this issue. Nevertheless, it struggles to provide accurate approximation when multiple optimal control inputs exist for each system state. In this case, the MPC control laws follow one-to-many mappings, which DNNs cannot correctly approximate as they can only provide one-to-one mappings. Therefore, we propose mixture network-based approximation methods. Mixture networks, with components of probability (density) distributions in the output layer, can approximate the MPC control laws through a combination of conditional probabilities provided by mixing several estimated probability distributions. This approach then generates multiple control inputs with the highest probabilities. Notably, the proposed method can be applied to various problems by selecting an appropriate probability distribution, such as using a Gaussian distribution for nonlinear problems and a Bernoulli distribution for MILP problems. In this thesis, we investigate two case studies: a benchmark problem for nonlinear problems and a scheduling problem in the steel-making process for MILP problems. The simulation results demonstrate that the mixture network-based approximation method outperforms the DNN-based approximation method.

View record

Clustering is a fundamental unsupervised machine learning task that aims to aggregate similar data into one cluster and separate those in diverse into different clusters. Cluster analysis can always be formulated as an optimization problem. Various objective functions may lead to different clustering problems. In this thesis, we concentrate on k-means and k-center problems. Each can be formulated as a mixed-integer nonlinear programming problem. The work about k-means clustering optimization has been published on ICML 2021 [30]. Moreover, we also submitted the work about global optimization of k-center clustering to ICML 2022 and the paper has been accepted in Phase 1 of reviewing. This thesis provides a practical global optimization algorithm for these two tasks based on a reduced-space spatial branch and bound (BB) scheme. This algorithm can guarantee convergence to the global optimum by only branching on the centers of clusters, which is independent of the dataset’s cardinality. We also design several methods to construct lower and upper bounds at each node in the BB scheme. In addition, for k-center problem, a set of feasibility-based bounds tightening techniques are proposed to narrow down the domain of centers and significantly accelerate the convergence. To demonstrate the capacity of this algorithm, we present computational results on UCI datasets and compare our proposed algorithms with the off-the-shelf global optimal solvers and classical local optimal algorithms. For k-means clustering, the numerical experiments demonstrated our algorithm’s ability to handle datasets with up to 200,000 samples. Besides, for k-center clustering, the serial implementation of the algorithm on the dataset with 14 million samples and 3 features can attain the global optimum to an optimality gap of 0.1% within 2 hours.

View record