• Author / Uploaded
• boilerqwerty81

Citation preview

Advanced Control using M ATLAB

or Stabilising the unstabilisable David I. Wilson Auckland University of Technology New Zealand May 15, 2015

Copyright © 2015 David I. Wilson

Auckland University of Technology New Zealand Creation date: May, 2015.

All rights reserved. No part of this work may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without prior permission.

C ONTENTS 1

Introduction 1.1 Notation and colour conventions . . . . . . . 1.2 Matlab for computer aided control design . . 1.2.1 Alternative computer design aids . . 1.3 Laboratory equipment for control tests . . . . 1.3.1 Plants with one input and one output 1.3.2 Multi-input and multi-output plants . 1.3.3 Slowing down Simulink . . . . . . . . 1.4 Economics of control . . . . . . . . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

1 2 3 4 5 5 7 8 10

2 From differential to difference equations 2.1 Computer in the loop . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 Sampling an analogue signal . . . . . . . . . . . . . . . . . 2.1.2 Selecting a sample rate . . . . . . . . . . . . . . . . . . . . . 2.1.3 The sampling theorem and aliases . . . . . . . . . . . . . . 2.1.4 Discrete frequency . . . . . . . . . . . . . . . . . . . . . . . 2.2 Finite difference models . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Difference equations . . . . . . . . . . . . . . . . . . . . . . 2.3 The z transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 z-transforms of common functions . . . . . . . . . . . . . . 2.4 Inversion of z-transforms . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 Inverting z-transforms with symbolically . . . . . . . . . . 2.4.2 The partial fraction method . . . . . . . . . . . . . . . . . . 2.4.3 Long division . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.4 Computational approach . . . . . . . . . . . . . . . . . . . 2.4.5 Numerically inverting the Laplace transform . . . . . . . . 2.5 Discretising with a sample and hold . . . . . . . . . . . . . . . . . 2.5.1 Converting Laplace transforms to z-transforms . . . . . . 2.5.2 The bilinear transform . . . . . . . . . . . . . . . . . . . . . 2.6 Discrete root locus diagrams . . . . . . . . . . . . . . . . . . . . . . 2.7 Multivariable control and state space analysis . . . . . . . . . . . . 2.7.1 States and state space . . . . . . . . . . . . . . . . . . . . . 2.7.2 Converting differential equations to state space form . . . 2.7.3 Interconverting between state space and transfer functions 2.7.4 Similarity transformations . . . . . . . . . . . . . . . . . . . 2.7.5 Interconverting between transfer functions forms . . . . . 2.7.6 The steady state . . . . . . . . . . . . . . . . . . . . . . . . . 2.7.7 Poles and zeros . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 Solving the vector differential equation . . . . . . . . . . . . . . . 2.8.1 Numerically computing the discrete transformation . . . . 2.8.2 Using M ATLAB to discretise systems . . . . . . . . . . . . . 2.8.3 Time delay in state space . . . . . . . . . . . . . . . . . . . 2.9 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

15 15 16 16 17 18 21 22 24 24 25 26 27 29 31 33 38 40 41 45 46 49 51 53 57 58 59 61 62 65 67 71 72

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

CONTENTS

iv 2.9.1 Stability in the continuous domain . . . . . . . . . . . . . . . . . . . 2.9.2 Stability of the closed loop . . . . . . . . . . . . . . . . . . . . . . . . 2.9.3 Stability of discrete time systems . . . . . . . . . . . . . . . . . . . . 2.9.4 Stability of nonlinear differential equations . . . . . . . . . . . . . . 2.9.5 Expressing matrix equations succinctly using Kronecker products . 2.9.6 Summary of stability analysis . . . . . . . . . . . . . . . . . . . . . . 2.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

73 76 77 78 84 86 86

3 Modelling dynamic systems with differential equations 3.1 Dynamic system models . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1 Steady state and dynamic models . . . . . . . . . . . . . . . 3.2 A collection of illustrative dynamic models . . . . . . . . . . . . . . 3.2.1 Simple models . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 A continuously-stirred tank reactor . . . . . . . . . . . . . . 3.2.3 A forced circulation evaporator . . . . . . . . . . . . . . . . . 3.2.4 A binary distillation column model . . . . . . . . . . . . . . 3.2.5 Rigorous tray-by-tray distillation column models . . . . . . 3.2.6 Interaction and the Relative Gain Array . . . . . . . . . . . . 3.3 Curve fitting experimental data . . . . . . . . . . . . . . . . . . . . . 3.3.1 Least-squares polynomial regression . . . . . . . . . . . . . . 3.3.2 Least squares improvements . . . . . . . . . . . . . . . . . . 3.3.3 Identification of nonlinear models . . . . . . . . . . . . . . . 3.3.4 Parameter confidence intervals . . . . . . . . . . . . . . . . . 3.4 Numerical tools for modelling . . . . . . . . . . . . . . . . . . . . . . 3.4.1 The solution of nonlinear ordinary differential equations . . 3.4.2 Differential/Algebraic equation systems and algebraic loops 3.5 Linearisation of nonlinear dynamic equations . . . . . . . . . . . . . 3.5.1 Linearising a nonlinear tank model . . . . . . . . . . . . . . 3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

89 89 90 90 91 95 97 98 102 107 109 111 113 115 119 124 125 128 130 132 134

4 The PID controller 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.1 P, PI or PID control . . . . . . . . . . . . . . . . . . . . . . 4.2 The industrial PID algorithm . . . . . . . . . . . . . . . . . . . . 4.2.1 Implementing the derivative component . . . . . . . . . 4.2.2 Variations of the PID algorithm . . . . . . . . . . . . . . . 4.2.3 Integral only control . . . . . . . . . . . . . . . . . . . . . 4.3 Simulating a PID process in S IMULINK . . . . . . . . . . . . . . . 4.4 Extensions to the PID algorithm . . . . . . . . . . . . . . . . . . . 4.4.1 Avoiding derivative kick . . . . . . . . . . . . . . . . . . . 4.4.2 Input saturation and integral windup . . . . . . . . . . . 4.5 Discrete PID controllers . . . . . . . . . . . . . . . . . . . . . . . 4.5.1 Discretising continuous PID controllers . . . . . . . . . . 4.5.2 Simulating a PID controlled response in Matlab . . . . . 4.5.3 Controller performance as a function of sample time . . 4.6 PID tuning methods . . . . . . . . . . . . . . . . . . . . . . . . . 4.6.1 Open loop tuning methods . . . . . . . . . . . . . . . . . 4.6.2 Closed loop tuning methods . . . . . . . . . . . . . . . . 4.6.3 Closed loop single-test tuning methods . . . . . . . . . . 4.6.4 Summary on closed loop tuning schemes . . . . . . . . . 4.7 Automated tuning by relay feedback . . . . . . . . . . . . . . . . 4.7.1 Describing functions . . . . . . . . . . . . . . . . . . . . . 4.7.2 An example of relay tuning . . . . . . . . . . . . . . . . . 4.7.3 Self-tuning with noise disturbances . . . . . . . . . . . . 4.7.4 Modifications to the relay feedback estimation algorithm

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

137 137 137 138 138 140 141 141 143 144 147 147 151 152 153 155 155 159 164 171 171 173 175 178 180

. . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . .

CONTENTS

v

4.8

Drawbacks with PID controllers . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8.1 Inverse response processes . . . . . . . . . . . . . . . . . . . . . . . . 4.8.2 Approximating inverse-response systems with additional deadtime 4.9 Dead time compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.10 Tuning and sensitivity of control loops . . . . . . . . . . . . . . . . . . . . . . 4.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

186 186 189 190 193 195

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

197 197 198 199 200 201 201 205 212 213 215 218 220 222 223 225 228 229 231 232 234 234 236

6 Identification of process models 6.1 The importance of system identification . . . . . . . . . . . . . . . . . . . . . 6.1.1 Basic definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.2 Black, white and grey box models . . . . . . . . . . . . . . . . . . . . 6.1.3 Techniques for identification . . . . . . . . . . . . . . . . . . . . . . . 6.2 Graphical and non-parametric model identification . . . . . . . . . . . . . . 6.2.1 Time domain identification using graphical techniques . . . . . . . . 6.2.2 Experimental frequency response analysis . . . . . . . . . . . . . . . 6.2.3 An alternative empirical transfer function estimate . . . . . . . . . . 6.3 Continuous model identification . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.1 Fitting transfer functions using nonlinear least-squares . . . . . . . . 6.3.2 Identification of continuous models using successive differentiation 6.3.3 Practical continuous model identification . . . . . . . . . . . . . . . . 6.4 Popular discrete-time linear models . . . . . . . . . . . . . . . . . . . . . . . 6.4.1 Extending the linear model . . . . . . . . . . . . . . . . . . . . . . . . 6.4.2 Output error model structures . . . . . . . . . . . . . . . . . . . . . . 6.4.3 General input/output models . . . . . . . . . . . . . . . . . . . . . . . 6.5 Regressing discrete model parameters . . . . . . . . . . . . . . . . . . . . . . 6.5.1 Simple offline system identification routines . . . . . . . . . . . . . . 6.5.2 Bias in the parameter estimates . . . . . . . . . . . . . . . . . . . . . . 6.5.3 Using the System Identification toolbox . . . . . . . . . . . . . . . . . 6.5.4 Fitting parameters to state space models . . . . . . . . . . . . . . . . 6.6 Model structure determination and validation . . . . . . . . . . . . . . . . . 6.6.1 Estimating model order and deadtime . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

239 239 240 241 242 242 243 249 255 257 258 259 262 263 265 267 268 269 272 273 274 278 280 282

5 Digital filtering and smoothing 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1 The nature of industrial noise . . . . . . . . . . . . . . . . 5.1.2 Differentiating without smoothing . . . . . . . . . . . . . 5.2 Smoothing measured data using analogue filters . . . . . . . . . 5.2.1 A smoothing application to find the peaks and troughs . 5.2.2 Filter types . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.3 Classical analogue filter families . . . . . . . . . . . . . . 5.3 Discrete filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.1 A low-pass filtering application . . . . . . . . . . . . . . . 5.3.2 Digital filter approximations . . . . . . . . . . . . . . . . 5.3.3 Efficient hardware implementation of discrete filters . . 5.3.4 Numerical and quantisation effects for high-order filters 5.3.5 Filter visualisation tool . . . . . . . . . . . . . . . . . . . . 5.4 The Fourier transform . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1 Fourier transform definitions . . . . . . . . . . . . . . . . 5.4.2 Orthogonality and frequency spotting . . . . . . . . . . . 5.4.3 Using M ATLAB’s FFT function . . . . . . . . . . . . . . . 5.4.4 Periodogram . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.5 Fourier smoothing . . . . . . . . . . . . . . . . . . . . . . 5.5 Numerically differentiating industrial data . . . . . . . . . . . . 5.5.1 Establishing feedrates . . . . . . . . . . . . . . . . . . . . 5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

CONTENTS

vi 6.6.2 Robust model fitting . . . . . . . . . . . . . . . . . . . . . . . 6.6.3 Common nonlinear model structures . . . . . . . . . . . . . 6.7 Online model identification . . . . . . . . . . . . . . . . . . . . . . . 6.7.1 Recursive least squares . . . . . . . . . . . . . . . . . . . . . . 6.7.2 Recursive least-squares in M ATLAB . . . . . . . . . . . . . . 6.7.3 Tracking the precision of the estimates . . . . . . . . . . . . . 6.8 The forgetting factor and covariance windup . . . . . . . . . . . . . 6.8.1 The influence of the forgetting factor . . . . . . . . . . . . . . 6.8.2 Covariance wind-up . . . . . . . . . . . . . . . . . . . . . . . 6.8.3 Maintaining a symmetric positive definite covariance matrix 6.9 Identification by parameter optimisation . . . . . . . . . . . . . . . . 6.10 Online estimating of noise models . . . . . . . . . . . . . . . . . . . 6.10.1 A recursive extended least-squares example . . . . . . . . . 6.10.2 Recursive identification using the SI toolbox . . . . . . . . . 6.10.3 Simplified RLS algorithms . . . . . . . . . . . . . . . . . . . . 6.11 Closed loop identification . . . . . . . . . . . . . . . . . . . . . . . . 6.11.1 Closed loop RLS in Simulink . . . . . . . . . . . . . . . . . . 6.12 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Adaptive Control 7.1 Why adapt? . . . . . . . . . . . . . . . . . . . . . . . . 7.1.1 The adaption scheme . . . . . . . . . . . . . . . 7.1.2 Classification of adaptive controllers . . . . . . 7.2 Gain scheduling . . . . . . . . . . . . . . . . . . . . . . 7.3 The importance of identification . . . . . . . . . . . . 7.3.1 Polynomial manipulations . . . . . . . . . . . 7.4 Self tuning regulators (STRs) . . . . . . . . . . . . . . 7.4.1 Simple minimum variance control . . . . . . . 7.5 Adaptive pole-placement . . . . . . . . . . . . . . . . 7.5.1 The Diophantine equation and the closed loop 7.5.2 Solving the Diophantine equation in Matlab . 7.5.3 Adaptive pole-placement with identification . 7.5.4 Adaptive pole-placement in S IMULINK . . . . 7.6 Practical adaptive pole-placement . . . . . . . . . . . 7.6.1 Dealing with non-minimum phase systems . . 7.6.2 Separating stable and unstable factors . . . . . 7.6.3 Experimental adaptive pole-placement . . . . 7.6.4 Minimum variance control with dead time . . 7.7 Summary of adaptive control . . . . . . . . . . . . . . 8 Multivariable controller design 8.1 Controllability and observability . . . . . . . . . . . 8.1.1 Controllability . . . . . . . . . . . . . . . . . . 8.1.2 Observability . . . . . . . . . . . . . . . . . . 8.1.3 Computing controllability and observability 8.1.4 State reconstruction . . . . . . . . . . . . . . 8.2 State space pole-placement controller design . . . . 8.2.1 Poles and where to place them . . . . . . . . 8.2.2 Deadbeat control . . . . . . . . . . . . . . . . 8.2.3 Pole-placement for tracking control . . . . . 8.3 Estimating the unmeasured states . . . . . . . . . . . 8.4 Combining estimation and state feedback . . . . . . 8.5 Generic model control . . . . . . . . . . . . . . . . . 8.5.1 The tuning parameters . . . . . . . . . . . . . 8.5.2 GMC control of a linear model . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

284 285 286 287 291 295 298 299 300 302 304 307 309 311 311 314 315 315

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . .

323 323 324 325 325 327 327 328 329 331 332 333 336 339 341 343 345 347 348 353

. . . . . . . . . . . . . .

357 358 358 359 361 363 366 369 371 373 375 377 379 379 382

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

CONTENTS

8.6

8.7

8.5.3 GMC applied to a nonlinear plant . . . . . . . . . . . Exact feedback linearisation . . . . . . . . . . . . . . . . . . . 8.6.1 The nonlinear system . . . . . . . . . . . . . . . . . . 8.6.2 The input/output feedback linearisation control law 8.6.3 Exact feedback example . . . . . . . . . . . . . . . . . Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

vii . . . . . .

9 Classical optimal control 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Parametric optimisation . . . . . . . . . . . . . . . . . . . . . . 9.2.1 Choosing a performance indicator . . . . . . . . . . . . 9.2.2 Optimal tuning of a PID regulator . . . . . . . . . . . . 9.2.3 Using S IMULINK inside an optimiser . . . . . . . . . . . 9.2.4 An optimal batch reactor temperature policy . . . . . . 9.3 The general optimal control problem . . . . . . . . . . . . . . . 9.3.1 A revision of multivariable calculus . . . . . . . . . . . 9.3.2 The optimal control formulation . . . . . . . . . . . . . 9.3.3 The two-point boundary problem . . . . . . . . . . . . 9.3.4 Optimal control examples . . . . . . . . . . . . . . . . . 9.3.5 Problems with a specified target set . . . . . . . . . . . 9.4 Linear quadratic control . . . . . . . . . . . . . . . . . . . . . . 9.4.1 Continuous linear quadratic regulators . . . . . . . . . 9.4.2 Analytical solution to the LQR problem . . . . . . . . . 9.4.3 The steady-state solution to the matrix Riccati equation 9.4.4 The discrete LQR . . . . . . . . . . . . . . . . . . . . . . 9.4.5 A numerical validation of the optimality of LQR . . . . 9.4.6 Tracking LQR control . . . . . . . . . . . . . . . . . . . 9.5 Estimation of state variables . . . . . . . . . . . . . . . . . . . . 9.5.1 Random processes . . . . . . . . . . . . . . . . . . . . . 9.5.2 Combining deterministic and stochastic processes . . . 9.5.3 The Kalman filter estimation scheme . . . . . . . . . . . 9.5.4 The steady-state form of the Kalman filter . . . . . . . . 9.5.5 Current and future prediction forms . . . . . . . . . . . 9.5.6 An application of the Kalman filter . . . . . . . . . . . . 9.5.7 The role of the Q and R noise covariance matrices . . . 9.5.8 Extensions to the basic Kalman filter algorithm . . . . . 9.5.9 The Extended Kalman Filter . . . . . . . . . . . . . . . . 9.5.10 Combining state estimation and state feedback . . . . . 9.5.11 Optimal control using only measured outputs . . . . . 9.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

383 389 389 390 392 396

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

397 397 397 398 401 404 406 408 409 410 412 413 417 421 421 423 427 431 437 441 444 444 449 451 455 456 460 461 465 467 470 471 472

10 Predictive control 10.1 Model predictive control . . . . . . . . . . . . . . . . . . . . . . . . 10.1.1 Constrained predictive control . . . . . . . . . . . . . . . . 10.1.2 Dynamic matrix control . . . . . . . . . . . . . . . . . . . . 10.2 A Model Predictive Control Toolbox . . . . . . . . . . . . . . . . . 10.2.1 A model predictive control GUI . . . . . . . . . . . . . . . 10.2.2 MPC toolbox in M ATLAB . . . . . . . . . . . . . . . . . . . 10.2.3 Using the MPC toolbox in S IMULINK . . . . . . . . . . . . 10.2.4 MPC control of an unstable 3 degree of freedom helicopter 10.2.5 Further readings on MPC . . . . . . . . . . . . . . . . . . . 10.3 Optimal control using linear programming . . . . . . . . . . . . . 10.3.1 Development of the LP problem . . . . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

475 475 478 482 487 487 488 490 491 495 496 496

A List of symbols

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

505

viii

CONTENTS

B Useful utility functions in Matlab

507

C Transform pairs

509

D Useful test models 511 D.1 A forced circulation evaporator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 511 D.2 Aircraft model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 512

L IST 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 1.10 1.11 1.12 1.13 1.14 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 2.10 2.11 2.12 2.13 2.14 2.15 2.16 2.17 2.18 2.19 2.20 2.21 2.22 2.23 2.24 2.25 2.26

OF

F IGURES

Standard color conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Closed loop color conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The standard colour and plot layout for a controlled response used in this book. Black Box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The “Black-box” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Balance arm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fan and flapper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Flapper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Helicopter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Helicopter control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Helicopter flying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Real-time Simulink simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Traditional vs. Advanced control . . . . . . . . . . . . . . . . . . . . . . . . . . . . Economic improvements of better control . . . . . . . . . . . . . . . . . . . . . . .

The computer in the control loop . . . . . . . . . . . . . . . . . . . . . . . . . . 3 bit sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A time series with unknown frequency components . . . . . . . . . . . . . . . The frequency component of a sampled signal . . . . . . . . . . . . . . . . . . Frequency aliases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Scarlet Letter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . H´enon’s attractor in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . H´enon’s attractor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Inverting z-transforms using dimpulse . . . . . . . . . . . . . . . . . . . . . . . Numerically inverting the Laplace transform using the Bromwich integral . . Visualising the Bromwich integral . . . . . . . . . . . . . . . . . . . . . . . . . Numerically inverting the Laplace transform . . . . . . . . . . . . . . . . . . . Numerically inverting Laplace transforms . . . . . . . . . . . . . . . . . . . . . Ideal sampler and zeroth-order hold. . . . . . . . . . . . . . . . . . . . . . . . . Rectangular pulse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zeroth-order hold effects on the discrete Bode diagram . . . . . . . . . . . . . Bode plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The discrete root locus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Various discrete closed loop responses . . . . . . . . . . . . . . . . . . . . . . . A binary distillation column block diagram . . . . . . . . . . . . . . . . . . . . Block diagrams of continuous and discrete state-space systems . . . . . . . . . A block diagram of a state-space system with direct feed-through . . . . . . . Unsteady and steady states for level systems . . . . . . . . . . . . . . . . . . . Submarine step response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Issues and approaches in assessing system stability . . . . . . . . . . . . . . . Nyquist diagram of Eqn. 2.101 in (a) three dimensions and (b) as typically sented in two dimensions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.27 Liapunov (1857–1918) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

. . . . . . . . . . . . . .

2 3 3 6 6 6 7 8 8 9 10 10 12 12

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . pre. . . . . .

16 16 20 20 21 21 23 23 30 34 35 37 37 38 38 44 45 47 47 48 50 50 60 70 73 76 78

LIST OF FIGURES

x

2.28 Regions of stability for continuous and discrete systems . . . . . . . . . . . . . . . . 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18 3.19 3.20 3.21 3.22 3.23 3.24 3.25 3.26

A stable and unstable pendulum . . . . . . . . . . . . Simple buffer tank . . . . . . . . . . . . . . . . . . . . The UK growth based on the GDP . . . . . . . . . . . A continually stirred tank reactor (CSTR) reactor . . . A forced circulation evaporator . . . . . . . . . . . . . Distillation column model . . . . . . . . . . . . . . . . Wood-Berry step response . . . . . . . . . . . . . . . . Wood-Berry column in S IMULINK . . . . . . . . . . . . Distillation tower . . . . . . . . . . . . . . . . . . . . . Sparsity of the distillation column model . . . . . . . Open loop distillation column control . . . . . . . . . Distillation column control . . . . . . . . . . . . . . . . Distillation column control (in detail) . . . . . . . . . . Distillation interactions . . . . . . . . . . . . . . . . . . Dynamic RGA . . . . . . . . . . . . . . . . . . . . . . . Dynamic RGA for a 3 term system . . . . . . . . . . . Density of Air . . . . . . . . . . . . . . . . . . . . . . . Fitting a high-order polynomial to some physical data A bio-chemical reaction . . . . . . . . . . . . . . . . . Model of compressed water . . . . . . . . . . . . . . . Experimental pressure-rate data . . . . . . . . . . . . . t distribution . . . . . . . . . . . . . . . . . . . . . . . . Experimental pressure-rate data . . . . . . . . . . . . . Parameter confidence regions . . . . . . . . . . . . . . Linear and nonlinear trajectories . . . . . . . . . . . . Linearising a nonlinear tank model . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

91 93 96 96 98 99 100 101 103 105 105 106 107 107 110 110 113 115 117 118 120 122 123 124 126 133

4.1 4.2 4.3

PID block diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A series PID controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Comparing PI and integral-only control for the real-time control of a noisy flapper plant with sampling time T = 0.08. . . . . . . . . . . . . . . . . . . . . . . . . . . . . PID simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PID internals in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PID controller variants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Realisable PID controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PID controller in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PID controller with anti-derivative kick. . . . . . . . . . . . . . . . . . . . . . . . . . Avoiding derivative kick . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Illustrating the improvement of anti-derivative kick schemes for PID controllers when applied to the experimental electromagnetic balance. . . . . . . . . . . . . . . Derivative control and noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Anti-windup comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discrete PID controller in S IMULINK . . . . . . . . . . . . . . . . . . . . . . . . . . . Headbox control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Headbox controlled response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A PID controlled process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sample time and discrete PID control . . . . . . . . . . . . . . . . . . . . . . . . . . ZN open loop tuning method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cohen-Coon model fitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cohen-Coon tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PID tuning using a GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Solving for the ultimate frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ziegler-Nichols tuned responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

139 140

4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11 4.12 4.13 4.14 4.15 4.16 4.17 4.18 4.19 4.20 4.21 4.22 4.23 4.24

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

86

141 141 142 142 143 143 144 145 146 146 148 150 152 152 154 154 157 158 159 160 162 163

LIST OF FIGURES 4.25 4.26 4.27 4.28 4.29 4.30 4.31 4.32 4.33 4.34 4.35 4.36 4.37 4.38 4.39 4.40 4.41 4.42 4.43 4.44 4.45 4.46 4.47 4.48 4.49 4.50 4.51 4.52 4.53

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

165 168 169 172 172 174 176 177 179 181 182 182 183 183 184 185 186 187 188 189 190 191 191 192 193 194 194 196 196

A filter as a transfer function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A noisy measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Noise added to a true signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Derivative action given noisy data . . . . . . . . . . . . . . . . . . . . . . . . . . . . Smoothing industrial data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Low-pass filter specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Three single low-pass filters cascaded together to make a third-order filter. . . . . . Amplitude response of filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Analogue Butterworth filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Analogue Chebyshev filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Butterworth and Chebyshev filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using a Butterworth filter to smooth noisy data . . . . . . . . . . . . . . . . . . . . . The frequency response for Butterworth filters . . . . . . . . . . . . . . . . . . . . . Various Butterworth filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Advantages of frequency pre-warping . . . . . . . . . . . . . . . . . . . . . . . . . . Hardware difference equation in Direct Form I . . . . . . . . . . . . . . . . . . . . . An IIR filter with a minimal number of delays . . . . . . . . . . . . . . . . . . . . . Cascaded second-order sections to realise a high-order filter . . . . . . . . . . . . . A second-order section (SOS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Comparing single precision second-order sections with filters in direct form II transposed form. Note that the direct form II filter is actually unstable when run in single precision. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.21 Magnitude response from fvtool . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.22 Approximating square waves with sine waves . . . . . . . . . . . . . . . . . . . . . 5.23 The Fourier approximation to a square wave . . . . . . . . . . . . . . . . . . . . . .

197 198 199 200 202 203 203 204 206 211 212 214 215 216 217 218 219 221 221

5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 5.11 5.12 5.13 5.14 5.15 5.16 5.17 5.18 5.19 5.20

A typical closed loop response of a stable system to a P-controller . A Yuwana-Seborg closed loop step test . . . . . . . . . . . . . . . . . Closed loop responses using the YS scheme . . . . . . . . . . . . . . A self-tuning PID controlled process . . . . . . . . . . . . . . . . . . A process under relay tuning with the PID regulator disabled. . . . An unknown plant under relay feedback . . . . . . . . . . . . . . . Nyquist & Bode diagrams . . . . . . . . . . . . . . . . . . . . . . . . PID Relay tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Relay tuning with noise . . . . . . . . . . . . . . . . . . . . . . . . . Relay tuning of the blackbox . . . . . . . . . . . . . . . . . . . . . . . Relay tuning results of the blackbox . . . . . . . . . . . . . . . . . . A relay with hysteresis. . . . . . . . . . . . . . . . . . . . . . . . . . . Relay with hysteresis . . . . . . . . . . . . . . . . . . . . . . . . . . . Relay feedback with hysteresis . . . . . . . . . . . . . . . . . . . . . Relay feedback with an integrator . . . . . . . . . . . . . . . . . . . 2-point Relay identification . . . . . . . . . . . . . . . . . . . . . . . Monotonicity index . . . . . . . . . . . . . . . . . . . . . . . . . . . . The J curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . An inverse response process . . . . . . . . . . . . . . . . . . . . . . . A NMP plant controlled with a PI controller . . . . . . . . . . . . . Approximating inverse-response systems with additional deadtime Smith Predictor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Smith Predictor with no mismatch . . . . . . . . . . . . . . . . . . . Smith predictor in Simulink . . . . . . . . . . . . . . . . . . . . . . . Dead time compensation . . . . . . . . . . . . . . . . . . . . . . . . . Deadtime compensation applied to the blackbox . . . . . . . . . . . Closed loop with plant subjected to disturbances . . . . . . . . . . . Sensitivity transfer functions . . . . . . . . . . . . . . . . . . . . . . Sensitivity robustness measure . . . . . . . . . . . . . . . . . . . . .

xi . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

223 223 227 227

LIST OF FIGURES

xii 5.24 5.25 5.26 5.27 5.28 5.29

Two signals . . . . . . . . . . . . . . . . . . . . . . Critical radio frequencies . . . . . . . . . . . . . . . Power spectrum for a signal . . . . . . . . . . . . . Smoothing by Fourier transform . . . . . . . . . . Differentiating and smoothing noisy measurement Filtering industrial data . . . . . . . . . . . . . . .

. . . . . .

228 232 233 234 236 237

6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 6.10 6.11 6.12 6.13 6.14 6.15 6.16 6.17

The prediction problem for modelling . . . . . . . . . . . . . . . . . . . . . . . . . . Comparing model and plant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . An experimental setup for input/output identification . . . . . . . . . . . . . . . . Typical open loop step tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Areas method for model identification . . . . . . . . . . . . . . . . . . . . . . . . . . Examples of the Areas method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Identification of the Blackbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Balance arm step test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Random signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A 5-element binary shift register to generate a pseudo-random binary sequence. . Pseudo-random binary sequence generator in S IMULINK . . . . . . . . . . . . . . . Black box experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Black box response analysis using a series of sine waves . . . . . . . . . . . . . . . . Black box response using an input chirp signal. . . . . . . . . . . . . . . . . . . . . . Black box frequency response analysis using a chirp signal. . . . . . . . . . . . . . . Flapper response to a chirp signal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Experimental setup to subject a random input into an unknown plant. The input/output data was collected, processed through Listing 6.2 to give the frequency response shown in Fig. 6.18. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The experimental frequency response compared to the true analytical Bode diagram. See the routine in Listing 6.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Black box response given a pseudo-random input sequence. . . . . . . . . . . . . . Black box frequency response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Empirical transfer function estimate . . . . . . . . . . . . . . . . . . . . . . . . . . . Experimental data from a continuous plant . . . . . . . . . . . . . . . . . . . . . . . A continuous-time model fitted to input/output data . . . . . . . . . . . . . . . . . Continuous model identification strategy . . . . . . . . . . . . . . . . . . . . . . . . Continuous model identification simulation . . . . . . . . . . . . . . . . . . . . . . . Continuous model identification of the blackbox . . . . . . . . . . . . . . . . . . . . Identification using Laguerre functions . . . . . . . . . . . . . . . . . . . . . . . . . A signal flow diagram of an auto-regressive model with exogenous input or ARX model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A signal flow diagram of a ARMAX model . . . . . . . . . . . . . . . . . . . . . . . Discrete time models implemented in S IMULINK . . . . . . . . . . . . . . . . . . . . A signal flow diagram of an output-error model . . . . . . . . . . . . . . . . . . . . A general input/output model structure . . . . . . . . . . . . . . . . . . . . . . . . . Kaos, Control and regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ARX estimation exhibiting bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Offline system identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fitting a polynomial to some (x, y) data . . . . . . . . . . . . . . . . . . . . . . . . . Using the AIC to establish model structure . . . . . . . . . . . . . . . . . . . . . . . Using the AIC to establish model structure using separate validation data . . . . . Identification of deadtime from the step response . . . . . . . . . . . . . . . . . . . Deadtime estimation at fast sampling . . . . . . . . . . . . . . . . . . . . . . . . . . Deadtime estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Blackbox step model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hammerstein and Wiener model structures . . . . . . . . . . . . . . . . . . . . . . . Ideal RLS parameter estimation. (See also Fig. 6.46(a).) . . . . . . . . . . . . . . . .

239 241 242 244 245 245 246 246 248 248 249 250 251 252 253 253

6.18 6.19 6.20 6.21 6.22 6.23 6.24 6.25 6.26 6.27 6.28 6.29 6.30 6.31 6.32 6.33 6.34 6.35 6.36 6.37 6.38 6.39 6.40 6.41 6.42 6.43 6.44

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

254 254 255 256 256 258 259 261 262 263 264 265 266 267 268 268 271 274 275 281 281 282 283 283 284 285 286 292

LIST OF FIGURES

xiii

6.45 6.46 6.47 6.48 6.49 6.50 6.51 6.52 6.53 6.54 6.55 6.56 6.57 6.58 6.59 6.60 6.61 6.62 6.63 6.64

Recursive least squares estimation . . . . . . . . . . . . . . . . . . . RLS under S IMULINK . . . . . . . . . . . . . . . . . . . . . . . . . . . RLS under S IMULINK (version 2) . . . . . . . . . . . . . . . . . . . . Estimation of a two parameter plant . . . . . . . . . . . . . . . . . . Confidence limits for estimation . . . . . . . . . . . . . . . . . . . . RLS and an abrupt plant change . . . . . . . . . . . . . . . . . . . . The ‘memory’ of the forgetting factor . . . . . . . . . . . . . . . . . . Identification using various forgetting factors . . . . . . . . . . . . . Covariance wind-up . . . . . . . . . . . . . . . . . . . . . . . . . . . Variable forgetting factor . . . . . . . . . . . . . . . . . . . . . . . . . The MIT rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Optimising the adaptation gain . . . . . . . . . . . . . . . . . . . . . Addition of coloured noise to a dynamic process. See also Fig. 6.32. RLS with coloured noise . . . . . . . . . . . . . . . . . . . . . . . . . Recursive extended least-squares estimation . . . . . . . . . . . . . Recursive extended least-squares estimation . . . . . . . . . . . . . A simplified recursive least squares algorithm . . . . . . . . . . . . Furnace input/output data . . . . . . . . . . . . . . . . . . . . . . . Closed loop estimation using Simulink . . . . . . . . . . . . . . . . . RLS in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

293 294 295 296 296 297 299 300 302 303 306 307 308 309 310 312 313 314 315 320

7.1 7.2 7.3 7.4 7.5 7.6 7.7

325 326 330 331 331 337

7.22

The structure of an indirect adaptive controller . . . . . . . . . . . . . . . . . . . . . Varying process gain of a spherical tank . . . . . . . . . . . . . . . . . . . . . . . . . Simple minimum variance control . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simple minimum variance control (zoomed) . . . . . . . . . . . . . . . . . . . . . . Pole-placement control structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adaptive pole-placement control structure with RLS identification . . . . . . . . . Control of multiple plants with an adapting controller. We desire the same closed loop response irrespective of the choice of plant. . . . . . . . . . . . . . . . . . . . . Three open loop plants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Desired closed loop response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adaptive pole-placement with identification . . . . . . . . . . . . . . . . . . . . . . Comparing the adaptive pole-placement with the reference trajectory . . . . . . . . Adaptive pole-placement with identification in S IMULINK . . . . . . . . . . . . . . Internals of the Adaptive pole-placement design routine . . . . . . . . . . . . . . . Bursting in adaptive control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A plant with poorly damped zeros . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adaptive pole-placement with an unstable B . . . . . . . . . . . . . . . . . . . . . . Migration of poles and zeros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Areas of well-damped poles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adaptive pole-placement of the black-box . . . . . . . . . . . . . . . . . . . . . . . . Adaptive pole-placement of the black-box (zoomed portion) . . . . . . . . . . . . . A non-minimum phase plant with an unstable zero which causes an inverse step response. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Moving average control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

352 353

8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 8.9

Reconstructing states . . . . . . . . . . . . . . . . . . . . . . . Pole-placement . . . . . . . . . . . . . . . . . . . . . . . . . . Pole-placement degradation . . . . . . . . . . . . . . . . . . . Deadbeat control . . . . . . . . . . . . . . . . . . . . . . . . . State feedback control system with an integral output state. . State feedback with integral states . . . . . . . . . . . . . . . Simultaneous control and state estimation . . . . . . . . . . . Control and estimation . . . . . . . . . . . . . . . . . . . . . . GMC tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . .

366 370 371 372 374 375 377 380 381

7.8 7.9 7.10 7.11 7.12 7.13 7.14 7.15 7.16 7.17 7.18 7.19 7.20 7.21

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

337 338 338 340 340 341 341 342 344 346 347 348 349 350

LIST OF FIGURES

xiv 8.10 8.11 8.12 8.13 8.14 8.15 8.16

GMC tuning characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . Linear GMC response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Linear GMC comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . GMC CSTR control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A CSTR phase plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The configuration of an input/output feedback linearisation control law Exact feedback linearisation . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . .

382 384 384 388 388 390 394

9.1 9.2 9.3 9.4 9.5 9.6 9.7 9.8 9.9 9.10 9.11 9.12 9.13 9.14 9.15 9.16 9.17 9.18 9.19 9.20 9.21 9.22 9.23 9.24 9.25 9.26 9.27 9.28 9.29 9.30 9.31 9.32 9.33 9.34 9.35 9.36 9.37

IAE areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ITAE breakdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Optimal responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Optimal PID tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Optimum PI tuning of the blackbox plant . . . . . . . . . . . . . . . . . . . . . . . S IMULINK model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Production of a valuable chemical in a batch reactor. . . . . . . . . . . . . . . . . . Temperature profile optimisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . Optimum temperature profile comparison for different number of temperatures Optimal control for the Rayleigh problem . . . . . . . . . . . . . . . . . . . . . . . Optimal control for the batch reactor . . . . . . . . . . . . . . . . . . . . . . . . . . Optimal control with targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Steady-state and time-varying LQR control . . . . . . . . . . . . . . . . . . . . . . Steady-state continuous LQR controller . . . . . . . . . . . . . . . . . . . . . . . . A block diagram of LQR control . . . . . . . . . . . . . . . . . . . . . . . . . . . . Comparing discrete and continuous LQR controllers . . . . . . . . . . . . . . . . . LQR control with varying tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pole-placement and LQR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pole-placement and LQR showing the input . . . . . . . . . . . . . . . . . . . . . Trial pole locations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Trial pole-placement performance . . . . . . . . . . . . . . . . . . . . . . . . . . . LQR servo control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LQI servo control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Black box servo control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A block diagram of a state estimator . . . . . . . . . . . . . . . . . . . . . . . . . . PDF of a random variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Correlated noisy x, y data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2D ellipse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A block diagram of a stochastic state-space system . . . . . . . . . . . . . . . . . . Kalman filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Prediction-type Kalman filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Current-type Kalman filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kalman filter demonstration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The performance of a Kalman filter for different q/r ratios . . . . . . . . . . . . . A random walk process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LQG in S IMULINK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LQG of an aircraft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

399 399 400 403 404 405 406 407 408 416 418 420 426 429 433 434 437 438 439 439 440 442 442 443 444 445 448 449 450 455 457 458 461 464 469 470 471

10.1 10.2 10.3 10.4 10.5 10.6 10.7 10.8 10.9

Horizons used in model predictive control . Predictions of the Reserve Bank . . . . . . . . Inverse plant . . . . . . . . . . . . . . . . . . . Predictive control . . . . . . . . . . . . . . . . Acausal response . . . . . . . . . . . . . . . . Varying the horizons of predictive control . . MPC on the blackbox . . . . . . . . . . . . . . Step response coefficients for a stable system DMC control details . . . . . . . . . . . . . .

. . . . . . . . .

476 476 479 480 481 482 482 483 485

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . .

. . . . . . . . .

. . . . . . .

. . . . . . . . .

. . . . . . .

. . . . . . . . .

. . . . . . .

. . . . . . . . .

. . . . . . . . .

LIST OF FIGURES 10.10DMC control . . . . . . . . . . . . . . . . . . . . 10.11Adaptive DMC of the blackbox . . . . . . . . . 10.12An MPC graphical user interface . . . . . . . . 10.13Multivariable MPC . . . . . . . . . . . . . . . . 10.14S IMULINK and MPC . . . . . . . . . . . . . . . 10.15MPC . . . . . . . . . . . . . . . . . . . . . . . . 10.16A 3 degree of freedom helicopter . . . . . . . . 10.17MPC control of a helicopter structure . . . . . . 10.18MPC control of a helicopter results . . . . . . . 10.19LP constraint matrix dimensions . . . . . . . . 10.20LP optimal control . . . . . . . . . . . . . . . . 10.21LP optimal control with active constraints . . . 10.22Non-square LP optimal control . . . . . . . . . 10.23LP optimal control showing acausal behaviour

xv . . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

486 488 489 490 491 492 492 493 495 498 502 502 503 503

xvi

LIST OF FIGURES

L IST

OF

TABLES

1.1

Computer aids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4

2.1 2.2 2.3

Final and initial value theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Inverting a z transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Laplace transform pairs used for testing . . . . . . . . . . . . . . . . . . . . . . . . .

26 32 36

3.1 3.2 3.3 3.4 3.5 3.6

Standard nomenclature used in modelling dynamic systems . . . . . . . . Parameters of the CSTR model . . . . . . . . . . . . . . . . . . . . . . . . . The important variables in the forced circulation evaporator from Fig. 3.5. Compressed water . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The parameter values for the CSTR model . . . . . . . . . . . . . . . . . . . The initial state and manipulated variables for the CSTR simulation . . . .

4.1 4.2 4.3

4.5 4.6

Alternative PID tuning parameter conventions . . . . . . . . . . . . . . . . . . . . . Ziegler-Nichols open-loop PID tuning rules . . . . . . . . . . . . . . . . . . . . . . . PID controller settings based on IMC for a small selection of common plants where the control engineer gets to chose a desired closed loop time constant, τc . . . . . . . Various alternative ‘Ziegler-Nichols’ type PID tuning rules as a function of the ultimate gain, Ku , and ultimate period, Pu . . . . . . . . . . . . . . . . . . . . . . . . . Closed-loop single-test PID design rules . . . . . . . . . . . . . . . . . . . . . . . . . Relay based PID tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5.1

Filter transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

6.1 6.2

Experimentally determined frequency response of the blackbox . . . . . . . . . . . 251 Identification in state-space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280

8.1 8.2

The relationship between regulation and estimation . . . . . . . . . . . . . . . . . . 376 Litchfield nonlinear CSTR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385

9.1

Common integral performance indices . . . . . . . . . . . . . . . . . . . . . . . . . . 400

4.4

xvii

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. 94 . 97 . 97 . 118 . 127 . 128 138 158 159 161 167 177

xviii

LIST OF TABLES

L ISTINGS 2.1 2.2 2.3 2.4

Numerically inverting Laplace transforms using the Gaver-Stehfest algorithm . . . Symbolic Laplace to z-transform conversion . . . . . . . . . . . . . . . . . . . . . . Symbolic Laplace to z-transform conversion with ZOH . . . . . . . . . . . . . . . . Extracting the gain, time constants and numerator time constants from an arbitrary transfer function format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Find the transmission zeros for a MIMO system . . . . . . . . . . . . . . . . . . . . 2.6 Submarine simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Example of the Routh array using the symbolic toolbox . . . . . . . . . . . . . . . . 2.8 Solve the continuous matrix Lyapunov equation using Kronecker products . . . . 2.9 Solve the matrix Lyapunov equation using the lyap routine . . . . . . . . . . . . . 2.10 Using Lyapunov to establish stability of a linear system . . . . . . . . . . . . . . . . 2.11 Solve the discrete matrix Lyapunov equation using Kronecker products . . . . . . 3.1 Computing the dynamic relative gain array analytically . . . . . . . . . . . . . . . . 3.2 Computing the dynamic relative gain array numerically as a function of ω. See also Listing 3.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Curve fitting using polynomial least-squares . . . . . . . . . . . . . . . . . . . . . . 3.4 Polynomial least-squares using singular value decomposition. This routine follows from, and provides an alternative to, Listing 3.3. . . . . . . . . . . . . . . . . . . . . 3.5 Curve fitting using a generic nonlinear optimiser . . . . . . . . . . . . . . . . . . . . 3.6 Curve fitting using the O PTI optimisation toolbox. (Compare with Listing 3.5.) . . 3.7 Fitting water density as a function of temperature and pressure . . . . . . . . . . . 3.8 Finding the optimum parameter for a nonlinear reaction rate model . . . . . . . . . 3.9 Routines for the cumulative probability distribution, pt, and the inverse CDF, qt, for the t-distribution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.10 Parameter confidence limits for a nonlinear reaction rate model. This routines follows from Listing 3.8 and uses the statistical functions from Listing 3.9. . . . . . . . 3.11 Comparing the dynamic response of a pendulum to the linear approximation . . . 3.12 The dynamics of a chaotic pendulum from Eqn. 3.69. . . . . . . . . . . . . . . . . . 3.13 Using linmod to linearise an arbitrary S IMULINK module. . . . . . . . . . . . . . . 4.1 Constructing a transfer function of a PID controller . . . . . . . . . . . . . . . . . . 4.2 Constructing a discrete (filtered) PID controller . . . . . . . . . . . . . . . . . . . . . 4.3 A simple continuous-time PID controller . . . . . . . . . . . . . . . . . . . . . . . . ¿ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Ziegler-Nichols PID tuning rules for an arbitrary transfer function . . . . . . . . . . 4.5 Identifies the characteristic points for the Yuwana-Seborg PID tuner from a trial closed loop response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Compute the closed loop model from peak and trough data . . . . . . . . . . . . . 4.7 Compute the ultimate gain and frequency from the closed loop model parameters. 4.8 Compute the open loop model, Gm , Eqn. 4.31. . . . . . . . . . . . . . . . . . . . . . 4.9 Compute appropriate PI or PID tuning constants based on a plant model, Gm , using the IMC schemes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.10 Calculates the period and amplitude of a sinusoidal time series using least-squares. 5.1 Designing Butterworth Filters using Eqn. 5.4. . . . . . . . . . . . . . . . . . . . . . . 5.2 Designing a low-pass Butterworth filter with a cut-off frequency of fc = 800 Hz. . xix

35 40 40 59 61 69 75 81 81 82 83 109 109 112 114 116 117 118 120 121 122 125 126 133 139 151 152 163 164 169 170 170 170 170 179 206 207

LISTINGS

xx 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 6.10 6.11 6.12 6.13 6.14 6.15 6.16 6.17 6.18 6.19 6.20 6.21 6.22 6.23 7.1 7.2 7.3 7.4 7.5 7.6 7.7 7.8 7.9 8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 8.9

Designing a high-pass Butterworth filter with a cut-off frequency of fc = 800 Hz. . 207 Designing Chebyshev Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 Computing a Chebyshev Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 Converting a 7th-order Butterworth filter to 4 second-order sections . . . . . . . . . 221 Comparing DFII and SOS digital filters in single precision. . . . . . . . . . . . . . . 222 Designing and visualising a 5th order elliptic band-stop filter. . . . . . . . . . . . . 223 Routine to compute the power spectral density plot of a time series . . . . . . . . . 230 Smoothing and differentiating a noisy signal . . . . . . . . . . . . . . . . . . . . . . 235 Identification of a first-order plant with deadtime from an openloop step response using the Areas method from Algorithm 6.1. . . . . . . . . . . . . . . . . . . . . . . 245 Frequency response identification of an unknown plant directly from input/output data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 Non-parametric frequency response identification using etfe. . . . . . . . . . . . . 255 Function to generate output predictions given a trial model and input data. . . . . 258 Optimising the model parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258 Validating the fitted model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 Continuous model identification of a non-minimum phase system . . . . . . . . . . 260 Generate some input/output data for model identification . . . . . . . . . . . . . . 272 Estimate an ARX model from an input/output data series using least-squares . . . 273 An alternative way to construct the data matrix for ARX estimation using Toeplitz matrices. See also Listing 6.9. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 Offline system identification using arx from the System Identification Toolbox . . 275 Offline system identification with no model/plant mismatch . . . . . . . . . . . . . 275 Demonstrate the fitting of an AR model. . . . . . . . . . . . . . . . . . . . . . . . . . 276 Create an input/output sequence from an output-error plant. . . . . . . . . . . . . 277 Parameter identification of an output error process using oe and arx. . . . . . . . 277 A basic recursive least-squares (RLS) update (without forgetting factor) . . . . . . . 290 Tests the RLS identification scheme using Listing 6.16. . . . . . . . . . . . . . . . . . 292 A recursive least-squares (RLS) update with a forgetting factor. (See also Listing 6.16.)299 Adaption of the plant gain using steepest descent . . . . . . . . . . . . . . . . . . . 306 Create an ARMAX process and generate some input/output data suitable for subsequent identification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 Identify an ARMAX process from the data generated in Listing 6.20. . . . . . . . . 310 Recursively identify an ARMAX process. . . . . . . . . . . . . . . . . . . . . . . . . 311 Kaczmarz’s algorithm for identification . . . . . . . . . . . . . . . . . . . . . . . . . 312 Simple minimum variance control where the plant has no time delay . . . . . . . . 329 A Diophantine routine to solve F A + BG = T for the polynomials F and G. . . . . 334 Alternative Diophantine routine to solve F A + BG = T for the polynomials F and G. Compare with Listing 7.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334 Constructing polynomials for the Diophantine equation example . . . . . . . . . . 335 Solving the Diophantine equation using polynomials generated from Listing 7.4. . 336 Adaptive pole-placement control with 3 different plants . . . . . . . . . . . . . . . . 338 The pole-placement control law when H = 1/B . . . . . . . . . . . . . . . . . . . . 343 Factorising an arbitrary polynomial B(q) into stable, B + (q), and unstable and poorly damped, B − (q), factors such that B = B + B − and B + is defined as monic. . . . . . 345 Minimum variance control design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351 A simple state reconstructor following Algorithm 8.1. . . . . . . . . . . . . . . . . . 365 Pole-placement control of a well-behaved system . . . . . . . . . . . . . . . . . . . . 369 A deadbeat controller simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372 Pole placement for controllers and estimators . . . . . . . . . . . . . . . . . . . . . . 378 GMC on a Linear Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383 GMC for a batch reactor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386 The dynamic equations of a batch reactor . . . . . . . . . . . . . . . . . . . . . . . . 387 Find the Lie derivative for a symbolic system . . . . . . . . . . . . . . . . . . . . . . 392 Establish relative degree, r (ignore degree 0 possibility) . . . . . . . . . . . . . . . . 393

LISTINGS 8.10 8.11 9.1 9.2 9.3 9.4 9.5 9.6 9.7 9.8 9.9 9.10 9.11 9.12 9.13 9.14 9.15 9.16 9.17 9.18 9.19 9.20 9.21 9.22 9.23 9.24 9.25 9.26 9.27 9.28 9.29 10.1 10.2 10.3 10.4 10.5 10.6 B.1 B.2 B.3

xxi

Design Butterworth filter of order r. . . . . . . . . . . . . . . . . . . . . . . . . . . . 393 Symbolically create the closed loop expression . . . . . . . . . . . . . . . . . . . . . 393 Returns the IAE performance for a given tuning. . . . . . . . . . . . . . . . . . . . . 402 Optimal tuning of a PID controller for a non-minimum phase plant. This script file uses the objective function given in Listing 9.1. . . . . . . . . . . . . . . . . . . . . . 402 Returns the ITSE using a S IMULINK model. . . . . . . . . . . . . . . . . . . . . . . . 405 Analytically computing the co-state dynamics and optimum input trajectory as a function of states and co-states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416 Solving the reaction profile boundary value problem using the boundary value problem solver, bvp4c.m. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417 Computes the full time-evolving LQR solution . . . . . . . . . . . . . . . . . . . . . 425 The continuous time differential Riccati equation. This routine is called from Listing 9.8. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425 Solves the continuous time differential Riccati equation using a numerical ODE integrator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425 Calculate the continuous optimal steady-state controller gain. . . . . . . . . . . . . 428 Closed loop simulation using an optimal steady-state controller gain. . . . . . . . . 428 Solving the algebraic Riccati equation for P∞ using Kronecker products and vectorisation given matrices A, B, Q and R. . . . . . . . . . . . . . . . . . . . . . . . . 431 Calculate the discrete optimal steady-state gain by ‘iterating until exhaustion’. Note it is preferable for numerical reasons to use lqr for this computation. . . . . . . . . 432 Comparing the continuous and discrete LQR controllers. . . . . . . . . . . . . . . . 433 An LQR controller for the blackbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437 Comparing an LQR controller from Listing 9.14 with a pole-placement controller . 438 Computing the closed loop poles from the optimal LQR controller from Listing 9.14. 440 Tracking LQR controller with integral states . . . . . . . . . . . . . . . . . . . . . . . 441 Comparing the actual normally distributed random numbers with the theoretical probability density function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445 Probability and inverse probability distributions for the F -distribution. . . . . . . . 447 Generate some correlated random data. . . . . . . . . . . . . . . . . . . . . . . . . . 447 Plot a 3D histogram of the random data from Listing 9.20. . . . . . . . . . . . . . . 448 Compute the uncertainty regions from the random data from Listing 9.21. . . . . . 449 Validating the uncertainty regions computed theoretically from Listing 9.22. . . . . 449 Solving the discrete time Riccati equation using exhaustive iteration around Eqn. 9.99 or alternatively using the dare routine. . . . . . . . . . . . . . . . . . . . . . . . . . 456 Alternative ways to compute the Kalman gain . . . . . . . . . . . . . . . . . . . . . 458 State estimation of a randomly generated discrete model using a Kalman filter. . . 460 Computing the Kalman gain using dlqe. . . . . . . . . . . . . . . . . . . . . . . . . 462 Demonstrating the optimality of the Kalman filter. . . . . . . . . . . . . . . . . . . . 463 Potter’s algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466 Predictive control with input saturation constraints using a generic nonlinear optimiser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479 Objective function to be minimised for the predictive control algorithm with input saturation constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480 Dynamic Matric Control (DMC) control . . . . . . . . . . . . . . . . . . . . . . . . . 486 Setting up an MPC controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 488 Nonlinear dynamics of a helicopter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493 Optimal control using linear programming . . . . . . . . . . . . . . . . . . . . . . . 500 Polynomial addition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 507 Convolution (polynomial multiplication) of two or more polynomials. . . . . . . . 507 Strip leading zeros from a polynomial. . . . . . . . . . . . . . . . . . . . . . . . . . . 508

xxii

LISTINGS

C HAPTER 1

I NTRODUCTION Mathematicians may flatter themselves that they posses new ideas which mere human language is as yet unable to express. Let them make the effort to express those ideas in appropriate words without the aid of symbols, and if they succeed, they will not only lay us laymen under a lasting obligation but, we venture to say, they will find themselves very much enlightened during the process, and will even be doubtful whether the ideas expressed as symbols had ever quite found their way out of the equations into their minds. James Clerk Maxwell, 1890

Control, in an engineering sense, is where actions are taken to ensure that a particular physical process responds in some desired manner. Automatic control is where we have relieved the human operator from the tedium of consistently monitoring the process and supplying the necessary corrections. Control as a technical discipline is therefore important not only in the fields of engineering, but also in economics, sociology and indeed in most aspects of our life. When studying control, we naturally assume that we do conceivably have some chance of influencing things. For example, it is worthwhile to study the operation of a coal fired power plant in order to minimise possibly polluting emissions, but it is not worth our time to save the world from the next ice age, or as the results of a special study group who investigated methods designed to protect the world from a stray comet (such as the one postulated to have wiped out the dinosaurs 80 million years ago) concluded, there was nothing feasible we could do, such as change the earth’s orbit, or blast the asteroid, to avoid the collision. In these latter examples, the problem exists, but our influence is negligible. The teaching of control has changed in emphasis over the last decade from that of linear singleinput/single-output systems elegantly described in the Laplace domain, to general nonlinear multiple-input/multiple-output systems best analysed in the state space domain. This change has been motivated by the increasing demands by industry and public to produce more, faster or cleaner and is now much more attractive due to the impressive improvements in computer aided tools, such as M ATLAB used in these notes. This new emphasis is called advanced (or modern) control as opposed to the traditional or classical control. This set of notes is intended for students who have previously attended a first course in automatic control covering the usual continuous control concepts like Laplace transforms, Bode diagrams, stability of linear differential equations, PID controllers, and perhaps some exposure to discrete time control topics like z transforms and state space. This book attempts to describe what advanced control is, and how it is applied in engineering applications with emphasis on the construction of controllers using computer aided design tools such as the numerical programming environment M ATLAB from the MathWorks, [136]. With this tool, we can concentrate on the intentions behind the design procedures, rather than the 1

CHAPTER 1. INTRODUCTION

2 mechanics to follow them.

Part one contains some revision material in z-transforms, modelling and PID controller tuning. The discrete domain, z-transforms, and stability concepts with a brief discussion of appropriate numerical methods are introduced in chapter 2. A brief potpourri of modelling is summarised in chapter 3. Chapter 4 is devoted to the most common industrial controller, the three term PID controller with emphasis on tuning, implementation and limitations. Some basic concepts from signal processing such as filtering and smoothing are introduced in chapter 5. Identification and the closely related adaptive control are together in chapters 6 and 7. State space analysis and optimal control design are given in the later chapters 8 and 9.

1.1

N OTATION AND COLOUR CONVENTIONS

Throughout these notes I have used some typographical and colour conventions. In mathematical expressions, scalar variables are written in italic such as a, b, c, or if Greek, γ, ϕ while vectors x, y are upright bold lower case and matrices, A, ∆ are bold upper case. More notation is introduced as required. M ATLAB instructions to be typed in as a command are given in a fixed-width font as A=chol(B*B’) and it is important to carefully note the typographical difference between similar symbols such as the number zero, 0 and the capital letter O, the number one (1) and the letter l, and the Greek letters ν, ρ and the English italic letters v and p. A section of M ATLAB code and the associated output is given as 1

>> G = pole(tf(1,[1 3])) % Find the poles of G(s) = G = -3

1 . s+3

However in many cases the output is not explicitly given and the M ATLAB prompt is omitted. The book is primarily concerned with control loops for which we will use, where practical, a consistent colour convention. The inputs or manipulated variables to the plant are coloured blue, while the plant outputs or measurements are coloured red. A suitable mnemonic is to think of a simple domestic hot water heater where the industrial plant being ‘dirty brown’ heats up the incoming input cold water (blue) to produce an output hot water (red) as shown in Fig. 1.1. Input u

Plant

output, y

cold water

Domestic Hot water heater

hot water

Figure 1.1: Standard colour conventions used for inputs (blue) and outputs (red) in this book. The default colouring convention for a closed loop control system is shown in Fig. 1.2. Here we colour the reference or setpoint in red (or pink), and disturbing signals such as noise (perhaps arising from nature) are coloured green. Fig. 1.3 shows an example of a typical controlled response given in this book. The top plot shows the output in red, with the setpoint in dotted pink. The lower trend shows the manipulated input in blue. Other information such as the constraint bounds for the manipulated variable are overlayed in grey.

1.2. MATLAB FOR COMPUTER AIDED CONTROL DESIGN

3

noise, e error e setpoint, r

+

u controller

Plant

b

output, y

Figure 1.2: Standard colour conventions used for closed loop block diagrams.

setpoint output y(t)

1

0.5

y(t)

0

Input u(t)

1 0.5 discrete input

0 0

1.2

10

20 time

30

40

Figure 1.3: The standard colour and plot layout for a controlled response used in this book.

M ATLAB FOR COMPUTER AIDED CONTROL DESIGN

Modern control design has heavy computing requirements. In particular one needs to: 1. manipulate symbolic algebraic expressions, and 2. perform intensive numerical calculations and simulations for proto-typing and testing quickly and reliably, and finally 3. to implement the controller at high speed in special hardware such as an embedded controller or a digital signal processing (DSP) chip perhaps using assembler. To use this new theory, it is essential to use computer aided design (CAD) tools efficiently as realworld problems can rarely be solved manually. But as [171] point out, ‘the use of computers in the design of control systems has a long and fairly distinguished history’. This book uses M ATLAB for the design, simulation and prototyping of controllers. M ATLAB, (which is short for MATrix LABoratory), is a programming environment that grew out of an effort to create an easy user-interface to the very popular and well regarded public domain F ORTRAN linear algebra collection of programmes, L INPACK and E ISPACK. With this direct interpretive interface, one can write quite sophisticated algorithms in a very high level language, that are consequently easy to read and maintain. M ATLAB is supported with a variety of toolboxes

CHAPTER 1. INTRODUCTION

4

comprising of collections of source code subroutines organised in areas of specific interest. The toolboxes we are most interested in, and used in this book are: Control toolbox containing functions for controller design, frequency domain analysis, conversions between various models forms, pole placement, optimal control etc. (Used throughout) Symbolic toolbox which contains a gateway to the symbolic capabilities of M APLE. Signal processing toolbox containing filters, wave form generation and spectral analysis. (Used principally in chapter 5.) System identification toolbox for identifying the parameters of various dynamic model types. (Used in chapter 6.) You may also find the following free statistics toolbox useful available at: www.maths.lth.se/matstat/stixbox/, or the R package from [163]. Optimisation which contains routines to maximise or minimise algebraic or dynamic functions. One suitable free toolbox is available from www.i2c2.aut.ac.nz/Wiki/OPTI/. This toolbox is used primarily in chapters 9 and 10. Real-time toolbox can be used to interface M ATLAB to various analogue to digital converters. A free version suitable with Measurement Computing’s MCC DAQs is available from www.i2c2.aut.ac.nz/Resources/Software/SysIDToolbox.html. Additional documentation to that supplied with M ATLAB is the concise and free summary notes [185] or the more recent [69]. Recently there has been exponential growth of other texts that heavily use M ATLAB (such as this one), and a current list is available from the Mathwork’s anonymous ftp server at www.mathworks.com. This server also contains many user contributed codes, as well as updates, bug fixes etc. If M ATLAB, or even programming a high level language is new to you, then [204] is a cheap recommended compendium, similar in form to this, covering topics in numerical analysis, again with many M ATLAB examples.

1.2.1

A LTERNATIVE

COMPUTER DESIGN AIDS

Table 1.1 lists a number of alternatives computer-aided design and modelling environments similar and complimentary to M ATLAB.

Product S CI L AB P YTHON O CTAVE RL AB V ISUAL M ODEL Q M ATH V IEWS M U PAD M APLE M ATHEMATICA

WWW site www.scilab.org code.google.com/p/pythonxy www.octave.org rlabplus.sourceforge.net www.qxdesign.com www.mathwizards.com www.mupad.de www.maplesoft.com www.mathematica.com

comment Free Matlab/Simulink clone Free programming environment Free Matlab clone, inactive Matlab clone, Linux shareware Simulink clone shareware Interfaces with S CI L AB commercial CAS commercial CAS

Table 1.1: Shareware or freeware Matlab lookalikes and computer algebra systems Unlike M ATLAB, symbolic manipulators are computer programs that by manipulating symbols can perform algebra. Such programs are alternatively known as computer algebra systems or

1.3. LABORATORY EQUIPMENT FOR CONTROL TESTS

5

CAS. The most well known examples are M ATHEMATICA, M APLE, M U PAD, and M ACSYMA, (see Table 1.1). These programs can find analytical solutions to many mathematical problems involving integrals, limits, special functions and so forth. They are particularly useful in the controller design stage. The Numerics in Control1 group in Europe have collect together a freeware F ORTRAN subroutine library S LICOT for routines relevant in systems and control. Problem 1.1 1. Familiarise yourself with the fundamentals of M ATLAB. Run the M ATLAB demo by typing demo once inside M ATLAB. 2. Try the M ATLAB tutorial (part 1). 3. Read through the M ATLAB primer, [185] or [69], and you should get acquainted with the M ATLAB users manual.

1.3

L ABORATORY EQUIPMENT FOR CONTROL TESTS

Obviously if we are to study automatic control with the aim to control eventually chemical plants, manufacturing processes, robots, undertake filtering to do active noise cancellation and so forth, we should practice, preferably on simpler, more well understood, and potentially less hazardous equipment. In the Automatic Control Laboratory in the Department of Electrical Engineering at the Karlstad University, Sweden we have a number of simple bench-scale plants to test identification and control algorithms on.

1.3.1 T HE

P LANTS

WITH ONE INPUT AND ONE OUTPUT

BLACKBOX

Fig. 1.4 and Fig. 1.5(a) shows what we perhaps unimaginatively refer to as a “black-box’. It is a box, and it is coloured black. Subjecting the box to an input voltage from 0 to 5 volts delivers an output voltage also spanning from around 0 to 5 volts, but lagging behind the input voltage since the internals of the blackbox are simply either 7 or 9 (depending on the switch position) low-pass passive filters cascaded together. The blackbox is a relatively well behaved underdamped stable system with dominant time constants of around 5 to 10 seconds. Fig. 1.5(b) shows the response of the blackbox to two input steps. The chief disadvantage of using this device for control studies is that the output response is not ‘visible’ to the naked eye, and that we cannot manually introduce disturbances. One complication you can do is to cascade two blackboxes together to modify the dynamics.

E LECTRO - MAGNETIC

BALANCE ARM

The electromagnetic balance arm shown in Fig. 1.6(a) is a fast-acting, highly oscillatory, plant with little noise. The aim is to accurately weigh small samples by measuring the current required to keep the balance arm level, or alternatively just to position the arm at different angles. The output response to a step in input shown in Fig. 1.6(b) indicates how long it would take for the oscillations to die away. 1 The

home page is located at http://www.win.tue.nl/niconet/niconet.html

CHAPTER 1. INTRODUCTION

6 Input

Output B LACK B OX

From computer

To computer

sluggish

D/A

A/D

Fast Input indicator

GND Figure 1.4: Blackbox configuration. The manual switch marked will toggle between either 7 or 9 low-pass filters.

Blackbox step response

0.45 0.4 0.35

ipnut/output

0.3 0.25 0.2 0.15

input output

0.1 0.05 0 −0.05 15

20

25

30

35

40

45

50

time (s)

(a) Black-box wiring to the National Instruments (b) The response of the blackbox to 2 step inputs LabPC terminator.

Figure 1.5: The “Black-box” Step response of the balance arm 1

arm position

0.8 0.6 0.4 0.2 0 −0.2 −0.4 20

30

40

50

30

40

50

60

70

80

90

100

60 70 time ∆T=0.05 seconds

80

90

100

0.3 0.2

input

0.1 0 −0.1 −0.2 20

(a) The electromagnetic balance arm

(b) The response of the arm to a step changes in input.

Figure 1.6: The electromagnetic balance arm

1.3. LABORATORY EQUIPMENT FOR CONTROL TESTS

7

FAN AND F LAPPER Contrary to the balance arm, the fan and flapper in Fig. 1.7 and Fig. 1.8(b) has few dynamics, but significant low-pass filtered measurement noise. The counter weight can be moved along the lever arm to mimic a disturbance variable. An interesting exercise is to place two flapper units in close proximity. The air from one then disturbs the flapper from the other which makes an interacting multivariable plant.

adjustable counter weight angle transducer

θ Input, u(t)

output angle

Motor Fan

Figure 1.7: The fan and flapper with an adjustable counter weight as a disturbance

S TEPPER MOTORS A stepper motor is an example of a totally discrete system.

1.3.2

M ULTI - INPUT

AND MULTI - OUTPUT PLANTS

It is possible to construct multivariable interacting plants by physically locating two plants close to each other. One possibility is to locate two flappers adjacent to each other, another possibility is one flapper and one balance arm. The extent of the interaction can be varied by adjusting the relative position of the two plants.

H ELICOPTER The model helicopter, Fig. 1.9, is an example of a highly unstable, multivariable (3 inputs, 2 outputs) nonlinear, strongly interacting plant. It is a good example of where we must apply control (or crash and burn). Fig. 1.10(b) shows the controlled response using 2 PID controllers to control the direction and altitude. Fig. 1.11(a) shows a 3 dimensional view of the desired and actual flight path. Section 10.2.4 illustrates the control of a different three degree of freedom helicopter.

CHAPTER 1. INTRODUCTION

8

Step response of the flapper 0.35

output

0.3 0.25 0.2 0.15 0.1 0

5

10

15

20

25

30

35

40

45

50

5

10

15

20 25 30 time (sec) dt=0.05

35

40

45

50

0.4 input

0.3 0.2 0.1 0 −0.1 0

(a) The fan/flapper equipment

(b) Step response of the flapper

Figure 1.8: The Fan and flapper.

top rotor

plant outputs

side rotor

moveable counter weight elevation angle azimuth angle

support stand

Figure 1.9: Helicopter plant with 2 degrees of freedom. See also Fig. 1.10(a).

1.3.3

S LOWING

DOWN

S IMULINK

For some applications like the development of PID controllers you want to be able to slow down the S IMULINK simulation to have time to manually introduce step changes, add disturbances, switch from automatic to manual etc. If left alone in simulation mode, S IMULINK will run as fast as possible, but it will slow down when it needs to sub-sample the integrator around discontinuities or periods when the system is very stiff. The S IMULINK Execution Control block allows you to specify that the simulation runs at a multiple of real-time. This is most useful when you want to slow down a simulation, or ensure that it runs at a constant rate.

1.3. LABORATORY EQUIPMENT FOR CONTROL TESTS

9

(a) The ‘flying’ helicopter balanced using 2 PID controllers since it is openloop unstable and would otherwise crash. Helicopter 1.5 1

output

0.5 0 −0.5 −1 −1.5

0

10

20

30

40

50

60

70

80

90

100

0

10

20

30

40

50 Time (sec)

60

70

80

90

100

1

input

0.5

0

−0.5

−1

(b) Multivariable PID control of the helicopter exhibiting mediocre controlled response and severe derivative kick.

Figure 1.10: Multivariable PID control of an unstable helicopter

CHAPTER 1. INTRODUCTION

10 Helicopter flight path

0.5

Up/Down

0

−0.5

−1 1.5 1 0.5 250

0 200

−0.5

150 100

−1 East/West

−1.5

50 0

time

(a) Multivariable helicopter control

(b) Model helicopter in the trees

Figure 1.11: Helicopter flying results The block is available from: http://www.mathworks.com/matlabcentral/fileexchange The implementation is shown in Fig. 1.12 where the Simulation Execution Control block looks like an A/D card but in fact is not connected to anything, although it does export some diagnostic timing information.

Scope1 Simulink Execution Control

1 s+1 Signal Generator

Transfer Fcn

Scope

Figure 1.12: Real-time Simulink simulations. A parameter in the S IMULINK E XECUTION C ON TROL block sets the speed of the simulation. An alternative method to slow down a Simulink simulation and force it to run at some set rate is to use the commercial Humusoft real-time toolbox, but not with the A/D card actually interfaced to anything.

1.4

E CONOMICS OF CONTROL

Most people would agree that Engineers apply technology, but what do these two words really mean? Technology is derived from two Greek words, techne which means skill or art, and logia which means science or study. The interesting point here is that the art component is included. The English language unfortunately confuses the word engine with engineering so that many people have the mistaken view that engineers drive engines (mostly). Actually engineer is derived from the Latin ingeniatorium which means one who is ingenious at devising. A far cry from the relatively simple act of piloting jumbos. An interesting American perspective of the professional engineer and modern technology is given as light reading in [2] and Florman’s The Existential

1.4. ECONOMICS OF CONTROL

11

Pleasures of Engineering, [67]. Chemical engineering is, succinctly put, chemistry constrained by cost. The chemist wants the reaction to proceed, the chemical engineer takes that for granted, but is interested in increasing the rate, or pushing the equilibrium, or in most cases both at the same time. As the incentive to produce better products increases, accompanied by an awareness of potent global competition driving one to reduce costs, process control becomes an important aspect of engineering. Obviously modern computerised process control systems are expensive. They are especially expensive compared with other computers such as office or financial computers because the market is smaller, the environment harsher, the graphical requirements more critical, the duties more varied, and the potential payoffs larger. Process control has at least two main duties to perform; first to ensure that the plant is operated safely (that is protect the plant, environment and people), and second that the product quality is consistent with some customer or regulatory body demanded specifications. There is always a trade off between how much control you apply and the benefits that result. An automobile manufacturer could produce an almost totally indestructible car (ie, tank), but the expense of raw materials required, and the high running costs would certainly deem the project an economic failure. On the other hand, in 1965 Ralph Nader complained in the aptly named Unsafe at Any Speed about the poor quality of American automotive engineering, the lack of controls and the unsafe result. This influential book challenged the balance from between commercial profits and more quality control. A product with less variation in quality may be worth more than a product that has a higher average quality, but more variation. A potentially dangerous production facility that regularly destroys equipment, people or surroundings is not usually tolerated by the licensing authorities.

Safety concerns motivate better control. Fig. 1.13, adapted from [7], gives an industrial perspective of the status of process control in 1994. The techniques are divided into those considered “classical” or traditional, which demand only modest digital online computing power, (if any), little in the way of explicit process models or understanding, and those termed loosely “advanced”. One of the major concerns for the process control engineer is to reduce the output variance. If the variation about the setpoint is small, then the setpoint can be shifted closer to the operating constraint, without increasing the frequency of alarms. Fig. 1.14 demonstrates the ideal case that while popular in the advertising literature, is harder to achieve unambiguously in practice. Many text books in the control field are very vague about the actual configuration of real process control systems used today. Other books that take a more trade oriented approach, are vague about the academic side of the control performance. There are a number of reasons for this. First many texts try hard to describe only the theoretical aspects of digital control, and anything remotely applied is not considered worthy of their attention. Secondly, the control systems are rapidly changing as the cost of micro-processors drop in price, and different programming methods come into flavour. Thirdly many industries are deliberately vague about publishing the details of their control system since they perceive that this information could help their competitors.

CHAPTER 1. INTRODUCTION

12

Advanced control

Optimisation

Direct search

Rule definition

Online simulation

Constraint

Steady state

dynamic methods

single variable

Regulatory

Signal condition

PID algorithm

multivariable

Deadtime

feedforward control

compensation

Basic

analogue control

Field

Valves

PLCs

Transmitters

DCS

Process computer

smart transmitters

Onstream analysers

Traditional control

Figure 1.13: A comparison of traditional vs. advanced process control techniques. Adapted from [7].

upper quality constraint violations

spt #3 loss ($)

spt #2

setpoint #1

process output

advanced control

regulatory

manual control time

Figure 1.14: Economic improvements owing to better control. If the control scheme can reduce the variance, the setpoint can be shifted closer to the operating or quality constraint, thereby decreasing operating costs.

1.4. ECONOMICS OF CONTROL

13

However some information of this type is given in [64, pp131-149] and [18]. One good balance for the practitioner is [145].

14

CHAPTER 1. INTRODUCTION

C HAPTER 2

F ROM

DIFFERENTIAL TO DIFFERENCE

EQUATIONS Excerpt from What a sorry state of affairs, Martin Walker, Guardian Weekly, June 29, 1997.

. . . But then he said something sensible, as he quite often does. . . . “We need to treat individuals as individuals and we need to address discrete problems for what they are, and not presume them to be part of some intractable racial issue.” Gingrich, properly understood, is a national treasure, and not only because he is one of the few Americans who understand the difference between “discreet” and “discrete”.

2.1

C OMPUTER IN THE LOOP

The cost and flexibility advantages of implementing a control scheme in software rather than fabricating it in discrete components today are simply too large to ignore. However by inserting a computer to run the software necessitates that we work with discrete regularly sampled signals. This added complexity, which by the way is more than compensated by the above mentioned advantages, introduces a whole new control discipline, that of discrete control. Fig. 2.1 shows a common configuration of the computer in the loop. For the computer to respond to any outside events, the signals must first be converted from an analogue form to a digital signal, say 1 to 5 Volts, which can, with suitable processing be wired to an input port of the computer’s processor. This device that accomplishes this conversion is called an Analogue to Digital converter or A/D. Similarly, any binary output from the pins on a processor must first be converted to an analogue signal using a Digital to Analogue converter, or D/A. In some microcontrollers, (rather than micro-processors), such as Intel’s 8048 or some versions of the 8051, these converters may be implemented on the microcontroller chip. Digital to analogue conversion is easy and cheap. One simply loads each bit across different resistors, and sums the resultant voltages. The conversion is essentially instantaneous. Analogue to digital is not nearly as easy nor cheap, and this is the reason that the common data acquisition cards you can purchase for your PC will often multiplex the analogue input channel. There are various schemes for the A/D, one using a D/A inside a loop using a binary search algorithm. Obviously this conversion is not instantaneous, although this is not normally considered a problem for process control applications. Any introductory electrical engineering text such as [172] will give further details on the implementation details. 15

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

16

Process control computer setpoint, r(t) A/D

+

Computer

Plant

D/A

output, y(t)

−

Figure 2.1: The computer in the control loop.

2.1.1

S AMPLING

AN ANALOGUE SIGNAL

It is the A/D converter that is the most interesting for our analysis of the discrete control loop. The A/D converter will at periodic intervals defined by the computer’s clock sample the continuous analogue input signal. The value obtained is typically stored in a device called a zeroth-order hold until it is eventually replaced by the new sample collected one sample time later. Given constraints on cost, the A/D converter will only have a limited precision, or limited number of bits in which to store the incoming discrete value. Common A/D cards such as the PCLabs card, [3], use 12 bits giving 212 or slightly over 4 thousand discretisation levels. The residual chopped away is referred to as the quantisation error. For a given number of bits, b, used in the converter, the amplitude quantisation is δ = 2−b Low cost analogue converters may only use 8 bits, while digital audio equipment use between 16 and 18 bits. Fig. 2.2 shows the steps in sampling an analogue signal with a three bit (8 discrete levels), A/D sampler. The dashed stair plot gives an accurate representation of the sampled signal, but owing to the quantisation error, we are left with the solid stair plot. You can reproduce Fig. 2.2 in M ATLAB using the fix command to do the chopping, and stairs to construct the stair plot. While Sampling a continuous signal 8 Analogue Sampled quantisized

7 6 5

Figure 2.2: Sampling an analogue signal (heavy solid) with a three bit (8 discrete levels) A/D converter and zeroth-order hold. The sampled values, •, are chopped to the next lowest discrete integer level giving the sampled and quantisied output

Signal

4 3 2 1 0 −1 −2

0

2

4

6 Sample time, (k∆T)

8

10

12

other types of hold are possible, anything higher than a first-order hold is rarely used.

2.1.2

S ELECTING

A SAMPLE RATE

Once we have decided to implement discrete control, rather than continuous control, we must decided on a reasonable sampling rate. This is a crucial parameter in discrete control systems.

2.1. COMPUTER IN THE LOOP

17

The sample time, (T or sometimes denoted ∆t ), is measured in time units, say seconds or in industrial applications, minutes. The reciprocal of the sample time is the sample frequency, f , and is usually measured in cycles/second or Hertz. The radial or angular velocity (which some confusingly also term frequency) is denoted ω and is measured in radians/second. The interrelationships between these quantities are     ω radians/s 1 cycles = (2.1) f= T second 2π radians/cycle The faster the sampling rate, (the smaller the sampling time, T ), the better our discretised signal approximates the real continuous signal. However, it is uneconomic to sample too fast, as the computing and memory hardware may become too expensive. When selecting an appropriate sampling interval, or sample rate, we should consider the following issues: • The maximum frequency of interest in the signal • The sampling theorem which specifies a lower limit required on the sampling rate to resolve any particular frequency unambiguously. (See §2.1.3 following.) • Any analogue filtering that may be required (to reduce the problem of aliasing) • The cost of the hardware and the speed of the A/D converters. Ogata discusses the selection of a sample time qualitatively in [150, p38]. However for most chemical engineering processes, which are dominated by relatively slow and overdamped processes, the sample time should lie somewhere between ten times the computational delay of the hardware tc and some small fraction of the process dominant time constant τ , say 10tc ≤ T ≤

τ 10

(2.2)

For most chemical engineering applications, the computational delay is negligible compared with the process time constant, (tc → 0), so we often choose T ≈ τ /10. Thus for a simple first order rise, we would expect to have about 20–30 data samples from 0 to 99%. Some may argue that even this sampling rate is too high, and opt for a more conservative (larger) sample time down to τ /6. Note that commonly used guidelines such as presented in Table 22.1 in [181, p535] span a wide range of recommended sample times.

O VERLY FAST SAMPLING Apart from the high cost of fast A/D converters, there is another argument against fast sampling. When one samples a continuous system, the poles of a stable continuous system map to the poles of a stable discrete system as T goes to zero. However the zeros in the LHP of a continuous system may not map to zeros inside the unit disk of a discrete system as T tends to zero. This nonintuitive result could create problems if one samples a system, and then uses the inverse of this system within a one step controller, since now the zeros outside the unit circle become unstable poles inside the controller for small sample times. Note that the inverse continuous system is stable, and the discrete inverse system will be stable for large sample times, and it will only be unstable for small sample times.

2.1.3

T HE

SAMPLING THEOREM AND ALIASES

“Dinsdale was a gentleman. And what’s more he knew how to treat a female impersonator” John Cleese in Monty Python (15-9-1970)

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

18

The sampling theorem gives the conditions necessary to ensure that the inverse sampling procedure is a “one to one” relationship. To demonstrate the potential problems when sampling, consider the case where we have two sinusoidal signals, but at different frequencies.1     7 1 y1 = − sin 2π t and y2 = sin 2π t 8 8 If we sample these two signals relatively rapidly, say at 25Hz (T =40ms) then we can easily see two distinct sine curves. However if we sample at T = 1, we obtain identical points. Aliasing 1

t = [0:0.04:8]'; y1= -sin(2*pi*7/8*t); y2= sin(2*pi/8*t); plot(t,[y1, y2])

0.5 signal

2

As shown opposite, we note that y1 (solid) and y2 (dashed) just happen to coincide at t = 0, 1, 2, 3, . . . (•).

0 −0.5 −1 0

2

4 time (s)

6

8

Consequently, at the slower sampling rate of 1 Hz, we cannot distinguish between the two different signals. This is the phenomenon known as aliasing since one of the frequencies “pretends” to be another. In conclusion, two specific sinusoids of different frequencies can have identical sampled signals. Thus in the act of taking samples from a continuous measurement, we have lost some information. Since we have experienced a problem when we sample too slowly, it is reasonable to ask what the minimum rate is so that no aliasing occurs. This question is answered by the sampling theorem which states:

To recover a signal from its sample, one must sample at least two times a period, or alternatively sample at a rate twice the highest frequency of interest in the signal. Alternatively, the highest frequency we can unambiguously reconstruct for a given sampling rate, 1/T , is half of this, or 1/(2T ). This is called the Nyquist frequency, fN . In the second example above when we sampled at 1 Hz, the sampling radial velocity was ωs = 2π = 6.28 rad/s. This was satisfactory to reconstruct the low frequency signal (f1 = 1/8 Hz) since 2ω1 = 1.58 rad/s. We are sampling faster than this minimum, so we can reconstruct this signal. However for the faster signal (f2 = 7/8 Hz), we cannot reconstruct this signal since 2ω2 = 11.0 rad/s, which is faster than the sampling radial velocity.

2.1.4

D ISCRETE

FREQUENCY

If we sample the continuous signal, x(t) = A cos(ωt) with a sample time of T , x(nT ) = A cos(ωnT ) = A cos(Ωn) 1 This

example was adapted from Franklin & Powell p81

2.1. COMPUTER IN THE LOOP

19

where the digital frequency, Ω, is defined as def

Ω = ωT = 2πf T = 2π

f fs

then the range of analogue frequencies is 0 < f < ∞ while the range of digital frequencies is limited by the Nyquist sampling limit, fs /2 giving the allowable range for the digital frequency as 0≤Ω≤π M ATLAB unfortunately decided on a slightly different standard in the S IGNAL P ROCESSING toolbox. Instead of a range from zero to π, M ATLAB uses a range from zero to 2 where 1 corresponds to half the sampling frequency or the Nyquist frequency. See [44]. In summary:

sample time sampling frequency angular velocity digital frequency

symbol T or ∆t fs = T1 ω = 2πf Ω = ωT = 2πf fs

units s Hz rad/s –

where the allowable ranges are: 0 ≤ω < ∞, 0 ≤Ω ≤ π,

continuous sampled

and Nyquist frequency, fN , and the dimensionless Nyquist frequency, ΩN are: 1 fs = [Hz] 2 2T 2πfN = =π fs

fN = ΩN

It is practically impossible to avoid aliasing problems when sampling, using only digital filters. Almost all measured signals are corrupted by noise, and this noise has usually some high or even infinite frequency components. Thus the noise is not band limited. With this noise, no matter how fast we sample, we will always have some reflection of a higher frequency component that appears as an impostor or alias frequency. If aliasing is still a problem, and you cannot sample at a higher rate, then you can insert a low pass analogue filter between the measurement and the analogue to digital converter (sampler). The analogue filter or in this case known as an anti-aliasing filter, will band-limit the signal, but not corrupt it with any aliasing. Expensive high fidelity audio equipment will still use analogue filters in this capacity. Analogue and digital filters are discussed in more detail in chapter 5. Detecting aliases Consider the trend y(t) in Fig. 2.3 where we wish to estimate the important frequency components of the signal. It is evident that y(t) is comprised of one or two dominating harmonics. The spectral density when sampling at T = 0.7s (• in Fig.2.3) given in the upper trend of Fig. 2.4 exhibits three distinct peaks. These peaks are the principle frequency components of the signal and are obtained by plotting the absolute value of the Fourier transform of the time signal2 ,

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

20

3 continuous Ts=0.7

output

2

Ts=1.05

1

0

Figure 2.3: Part of a noisy time series with unknown frequency components. The continuous underlying signal (solid) is sampled at T = 0.7, (•), and at T = 1.05 s (△).

2

−1

−2

0

1

2

3

4

5

6

7

time (s)

Ts = 0.7 s

power

10

0

10

fN=0.71

2

Ts = 1.05 s

power

10

0

10

fN=0.48 0

0.1

0.2

0.3

0.4 0.5 frequency (Hz)

0.6

0.7

0.8

Figure 2.4: The frequency component of a signal sampled at Ts = 0.7s (upper) and Ts = 1.05s (lower). The Nyquist frequencies for both cases are shown as vertical dashed lines. See also Fig. 2.5. |DF T {y(t)}|. Reading off the peak positions, and for the moment overlooking any potential problems with undersampling, we would expect y(t) to be something like y(t) ≈ sin(2π0.1t) + sin(2π0.5t) + sin(2π0.63t) However in order to construct Fig. 2.4 we had to sample the original time series y(t) possibly introducing spurious frequency content. The Nyquist frequency fN = 1/(2Ts ) is 0.7143 and is shown as a vertical dashed line in Fig. 2.4(top). The power spectrum is reflected in this line, but is not shown in Fig. 2.4. If we were to re-sample the process at a different frequency and re-plot the power density plot then the frequencies that were aliased will move in this second plot. The △ points in Fig. 2.3 are sampled at ∆t = 1.05s with corresponding spectral power plot in Fig. 2.4. The important data from Fig. 2.4 is repeated below.

Curve top bottom

Ts (s) 0.7 1.05

fN (Hz) 0.7143 0.4762

peak 1 0.1 0.1

peak 2 0.50 0.152

peak 3 0.63 0.452

First we note that the low frequency peak (f1 = 0.1Hz) has not shifted from curve a (top) to curve b (bottom), so we would be reasonably confident that f1 = 0.1Hz and is not corrupted by the sampling process. 2 More

about the spectral analysis or the power spectral density of signals comes in chapter 5.

2.2. FINITE DIFFERENCE MODELS

21

However, the other two peaks have shifted, and this shift must be due to the sampling process. Let us hypothesize that f2 = 0.5Hz. If this is the case, then it will appear as an alias in curve b since the Nyquist frequency for curve b (fN (b) = 0.48) is less than the proposed f2 = 0.5, but it will appear in the correct position on curve a. The apparent frequency fˆ2 (b) on curve b will be fˆ2 (b) = 2fN (b) − f2 = 2 × 0.4762 − 0.5 = 0.4524 which corresponds to the third peak on curve b. This would seem to indicate that our hypothesis is correct for f2 . Fig. 2.5 shows this reflection. 2

T = 0.7 s

10 power

s

0

10

f =0.71 N

2

T = 1.05 s

10 power

s

0

10

f =0.48 N

0

0.1

0.2

0.3 0.4 0.5 frequency (Hz)

0.6

0.7

0.8

Figure 2.5: Reflecting the frequency response in the Nyquist frequency from Fig. 2.4 shows why one of the peaks is evident in the frequency spectrum at Ts = 1.05, but what about the other peak?

Now we turn our attention to the final peak. f3 (a) = 0.63 and f2 (b) = 0.152. Let us again try the fact that f3 (a) is the true frequency = 0.63Hz. If this were the case the apparent frequency on curve b would be fˆ3 (b) = 2fN (b) − f2 = 0.3224Hz. There is no peak on curve b at this frequency so our guess is probably wrong. Let us suppose that the peak at f3 (a) is the first harmonic. In that case the true frequency will be f3 = 2fN (a) − fˆ3 (a) = 0.8Hz. Now we check using curve b. If the true third frequency is 0.8Hz, then the apparent frequency on curve b will be fˆ3 (b) = 2fN (b) − f3 = 0.153Hz. We have a peak here which indicates that our guess is a good one. In summary, a reasonable guess for the unknown underlying function is y(t) ≈ sin(2π0.1t) + sin(2π0.5t) + sin(2π0.8t) although we can never be totally sure of the validity of this model. At best we could either resample at a much higher frequency, say fs > 10Hz, or introduce an analogue low pass filter to cut out, or at least substantially reduce, any high frequencies that may be reflected.

2.2

F INITE DIFFERENCE MODELS

To compute complex mathematical operations such as differentiation or integration, on a digital computer, we must first discretise the continuous equation into a discrete form. Discretising is where we convert a continuous time differential equation into a difference equation that is then possible to solve on a digital computer. In control applications, we are often required to solve ordinary differential equations such as dy = f (t, y) dt

(2.3)

for y(t). The derivative can be approximated by the simple backward difference formula yt − yt−T dy ≈ dt T

(2.4)

Figure 2.6: In Nathaniel Hawthorne’s The Scarlet Letter, Hester Prynne is forced to wear a large, red “A” on her chest when she is found guilty of adultery and refuses

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

22

where T is the step size in time or the sampling interval. Provided T is kept sufficiently small, (but not zero) the discrete approximation to the continuous derivative is reasonable. Inserting this approximation, Eqn. 2.4 into our original differential equation, Eqn. 2.3, and rearranging for yt results in the famous Euler backward difference scheme; yt = yt−T + T f (t − T, yt−T )

(2.5)

Such a scheme is called a recurrence scheme since a previous solution, yt−T , is used to calculate yt , and is suitable for computer implementation. The beauty of this crude method of differentiation is the simplicity and versatility. Note that we can discretise almost any function, linear or nonlinear without needing to solve analytically the original differential equation. Of course whether our approximation to the original problem is adequate is a different story and this complex, but important issue is addressed in the field of numerical analysis. Problem 2.1

1. What is the finite difference approximation for a third order derivative?

2. Write down the second order finite difference approximation to τ2

2.2.1

D IFFERENCE

dy d2 y + 2ζτ + y = ku 2 dt dt

EQUATIONS

Difference equations are the discrete counterpart to continuous differential equations. Often the difference equations we are interested in are the result of the discretisation of a continuous differential equation, but sometimes, as in the case below, we may be interested in the difference equation in its own right. Difference equations are much easier to simulate and study in a digital computer than a differential equation for example, since they are already written in a form where we can just step along following a relation xk+1 = f (xk ) rather than resorting to integration. This step by step procedure is also known as a mathematical mapping since we map the old vector xk to a new vector xk+1 .

H E´ NON ’ S

CHAOTIC ATTRACTOR

The system of discrete equations known as H´enon’s attractor, xk+1 = (yk + 1) − 1.4x2k , yk+1 = 0.3xk ,

x0 = 1

(2.6)

y0 = 1

(2.7)

is an interesting example of a discrete mapping which exhibits chaotic behaviour. We will start from point x0 = (x0 , y0 ) = (1, 1) and investigate the subsequent behaviour using S IMULINK. Since H´enon’s attractor is a discrete system, we will use unit delays to shift back from xk+1 → xk and to better visualise the results we will plot an (x, y) phase plot. The S IMULINK simulation is given in Fig. 2.7. Compare this S IMULINK construction (which I have drawn deliberately to flow from right to left) so that it matches the way we read the system equations, Eqn. 2.6-Eqn. 2.7.

2.2. FINITE DIFFERENCE MODELS

23

1.4 Product

Gain2 x

1/z

1

To Workspace

x1

Sum1

y

1/z

0.3

To Workspace2

y1

Constant

XY Graph

Gain

Figure 2.7: H´enon’s attractor, (Eqns 2.6–2.7), modelled in S IMULINK. See also Fig. 2.8.

2 1

x

k

1

Start point

0 −1 yk

0.5

−2 1 0.5 yk

0

0 −0.5

0

5

10

15 20 time counter

(a) The time response

25

30

35

−0.5 −1.5

−1

−0.5

0 x

0.5

1

1.5

k

(b) The (x, y) or phase response

Figure 2.8: The time response (left) and phase plot (right) of H´enon’s attractor as computed by the S IMULINK block diagram in Fig. 2.7.

The time response of x and y, (left figures in Fig. 2.8), of this mapping is not particularly interesting, but the result of the phase plot (without joining the dots), (right figure in Fig. 2.8), is an interesting mapping. Actually what the chaotic attractor means is not important at this stage, this is after all only a demonstration of a difference equation. However the points to note in this exercise are that we never actually integrated any differential equations, but only stepped along using the unit delay blocks in S IMULINK. Consequently these types of simulations are trivial to program, and run very fast in a computer. Another example of a purely discrete system is given in later Fig. 6.10.

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

24

2.3

T HE z TRANSFORM

The sampling of a continuous function f (t) at regular sampling intervals equally spaced T time units apart creates a new function f ∗ (t), f ∗ (t) =

∞ X

n=0

f (kT ) δ(t − kT )

(2.8)

where δ(x) is the Dirac delta function which is defined as being a perfect impulse at x = 0, and zero elsewhere. Note that the actual value of δ(0) is infinity, but the integral of δ(t) is 1.0. Expanding the summation term in Eqn 2.8 gives f ∗ (t) = f0 δ(t) + fT δ(t − T ) + f2T δ(t − 2T ) + f3T δ(t − 3T ) + · · · Now suppose we wish to know the value of the third sampled function f ∗ (t = 3T ). For simplicity we will assume the sample time is T = 1. f ∗ (3) = f0 δ(3) + f1 δ(2) + f2 δ(1) + f3 δ(0) + f4 δ(−1) + · · · | | {z } {z } all zero

all zero

All the terms except for the term containing δ(0) are zero, while δ(0) = ∞. Thus the “value” of the function f ∗ (3) = ∞. Often you will see a graph of f ∗ (t) depicted where the heights of the values at the sample times are the same as the height of the continuous distribution such as sketched in Fig. 2.14. Strictly this is incorrect, as it is the ‘strength’ or integral of f ∗ (t) which is the same as the value of the continuous distribution f (t). I think of the “function” f ∗ (t) as a series of impulses whose integral is equal to the value of the continuous function f (t). If we take the Laplace transform of the sampled function f ∗ (t), we get (∞ ) X ∗ L {f (t)} = L fkT δ(t − kT )

(2.9)

n=0

Now the function fkT is assumed constant so it can be factored out of the Laplace transform, and the Laplace transform of the δ(0) is simply 1.0. If the impulse is delayed kT units, then the Laplace transform is e−kT × 1. Thus Eqn. 2.9 simplifies to L {f ∗ (t)} = where we have defined

∞ X

fkT e−kT s =

n=0

∞ X

fk z −k

k=0 def

z = esT

(2.10)

In summary, the z-transform of the sampled function f ∗ (t) is defined as def

F (z) = Z {f ∗ (t)} =

∞ X

fk z −k

(2.11)

k=0

The function F (z) is an infinite series, but can be written in a closed form if f (t) is a rational function.

2.3.1

z- TRANSFORMS

OF COMMON FUNCTIONS

We can use the definition of the z-transform, Eqn. 2.11, to compute the z-transform of common functions such as steps, ramps, sinusoids etc. In this way we can build for ourselves a table of z-transforms such as those found in many mathematical or engineering handbooks.

2.4. INVERSION OF Z -TRANSFORMS

25

S AMPLED STEP FUNCTION k ≥ 0. The z-transform of s(kT ) following

The unit sampled step function is simply s(kT ) = 1, the definition of Eqn. 2.11 is S(z) =

∞ X

k=0

kn z −k = 1 + 1 · z −1 + 1 · z −2 + 1 · z −3 + · · ·

(2.12)

By using the sum to infinity for a geometric series,3 we obtain the closed form equivalent for Eqn. 2.12 as 1 S(z) = (2.13) 1 − z −1 R AMP

AND EXPONENTIAL FUNCTIONS

For the ramp function, x(k) = n for k ≥ 0, and the exponential function, x(k) = e−an , k ≥ 0, we could go through the same formal analytical procedure, but in this case we can use the ztrans command from the symbolic toolbox in M ATLAB.

The z-transform of the ramp function, Z {k}, using the Symbolic toolbox for M ATLAB is

The of the exponential function,  z-transform Z e−ak is 1

1

>> syms k T >> ztrans(k*T) ans = T*z/(z-1)ˆ2

6

>> syms k T a >> ztrans(exp(-a*k*T)) ans = z/exp(-a*T)/(z/exp(-a*T)-1) >> simplify(ans) ans = z*exp(a*T)/(z*exp(a*T)-1)

In a similar manner we can build our own table of z-transforms of common functions.

F INAL

AND INITIAL VALUE THEOREMS

Tables and properties of z-transforms of common functions are given in many handbooks and texts such as [150, p49,67], but two of the more useful theorems are the initial value and the final value theorems given in Table 2.1. As in the continuous case, the final value theorem is only applicable if the transfer function is stable.

2.4

I NVERSION OF z- TRANSFORMS

The usefulness of the z-transform is that we can do algebraic manipulations on discrete difference equations in the same way we can do algebraic manipulations on differential equations using the 3 The

sum of a geometric series of n terms is a + ar + ar 2 + · · · + ar n−1 =

a (1 − r n ) ; r 6= 1 1−r

and if |r| < 1, then the sum for an infinite number of terms converges to a S∞ = 1−r

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

26

Table 2.1: Comparing final and initial value theorems for continuous and discrete systems

Initial value, limt→0 y(t) Final value, limt→∞ y(t)

Continuous lims→∞ sY (s) lims→0 sY (s)

Discrete lim  z→∞ Y (z)
 limz→1 1 − z −1 Y (z)

Laplace transform. Generally we: 1. Convert to z-transforms, then 2. do some relatively simple algebraic manipulations to make it easier to 3. invert back to the time domain. The final step, the inversion, is the tricky part. The process of obtaining f ∗ (t) back from F (z) is called inverting the z-transform, and is written as f ∗ (t) = Z −1 {F (z)}

(2.14)

Note that only the sampled function is returned f ∗ (t), not the original f (t). Thus the full inversion process to the continuous domain f (t) is a “one to many” operation. There are various ways to invert z-transforms: 1. Use tables of z-transforms and their inverses in handbooks, or use a symbolic manipulator. (See §2.4.1.) 2. Use partial-fraction expansion and tables. In M ATLAB use residue to compute the partial fractions. (See §2.4.2.) 3. Long division for rational polynomial expressions in the discrete domain. In M ATLAB use deconv to do the polynomial division. (See §2.4.3.) 4. Computational approach while suitable for computers, has the disadvantage that the answer is not in a closed form. (See §2.4.4.) 5. Analytical formula from complex variable theory. (Not useful for engineering applications, see §2.4.5.)

2.4.1

I NVERTING z- TRANSFORMS

WITH SYMBOLICALLY

The easiest, but perhaps not the most instructive, way to invert z-transforms is to use a computer algebra package or symbolic manipulator such as the symbolic toolbox or M U PAD. One simply enters the z-transform, then requests the inverse using the iztrans function in a manner similar to the forward direction shown on page 25.

3

>> syms z >> G = (10*z+5)/(z-1)/(z-1/4) G = (10*z+5)/(z-1)/(z-1/4) >> pretty(G) 10 z + 5 -----------------

% Define a symbolic variable z % Construct G(z) = (10z + 5)/(z − 1)/(z − 1/4) % check it

2.4. INVERSION OF Z -TRANSFORMS 8

27

(z - 1) (z - 1/4) >> iztrans(G) % Invert the z-transform, Z −1 {G(z)} ans = 20*kroneckerDelta(n, 0) - 40*(1/4)ˆn + 20

The Kronecker δ function, or kroneckerDelta(n, 0) is a shorthand way of expressing piecewise functions in M ATLAB. The expression kroneckerDelta(n, 0) returns 1 when n = 0, and 0 for all other values. δ(n)

1 0.5 0 −5

0 n

5

This is a rather messy and needlessly complicated artifact due to the fact that the symbolic toolbox does not know that n is defined as positive. Also M ATLAB automatically choose n for the indexing variable, but we might prefer to use k and explicitly inform M ATLAB that k > 0. This gives us a cleaner solution. >> syms z >> syms k positive >> G = (10*z+5)/(z-1)/(z-1/4); 4

9

>> y=iztrans(G,k) y = 20-40*(1/4)ˆk

% Invert to get y[k] = Z −1 {G(z)}

>> limit(y,k,inf) ans = 20

% Compute final value: y[∞]

The last command computed the steady-state by taking the limit of y[k] as k → ∞.

2.4.2

T HE

PARTIAL FRACTION METHOD

Inverting transforms by partial fractions is applicable for both discrete z-transforms and continuous Laplace transforms. In both cases, we find an equivalent expression for the transform in simple terms that are summed together. Hopefully we can then consult a set of tables, such as [150, Table 2.1 p49], to invert each term individually. The inverse of the sum, is the sum of the inverses meaning that these expressions when summed together give the full inversion. The M ATLAB function residue forms partial fractions from a continuous transfer function, although the help file warns that this operation is numerically poorly conditioned. Likewise the routine residuez (that is residue with a trailing ‘z’) is designed to extract partial fractions in a form suitable when using z−transforms. As an example, suppose we wish to invert G(s) =

−4s2 − 58s − 24 −4s2 − 58s − 24 = 3 2 s + 2s − 24s s(s + 6)(s − 4)

using partial fractions noting that it contains no multiple or complex roots. It is easy to spot the pole at s = 0, and the two remaining can be found by synthetic division, or by factorising using the roots([1 2 -24 0]) command. The partial fraction decomposition

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

28 of G(s) can be written as

G(s) =

A B C −4s2 − 58s − 24 = + + s(s + 6)(s − 4) s s+6 s−4

where the coefficients A, B and C can be found either by equating the coefficients or using the ‘cover-up’ rule shown below. −4s2 − 58s − 24 −4s2 − 58s − 24 A= = 1, B= =3 (s + 6)(s − 4) s=0 s(s − 4) s=−6 −4s2 − 58s − 24 C= = −8 s(s + 6) s=4

In M ATLAB we can use the residue command to extract the partial fractions.

4

9

>> B = [-4 -58 -24]; >> A = [ 1 2 -24 0]; >> [r,p,k] = residue(B,A) r = 3 -8 1 p = -6 4 0 k = []

% Numerator of G(s) = (−4s2 − 58s − 24)/(s(s + 6)(s − 4)) % Denominator of G(s) % Find partial fractions % residues (top line)

% factors or poles

% No extra parts

The order of the residue and pole coefficient produced by residue is the same for each, so the partial fraction decomposition is again G(s) =

3 −8 1 −4s2 − 58s − 24 = + + s3 − 2s − 24s s+6 s−4 s

and we can invert each term individually perhaps using tables to obtain the time domain solution g(t) = 3e−6t − 8e4t + 1 If the rational polynomial has repeated roots or complex poles, slight modifications to the procedure are necessary so you may need to consult the help file.

S PECIAL CASES FOR z- TRANSFORMS We can invert z-transforms in a manner similar to Laplace transforms, but if you consult a table of z-transforms and their inverses in standard control textbooks such as Table 2-1 in [151, p49], you will discover that the table is written in terms of factors of the form z/(z + a) or alternatively 1/(1 − z −1 ) rather than say 1/(z + a). This means that we should first perform the partial fraction decomposition on X(z)/z, rather than just X(z) as in the continuous case. An outline to follow is given in Algorithm 2.1. Algorithm 2.1 Inverting z-transforms by partial fractions To invert a z transform, X(z), using partial fractions, do the following:

2.4. INVERSION OF Z -TRANSFORMS

29

1. Divide X(z) by z obtaining X(z)/z. 2. Find the partial fractions of X(z)/z perhaps using residue. 3. Multiply each partial fraction by z to obtain a fraction of the form z/(z + a) 4. Find the inverse of each fraction separately using tables, and sum together to find x(t).

S YMBOLIC

PARTIAL FRACTIONS IN

M ATLAB

Suppose we want to invert

10z + 4z + 3 to g[k] using partial fractions. Unfortunately there is no direct symbolic partial fraction command, but Scott Budge from Utah University found the following sneaky trick which involves first integrating, then differentiating the polynomial. This round-about way works because M ATLAB integrates the expression by internally forming partial fractions and then integrating term by term. Taking the derivative brings you back to the original expression, but now in partial fraction form. G(z) =

2

7

12

z2

>> syms z >> G = 10*z/(zˆ2 + 4*z + 3); >> G= diff(int(G/z)) % Extract partial fractions of G(z)/z G = 5/(z+1)-5/(z+3) >> G=expand(G*z) G = 5z 5z − z+3 5*z/(z+1)-5*z/(z+3) % Gives us G(z)/z = z+1 >> syms k positive; iztrans(G,z,k) ans = -5*(-3)ˆk+5*(-1)ˆk

While this strategy currently works, we should take note that in future versions of the symbolic toolbox this may not continue to work. In the above example we also have the option of using residuez.

2.4.3

L ONG

DIVISION

Consider the special case of the rational polynomial where the denominator is simply 1, c0 + c1 z −1 + c2 z −2 + c3 z −3 + · · · 1 The inverse of this transform is trivial since the time solution at sample number k or t = kT is simply the coefficient of the term z −k . Consequently it follows that f (0) = c0 , f (T ) = c1 , f (2T ) = c2 , . . . , f (kT ) = ck etc. Thus if we have a rational transfer function, all we must do is to synthetically divide out the two polynomials to obtain a single, albeit possibly infinite, polynomial in z. F (z) =

This approach is known as “direct division” [150, p69] and in general will result in an infinite power series in z. As such it is not a particularly elegant closed-form solution, but for most stable systems, the power series can be truncated eventually with little error. So to invert a z-transform using long-division, we:

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

30

1. Convert the z-transform to a (possibly infinite) power series by dividing the denominator into the numerator using long division or deconv. 2. The coefficients of z k are the solutions at the sample times kT . Long division by hand is tedious and error-prone, but we can use the polynomial division capabilities of M ATLAB’s deconv to do this long division automatically. Since deconv only performs “integer” division returning a quotient and remainder, we must fool it into giving us the “infinite” series by padding the numerator with zeros. To invert Y (z) =

z2 + z z2 + z = (z − 1)(z 2 − 1.1z + 1) z 3 − 2.1z 2 + 2.1z − 1

using deconv for the long division, we pad the numerator on the right with say 5 zeros, (that is, we multiply by z 5 ), and then do the division. z7 + z6 2 = z 4 + 3.1z 3 + 4.41z{z + 3.751z + 1.7161} + remainder z 3 − 2.1z 2 + 2.1z − 1 | Q

We are not particularly interested in the remainder polynomial. The more zeros we add, (5 in the above example), the more solution points we get.

3

>> Yn = [1,1,0]; % Numerator of G(z) >> Yd = conv([1,-1],[1 -1.1 1]); % Denominator of G(z) is (z − 1)(z 2 − 1.1z + 1) >> [Q,R] = deconv([Yn,0,0,0,0,0],Yd) % Multiply by z 5 to zero pad, then do long division Q = 1.0000 3.1000 4.4100 3.7510 1.7161 >> dimpulse(Yn,Yd,7);

% Matlab's check

We can also numerically verify the inversion using dimpulse for six samples as shown in Fig. 2.9. 5

Output

4

Figure 2.9: Inverting z-transforms using dimpulse for the first six samples. It is possible also to construct a discrete transfer function object and then use the generic impulse routine. Problem 2.2

3 2 1 0 0

1

2

3 sample #

1. The pulse transfer function of a process is given by Y (z) 4(z + 0.3) = 2 X(z) z − z + 0.4

Calculate the response of y(nT ) to a unit step change in x using long division. 2. Determine the inverse by long division of G(z) G(z) =

z(z + 1) (z − 1)(z 2 − z + 1)

4

5

6

2.4. INVERSION OF Z -TRANSFORMS

2.4.4

C OMPUTATIONAL

31

APPROACH

The computational method is so-called because it is convenient for a computer solution technique such as M ATLAB as opposed to an analytical explicit solution. In the computational approach we convert the z-transform to a difference equation and use a recurrence solution. Consider the following transform (used in the example from page 26) which we wish to invert back to the time domain. 10z + 5 10z −1 + 5z −2 X(z) 10z + 5 = 2 = = U (z) (z − 1)(z − 0.25) z − 1.25z + 0.25 1 − 1.25z −1 + 0.25z −2

(2.15)

where we have coloured the input values blue and the output values red. (We will again use this colour convention in chapter 6). The inverse is equivalent to solving for x(t) when u(t) is an impulse function. The transform can be expanded and written as a difference equation xk = 1.25xk−1 − 0.25xk−2 + 10uk−1 + 5uk−2

(2.16)

Since U (z) is an impulse, U (z) = 1 which means that only u(0) = 1 while all other samples u(k) = 0, k 6= 0. Now we substitute k = 0 to start, and note that u(k) = x(k) = 0 when k < 0 by definition. We now have enough information to compute x(0). x0 = 1.25x−1 − 0.25x−2 + 10u−1 + 5u−2 = 1.25 × 0 − 0.25 × 0 + 10 × 0 + 5 × 0 =0

Continuing we substitute k = 1 into Eqn. 2.16 and using our previously computed x0 and noting that now u0 is nonzero, we can find the next term. x1 = 1.25x0 − 0.25x−1 + 10u0 + 5u−1 = 1.25 × 0 − 0.25 × 0 + 10 × 1 + 5 × 0 = 10 Just to clarify this recurrence process further, we can try the next iteration of Eqn. 2.16 with k = 2, x2 = 1.25x1 − 0.25x0 + 10u1 + 5u0 = 1.25 × 10 − 0.2 × 0 + 10 × 0 + 5× 1 = 17.5 Now we can use the full recurrence relation, (Eqn 2.16), to obtain the solution in a stepwise manner to build up the solution as shown in Table 2.2. All the terms on the right hand side of Eqn 2.16 are either known initially (u(t)), or involve past information, hence it is possible to solve. Note that in this inversion scheme, u(t) can be any known input series. M ATLAB is well suited for this type of computation. To invert Eqn. 2.15 we can use the discrete filter command. We rewrite Eqn. 2.15 in the form of a rational polynomial in z −1 , G(z) =

b0 + b1 z −1 + b2 z −2 1 + a1 z −1 + a2 z −2

where the numerator and denominator are entered as row vectors in decreasing powers of z. We then create an impulse input vector u with say 6 samples and then compute the output of the system

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

32

Table 2.2: The inversion of a z transform using the computation method. Only the first 5 samples and the final value are calculated. The final value is as calculated using the symbolic manipulator earlier on page 27. time (kT ) −2 −1 0 1 2 3 4 .. . ∞

3

8

u(k) 0 0 1 0 0 0 0 .. . 0

x(k) 0 0 0 10 17.5 19.375 19.8438 .. . 20

>> b = [0.0, 10.0, 5.0] % Numerator (10z −1 + 5z −2 ) >> a = [1.0, -1.25, 0.25] % Denominator (1 − 1.25z −1 + 0.25z −2 ) >> u = [1, zeros(1,5)] % impulse input, u(k) = δ(k) u = 1 0 0 0 0 0 >> x = filter(b,a,u) % compute y(k) x = 10.0000 17.5000 19.3750 19.8438 19.9609

19.9902

which gives the same results as Table 2.2. The control toolbox has a special object for linear time invariant (LTI) systems. We can create a discrete system with the transfer function command, tf, and by specifying a sampling time, we imply to M ATLAB that it is a discrete system. 1

6

>> sysd = tf([10 5],[1 -1.25 0.25],1) % System of interest (10z + 5)/(z 2 − 1.25z + 0.25) Transfer function: 10 z + 5 ------------------zˆ2 - 1.25 z + 0.25 Sampling time: 1

11

16

>> y=impulse(sysd) % compute the impulse response y = 0 10.0000 17.5000 19.3750 19.8438 19.9609

Finally, the contour integral method can also be used to invert z-transforms, [150, §3–3], but Seborg et al maintains it is seldom used in engineering practice, [181, p571] because it is fraught with numerical implementation difficulties. These difficulties are also present in the continuous equivalent; some of which are illustrated in the next section.

2.4. INVERSION OF Z -TRANSFORMS

2.4.5

N UMERICALLY

INVERTING THE

33

L APLACE

TRANSFORM

Surprisingly, the numerical inversion of continuous transfer functions is considered far less important than the computation of the inverse of discrete transfer functions. This is fortunate because the numerical inversion of Laplace transforms is devilishly tricky. Furthermore, it is unlikely that you would ever want to numerically invert a Laplace transform in control applications, since most problems involver little more than a rational polynomial with a possible exponential term. For these problems we can easily factorise the polynomial and use partial fractions or use the step orimpulse routines for continuous linear responses. However in the rare cases where we have a particularly unusual F (s) which we wish to convert back to the time domain, we might be tempted to use the analytical expression for the inverse directly Z σ+j∞ 1 F (s) est ds (2.17) f (t) = 2πj σ−j∞ where σ is chosen to be larger than the real part of any singularity of F (s). Eqn. 2.17 is sometimes known as the Bromwich integral or Mellin’s inverse formula. The following example illustrates a straight forward application of Eqn. 2.17 to invert a Laplace transform. However be warned that for all but the most well behaved rational polynomial examples this strategy is not to be recommended as it results in severe numerical roundoff error due to poor conditioning. Suppose we try to numerically invert a simple Laplace transform, F (s) =


3 3 1 − e−5t ←→ f (t) = s(s + 5) 5

with a corresponding simple time solution. We can start by defining an anonymous function of the Laplace transform to be inverted. Fs = @(s) 3.0./(s+5)./s; % Laplace function to invert F (s) = 3/(s(s + 5)) . . . Fsexp = @(s,t) Fs(s).*exp(s*t) % and associated integrand F (s)est

Now since the largest singularity of F (s) is 0, we can choose the contour path safely to be say σ = 0.1. We can also approximate the infinite integration interval by reasonably large numbers. It is convenient that the M ATLAB routine integral works directly in the complex domain.

3

c = 0.1; % Value σ > than any singularities of F (s) a = c-100j; b = c+200j; % Contour path approx: σ − j∞ to σ + j∞ t=linspace(0,8);

8

% time range of interest

% Numerically approximate Eqn. 2.17. ft = 1./(2*pi*1j)*arrayfun(@(t) integral(@(x) Fsexp(x,t),a,b), ti); plot(t,ft) % Compare results in Fig. 2.10(a).

Due to small numerical roundoff however, the returned solution has a small imaginary component, which we can in this instance safely ignore. In this rather benign example, where we know the analytical solution, we can validate the accuracy as shown in Fig. 2.10(a), which it has to be admitted, is not that wonderful. We should also note in the simplistic implementation above we have not adequately validated that our algorithmic choices such as the finite integration limits are appropriate, so we should treat any subsequent result with caution. In fact if you repeat this calculation with a larger value of σ, or a smaller integration range such as say 1 − 10j to 1 + 10j

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

34 0.7

2

0.6

1.5

L−1

0.5 1

n

√ 1 s 2 +1

o

f(t)

f(t)

0.4 0.5

0.3 0 0.2

L 0.1

−1

n

o

3 s(s+5)

−0.5 −1 2 error

error

0 0.02 0 −0.02

0

2

(a) Inverting o a n 3 L−1 s(s+5)

4 t

benign

6

transfer

8

function,

1 0 −1

0

5

10 t

15

20

(b) Inverting a challenging transfer function,  1 −1 √ L s2 +1

Figure 2.10: Numerically inverting Laplace transforms by direct evaluation of the Bromwich integral. Since this strategy is suspectable to considerable numerical errors, it is not to be recommended. then the quality of the solution drops alarmingly. One way to improve the integration accuracy is to use the waypoints option in the integral routine. How the Bromwich integral, Eqn. 2.17, actually works can be illustrated with the somewhat unusual pair √   e− s 1 √ (2.18) ←→ f (t) = erfc F (s) = s 2 t The top two plots in Fig. 2.11(a) plot the two surfaces of the complex function F (s)est (which is the integrand in Eqn. 2.17, the thing we need to integrate) for the specific case when t = 1.5. We need to plot two surfaces, because since s is a complex variable, F (s)est will in general be a complex variable as well. Consequently we can plot the real surface on one plot and the imaginary surface on another. Now to perform the integration in Eqn. 2.17, we need to integrate this complex function along the imaginary axis, or in this case just slightly ‘east’ of the imaginary axis in order to miss the singularity in F (s) at the origin. The two semi-transparent windows vertically cutting the surfaces show the path along which we need to integrate. To better visualise exactly what we are integrating, the plots of the function just along the wall are repeated in the figures below in Fig. 2.11(b). Note how the integrand for ℑ{G(s)} is odd, so the definite integral for symmetric limits will turn out to be zero. However the integral for the real part (left hand plot in Fig. 2.11(b)) is definitely not zero, but clearly seems to converge as s tends towards ±∞i. The total integral will therefore be a pure complex number which is scaled by 1/(2πi) to give a pure real number representing the time domain solution y(t = 1.5). Over the years there have been many attempts to improve the inversion strategy as reviewed in [1], but nearly all of the proposals run into numerical problems of precision. One popular approximation of order M for inverse Laplace transforms introduced by Gaver and subsequently

2.4. INVERSION OF Z -TRANSFORMS

35

(a) The real and imaginary surfaces of the complex function F (s)est where F (s) = e− case when t = 1.5. Note the singularity at the origin. 2.5

√

s /s

for the

0.6

0.4

2

0.2

ℑ(G(s))

ℜ(G(s))

1.5

1

0

−0.2

0.5 −0.4 0

−0.6

−0.5 −5

0 ℑ{s}

−0.8 −5

5

0 ℑ{s}

5

(b) The shaded part represents the integral of F (s)est along the imaginary axis. The real part of is even (so the integral is non-zero), while the imaginary part is odd, so the integral is zero.

Figure 2.11: Visualising the Bromwich integral for F (s) = e−

√ s

/s for the case when t = 1.5.

improved by Stehfest is f (t) = L−1 {F (s)} ≈

  2M ln(2) X k ln(2) wk F t t

(2.19)

k=1

where the weights, wk , are given by wk =

(−1)M+k M!

min(k,M)

X

j=⌊(k+1)/2⌋

j M+1

    M 2j j j j k−j

(2.20)

and the notation ⌊x⌋ means the greatest integer less than or equal to x, [111]. The binomial coeffi
cients nk can be generated in M ATLAB using nchoosek(n,k).

Listing 2.1 implements this algorithm for scalar t. The routine could be made more efficient by pre-computing the weights just once for a given approximation order M so in the common case where t is a vector, we do not waste time recomputing the weights. Note also that the approximation assumes that t is strictly greater than zero, so for t = 0 we could use the initial value theorem. Listing 2.1: Numerically inverting Laplace transforms using the Gaver-Stehfest algorithm

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

36

2

function [y, A] = GSinvLT(Fs,t,M) % Invert a laplace transform, F (s), to f (t) using the Gaver-Stehfest algorithm of order M

12

% Compute weights Ak = f (M ) following Eqn. 2.20 A = zeros(2*M,1); % placeholder for weights for k=1:2*M B=0; for j=floor((k+1)/2):min([k,M]) B=B+jˆ(M+1)*nchoosek(M,j)*nchoosek(2*j,j)*nchoosek(j,k-j); end % j A(k) = (-1)ˆ(M+k)*B/factorial(M); end % k

17

ln2t = log(2)/t; % Now compute solution k = [1:2*M]'; y = ln2t*sum(A.*Fs(k*ln2t)); % Eqn. 2.19 return

7

The performance of this approximation is given in Fig. 2.12 which shows the problem when at√ tempting to invert F (s) = 1/ s2 + 1 with a varying number of terms. The Gaver-Stehfest family of schemes are well known to produce inaccurate results for underdamped systems, [111, p2986], but when we increase the number of terms in an attempt to improve the accuracy, the results deteriorate due to numerical round off. This is because the weights grow rapidly and have alternating signs. Directly applying the Bromwich integral to this transform as shown in Fig. 2.10(b) is no better. While the strategy suggested in [1] seemingly circumvents the problem of precision, it does so by brute-force using multi-precision arithmetic which is hardly elegant. An alternative numerical algorithm is the inverse Laplace transform function invlap function contributed by Karl Hollenbeck from the Technical University of Denmark. You can test this routine using one of the following Laplace transform pairs in Table 2.3. The first two are standard, those following are more unusual. A more challenging collection of test transforms is given in [1]. Table 2.3: A collection of Laplace transform pairs suitable for testing numerical inversion strategies.

Description Easy test Control TF Oscillatory Nasty

F (s)

f (t)

2

1/s 1 (s + 2)(s + 3) 1 √ s2 + 1 1 √ e−1/s s

The numerical solution to L−1

n

Comment

t 1 2

e−t − e−3t J0 (t)

√1 πt




√ cos( 4t),

√1 e−1/s s

o

Bessel function, see Fig. 2.10(b) & Fig. 2.12. See Fig. 2.13.

is compared with the analytical solution from Table 2.3

in Fig. 2.13 using a modest tolerance of 10−2 . The routine splits the time axis up into decades, and inverts each separately which is why we can see some numerical noise starting at t = 10. All of these numerical inversion strategies, starting from the direct application of the Bromwich integral in Eqn. 2.17, the Gaver-Stehfest algorithm and variants and the implementation invlap require some care to get reasonable results. Unfortunately in practical cases it can be difficult to

2.4. INVERSION OF Z -TRANSFORMS

37

1 # of terms = 8

0.5 0 1

# of terms = 16

0.5 0 1

# of terms = 20

0.5 0 1

# of terms = 26

0.5 0

Figure 2.12: Demonstrating the precision problems when numerically inverting the Laplace transform using the Gaver-Stehfest algorithm with a varying number of terms. The exact inversion is the red solid line while the approximate inversion is given by –◦–.

−0.5 0

5 time, t

10

1.5

f(t)

1

L−1

0.5

n

exp(−1/s) √ s

o

0

0 −0.5 10

−2

error

10

−4

10

−6

10

0

5

10

15

20 time

25

30

35

40

Figure 2.13: Testing the numerical inversion √ of the Laplace transform e−1/s / s using the invlap routine.

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

38

choose suitable algorithmic tuning constants, which mean that it is difficult to generate any sort of error bound on the computed curve. samhold

2.5

D ISCRETISING WITH A SAMPLE AND HOLD

In §2.1.1 we saw how the continuous analogue signal must be sampled to be digestible by the process control computer. To analyse this sampling operation, we can approximate the real sampler with an ideal sampler in cascade with a hold element such as given in Fig. 2.14. f ⋆ (t) f (t) b

fh (t)

Hold b

Ideal sampler

continuous signal

sampled signal

sampled & held signal

Figure 2.14: Ideal sampler and zeroth-order hold. When we sample the continuous function f (t), we get a sampled function f ∗ (t), that is a series of spikes that exist only at the sampling instant as shown in the middle plot of Fig. 2.14. The sampled function, f ∗ (t), is undefined for the time between the sampling instants which is inconvenient given that this is the bulk of the time. Obviously one sensible solution is to retain the last sampled value until a new one is collected. During the sample period, the sampled value f ∗ (t) is the same as the last sampled value of the continuous function; f (kT + t) = f (kT ) where 0 ≤ t < T shown diagrammatically in the third plot in Fig. 2.14. Since the last value collected is stored or held, this sampling scheme is referred to as a zeroth-order hold. The zeroth-order refers to the fact that the interpolating function used between adjacent values is just a horizontal line. Higher order holds are possible, but the added expense is not typically justified even given the improved accuracy. We can find the Laplace transform of the zeroth-order hold element by noting that the input is an impulse function, and the output is a rectangular pulse of T duration, and y(t) high, as shown in Fig. 2.15. output

rectangular pulse

y(t)

time, t 0

sample time, T

Figure 2.15: A Rectangular pulse.

2.5. DISCRETISING WITH A SAMPLE AND HOLD

39

Directly applying the definition of the Laplace transform, L{zoh} =

Z

∞

y(t) e−st dt = yt

0

= yt ×

Z

0

1 − e−sT s } | {z

T

1 × e−st dt

zoh

we obtain the transform for the zeroth-order hold. In summary, the transformation of a continuous plant G(s) with zero-order hold is   G(s) (2.21) G(z) = (1 − z −1 ) Z s Note that this is not the same as simply the z-transform of G(s). Suppose we wish to discretise the continuous plant G(s) =

8 s+3

(2.22)

with a zero-order hold at a sample time of T = 0.1 seconds. We can do this analytically using tables, or we can trust it to M ATLAB. First we try analytically and apply Eqn 2.21 to the continuous process. The difficult bit is the z-transformation. We note that     G(s) 8 3 Z = Z (2.23) s 3 s(s + 3) and that this is in the form given in tables of z-transforms such as [150, p50], part of which is repeated here: X(s)

X(z)

a s(s + a)

(1 − e−aT )z −1 (1 − z −1 )(1 − e−aT z −1 )

Inverting using the tables, and substituting in Eqn. 2.21 gives     8 3 G(s) = (1 − z −1 ) · Z G(z) = (1 − z −1 ) · s 3 s(s + 3) 8 (1 − e−3T )z −1 (1 − z −1 ) · 3 (1 − z −1 )(1 − e−3T z −1 ) −0.3 −1 )z 8 (1 − e = · 3 (1 − e−0.3 z −1 ) 0.6912z −1 , for T = 0.1 = 1 − 0.7408z −1 =

(2.24)

We can repeat this conversion numerically using the continuous-to-discrete function, c2d. We first define the continuous system as an LTI object, and then convert to the discrete domain using a zeroth order hold. >>sysc = tf(8,[1 3]) 3

Transfer function: 8

% Continuous system: Gc (s) = 8/(s + 3)

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

40 ----s + 3 8

13

>> sysd = c2d(sysc,0.1,'zoh') % convert to discrete using a zoh Transfer function: 0.6912 ---------z - 0.7408 Sampling time: 0.1

which once again is Eqn. 2.24. In fact we need not specify the ’zoh’ option when constructing a discrete model using c2d or even in S IMULINK, since a zeroth-order hold is employed by default. Methods for doing the conversion symbolically are given next in section 2.5.1.

2.5.1

C ONVERTING L APLACE

TRANSFORMS TO

z- TRANSFORMS

In many practical cases we may already have a continuous transfer function of a plant or controller which we wish to discretise. In these circumstances we would like to convert from a continuous transfer function in s, (such as say a Butterworth filter) to an equivalent discrete transfer function in z. The two common ways to do this are: 1. Analytically by first inverting the Laplace transform back to the time domain, and then take the (forward) z-transform.  def G(z) = Z L−1 {G(s)} (2.25) 2. or by using an approximate method known as the bilinear transform.

The bilinear method whilst approximate has the advantage that it just involves a simple algebraic substitution for s in terms of z. The bilinear transform is further covered in §2.5.2. The formal method we have already seen previously, but since it is such a common operation, we can write a symbolic M ATLAB script to do it automatically for us for any given transfer function. However we should be aware whether the transformation includes the sample and hold, or if that inclusion is left up to the user. Listing 2.2 converts a Laplace expression to a z-transform expression without a zeroth-order hold. Including the zeroth-order hold option is given in Listing 2.3. Listing 2.2: Symbolic Laplace to z-transform conversion

5

function Gz = lap2ztran(G) % Convert symbolically G(s) to G(z) syms t k T z Gz = simplify(ztrans(subs(ilaplace(G),t,k*T),k,z)); return

Listing 2.3: Symbolic Laplace to z-transform conversion with ZOH function Gz = lap2ztranzoh(G) % Convert symbolically G(s) to G(z) with a ZOH.

2.5. DISCRETISING WITH A SAMPLE AND HOLD

5

41

syms s t k T z Gz = simplify((1-1/z)*ztrans(subs(ilaplace(G/s),t,k*T),k,z)); return

We can test the conversion routines in Listings 2.2 and 2.3 on the trial transfer function G(s) = 1/s2 .

5

10

>> Gz =lap2ztran(1/sˆ2); % Convert G(s) = 1/s2 to the discrete domain G(z). >> pretty(Gz) T z -------2 (z - 1) >> Gz = lap2ztranzoh(1/sˆ2);% Do the conversion again, but this time include a ZOH. >> pretty(Gz) 2 (z + 1) T 1/2 ---------2 (z - 1)

2.5.2

T HE

BILINEAR TRANSFORM

The analytical ‘one step back–one step forward’ procedure while strictly correct, is a little tedious, so a simpler, albeit approximate, way to transform between the Laplace domain and the z-domain is to use the bilinear transform method, sometimes known as Tustin’s method. By definition def

z = esT or z −1 = e−sT , (from Eqn. 2.10). If we substituted directly natural logs would appear in the rational polynomial in z, ln(z) s= (2.26) T making the subsequent analysis difficult. For example the resulting expression could not be transformed into a difference equation which is what we desire. We can avoid the troublesome logarithmic terms by using a Pad´e approximation for the exponential term as 2 − sT e−sT = z −1 ≈ (2.27) 2 + sT or alternatively s≈

2 (1 − z −1 ) T (1 + z −1 )

(2.28)

This allows us to transform a continuous time transfer function G(s) to a discrete time transfer function G(z), G(z) = G(s)|s= 2 1−z−1 (2.29) T 1+z −1

Eqn. 2.29 is called the bilinear transform owing to the linear terms in both the numerator and denominator and it has the advantage that a stable continuous time filter will be stable in the discrete domain. The disadvantage is that the algebra required soon becomes unwieldy if attempted manually. Other transforms are possible and are discussed in §4-2 p308 of Ogata [150]. Always remember that this transformation is approximate, being equivalent to a trapezoidal integration.

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

42

Here we wish to approximately discretise the continuous plant G(s) =

1 (s + 1)(s + 2)

at a sample time of T = 0.1 using the bilinear transform. The discrete approximate transfer function is obtained by substituting Eqn. 2.28 for s and simplifying. 1 G(z) = (s + 1)(s + 2) s= 2 1−z−1 T 1+z −1

=  =

2 T

1−z −1 1+z −1

+1

1 

2 T

1−z −1 1+z −1

+2

0.0022z 2 + 0.0043z + 0.0022 z 2 − 1.723z + 0.74

=

T 2 (z + 1)2 1 2 2 2 (2T + 6T + 4)z + (4T 2 − 8)z + 2T 2 − 6T + 4

for T = 0.1

Since this transformation is just an algebraic substitution, it is easy to execute it symbolically in M ATLAB. 1

6

>> >> >> >>

syms s T z G = 1/(s+1)/(s+2); Gd = simplify(subs(G,s,2/T*(1-1/z)/(1+1/z))); % Substitute Eqn. 2.28 for s. pretty(Gd) 2 2 T (z + 1) 1/2 ------------------------------------(2 z - 2 + T z + T) (z - 1 + T z + T)

Alternatively we could use M ATLAB to numerically verify our symbolic solution given that T = 0.1.

2

7

12

>> G = zpk([],[-1 -2],1) Zero/pole/gain: 1 ----------(s+1) (s+2) >> T=0.1; >> Gd = c2d(G,T,'tustin') Zero/pole/gain: 0.0021645 (z+1)ˆ2 --------------------(z-0.9048) (z-0.8182) Sampling time: 0.1 >> tf(Gd)

17

Transfer function: 0.002165 zˆ2 + 0.004329 z + 0.002165 -----------------------------------zˆ2 - 1.723 z + 0.7403 22

Sampling time: 0.1

2.5. DISCRETISING WITH A SAMPLE AND HOLD

43

The bilinear command in the S IGNAL P ROCESSING toolbox automatically performs this mapping from s to z. This is used for example in the design of discrete versions of common analogue filters such as the Butterworth or Chebyshev filters. These are further described in §5.2.3. Problem 2.3 1. Use Tustin’s method (approximate z-transform) to determine the discrete time response of 4 G(s) = (s + 4)(s + 2) to a unit step change in input by long division. The sample time T = 1 and solve for 7 time steps. Compare with the exact solution.

T HE

FREQUENCY RESPONSE CHARACTERISTICS OF A HOLD ELEMENT

The zeroth order hold element modifies the transfer function, so consequently influences the closed loop stability, and frequency characteristics of the discrete process. We can investigate this influence by plotting the discrete Bode and Nyquist plots for a sampled process with and without a zeroth order hold. Suppose we have the continuous process Gp (s) =

1.57 s(s + 1)

(2.30)

which is sampled at a frequency of ω = 4 rad/s. (This corresponds to a sample time of T = π/2 seconds.) First we will ignore the zeroth order hold and transform Eqn 2.30 to the z domain by using tables such as say Table 2–1, p49 #8 in DCS. (1 − e−aT )z −1 (1 − z −1 )(1 − e−aT z −1 ) 1.243z = (z − 1)(z − 0.208)

Gp (z) = 1.57

(2.31) (2.32)

To get the discrete model with the zeroth order hold, we again use Eqn 2.21, and the tables. Doing this, in a similar manner to what was done for Eqn 2.24, we get Gh0 Gp (z) =

1.2215z + 0.7306 (z − 1)(z − 0.208)

(2.33)

Now we can plot the discrete frequency responses using M ATLAB to duplicate the figure given in [109, p649] as shown in Fig. 2.16. In the listing below, I first construct the symbolic discrete transfer function, substitute the sampling time, then convert to a numeric rational polynomial. This rational polynomial is now in a suitable form to be fed to the control toolbox routines such as bode.

2

7

syms s T z G = 1.57/(s*(s+1)) Gd = lap2ztran(G) Gd2 = subs(Gd,T,pi/2) [num,den] = numden(Gd2)

% convert to a z−transform without ZOH % extract top & bottom lines

B= sym2poly(num); A = sym2poly(den); % convert to polynomials Gd = tf(B,A,pi/2)

% construct a transfer function

44

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

To generate the discretisation including the zeroth-order hold, we could use the symbolic lap2ztranzoh routine given in Listing 2.3, or we could use the built-in c2d function. 1

[num,den] = numden(G) % extract numerator & denominator Bc= sym2poly(num); Ac = sym2poly(den); Gc = tf(Bc,Ac) Gdzoh = c2d(Gc,pi/2,'zoh')

6

bode(Gc,Gd,Gdzoh) legend('Continuous','No ZOH','with ZOH')

In this case I have used vanilla Bode function which will automatically recognise the difference between discrete and continuous transfer functions. Note that it also automatically selects both a reasonable frequency spacing, and inserts the Nyquist frequency at fN = 1/2T = 1/π Hz or ωN = 2 rad/s. These Bode plots shown in Fig. 2.16, or the equivalent Nyquist plots, show that the zeroth order hold destabilises the system. The process with the hold has a larger peak resonance, and smaller gain and phase margins. Bode Diagram

Magnitude (dB)

100

50

0

−50

Figure 2.16: The Bode diagram showing the difference between the continuous plant, a discretisation with and without the zeroth-order hold.

Phase (deg)

−100 −90 Continuous No ZOH with ZOH

−135

−180

−225 −2

10

−1

10

0

10 Frequency (rad/sec)

1

10

2

10

Of course we should compare the discrete Bode diagrams with that for the original continuous process. The M ATLAB bode function is in this instance the right tool for the job, but I will construct the plot manually just to demonstrate how trivial it is to substitute s = iω, and compute the magnitude and phase of G(iω) for all frequencies of interest.

2

num = 1.57; den = [1 1 0]; w = logspace(-2,1)'; iw = 1i*w;

% Plant of interest G(s) = 1.57/(s2 + s + 0) % Frequency range 10−2 < ω < 101 rad/s. % s = jω

2.6. DISCRETE ROOT LOCUS DIAGRAMS

7

45

G = polyval(num,iw)./polyval(den,iw); % G(s = jω) loglog(w,abs(G)) % Plot magnitude |G(iω)|, see Fig. 2.17. semilogx(w,angle(G)*180/pi) % and phase φ(iω)

We can compare this frequency response of the continuous system with the discretised version in Fig. 2.16. 10 4

AR

10 2

phase angle, φ [deg]

10 0

10 -2 -80 -100 -120 -140 -160 -180 10 -2

10

-1

10

0

10

frequency ω [rad/s]

2.6

1

Figure 2.17: A frequency response plot of G(s) where s = jω constructed manually without using the bode command.

D ISCRETE ROOT LOCUS DIAGRAMS

The root locus diagram is a classical technique used to study the characteristic response of transfer functions to a single varying parameter such as different controller gains. Traditionally the construction of root locus diagrams required tedious manual algebra. However now modern programs such as M ATLAB can easily construct reliable root locus diagrams which makes the design procedure far more attractable. For many controlled systems, the loop is stable for small values of controller gain Kc , but unstable for a gain above some critical value Ku . It is therefore natural to ask how does the stability (and response) vary with controller gain? The root locus gives this information in a plot form which shows how the poles of a closed loop system change as a function of the controller gain Kc . Given a process transfer function G and a controller Kc Q, the closed loop transfer function is the familiar Gcl =

Kc Q(z)G(z) 1 + Kc Q(z)G(z)

(2.34)

where the stability is dependent on the denominator (characteristic equation) 1 + Kc Q(z)G(z), which is of course itself dependent on the controller gain Kc . Plotting the roots of 1 + Kc Q(z)G(z) = 0

(2.35)

as a function of Kc creates the root locus diagram. In this section4 we will plot the discrete root locus diagram for the process Gp (z) = 4 Adapted

from [71, p124]

0.5(z + 0.6) z 2 (z − 0.4)

(2.36)

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

46

which is a discrete description of a first order process with dead time. We also add a discrete integral controller of the form z Gc (z) = Kc z−1 We wish to investigate the stability of the characteristic equation as a function of controller gain. With no additional arguments apart from the transfer function, rlocus will draw the root locus selecting sensible values for Kc automatically. For this example, I will constrain the plots to have a square aspect ratio, and I will overlay a grid of constant shape factor ζ and natural frequency ωn using the zgrid function. Q = tf([1 0],[1 -1],-1); % controller without gain, Q(z) G = tf(0.5*[1 0.6],[1 -0.4 0 0],-1); % plant, Gp (z) 3

Gol = Q*G; % open loop Q(z)Gp (z) zgrid('new'); xlim([-1.5 1.5]); axis equal rlocus(Gol) % plot root locus, see Fig. 2.18. rlocfind(Gol)

Once the plot such as Fig. 2.18 is on the screen, I can use rlocfind interactively to establish the values of Kc at different critical points in Fig. 2.18. Note that the values obtained are approximate.

Pole description border-line stability cross the ζ = 0.5 line critically damped overdamped, ωn = 36◦

Name ultimate gain design gain breakaway gain

≈ Kc 0.6 0.2 0.097 0.09

Once I select the gain that corresponds to the ζ = 0.5 crossing, I can simulate the closed loop step response.

3

Gol.Ts = 1; % Set sampling time T = 1 Kc = 0.2 % A gain where we expect ζ = 0.5 step(Kc*Gol/(1 + Kc*Gol),50) % Simulate closed loop, see Fig. 2.19.

A comparison of step responses for various controller gains is given in Fig. 2.19. The curve in Fig. 2.19 with Kc = 0.2 where ζ ≈ 0.5 exhibits an overshoot of about 17% which agrees well with the expected value for a second order response with a shape factor of ζ = 0.5. The other curves do not behave exactly as one would expect since our process is not exactly a second order transfer function.

2.7

M ULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS

In the mid 1970s, the western world suffered an ‘oil shock’ when the petroleum producers and consumers alike realised that oil was a scarce, finite and therefore valuable commodity. This had a number of important implications, and one of these in the chemical processing industries was the increased use of integrated heat plants. This integration physically ties separate parts of the plant together and demands a corresponding integrated or ‘plant-wide’ control system. Classical single-input/single-output (SISO) controller design techniques such as transfer functions, frequency analysis or root locus were found to be deficient, and with the accompanying development of affordable computer control systems that could administer hundreds of inputs and

2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS

47

Root Locus

1.5

1

0.1 0.3π/T 0.2 0.3 0.4 0.2π/T 0.5 0.6 0.7 0.1π/T 0.8 0.9

0.7π/T 0.8π/T

Imaginary Axis

0.5

0.9π/T 0

Ultimate gain

0.5π/T 0.6π/T 0.4π/T

Design gain

π/T π/T 0.1π/T

0.9π/T −0.5

0.8π/T

0.2π/T

breakaway gain

0.3π/T

0.7π/T 0.6π/T 0.4π/T 0.5π/T

−1

−1.5

−1.5

−1

−0.5

0

0.5

1

1.5

Real Axis

Figure 2.18: The discrete root locus of Eqn. 2.36 plotted using rlocus. See also Fig. 2.19 for the resultant step responses for various controller gains.

2

1 K =0.1 c

0 2 1 K =0.2 c

0 2

1

0

0

10

20 30 time [s]

Figure 2.19: Various discrete closed loop responses for gains Kc = 0.6, 0.2 and 0.1. Note that with Kc = 0.6 we observe a response on the staK =0.6 c bility boundary, while for Kc = 0.1 we observe a 40 50 critically damped response as anticipated from the root locus diagram given in Fig. 2.18.

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

48

outputs, many of which were interacting, nonlinear and time varying, new tools were required and the systematic approach offered by state space analysis became popular.

Manipulated

Disturbances

Many physical processes are multivariable. A binary distillation column such as given in Fig. 2.20 typically has four inputs; the feed flow and composition and reflux and boil-up rates, and at least four outputs, the product flow and compositions. To control such an interacting system, multivariable control such as state space analysis is necessary. The Wood-Berry distillation column model (Eqn 3.24) and the Newell & Lee evaporator model are other examples of industrial process orientated multivariable models. distillate composition, xD

feed rate

distillate rate, D

feed composition Distillation Column

Tray 5 temperature, T5 Tray 15 temperature, T15

reflux rate Bottoms flowrate, B reboiler heat Plant Inputs

Bottoms composition, xB Plant

Plant Outputs

Figure 2.20: A binary distillation column with multiple inputs, disturbances, and multiple outputs State space analysis only considers first order differential equations. To model higher order systems, one needs only to build systems of first order equations. These equations are conveniently collected, if linear, in one large matrix. The advantage of this approach is a compact representation, and a wide variety of good mathematical and robust numerical tools to analyse such a system.

A FEW WORDS

OF CAUTION

The state space analysis is sometimes referred to as the “modern control theory”, despite the fact that is has been around since the early 1960s. However by the end of the 1970s, the promise of the academic advances made in the previous decade were turning out to be ill-founded, and it was felt that this new theory was ill-equipped to cope with the practical problems of industrial control. In many industries therefore, ‘Advanced Process Control’ attracted a bad smell. Writing a decade still later in 1987, Morari in [138] attempts to rationalise why this disillusionment was around at the time, and whether the subsequent decade of activity alleviated any of the concerns. Morari summarises that commentators such as [70] considered theory such as linear multivariable control theory, (i.e. this chapter) which seemed to promise so much, actually delivered very little ‘and had virtually no impact on industrial practice’. There were other major concerns such as the scarcity of good process models, the increasing importance of operating constraints, operator acceptance etc, but the poor track record of linear multivariable control theory landed top billing. Incidentally, Morari writing almost a decade later still in 1994, [141], revisits the same topic giving the reader a nice linear perspective.

2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS

2.7.1

S TATES

49

AND STATE SPACE

State space equations are just a mathematical equivalent way of writing the original common differential equation. While the vector/matrix construction of the state space approach may initially appear intimidating compared with high order differential equations, it turns out to be more convenient and numerically more robust to manipulate these equations when using programs such as M ATLAB. In addition the state space form can be readily expanded into multivariable systems and even nonlinear systems. The state of a system is the smallest set of values such that knowledge of these values, knowledge of any future inputs and the governing dynamic equations, it is possible to predict everything about the future (and past) output of the system. The state variables are often written as a column vector of length n and denoted x. The state space is the n dimensional coordinate space that all possible state vectors must lie in. The input to a dynamic system is the vector of m input variables, u, that affect the state variables. Historically control engineers further subdivided the input variables into those they could easily and consciously adjust (known as control or manipulated variable inputs), and those variables that may change, but are outside the engineer’s immediate sphere of influence (like the weather) known as disturbance variables. The number of input variables need not be the same as the number of state variables, and indeed m is typically less than n. The state space equations are the n ordinary differential equations that relate the state derivatives to the state themselves, the inputs if any, and time. We can write these equations as x˙ = f (x, u, t),

x(t = 0) = x0 ,

x ∈ ℜn×1

(2.37)

where f (·) is a vector function, the n initial conditions at t = 0 for the system is some known point, x0 , and that the vector x has dimensions (n × 1). In control applications, the input is given by a control law which is often itself a function of the states, u = h(x) (2.38) which if we substitute into Eqn. 2.37, we get the closed loop response x˙ = f (x, h(x), t) which is now no longer an explicit function of the input u. For autonomous closed loop systems there is no explicit dependence on time, so the differential equation is simply x˙ = f (x)

(2.39)

For the continuous linear time invariant case, Eqn. 2.37 simplifies to x˙ = Ax + Bu

(2.40)

xk+1 = Φxk + ∆uk

(2.41)

or in discrete form at time t = kT where we have taken the convention that we use Roman variable names, A, B, for the continuous case, and Greek variable names, Φ, ∆, in the discrete case. (Note that M ATLAB does not follow this convention.) Block diagrams of both the continuous and discrete formulations of the state space model are shown in Fig. 2.21 and both are conceptually very similar. Such a form is suitable to implement state space systems in a simulator such as S IMULINK for example.

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

50

x˙ u(t)

B

+

R

x(t)

uk

xk+1 + z −1

∆

A

xk

Φ

Continuous plant

Discrete plant

Figure 2.21: A block diagram of a state space dynamic system, (a) continuous system: x˙ = Ax + Bu, and (b) discrete system: xk+1 = Φxk + ∆uk . (See also Fig. 2.22.) The output or measurement vector, y, are the p variables that are directly measured from the operating plant, since often the true states cannot themselves be directly measured. These outputs are related to the states by y = g(x), y ∈ ℜp×1 (2.42) or if the measurement relation is linear, then y = Cx.

(2.43)

Sometimes the input may directly affect the output bypassing the dynamic elements, in which case the full linear system in state space is described by x˙ = Ax + Bu y = Cx + Du

(2.44)

which is the standard starting point for the analysis of continuous linear dynamic systems, and is shown in block diagram form in Fig. 2.22.

D

u(t)

x˙ B

+

R

x(t) C

+

y(t)

A

Figure 2.22: A complete block diagram of a continuous state space dynamic system with outputs and direct measurement feed-through, Eqn. 2.44. Note that in Fig. 2.22, the internal state vector (bold lines), has typically more elements than either the input or output vectors (thin lines). The diagram also highlights the fact that as an observer to the plant, we can relatively easily measure our outputs (or appropriately called measurements), we presumably know our inputs to the plant, but we do not necessarily have access to the internal state variables since they are always contained inside the dashed box in Fig. 2.22. Strategies to estimate these hidden internal states is known as state estimation and are described in section 9.5. Fig. 2.22 also introduces a convention we will use later in system identification, where I colour the input signals blue, and the outputs red.

2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS D IMENSIONS

51

OF THE STATE - SPACE DESCRIPTION

It is important to be clear on the dimensions of the standard state-space system description given by Eqn. 2.44. We will take the convention that the state vector x is an (n × 1) column vector, or alternatively written as x ∈ ℜ(n×1) . That means we have n states, m inputs, and p measurements, or x ∈ ℜ(n×1) , u ∈ ℜ(m×1) , u ∈ ℜ(p×1) (2.45) Consequently, the dimensions of the model (transition) matrix, A, the input matrix, B, and the output or measurement matrix, C are A ∈ ℜ(n×n) ,

B ∈ ℜ(n×m) ,

C ∈ ℜ(p×n) ,

D ∈ ℜ(p×m) .

(2.46)

These four matrices in Eqn. 2.44 can be concatenated into one large block thus obtaining a shorthand way of storing these numbers

# of inputs m

n

A

B n # of states

(n × n)

(n × m)

C (p × n)

D (p × m)

p # of measurements

which is often called a packed matrix notation of the quadruplet (A,B,C,D). If you get confused about the appropriate dimensions of A, B etc, then this is a handy check, or alternatively you can use the diagnostic routine abcdchk. Note that in practice, the D matrix is normally the zero matrix since the input does not generally effect the output immediately without first passing through the system dynamics. Following our coloring convention introduced previously, we have coloured the control matrix B blue in the packed matrix because it is associated with the input, coloured the output matrix C red because it is associated with the outputs, and the direct term D purple because it is associated with both inputs and outputs.

2.7.2

C ONVERTING

DIFFERENTIAL EQUATIONS TO STATE SPACE FORM

If we have a collection or system of interlinked differential equations, it is often convenient and succinct to group them together in a vector/matrix collection. Alternatively if we start with an nth order differential equation, for the same reasons it is advisable to rewrite this as a collection of n first order differential equations known as Cauchy form. Given a general linear nth order differential equation y (n) + a1 y (n−1) + a2 y (n−2) + · · · + an−1 y˙ + an y = b0 u(n) + b1 u(n−1) + · · · + bn−1 u˙ + bn u (2.47)

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

52

by inspection, we can create the equivalent transfer function as Y (s) b0 sn + b1 sn−1 + · · · + bn−1 s + bn = n U (s) s + a1 sn−1 + · · · + an−1 s + an

(2.48)

We can cast this transfer function (or alternatively the original differential equation) into state space in a number of equivalent forms. They are equivalent in the sense that the input/output behaviour is identical, but not necessarily the internal state behaviour. The controllable canonical form is    x˙ 1 0 1 0  x˙ 2   0 0 1     ..   .. .. ..  . = . . .     x˙ n−1   0 0 0 −a0 −a1 −a2 x˙ n y=

h

··· ··· .. .



0 0 .. .

··· 1 · · · −an−1

.. . bn−1 − an−1 b0

b n − an b 0

     

x1 x2 .. .

      xn−1 xn





0 0 .. .

      +     0 1



   u   

(2.49)

x1 x2 .. .

 i  b 1 − a1 b 0    xn−1 xn

.. . . · · · ..



    + b0 u  

(2.50)

which is useful when designing pole-placement controllers. The observable canonical form is        0 0 · · · 0 −an b n − an b 0 x˙ 1 x1  x˙ 2   1 0 · · · 0 −an−1   x2   bn−1 − an−1 b0         (2.51)  ..  =  .. . .   ..  +  u .. .. .. ..    .   .    . . . . . . 0

x˙ n

y=



0

0

··· 1

0 ··· 0

and is useful when designing observers.

−a1 

b 1 − a1 b 0

xn

x1 x2 .. .

   1    xn−1 xn



    + b0 u  

(2.52)

If the transfer function defined by Eqn. 2.48 has real and distinct poles, Y (s) b0 sn + b1 sn−1 + · · · + bn−1 s + bn = U (s) (s + p1 )(s + p2 ) · · · (s + pn ) c1 c2 cn = b0 + + + ···+ s + p1 s + p2 s + pn then we can derive an especially elegant state space form     −p1 0 x˙ 1   x˙ 2   −p2       ..  =  . ..   .   0

x˙ n

y=



c1

c2

· · · cn

−pn  x1    x2  ..  .

xn

    



x1 x2 .. . xn





    +  

   + b0 u 

1 1 .. . 1



  u 

(2.53)

(2.54)

2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS

53

that is purely diagonal. As evident from the diagonal system matrix, this system is totally decoupled and possess the best numerical properties for simulation. For systems with repeated roots, or complex roots, or both, the closest we can come to a diagonal form is the block Jordan form. We can interconvert between all these above forms using the M ATLAB canon command. >> G = tf([5 6],[1 2 3 4]); % Define transfer function G = (5s2 + 6s)/(s3 + 2s2 + 3s + 4) 2

>> canon(ss(G),'compan') % Convert TF to the companion or observable canonical form a = 7

x1 x2 x3

x1 0 1 0

x1 x2 x3

u1 1 0 0

y1

x1 0

y1

u1 0

x2 0 0 1

x3 -4 -3 -2

x2 5

x3 -4

b = 12

17

c =

d = 22

These transformations are discussed further in [150, p515]. M ATLAB uses a variation of this form when converting from a transfer function to state space in the tf2ss function.

2.7.3

I NTERCONVERTING BETWEEN STATE SPACE AND TRANSFER FUNCTIONS

The transfer function form and the state space form are, by in large, equivalent, and we can convert from one representation to the other. Starting with our generic state space model, Eqn. 2.40, x˙ = Ax + Bu

(2.55)

with initial condition x(0) = 0 and taking Laplace transforms gives sx(s) = Ax(s) + Bu(s) x(s) (sI − A) = Bu(s)

−1

x(s) = (sI − A) B u(s) | {z } Gp (s)

(2.56)

where Gp (s) is a matrix of expressions in s and is referred to as the transfer function matrix. The M ATLAB command ss2tf (state space to transfer function) performs this conversion. In the common single-input/single output case, x˙ = Ax + Bu y = Cx

54

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

then the single, scalar transfer function is G(s) = C (sI − A)−1 B Now if we express this transfer function as a numerator and a denominator, G(s) = N (s)/D(s), then the denominator polynomial is D(s) = det (sI − A)

(2.57)

which we recognise as the characteristic polynomial for the matrix A. The roots of this polynomial are the poles of G(s) and are given by the eigenvalues of A. The numerator polynomial is N (s) = C adj (sI − A) B

(2.58)

where adj(·) is the adjugate, (or sometimes known as the adjoint), of a matrix. The roots of this polynomial are the zeros of G(s). In M ATLAB we can use the zero command for SISO systems or tzero for the transmission zeros for MIMO systems. It is interesting to note that the stability, oscillation and decay rate are governed only by the poles and hence the system dynamic matrix A, while the zeros are determined by the dynamics as well as the B and C matrices which in turn are a function of the sensor and manipulated variable positions, [87, p46]. State space to transfer function conversion Suppose we wish to convert the following state space description to transfer function form.     −7 10 4 −1 x˙ = x+ u (2.59) −3 4 2 −3 From Eqn. 2.56, the transfer function matrix is defined as −1

Gp (s) = (sI − A) B     −1  1 0 −7 10 4 = s − 0 1 −3 4 2  −1   (s + 7) −10 4 −1 = 3 (s − 4) 2 −3

−1 −3

 (2.60)

Now at this point, the matrix inversion in Eqn. 2.60 becomes involved because of the presence of the symbolic s in the matrix. Inverting the symbolic matrix expression gives the transfer function matrix.    1 s − 4 −10 4 −1 Gp (s) = −3 s + 7 2 −3 (s + 2)(s + 1)   4 s + 26 − 2  s+2 s + 3s + 2   =  2 3(6 + s)  − 2 s+2 s + 3s + 2

We can directly apply Eqn. 2.56 using the symbolic toolbox in M ATLAB.

2

>> syms s >> A = [-7 10; -3 4]; B = [4 -1; 2 -3]; >> G = (s*eye(2)-A)\B % Pulse transfer function Gp (s) = (sI − A)−1 B. G = [ 4/(s+2), -(s+26)/(2+sˆ2+3*s)] [ 2/(s+2), -3*(s+6)/(2+sˆ2+3*s)]

2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS

55

The method for converting from a state space description to a transfer function matrix described by Eqn. 2.56 is not very suitable for numerical computation owing to the symbolic matrix inversion required. However [175, p35] describes a method due to Faddeeva that is suitable for numerical computation in M ATLAB. Faddeeva’s algorithm to convert a state space description, x˙ = Ax + Bu into transfer function form, Gp (s).

1. Compute a, the characteristic equation of the n × n matrix A. (Use poly in M ATLAB.) 2. Set En−1 = I. 3. Compute recursively the following n − 1 matrices En−1−k = AEn−k + an−k I,

k = 1, 2, · · · , n − 1

4. The transfer function matrix is then given by Gp (s) =

sn−1 En−1 + sn−2 En−2 + · · · + E0 B an sn + an−1 sn−1 + · · · + a1 s + a0

(2.61)

We can repeat the state space to transfer function example, Eqn. 2.59, given on page 54 using Faddeeva’s algorithm starting by computing the characteristic polynomial of A a(s) = s2 + 3s + 2 = (s + 1)(s + 2) and with n = 2 we can compute  1 E1 = 0

0 1



,

and

E0 =



5 −1 2 2



Following Eqn. 2.61 gives a matrix of polynomials in s as,      1 1 0 −4 10 (sI − A)−1 = 2 s + 0 1 −3 7 s + 3s + 2   1 s−4 10 = s+7 (s + 1)(s + 2) −3 which we can recognise as the same expression as we found using the symbolic matrix inverse on page 54. All that remains is to post multiply by B to obtain the pulse transfer function matrix. We can implement Faddeeva’s algorithm using the symbolic toolbox as

4

9

syms s a = poly(A); n = size(A,1); as = poly2sym(a,'s'); E = eye(n,n); I = eye(n,n); Gp = sˆ(n-1)*E; for i=n-2:-1:0 E = A*E + a(i+2)*I; Gp = Gp + sˆi*E; end Gp = simplify(Gp*B/as);

% Construct characteristic polynomial, a(s). % En−1 = I

% Eqn. 2.61.

Finally, we can also use the control toolbox for the conversion from state space to transfer function form. First we construct the state space form,

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

56

A = [-7 10; -3,4]; B = [4 -1; 2 -3]; C = eye(size(A)); sys = ss(A,B,C,[])

% Continuous state space example from Eqn. 2.59. % construct system object

where in this case if leave out the D matrix, M ATLAB assumes no direct feed though path. We now can convert to the transfer function form: >> sys_tf = minreal(tf(sys)) % Convert to TF & remember to cancel common factors 2

Transfer function from input 1 to output... 4 #1: ----s + 2 7

#2:

12

2 ----s + 2

Transfer function from input 2 to output... -s - 26 #1: ------------sˆ2 + 3 s + 2

17

#2:

-3 s - 18 ------------sˆ2 + 3 s + 2

Once again we get the same pulse transfer function matrix. Note however that it is also possible to use ss2tf.

T RANSFER

FUNCTION TO STATE SPACE

To convert in the reverse direction, we can use the M ATLAB function tf2ss (transfer function to state space) thus converting an arbitrary transfer function description to the state space format of Eqn 2.40. For example starting with the transfer function G(s) =

s2 + 7s + 12 (s + 3)(s + 4) = 3 (s + 1)(s + 2)(s + 5) s + 8s + 17s + 10

we can convert to state space form using tf2ss. 1

>>num = [1 7 12]; % Numerator: s2 + 7s + 12 >>den = [1 8 17 10]; % Denominator: s3 + 8s + 17s + 10 >>[A,B,C,D] = tf2ss(num,den) % convert to state space

which returns the following four matrices 

 −8 −17 −10 0 0 , A= 1 0 1 0   C = 1 7 12 ,



 1 B= 0  0

D=0

2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS

57

This form is a variation of the controllable canonical form described previously in §2.7.2. This is not the only state space realisation possible however as the C ONTROL T OOLBOX will return a slightly different ABCD package

2

7

12

17

>> Gc = tf([1 7 12],[1 8 17 10]) % Transfer function in polynomial form Transfer function: sˆ2 + 7 s + 12 ----------------------sˆ3 + 8 sˆ2 + 17 s + 10 >>[A,B,C,D] = ssdata(Gc) % extract state space matrices A = -8.0000 -4.2500 -1.2500 4.0000 0 0 0 2.0000 0 B = 2 0 0 C = 0.5000 0.8750 0.7500 D = 0

Alternatively we could start with zero-pole-gain form and obtain yet another equivalent state space form. 1

G = zpk([-3 -4],[-1 -2 -5],1) % Transfer function model in factored format Gss = ss(G)

2.7.4

S IMILARITY

TRANSFORMATIONS

The state space description of a differential equation is not unique. We can transform from one description to another by using a linear invertible transformation such as x = Tz. Geometrically in 2 dimensions, this is equivalent to rotating the axes or plane. When one rotates the axes, the inter-relationship between the states do not change, so the transformation preserves the dynamic model. Suppose we have the dynamic system x˙ = Ax + Bu

(2.62)

which we wish to transform in some manner using the non-singular transformation matrix, T, where x = Tz. Naturally the reverse transformation z = T−1 x exists because we have restricted ourselves to consider only the cases when the inverse of T exists. Writing Eqn. 2.62 in terms of our new variable z we get d(Tz) = ATz + Bu dt z˙ = T−1 ATz + T−1 Bu

(2.63)

Eqn 2.63 and Eqn 2.62 represent the same dynamic system. They have the same eigenvalues hence the similarity transform, but just a different viewpoint. The mapping from A to T−1 AT is called

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

58

a similarity transform and preserves the eigenvalues. These two matrices are said to be similar. The proof of this is detailed in [85, p300] and [150, p513–514]. The usefulness of these types of transformations is that the dynamics of the states are preserved (since the eigenvalues are the same), but the shape and structure of the system has changed. The motivation is that for certain operations (control, estimation, modelling), different shapes are more convenient. A pure, (or nearly), diagonal shape of the A matrix for example has much better numerical properties than a full matrix. This also has the advantage that less computer storage and fewer operations are needed. To convert a system to a diagonal form, we use the transformation matrix T, where the columns of T are the eigenvectors of A. Systems where the model (A) matrix is diagonal are especially easy to manipulate. We can find the eigenvectors T and the eigenvalues e of a matrix A using the eig command, and construct the new transformed, and hopefully diagonal system matrix, [T,e] = eig(A) % find eigenvectors & eigenvalues of A V = T\A*T % New system matrix, T−1 AT, or use canon

Another use is when testing new control or estimation algorithms, it is sometimes instructive to devise non-trivial systems with specified properties. For example you may wish to use as an example a 3 × 3 system that is stable and interacting, and has one over-damped mode and two oscillatory modes. That is we wish to construct a full A matrix with specified eigenvalues. We can use the similarity transformations to obtain these models. Other useful transformations such as the controllable and observable canonical forms are covered in [150, §6–4 p646 ]. The M ATLAB function canon can convert state space models to diagonal or observable canonical form (sometimes known as the companion form). Note however the help file for this routine discourages the use of the companion form due to its numerical ill-conditioning. M ATLAB comes with some utility functions to generate test models such as ord2 which generates stable second order models of a specified damping and natural frequency, and rmodel is a flexible general stable random model builder or arbitrary order.

2.7.5

I NTERCONVERTING

BETWEEN TRANSFER FUNCTIONS FORMS

The previous section described how to convert between different representations of linear dynamic systems such as differential equations, transfer functions and state space descriptions. This section describes the much simpler task of converting between the different ways we can write transfer functions. Modellers tend to think in continuous time systems, G(s), and in terms of process gain and time constants, so will naturally construct transfer functions in factored form as K Πi (αi s + 1) Πj (τj s + 1)

(2.64)

where the all the variables of interest such as time constants τj are immediately apparent. On the other hand, system engineers tend to think in terms of poles and zeros, so naturally construct transfer functions also in factored form, but with a different scaling K ′ Πi (s + zi ) Πj (s + pi )

(2.65)

so where now the poles, pj , and zeros zi are immediately apparent. This is the form that M ATLAB uses in the zeros-pole-gain format, zpk.

2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS

59

Finally the hardware engineer would prefer to operate in the expanded polynomial form, (particularly in discrete cases), where the transfer function is of the form bm sm + bm−1 sm−1 + · · · + b0 sn + an−1 sn−1 + · · · + a0

(2.66)

This is the form that M ATLAB uses in the transfer-function format, tf. Note that the leading coefficient in the denominator is set to 1. As a M ATLAB user, you can define a transfer function that does not follow this convention, but M ATLAB will quietly convert to this normalised form if you type something like G = tf(zpk(G)). The inter-conversions between the forms is not difficult; between expressions Eqn. 2.64 and Eqn. 2.65 simply require some adjusting of the gains and factors, while to convert from Eqn. 2.66 requires one to factorise polynomials. For example, the following three transfer function descriptions are all equivalent −1.5 (s − 0.3333)(s + 0.1) −60s2 + 14s + 2 2 (10s + 1)(−3s + 1) = = (20s + 1)(2s + 1)(s + 1) (s + 1)(s + 0.5)(s + 0.05) 40s3 + 62s2 + 23s + 1 {z } {z } | {z } | | time constant

zero-pole-gain

expanded polynomial

It is trivial to interconvert between the zero-pole-gain, Eqn. 2.65, and the transfer function formats, Eqn. 2.66, in M ATLAB, but it is less easy to convert to the time constant description. Listing 2.4 extracts from an arbitrary transfer function form the time constants, τ , the numerator time constants, α, and the plant gain K. Listing 2.4: Extracting the gain, time constants and numerator time constants from an arbitrary transfer function format Gplant = tf(2*mconv([10 1],[-3 1]),mconv([20 1],[2 1],[1 1])) % TF of interest G = zpk(Gplant); % Convert TF description to zero-pole-gain 3

Kp = G.k; p = cell2mat(G.p); z = cell2mat(G.z); % Extract poles & zeros delay0 = G.iodelay ; % Extract deadtime (if any) 8

tau = sort(-1./p,'descend'); % Convert (& sort) to time constants, 1/(τj s + 1) ntau = sort(-1./z,'descend'); % Convert to numerator time constants, (αi s + 1) K = Kp*prod(tau)/prod(ntau); % Adjust plant gain

We could of course use the control toolbox functions pole and zero to extract the poles and zeros from an arbitrary LTI model.

2.7.6

T HE

STEADY STATE

The steady-state, xss , of a general nonlinear system x˙ = f (x, u) is the point in state space such that all the derivatives are zero, or the solution of 0 = f (xss , u)

(2.67)

If the system is linear, x˙ = Ax + Bu, then the steady state can be evaluated algebraically in closed form xss = −A−1 Bu (2.68) Consequently to solve for the steady-state one could invert the model matrix A but which may be illconditioned or computationally time consuming. Using M ATLAB one should use xss = -A\B*u.

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

60

If A has no inverse, then no (or alternatively infinite) steady states exist. An example of a process that has no steady state could be a tank-flow system that has a pump withdrawing fluid from the outlet at a constant rate independent of liquid height say just exactly balancing an input flow shown in the left hand schematic in Fig. 2.23. If the input flow suddenly increased, then the level will rise until the tank eventually overflows. If instead the tank was drained by a valve partially open at a constant value, then as the level rises, the increased pressure (head) will force more material out through the valve, (right-hand side of Fig. 2.23.) Eventually the system will rise to a new steady state. It may however overflow before the new steady state is reached, but that is a constraint on the physical system that is outside the scope of the simple mathematical description used at this time. Flow in

height, h h

flow out = f(height)

flow out is constant

restriction

constant flow pump

Figure 2.23: Unsteady (left) and steady (right) states for level systems If the system is nonlinear, there is the possibility that multiple steady states may exist. To solve for the steady state of a nonlinear system, one must use a nonlinear algebraic solver such as described in chapter 3. Example The steady state of the differential equation d2 y dy +7 + 12y = 3u 2 dt dt where dy dt = y = 0 at t = 0 and u = 1 , t ≥ 0 can be evaluated using Laplace transforms and the final value theorem. Transforming to Laplace transforms we get s2 Y (s) + 7sY (s) + 12Y (s) = 3U (s) Y (s) 3 3 = 2 = U (s) s + 7s + 12 (s + 3)(s + 4) while for a step input Y (s) =

3 1 · 2 s s + 7s + 12

2.7. MULTIVARIABLE CONTROL AND STATE SPACE ANALYSIS

61

The final value theorem is only applicable if the system is stable. To check, we require that the roots of the denominator, s2 + 7s + 12, lie in the left hand plane, √ −6 ± 72 − 4 × 12 = −4 and − 3 s= 2 Given that both roots have negative real parts, we have verified that our system is stable and we are allowed to apply the final value theorem to solve for the steady-state, y(∞), 3 3 = = 0.25 s2 + 7s + 12 12 Using the state space approach to replicate the above, we first cast the second order differential equation into two first order differential equations using the controllable canonical form given in Eqn. 2.50. Let z1 = y and z2 = dy dt , then     0 1 0 z˙ = z+ u −12 −7 3 lim y(t) = lim sY (s) = lim

t→∞

s→0

s→0

and the steady state is now zss = −A−1 Bu = −



−7/12 −1/12 1 0



0 3



1=



0.25 0



Noting that z1 = y, we see that the steady state is also at 0.25. Furthermore, the derivative term (z2 ) is zero, which is as expected at steady state.

2.7.7

P OLES

AND ZEROS

As mentioned in section 2.7.3, the poles of a state space dynamic system are the roots of the polynomial det(sI − A)−1 and the zeros are the roots of C adj(sI − A)−1 B. The concept of zeros, however in multivariable systems is more complicated than poles. Because forming the adjugate is awkward, it is easier to note the identity C adj(sI − A)−1 B = − det(R(s)) where def

R(s) =



sI − A B C 0



That means we can find the transmission zeros of a state space system by computing generalised eigenvalues of the matrix pencil      −A B −I 0 , , C 0 0 0 as detailed in [87, p55]. Listing 2.5 does this calculation which essentially duplicates tzero in M ATLAB. Listing 2.5: Find the transmission zeros for a MIMO system function [tzeros ] = tzero2(G) % Transmission zeros for a MIMO system (See also tzero.m)

5

10

[A,B,C,D] = ssdata(ss(G)); % convert to state space R1 = [-A, B; C, D]; R2 = [-eye(size(A)), zeros(size(B)); ... zeros(size(C)), zeros(size(D))]; tzeros = eig(R1,R2); % Extract generalised eigenvalues tzeros(isinf(tzeros)) = []; % Drop zeros at ∞ end

62

2.8

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

S OLVING THE VECTOR DIFFERENTIAL EQUATION

Since we can solve differential equations by inverting Laplace transforms, we would expect to be able to solve state space differential equations such as Eqn. 2.40 in a similar manner. If we look at the Laplace transform of a simple linear scalar differential equation, x˙ = ax + bu we find two terms, sX(s) − x0 = aX(s) + bU (s) x0 b X(s) = + U (s) s−a s−a

(2.69)

One of these terms is the response of the system owing to the initial condition, x0 eat , and is called the homogeneous solution, and the other term is due to the particular input we happen to be using. This is called the particular integral, and we must know the form of the input, u(t), before we solve this part of the problem. The total solution is the sum of the homogeneous and particular components.

T HE

HOMOGENEOUS SOLUTION

For the moment we will consider just the homogeneous solution to our vector differential equation. That is, we will assume no driving input, or u(t) = 0. (In the following section we will add in the particular integral due to a non-zero input.) Our vector differential equation, ignoring any input, is simply x˙ = Ax,

x(t = 0) = x0

(2.70)

Taking Laplace transforms, and not forgetting the initial conditions, we have sx(s) − x0 = A x(s)

x(s) (sI − A) = x0 (s)

x(s) = (sI − A)−1 x0

Finally inverting the Laplace transform back to the time domain gives o n x(t) = L−1 (sI − A)−1 x0

(2.71)

Alternatively we can solve Eqn. 2.70 by separating the variables and integrating obtaining x(t) = eAt x0

(2.72)

where the exponent of a matrix, eAt , is itself a matrix of the same size as A. We call this matrix exponential the transition matrix because it transforms the state vector at some initial time x0 to some point in the future, xt . We will give it the symbol Φ(t). The matrix exponential is defined just as in the scalar case as a Taylor series expansion, eAt or Φ(t) = I + At + def

=

∞ X A k tk k=0

k!

A 2 t2 A3 t3 + + ··· 2! 3! (2.73)

although this series expansion method is not recommended as a reliable computational strategy. Better strategies are outlined on page 66.

2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION

63

o n Comparing Eqn. 2.71 with Eqn. 2.72 we can see that the matrix eAt = L−1 (sI − A)−1 .

So to compute the solution, x(t), we need to know the initial condition and a strategy to numerically compute a matrix exponential.

T HE

PARTICULAR SOLUTION

Now we consider the full differential equation with nonzero input x˙ = Ax + Bu. Building on the solution to the homogeneous part in Eqn. 2.72, we get x(t) = Φt x0 +

Z

t

Φt−τ Buτ dτ

(2.74)

0

where now the second term accounts for the particular input vector u. Eqn. 2.74 is not particularly useful as written as both terms are time varying. However the continuous time differential equation can be converted to a discrete time differential equation that is suitable for computer control implementation provided the sampling rate is fixed, and the input is held constant between the sampling intervals. We would like to convert Eqn. 2.40 to the discrete time equivalent, Eqn. 2.41, repeated here xk+1 = Φxk + ∆uk

(2.75)

where xk is the state vector x at time t = kT where T is the sample period. Once the sample period is fixed, then Φ and ∆ are also constant matrices. We have also assumed here that the input vector u is constant, or held, over the sample interval, which is the norm for control applications. So starting with a known xk at time t = kT , we desire the state vector at the next sample time, xk+1 , or Z (k+1)T A T A (k+1)T xk+1 = e xk + e e−Aτ Buτ dt (2.76) kT

But as we have assumed that the input u is constant using a zeroth-order hold between the sampling intervals kT and (k + 1)T , Eqn. 2.76 simplifies to xk+1 = eAT xk +

Z

T

eAλ Buk dλ

(2.77)

0

where λ = T − t. For convenience, we can define two new matrices as def Φ = eAT ! Z T def A λ ∆= e dλ B

(2.78) (2.79)

0

which gives us our desired transformation in the form of Eqn. 2.75. In summary, to discretise x˙ = Ax+Bu at sample interval T , we must compute a matrix exponential, Eqn. 2.78, and integrate a matrix exponential, Eqn. 2.79. Note that Eqn. 2.77 involves no approximation to the continuous differential equation provided the input is constant over the sampling interval. Also note that as the sample time tends to zero, the state transition matrix Φ tends to the identity, I. If the matrix A is non-singular, then   ∆ = eAT − I A−1 B

(2.80)

64

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

is a simpler expression than the more general Eqn. 2.79. Double integrator example. The single-input/single output double integrator system, G(s) = 1/s2 , can be represented in continuous state space using two states as     0 0 1 x˙ = x+ u (2.81) 1 0 0   y= 0 1 x (2.82) At sample time T , the state transition matrix is from Eqn. 2.78,    0 0 1 Φ = eAT = exp = T 0 T

0 1



and the control matrix is given by Eqn. 2.79 T

∆=

Z

T

=

Z

0

0



T =  T2 2

! A λ e dλ B 

 !  1 0 1 dλ λ 1 0    T 0  1   =  T2  0 T 2

For small problems such as this, the symbolic toolbox helps do the computation.

5

10

>> A = [0 0; 1 0]; B = [1; 0]; >> syms T lambda % Symbolic sample time T and λ >> Phi = expm(A*T) % Φ(T ) Phi = [ 1, 0] [ T, 1] >> Delta = int(expm(A*lambda),lambda,0,T)*B % ∆(T ) Delta = [ T] [ 1/2*Tˆ2]

Symbolic example with invertible A matrix. We can discretise the continuous state space system x˙ =



−1.5 −0.5 1 0



x+



1 0



u

analytically at a sample time T by computing matrix exponentials symbolically. >> A = [-3/2, -1/2; 1, 0]; B = [1;0];

5

>> syms T >> Phi = expm(A*T) Phi = [ -exp(-1/2*T)+2*exp(-T), [ -2*exp(-T)+2*exp(-1/2*T),

exp(-T)-exp(-1/2*T)] 2*exp(-1/2*T)-exp(-T)]

Since A is invertible in this example, we can use the simpler Eqn. 2.80

2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION

3

8

65

  >> Delta = (Phi - eye(2))*A\B % ∆ = eAT − I A−1 B Delta = [ 2/(exp(-1/2*T)*exp(-T)-exp(-T)-exp(-1/2*T)+1)*(exp(-T)-exp(-1/2*T))] [ -2*(-exp(-1/2*T)+2*exp(-T)-1)/(exp(-1/2*T)*exp(-T)-exp(-T)-exp(-1/2*T)+1)] >> simplify(Delta) ans = [ 2*exp(-1/2*T)/(exp(-T)-1)] [ -2*(2*exp(-1/2*T)+1)/(exp(-T)-1)]

Of course it is evident from the above example that the symbolic expressions for Φ(T ) and ∆(T ) rapidly become unwieldy for dimensions much larger than about 2. For this reason, analytical expressions are of limited practical worth. The alternative numerical schemes are discussed in the following section.

2.8.1

N UMERICALLY

COMPUTING THE DISCRETE TRANSFORMATION

Calculating numerical values for the matrices Φ and ∆ can be done by hand for small dimensions by converting to a diagonal or Jordan form, or numerically using the exponential of a matrix. Manual calculations are neither advisable nor enjoyable, but [20, p35] mention that if you first compute Z T AT 2 A2 T 3 Ak−1 T k eAτ dτ = IT + Ψ= + + ···+ + ··· 2! 3! k! 0 then

Φ = I + AΨ and ∆ = ΨB

(2.83)

A better approach, at least when using M ATLAB, follows from Eqn. 2.78 and Eqn. 2.79 where dΦ = ΦA = AΦ dt d∆ = ΦB dt These two equations can be concatenated to give    d Φ ∆ Φ = 0 dt 0 I

(2.84) (2.85)



 A B (2.86) 0 0 R R which is in the same form as da/dt = ab. Rearranging this to da/a = b dt leads to the analytical solution      Φ ∆ A B = exp T (2.87) 0 I 0 0 ∆ I

enabling us to extract the required Φ and ∆ matrices provided we can reliably compute the exponential of a matrix. The M ATLAB C ONTROL T OOLBOX routine to convert from continuous to discrete systems, c2d, essentially follows this algorithm. We could try the augmented version of Eqn. 2.87 to compute both Φ and ∆ with one call to the matrix exponential function for the example started on page 64.

2

>> A = [-3/2, -1/2; 1, 0]; B = [1;0]; >> syms T >> [n,m] = size(B);

% extract dimensions of A & B. 

˜ = >> Aa = [A,B;zeros(m,n+m)] % Augmented A matrix, A

A 0

B 0



CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

66

7

12

17

Aa = -1.5 -0.5 1.0 1.0 0.0 0.0 0.0 0.0 0.0 ˜ = exp(AT ˜ ) >> Phi_a = expm(Aa*T); % compute exponential, Φ >> Phi = Phi_a(1:n,1:n) % Extract upper left block of composite solution Phi = [ -exp(-1/2*T)+2*exp(-T), exp(-T)-exp(-1/2*T)] [ -2*exp(-T)+2*exp(-1/2*T), 2*exp(-1/2*T)-exp(-T)] >> Delta = Phi_a(1:n,n+1:end) Delta = [ -2*exp(-T)+2*exp(-1/2*T)] [ 2*exp(-T)-4*exp(-1/2*T)+2]

R ELIABLY COMPUTING

MATRIX EXPONENTIALS

There are in fact several ways to numerically compute a matrix exponential. Ogata gives three computational techniques, [150, pp526–533], and a paper by Cleve Moler (one of the original M ATLAB authors) and Charles Van Loan is titled Nineteen dubious ways to compute the exponential of a matrix5 , and contrasts methods involving approximation theory, differential equations, matrix eigenvalues and others. Of these 19 methods, two found their way into M ATLAB, namely expm, which is the recommended default strategy, and expm1 intended to compute ex − 1 for small x . The one time when matrix exponentials are trivial to compute is when the matrix is diagonal. Physically this implies that the system is totally decoupled since the matrix A is comprised of only diagonal elements and the corresponding exponential matrix is simply the exponent of the individual elements. So given the diagonal matrix   λ1 0 0 D =  0 λ2 0  0 0 λ3 then the matrix exponential is simply



eλ1 D  e = 0 0

0 eλ2 0

 0 0  eλ3

which is trivial and reliable to compute. So one strategy then is to transform our system to a diagonal form (if possible), and then simply find the standard scalar exponential of the individual elements. However some matrices, such as those with multiple eigenvalues, is is impossible to convert to diagonal form, so in those cases the best we can do is convert the matrix to a Jordan block form as described in [150, p527], perhaps using the jordan command from the Symbolic toolbox. However this transformation is very sensitive to numerical roundoff and for that reason is not used for serious computation. For example the matrix   1 1 A= ǫ 1 with ǫ = 0 has a Jordan form of



1 0

1 1



5 SIAM Review, vol. 20, A 1978, pp802–836 and reprinted in [158, pp649–680]. In fact 25 years later, an update was published with some recent developments.

2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION

67

but for ǫ 6= 0, then the Jordan form drastically changes to the diagonal matrix   √ 0√ 1+ ǫ 0 1− ǫ In summary, for serious numerical calculation we should use the matrix exponential function expm. Remember not to confuse finding the exponent of a matrix, expm, with the M ATLAB function exp which simply finds the exponent of all individual elements in the matrix.

2.8.2

U SING M ATLAB

TO DISCRETISE SYSTEMS

All of the complications of discretising continuous systems to their discrete equivalent can be circumvented by using the M ATLAB command c2d which is short for continuous to discrete. Here we need only to pass the continuous system of interest, and the sampling time. As an example we can verify the conversion of the double integrator system shown on page 2.8.

2

G = tf(1,[1 0 0]) % Continuous system G(s) = 1/s2 in transfer function form Gc = ss(G) % Convert to continuous state space Gd = c2d(Gc,2) % Convert to discrete state space with a sample time of T = 2 a =

7

x1 x2

x1 1 2

x1 x2

u1 2 2

y1

x1 0

y1

u1 0

x2 0 1

b = 12

c = 17

x2 1

d =

22

Sampling time (seconds): 2 Discrete-time state space model.

Unlike in the analytical case presented previously, here we must specify a numerical value for the sample time, T . Example: Discretising an underdamped second order system with a sample time of T = 3 following Eqn. 2.87 using M ATLAB to compute the matrix exponential of the augmented system. 1

6

[A,B,C,D] = ord2(3, 0.5); Ts = 3.0;

% generate a second order system % sample time

[na,nb] = size(B); X = expm([A,B; zeros(nb,na+nb)]*Ts); Phi = X(1:na,1:na); Del = X(1:na,na+1:end);

% matrix exponential % Pull off blocks to obtain Φ & ∆.

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

68

We can use M ATLAB to numerically verify the expression found for the discretised double integrator on page 64 at a specific sample time, say T = 4. The double integrator in input/output form is 1 G(s) = 2 s which in packed state space notation (refer page 51) is     0 0 1 A B = 1 0 0  C D 0 1 0

3

8

Using a sample time of T = 4, we can compute     1 0 1 0 Φ= = T 1 4 1     T 4 ∆ = T 2 = 16

>>Gd_ss = c2d(Gc_ss,dt) a = x1 x2 x1 1.000 0 x2 4.000 1.000 b = x1 x2

13

u1 4.000 8.000

c =

2

2

>>Gc = tf(1,[1,0,0])% G(s) = 1/s2 >>Gc_ss = ss(Gc);% state space >>dt = 4; % sample time

y1

18

We can verify this using c2d in M ATLAB as demonstrated in the M ATLAB script opposite.

x1 0

x2 1.000

d = y1

u1 0

Once the two system matrices Φ and ∆ are known, solving the difference equation for further values of k just requires the repeated application of Eqn. 2.41. Starting from known initial conditions x0 , the state vector x at each sample instant is calculated thus x1 = Φx0 + ∆u0 x2 = Φx1 + ∆u1 = Φ2 x0 + Φ∆u0 + ∆u1 x3 = Φx2 + ∆u2 = Φ3 x0 + Φ2 ∆u0 + Φ∆u1 + ∆u2 .. . n

xn = Φ x0 +

n−1 X

Φn−1−k ∆uk

.. . (2.88)

k=0

The M ATLAB function ltitr which stands for linear time invariant time response or the more general dlsim will solve the general linear discrete time model as in 2.88. In special cases such as step or impulse tests, you can use dstep, or dimpulse. Problem 2.4

1. Evaluate An where A=



1 1

1 0



for different values of n. What is so special about the elements of An ? (Hint: find the eigenvalues of A.) 2. What is the determinant of An ? 3. Write a couple of M ATLAB m-functions to convert between the A, B, C and D form and the packed matrix form.

2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION

69

4. Complete the state space to transfer function conversion analytically started in §2.7.3, Eqn. 2.60. Compare your answer with using ss2tf. A submarine example A third-order model of a submarine from [63, p416 Ex9.17] is 

   0 1 0 0 x˙ =  −0.0071 −0.111 0.12  x +  −0.095  u 0 0.07 −0.3 0.072

(2.89)

where the state vector x is defined as

x⊤ =



θ

dθ dt

α



The state θ is the inclination of the submarine and α is the angle of attack above the horizontal. The scalar manipulated variable u is the deflection of the stern plane. We will assume that of the three states, we can only actually measure two of them, θ and α, thus   1 0 0 C= 0 0 1 In this example, we consider just three states x, one manipulated variable u, and two outputs, y. The stability of the open-loop system is given by the eigenvalues of A. In M ATLAB, the command eig(A) returns   −0.0383 + 0.07i eig(A) =  −0.0383 − 0.07i  −0.3343

showing that all eigenvalues have negative real parts, indicating that the submarine is oscillatory, though stable. In addition, the complex conjugate pair indicates that there will be some oscillation to the response of a step change in stern plane.

We can simulate the response of our submarine system (Eqn 2.89) to a step change in stern plane movement, u, using the step command. The smooth plot in Fig. 2.24 shows the result of this continuous simulation, while the ‘staired’ plot shows the result of the discrete simulation using a sample time of T = 5. We see two curves corresponding to the two outputs. Listing 2.6: Submarine simulation A = [0,1,0;-0.0071,-0.111,0.12;0,0.07,-0.3]; B = [0,-0.095,0.072]'; C = [1 0 0;0 0 1]; 3

Gc = ss(A,B,C,0)

% Continuous plant, Gc (s)

dt = 5; % sample time T = 5 (rather coarse) Gd = c2d(Gc,dt) % Create discrete model 8

step(Gc,Gd);

% Do step response and see Fig. 2.24.

Fig. 2.24 affirms that the openloop process response is stable, supporting the eigenvalue analysis. Notice how the step command automatically selected appropriate time scales for the simulation. How did it do this? The steady state of the system given that u = 1 is, using Eqn 2.68, 1

>> u = 1;

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

70

0 G

c

G

−5 x

1

d

−10

−15

0.4

x

2

0.3 0.2 0.1

Figure 2.24: Comparing the continuous and discrete step responses of the submarine model.

6

>>xss = -A\B*u; >>yss = C*xss yss = -9.3239 0.2400

0

0

20

40 time

60

80

% Steady-states given by xss = −ABu. % See steady-state outputs in Fig. 2.24.

which corresponds to the final values given in Fig. 2.24. We can verify the results from the c2d routine using Eqns 2.78 and 2.79, or in this case since A is invertible, Eqn. 2.80. >> Phi = expm(A*5.0); % Don’t forget the ‘m’ in expmatrix()   >> Delta = (Phi-eye(size(A)))*(A\B) % ∆ = eAT − I A−1 B

This script should give the same Φ and ∆ matrices as the c2d function since A is non-singular. Given a discrete or continuous model, we can compute the response to an arbitrary input sequence using lsim: t = [0:dt:100]'; U = sin(t/20); % Arbitrary input U (t) 3

lsim(Gc,U,t) lsim(Gd,U)

% continuous system simulation, y(t) % discrete system simulation, yk

We could explicitly demand a first-order-hold as opposed to the default zeroth order by setting the options for c2d. 1

optn = c2dOptions('Method','foh') % Ask for a first-order hold Gd2 = c2d(Gc,5,optn) % Compare the step response of this with a zeroth-order hold

2.8. SOLVING THE VECTOR DIFFERENTIAL EQUATION

71

Problem 2.5 By using a suitable state transformation, convert the following openloop system x˙ 1 = x2 x˙ 2 = 10 − 2x2 + u where the input u is the following function of the reference r,   u = 10 + 9r − 9 4 x

into a closed loop form suitable for simulation in M ATLAB using the lsim function. (ie: x˙ = Ax + Br) Write down the relevant M ATLAB code segment. (Note: In practice, it is advisable to use lsim whenever possible for speed reasons.) Problem 2.6 1. Write an m-file that returns the state space system in packed matrix form that connects the two dynamic systems G1 and G2 together, Gall = G1 · G2 Assume that G1 and G2 are already given in packed matrix form. 2. (Problem from [184, p126]) Show that the packed matrix form of the inverse of Eqn. 2.44, u = G−1 y, is   A − BD−1 C −BD−1 (2.90) G−1 = D−1 C D−1 assuming D is invertible, and we have the same number of inputs as outputs.

2.8.3

T HE

DISCRETE STATE SPACE EQUATION WITH TIME DELAY

The discrete state space equation, Eqn. 2.40, does not account for any time delay or dead time between the input u and the output x. This makes it difficult to model systems with delay. However, by introducing new variables we can accommodate dead time or time delays. Consider the continuous differential equation, x˙ t = Axt + But−θ (2.91) where θ is the dead time and in this particular case, is exactly equal to 2 sample times (θ = 2T ). The discrete time equivalent to Eqn 2.91 for a two sample time delay is xk+3 = Φxk+2 + ∆uk

(2.92)

which is not quite in our standard state space form of Eqn. 2.41 owing to the difference in time subscripts between u and x. Now let us introduce a new vector of state variables, z where   xk def zk =  xk+1  xk+2

(2.93)

is sometimes known as a shift register or tapped delay line of states. Using this new state vector, we can write the dynamic system, Eqn. 2.92 compactly as     xk+1 Ixk+1  zk+1 =  xk+2  =  Ixk+2 (2.94) xk+3 Φxk+2 + ∆uk

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

72

or in standard state space form as zk+1



0 = 0 0

   I 0 0 0 I  z k +  0  uk 0 Φ ∆

(2.95)

This augmented system, Eqn. 2.95 (with 2 units of dead time) is now larger than the original Eqn. 2.92 given that we have 2n extra states. Furthermore, note that the new augmented transition matrix in Eqn. 2.95 is now no longer of full rank (provided it was in the first place). If we wish to incorporate a delay θ, of β sample times (θ = βT ), then we must introduce β dummy state vectors into the system creating a new state vector z with dimensions (nβ × 1); ⊤  zk = xk xk+1 xk+2 · · · xk+β

The augmented state transition matrix Ξ and augmented control matrix Ψ are of the form     0 I 0 ··· 0 0  0 0 I ··· 0   0       0 0 0 ··· 0   0      Ξ= . . . . , and Ψ =  0  (2.96) . . ...   .. .. ..       ..   0 0 0 ··· I   .  0 0

0 ··· Φ

∆

Now the discrete time equation is

zk+1 = Ξzk + Ψuk

(2.97)

Since all the introduced states are unobservable, we have   xk = I 0 0 · · · 0 zk

and that the output equation is amended to  yk = C I 0

0 ··· 0



zk

(2.98)

(2.99)

If the dead time is not an exact multiple of the sample time, then more sophisticated analysis is required. See [71, p174]. Systems with a large value of dead time become very unwieldy as a dummy vector is required for each multiple of the sample time. This creates large systems with many states that possibly become numerically ill-conditioned and difficult to manipulate. Problem 2.7 1. Simulate the submarine example (Eqn 2.89) but now introducing a dead time of 2 sample time units. 2. Explain how the functions step and dstep manage to guess appropriate sampling rates, and simulation horizons. Under what circumstances will the heuristics fail?

2.9

S TABILITY

Stability is a most desirable characteristic of a control system. As a control designer, you should only design control systems that are stable. For this reason, one must at least be able to analyse a potential control system in order to see if it is indeed theoretically stable at least. Once the controller has been implemented, it may be too late and costly to discover that the system was actually unstable. One definition of stability is that a system of differential equations is stable if the output is bounded for all time for any given bounded input, otherwise it is unstable. This is referred to as bounded input bounded output (BIBO) stability.

2.9. STABILITY

73

While for linear systems, the concept of stability is well defined and relatively straight forward to evaluate6 , this is not the case for nonlinear systems which can exhibit complex behaviour. The next two sections discuss the stability for linear systems. Section 2.9.4 discusses briefly techniques that can be used to assess stability for nonlinear systems. Before any type of stability analysis can be carried out, it is important to define clearly what sort of analysis is desired. For example is a ‘Yes/No’ result acceptable, or do you want to quantify how close to instability the current operating point is? Is the system linear? Are the nonlinearities ‘hard’ or ‘soft’? Fig. 2.25 highlights some of these concerns. System stability

Transfer function LTIs

Nonlinear

Linearise nonpolynomial yes/no

Accuracy ?

Pad´e marginal

Routh array Jury Test

eigenvalues roots of denom. Bode plots Gain & phase margins

Lyapunov Simulate May be hard

Extensive testing?

Figure 2.25: Issues and approaches in assessing system stability

2.9.1

S TABILITY

IN THE CONTINUOUS DOMAIN

The most important requirement for any controller is that the controller is stable. Here we describe how one can analyse a continuous transfer function to determine whether it is stable or not. First recall the conditions for BIBO stability. In the Laplace domain, the transfer function system is stable if all the poles (roots of the denominator) have negative real parts. Thus if the roots of the denominator are plotted on the complex plane, then they must all lie to the left of the imaginary axis or in the Left Hand Plane (LHP). In other words, the time domain solution of the differential equation must contain only e−at terms and no terms of the form e+at , assuming a is positive. Why is the stability only determined by the denominator of the transfer function and not by the numerator or the input? 1. First the input is assumed bounded and stable (by BIBO definition). 6 Actually, the stability criterion can be divided into 3 categories; Stable, unstable and critically (un)stable. Physically critical stability never really occurs in practice.

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

74

2. If the transfer function is expanded as a sum of partial fractions, then the time solution terms will be comprised of elements that are summed together. For the system to be unstable, at least one of these individual terms must be unstable. Conversely if all the terms are stable themselves, then the summation of these terms will also be stable. The input will also contribute separate fractions that are summed, but these are assumed to be stable by definition, (part 1). Note that it is assumed for the purposes for this analysis, that the transfer function is written in its simplest form, that it is physically realisable, and that any time delays are expanded as a Pad´e approximation. In summary to establish the stability of a transfer function, we could factorise the denominator polynomial using a computer program to compute numerically the roots such as M ATLAB’s roots routine. Alternatively, we can just hunt for the signs of the real part of the roots, a much simpler operation, and this is the approach taken by the Routh and Jury tests described next.

R OUTH STABILITY

CRITERION

The easiest way to establish the absolute stability of a transfer function is simply to extract the roots of the denominator perhaps using M ATLAB ’ S root command. In fact this is an overkill since to establish absolute stability we need only to know the presence of any roots with positive real parts, not the actual value. Aside from efficiency, there are two cases where this strategy falls down. One is where we do not have access to M ATLAB or the polynomial is particularly ill-conditioned such that simple root extraction techniques fail for numerical reasons, and the other case is where we may have a free parameter such as a controller gain in the transfer function. This has prompted the development of simpler exact algebraic methods such as the Routh array or Lyapunov’s method to assess stability. The Routh criterion states that all the roots of the polynomial characteristic equation, an sn + an−1 sn−1 + an−2 sn−2 + an−3 sn−3 + · · · + a1 s + a0 = 0

(2.100)

have negative real parts if and only if the first column of the Routh table (as described below) have the same sign. Otherwise the number of sign changes is equal to the number of right-hand plane roots. The first two rows of the Routh table is simply formed by listing the coefficients of the characteristic polynomial, Eqn. 2.100, in the manner following sn an an−2 an−4 · · · sn−1 an−1 an−3 an−5 · · · sn−2 b1 b2 b3 ··· c2 c3 ··· sn−3 c1 .. .. .. . . . and where the new entries b and c in the subsequent rows are defined as an−1 an−2 − an an−3 , an−1 b1 an−3 − an−1 b2 c1 = , b1

b1 =

an−1 an−4 − an an−5 , an−1 b1 an−5 − an−1 b3 c2 = , b1

b2 =

The Routh table is continued until only zeros remain.

etc. etc.

2.9. STABILITY

75

The Routh array is only applicable for continuous time systems, but for discrete time systems, the Jury test can be used in a manner similar to the Routh table for continuous systems. The construction of the Jury table is slightly more complicated than the Routh table and is described in [61, 117–118]. The Routh array is most useful when investigating the stability as a function of a variable such as controller gain. Rivera-Santos has written a M ATLAB routine to generate the Routh Array with the possibility of including symbolic variables.7 Suppose we wish to determine the range of stability for a closed loop transfer function with characteristic equation s4 + 3s3 + 3s2 + 2s + K = 0 as a function of the unknown gain K.8 Listing 2.7: Example of the Routh array using the symbolic toolbox

3

8

>> syms K % Define symbolic gain K >> ra = routh([1 3 3 2 K],K) % Build Routh array for s4 + 3s3 + 3s2 + 2s + K ra = [ 1, 3, K] [ 3, 2, 0] [ 7/3, K, 0] [ -9/7*K+2, 0, 0] [ K, 0, 0]

Since all the elements in the first column of the table must be positive, we know that for stability −(9/7)K + 2 > 0 and that K > 0, or 0 < K < 14/7

S TABILITY AND

TIME DELAYS

Time delay or dead time does not affect the stability of the open loop response since it does not change the shape of the output curve, but only shifts it to the right. This is due to the nonpolynomial term e−sθ in the numerator of the transfer function. Hence dead time can be ignored for stability considerations for the open loop. However in the closed loop the dead time term appears in both the numerator and the denominator and now does affect the stability characteristics and can now no longer be ignored. Since dead time is a nonpolynomial term, we cannot simply find the roots of the denominator, since now there are an infinite number of them, but instead we must approximate the exponential term with a truncated polynomial such as a Pad´e approximation and then apply either a Routh array in the continuous time case, or a Jury test in the discrete time case. Note however that the Nyquist stability criteria yields exact results, even for those systems with time delays. The drawback is that the computation is tedious without a computer. Problem 2.8 Given G(s) =

15(4s + 1)e−θs (3s + 1)(7s + 1)

Find the value of deadtime θ such that the closed-loop is just stable using both a Pade´ approximation and a Bode or Nyquist diagram. 7 The routine, routh.m, is available from www.mathworks.com/matlabcentral/fileexchange/. 8 Problem adapted from [152, p288]

the

MathWorks

user’s

group

collection

at

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

76

A first order Pade´ approximation is e−θs =

2.9.2

S TABILITY

1 − 2θ s 1 + 2θ s

OF THE CLOSED LOOP

Prior to the days of cheap computers that can readily manipulate polynomial expressions and extract the roots of polynomials, one could infer the stability of the closed loop from a Bode or Nyquist diagram of the open loop. The openloop system G(s) =

100(s + 2) e−0.6s (s + 5)(s + 4)(s + 3)

(2.101)

is clearly stable, but the closed loop system, G/(1 + G) may, or may not be stable. If we substitute s = iω, and compute the complex quantity G(iω) as a function of ω, we can apply either the Bode stability criteria, or the equivalent Nyquist stability criteria to establish the stability of the closed loop without deriving an expression for the closed loop, and subsequently solving for the closed loop poles. The Bode diagram consists of two plots; the magnitude of G(iω) versus ω, and the phase of G(iω) versus ω. Alternatively we could plot the real and imaginary parts of G(iω) as a function of frequency, but this results in a three dimensional plot. Such a plot for the system given in Eqn. 2.101 is given in Fig. 2.26(a).

(a) The Nyquist diagram with frequency information presented in the third dimension. A ‘flagpole’ is drawn at (−1, 0i). Since the curve encircles the flagpole, the closed loop is unstable.

(b) A 2D Nyquist diagram which shows the closed loop is unstable since it encircles the (−1, 0i) point. In this case both positive and negative frequencies are plotted.

Figure 2.26: Nyquist diagram of Eqn. 2.101 in (a) three dimensions and (b) as typically presented in two dimensions. However three dimensional plots are difficult to manipulate, so we normally ignore the frequency component, and just view the shadow of the plot on the real/imaginary plane. The two dimensional Nyquist curve corresponding to Fig. 2.26(a) is given in Fig. 2.26(b). It is evident from either plot that the curve does encircle the critical (0, −1i) point, so the closed loop system will be unstable. Establishing the closed loop stability using the Nyquist criteria is slightly more general than when using the Bode criteria. This is because for systems that exhibit a non-monotonically decreasing curves may cross the critical lines more than once leading to misleading results. An interesting example from [80] is use to illustrate this potential problem.

2.9. STABILITY

2.9.3

S TABILITY

77 OF DISCRETE TIME SYSTEMS

The definition of stability for a discrete time system is similar to that of the continuous time system except that the definition is only concerned with the values of the output and input at the sample times. Thus it is possible for a system to be stable at the sample points, but be unstable between the sample points. This is called a hidden oscillation, although rarely occurs in practice. Also note that the stability of a discrete time system is dependent on the sample time, T . Recall that at sample time T , by definition z = exp(sT ), the poles of the continuous transfer function pi map to exp(pi T ) in discrete space. If pi is negative, then exp(pi T ) will be less than 1.0. If pi is positive, then the corresponding discrete pole zi will be larger than 1.0. Strictly if pi is complex, then it is the magnitude of zi that must be less than 1.0 for the system to be stable. In summary, the discrete time transfer function G(z) is stable if G(z) has no discrete poles on, or outside, the unit circle. This circle of radius 1 is called the unit circle. Now the evaluation of stability procedure is the same as that for the continuous case. One simply factorises the characteristic equation of the transfer function and inspects the roots. If any of the roots zi lie outside the unit circle, the system will be unstable. If any of the poles lie exactly on the unit circle, there can be some ambiguity about the stability, see [119, p120]. For example the transfer function which has poles on the unit circle,

G(z) =

1 z2 + 1

given a bounded input such as a step, u(k) = {1, 1, 1, 1 . . .} produces y(k) = {0, 0, 1, 1, 0, 0, 1, 1, 0, 0, . . .} which is bounded and stable. However if we try a different bounded input such as a cycle u(k) = {1, 0, −1, 0, 1, 0, −1, . . .} we observe y(k) = {0, 0, 1, 0, −2, 0, 3, 0, −4, 0, 5, 0, −6, 0, . . .} which is unbounded. Just as in the continuous case, we can establish the absolute stability of a discrete transfer function without explicitly solving for the roots of the denominator D(z) = 0, which, in the days before computers was a main consideration. The main methods for discrete systems are the Jury test, the bilinear transform coupled with the Routh array, and Lyapunov’s method. All these methods are analytical and hence exact. The Jury test is the discrete equivalent to the Routh array and is covered in Ogata [150, p242]. Lyapunov’s method has the special distinction that it is also applicable to nonlinear ODEs and is covered in §2.9.4. Suppose we wish to establish the stability of the discretised version of

G(s) =

1 6s2 + 5s + 1

at a sampling period of T = 2 and a first-order hold.

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

78

The discrete transfer function is G(z) =

0.0746 + 0.2005z −1 + 0.0324z −2 1 − 0.8813z −1 + 0.1889z −2

2

with discrete poles given by the solution of 1 − 0.8813z −1 + 0.1889z −2 = 0 or z=



7

>> G = tf(1,[6 5 1]); >> Gd = c2d(G,2,'foh') >> pole(Gd) ans = 0.5134 0.3679 >> pzmap(Gd); axis('equal');

0.5134 0.3679

Since both discrete poles lie inside the unit circle, the discretised transfer function with a firstorder hold is still stable. If you are using the state space description, then simply check the eigenvalues of the homogeneous part of the equation (the A or Φ matrix).

2.9.4

S TABILITY

OF NONLINEAR DIFFERENTIAL EQUATIONS

Despite the fact that nonlinear differential equations are in general very difficult, if not impossible to solve, one can attempt to establish the stability without needing to find the solution. Studies of this sort fall into the realm of nonlinear systems analysis which demands a high degree of mathematical insight and competence. [189] is a good introductory text for this subject that avoids much of the sophisticated mathematics. Two methods due to the Russian mathematician Lyapunov9 address this nonlinear system stability problem. The indirect or linearisation method is based on the fact that the stability near the equilibrium point will be closely approximated to the linearised approximation at the point. However it is the second method, or direct method that being exact is a far more interesting and powerful analysis. Since the Lyapunov stability method is applicable for the general nonlinear differential equation, it is of course applicable for linear systems as well.

Figure 2.27: “Alexander Mikhailovich Liapunov (1857–1918) was a Russian mathematician and mechanical engineer. He had the very rare merit of producing a doctoral dissertation of lasting value. This classic work was originally published in 1892 in Russian but is now available in an English translation Stability of Motion, Academic Press, NY, 1966. Liapunov died by violence in Odessa, which cannot be considered a surprising fate for a middle class intellectual in the chaotic aftermath of the Russian revolution.” Excerpt from Differential Equations and historical applications, Simmons, p465.

The Lyapunov stability theorem says that if such a differential system exists x˙ = f (x, t)

(2.102)

where f (0, t) = 0 for all t, and if there exists a scalar function V (x) having continuous first partial derivatives satisfying V (x) is positive definite and the time derivative, V˙ (x), is negative definite, then the equilibrium position at the origin is uniformly asymptotically stable. If V (x, t) → ∞ as x → ∞, then it is stable “in the large”. V (x) is called a Lyapunov function of the system. 9 Some

authors, e.g. Ogata, tend to it spell as ‘Liapunov’. Note however that M ATLAB uses the lyap spelling.

2.9. STABILITY

79

If the Lyapunov function is thought of as the total energy of the system, then we know that the total energy must both always be positive, hence the restriction that V (x) is positive definite, and also for the system to be asymptotically stable, this energy must slowly with time die away, hence the requirement that V˙ (x) < 0. The hard part about this method is finding a suitable Lyapunov function in the first place. Testing the conditions is easy, but note that if your particular candidate V (x) function failed the requirements for stability, this either means that your system is actually unstable, or that you just have not found the right Lyapunov function yet. The problem is that you don’t know which. In some cases, particularly vibrating mechanical ones, good candidate Lyapunov functions can be found by using the energy analogy, however in other cases this may not be productive. Algorithm 2.2 Lyapunov stability analysis of nonlinear continuous and discrete systems. The second method of Lyapunov is applicable for both continuous and discrete systems, [189, p65], [150, p557].

Discrete difference equations, Given a nonlinear difference equation, Continuous differential equations Given a nonlinear differential equation, x˙ = f (x, t) and a scalar function V (x), then if 1. V (x) is positive definite, and 2. V˙ (x), the derivative of V (x), is negative definite, and

xk+1 = f (xk ),

and a scalar function V (x), continuous in x, then if 1. V (x) > 0 for x 6= 0 (or V (x) is positive definite), and 2. ∆V (x), the difference of V (x), is negative definite, where ∆V (xk ) = V (xk+1 ) − V (xk )

3. V (x) → ∞ as kxk → ∞

then the system x˙ = f (x, t) is globally asymptotically stable.

f (0) = 0

= −V (f (xk )) − V (xk )

and 3. V (x) → ∞ as kxk → ∞

then the system xk+1 = f (xk ) is globally asymptotically stable.

Example of assessing the stability of a discrete nonlinear system using Lyapunov’s second method. Consider the dynamic equation from Ogata [152, Ex 9-18, p729]
x˙ 1 = x2 − x1 x21 + x22
x˙ 2 = −x2 − x2 x21 + x22

and we want to establish if it is stable or not. The system is nonlinear and the origin is the only equilibrium state. We can propose a trial Lyapunov function V (x) = x21 + x22 which is positive definite. (Note that that was the hard bit!) Now differentiating gives dx1 dx2 V˙ (x) = 2x1 + 2x2 = −2 x21 + x22 dt dt




which is negative definite. Since V (x) → ∞ as x → ∞, then the system is “stable in the large”. Relevant problems in [152, p771] include B-9-15, B-9-16 and B-9-17.

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

80

The problem with assessing the stability of nonlinear systems is that the stability can change depending on the values of the states and/or inputs. This is not the case for linear systems, where the stability is a function of the system, not the operating point. An important and relevant chemical engineering example of a nonlinear system that can either be stable or unstable is a continuously stirred tank reactor. The reactor can either be operated in the stable or unstable regime depending on the heat transfer characteristics. The stability of such reactors is important for exothermic reactions, since the thermal run-away is potentially very hazardous.

S TABILITY OF

LINEAR TIME INVARIANT SYSTEMS USING

LYAPUNOV

Sections 2.9.1 and 2.9.3 established stability criteria for linear systems. In addition to these methods, the second method of Lyapunov can be used since the linear system is simply a special case of the general nonlinear system. The advantages of using the method of Lyapunov are: 1. The Lyapunov method is analytical even for nonlinear systems and so the criteria is, in principle, exact. 2. It is computationally simple and robust and does not require the solution to the differential equation. 3. We do not need to extract eigenvalues of an n × n matrix which is an expensive numerical computation, (although we may need to check the definiteness of a matrix which does require at least a Cholesky decomposition). Of course similar advantages apply to the Routh array procedure, but unlike the Routh array, this approach can be extended to nonlinear problems. To derive the necessary condition for stability of x˙ = Ax using the method of Lyapunov, we follow the procedure outlined in Algorithm 2.2 where we choose a possible Lyapunov function in quadratic form, V (x) = xT Px The Lyapunov function will be positive definite if the matrix P is positive definite. The time derivative is given by V˙ (x) = x˙ T Px + xT Px˙ = (Ax)T Px + xT P(Ax)   = xT AT P + PA x | {z } −Q

where we have defined for convenience the matrix Q = −(AT P + PA). So if −Q is negative definite, (or alternatively Q is positive definite, Q ≻ 0), then the system is stable at the origin and hence asymptotically stable in the large. Note that this solution procedure is analytical. Unfortunately there is no cheap and easy computational way to establish if an arbitrary matrix is positive definite or not, but using M ATLAB we would usually attempt a Cholesky factorisation. Apart from the difficulty in deciding if a matrix is positive definite or not, we also have the problem that a failure of Q being positive definite does not necessarily imply that the original system is unstable. All it means is that if the original system does turn out to be stable, then the postulated P was actually not a Lyapunov function.

2.9. STABILITY

81

A solution procedure that avoids us testing many different Lyapunov candidates is to proceed in reverse; namely choose an arbitrary symmetric positive definite Q (say the identity matrix), and then solve AT P + PA = −Q (2.103) for the matrix P. If P is positive definite, then the system x˙ = Ax is stable. Eqn. 2.103 is known as the matrix Lyapunov equation or the more general form, AP + PB = −Q, is known as Sylvester’s equation. One obvious way to solve for the symmetric matrix P in Eqn. 2.103 is by equating the coefficients which results in a n(n + 1)/2 system of linear algebraic equations. It is convenient in M ATLAB to use the Kronecker tensor product to set up the equations and then use M ATLAB’s backslash operator to solve them. This has the advantage that it is expressed succinctly in M ATLAB, but it has poor storage and algorithmic characteristics and is recommended only for small dimensioned problems. Section 2.9.5 explains Kronecker tensor products and this analytical solution strategy in further detail. Listing 2.8: Solve the continuous matrix Lyapunov equation using Kronecker products

3

n= size(Q,1); % Solve AT P + PA = −Q, Q = QT I= eye(n); % Identity In , the same size as A P=reshape(-(kron(I,A')+kron(A',I))\Q(:),n,n);% −vec(I ⊗AT +AT ⊗I)vec(P) = vec(Q) P= (P+P')/2; % force symmetric (hopefully unnecessary)

The final line, while strictly unnecessary, simply forces the computed result to be symmetric, which of course it should be in the first case. Less restrictive conditions on the form of Q exist that could make the solution procedure easier to do manually and are discussed in [152, p766], and [150, p554]. Note however, using a computational tool such as M ATLAB, we would not normally bother with these modifications. The C ONTROL T OOLBOX for M ATLAB includes the function lyap that solves the matrix Lyapunov equation which is more numerically robust and efficient that equating the coefficients or using the Kronecker tensor product method given on page 81. However note that the definition of the Lyapunov matrix equation used by M ATLAB, AP + PAT = −Q

(2.104)

is not exactly the same as that defined by Ogata and used previously in Eqn. 2.103. (See the M ATLAB help file for clarification of this and compare with example 9–20 in [152, p734].) Suppose we wish to establish the stability of the continuous system x˙ = Ax   0 1 = x −1 0.5 using lyap, as opposed to say extracting eigenvalues. Now we wish to solve A⊤ P + PA = −Q for P where Q is some conveniently chosen positive definite matrix such as say Q = I. Since M ATLAB’s lyap function solves the equivalent matrix equation given by Eqn. 2.104, we must pass A⊤ rather than A as the first argument to lyap. Listing 2.9: Solve the matrix Lyapunov equation using the lyap routine 1

>> A = [0,1;-1,-0.4];

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

82

6

>> P = lyap(A',eye(size(A))) % Use AT instead of A P = 2.7000 0.5000 0.5000 2.5000 >> A'*P + P*A + eye(size(A)) % Does this equal zero ? ans = 1.0e-015 * 0 -0.4441 -0.4441 0.2220

11

16

>> Q = eye(size(A)); % alternative method >> n = max(size(Q)); % solve AT X + XA = C where C ≻ 0 >> I = eye(n); >> P=reshape(-(kron(I,A')+ kron(A',I))\Q(:),n,n) P = 2.7000 0.5000 0.5000 2.5000

Both methods return the same matrix P, P=



2.7 0.5 0.5 2.5



which has minor determinants of 2.7 and 6.5. Since both are positive, following Sylvester’s criteria, P is positive definite and so the system is stable. While we can use Sylvester’s criteria to establish if P is positive definite or not, and hence the system’s stability, it is not efficient for large systems. To establish if a symmetric matrix is positive definite, the easiest way is to look at the eigenvalues. If they are all positive, the matrix is positive definite. Unfortunately we were originally trying to avoid to solve for the eigenvalues, so this defeats the original purpose somewhat. Another efficient strategy to check for positive definiteness with M ATLAB is to attempt a Cholesky decomposition using the [R,p]=chol(A) command. The decomposition will be successful if A is positive definite, and will terminate early with a suitable error message if not. For example suppose we wanted to establish the stability of a transfer function G=

s−1 (s + 1)(s + 2)(s − 3)

which is clearly unstable due to the pole at s = +3. In this case we convert to state space, solve for P and then attempt a Cholesky decomposition to establish if P ≻ 0. Listing 2.10: Using Lyapunov to establish stability of a linear system 2

>> G = ss(zpk(1,[-1 -2 3],1)); % System is clearly unstable since one pole is s = +3 >> P = lyap(G.a, eye(size(G.a))) % Extract the A matrix & solve Lyapunov P = 0.6667 0.1179 -0.1179 0.1179 0.3333 0.1667 -0.1179 0.1667 -0.1667

7

>> chol(P) Error using chol Matrix must be positive definite.

12

>> [¬,p] = chol(P) p = 3

% If p 6= 0, then P is not positive definite.

2.9. STABILITY

83

As a final check, we could evaluate the eigenvalues of A in M ATLAB by typing eig(A). Of course in the above case we will get −1, −2 and +3. D ISCRETE TIME

SYSTEMS

For linear discrete time systems, the establishment of stability is similar to the continuous case given previously. Given the discrete time system xk+1 = Φxk , we will again choose a positive definite quadratic function for the trial Lyapunov function, V (x) = xT Px where the matrix P is positive definite, and compute the forward difference ∆V (x) = V (xk+1 ) − V (xk )

= xTk+1 Pxk+1 − xTk Pxk

Now if the matrix Q defined as

= (Φxk )T PΦxk − xTk Pxk   = xTk ΦT PΦ − P xk | {z } −Q

  Q = − ΦT PΦ − P

(2.105)

is positive definite, then the system is stable.

The reverse solution procedure, i.e. to solve for the matrix P given Q and Φ is analogous to the continuous time case given in Listing 2.8 using the Kronecker tensor product. Listing 2.11: Solve the discrete matrix Lyapunov equation using Kronecker products 1

n I P P

= = = =

max(size(Q)); % Solve AT PA − P = −Q, with Q ≻ 0 eye(n); % identity reshape((kron(I,I)-kron(Phi',Phi'))\Q(:),n,n) (P+P')/2; % force symmetric (hopefully unnecessary)

Once again, the more efficient dlyap routine from the C ONTROL T OOLBOX can also be used. In fact dlyap simply calls lyap after some minor pre-processing. We can demonstrate this establishing the stability of   3 −0.5 xk+1 = xk 0 0.8 In this case since Φ is upper triangular, we can read the eigenvalues by inspection, (they are the diagonal elements, 3 and 0.8). Since one of the eigenvalues is outside the unit circle we know immediately that the process is unstable. Notwithstanding, by following the method of Lyapunov, 1

6

11

>> Phi = [3, -0.5; 0, 0.8] >> Q = eye(size(Phi)); % Ensure Q ≻ 0 >> P = dlyap(Phi',Q) % Note Φ is transposed. P = -0.1250 -0.1339 -0.1339 2.9886 >> [R,rc] = chol(P) % Is P +ve definite? R = [] rc = 1 % No it isn’t, unstable!

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

84

we verify that the system is indeed unstable. We can both check the definiteness of P using Sylvester’s criteria [152, pp924-926], or by using the chol function as in the above example, or even by finding the eigenvalues of P. We then can compare the stability criteria thus established with that obtained by solving for the eigenvalues of Φ. Algorithm 2.3 Stability of linear systems using Lyapunov We wish to establish the absolute stability of the dynamic linear continuous system, x˙ = Ax or the discrete counterpart, xk+1 = Φxk 1. Choose a convenient positive definite n × n matrix Q, say the identity, In×n , then 2. Solve either

AT P + PA = −Q

in the case of a continuous system, or ΦT PΦ − P = −Q in the case of a discrete system for P by either equating the coefficients or using lyap or dlyap. 3. The Lyapunov function is V (x) = xT Px, so check the definiteness of P. The system is stable if and only if P is positive definite. Check using Sylvester’s criteria or attempt a Cholesky factorisation.

2.9.5

E XPRESSING MATRIX EQUATIONS SUCCINCTLY USING K RONECKER PROD UCTS

The strategy employed in Listings 2.8 and 2.11 to solve the Lyapunov equation used Kronecker products and vectorisation or stacking to express the matrix equation succinctly. This made it easy to solve since the resulting expression was now reformulated into a system of linear equations which can be solved using standard linear algebra techniques. Further details on the uses and properties of Kronecker products (or tensor products) are given in [88, p256], [115, Chpt 13] and [130]. The review in [36] concentrates specifically on Kronecker products used in control applications. The Kronecker product, given the symbol ⊗, B ∈ ℜp×q , results in a new (large) matrix  a11 B  .. A⊗B=  . am1 B

of two arbitrarily sized matrices, A ∈ ℜm×n and  · · · a1n B  .. mp×nq .. ∈ℜ . . · · · amn B

(2.106)

of size (mp × nq). In M ATLAB we would write kron(A,B). Note in general A ⊗ B 6= B ⊗ A.

The vectorisation of a matrix A is an operation that converts a rectangular matrix into a single column by stacking the columns of A on top of each other. In other words, if A is an (n × m) matrix, then vec(A) is the resulting (nm×1) column vector. In M ATLAB we convert block matrices or row vectors to columns simply using the colon operator, or A(:). We can combine this vectorisation operation with Kronecker products to express matrix multiplication as a linear transformation. For example, for the two matrices A and B of compatible

2.9. STABILITY

85

dimensions, then vec (AB) = (I ⊗ A) vec(B)   = BT ⊗ I vec(A)

and for the three matrices A, B and C of compatible dimensions, then   vec (ABC) = CT ⊗ A vec(B) = (I ⊗ AB) vec(C)   = CT BT ⊗ I vec(A)

(2.107)

(2.108)

Table II in [36] summarises these and many other properties of the algebra of Kronecker products and sums. This gives us an alternative way to express matrix expressions such as the Sylvester equation AX − XA = Q where we wish to solve for the matrix X given matrix A. In this case, using Eqn. 2.107, we can write   vec(AX − XA) = I ⊗ A − AT ⊗ I vec(X) = vec(Q) which is in the form of a system of linear equations Gx = q where the vectors  x and q are simply 

the stacked columns of the matrices X, and Q, and the matrix G is given by I ⊗ A − AT ⊗ I .

We first solve for the unknown vector x using say x = G−1 q or some numerically sound equivalent, and then we reassemble the matrix X by un-stacking the columns from x. Of course this strategy is memory intensive because the size of the matrix G is (n2 × n2 ). However [36] describes some modifications to this approach to reduce the dimensionality of the problem.

U SING fsolve TO

SOLVE MATRIX EQUATIONS

The general nonlinear algebraic equation solver fsolve within the O PTIMISATION T OOLBOX has the nice feature that it can solve matrix equations such as the continuous time Lyapunov equation directly. Here we wish to find the square matrix X such that AX + XAT + Q = 0 given an arbitrary square matrix A and a positive definite matrix Q. For an initial estimate for X we should start with a positive definite matrix such as I. We can compare this solution with the one generated by the dedicated lyap routine. >> n = 4; >> A = rand(n); >> Q1 = randn(n); Q = Q1'*Q1;

% Create a ‘random’ matrix A of dimensions (n × n) % Create a ‘random’ positive definite matrix Q

4

9

>> LyEqn = @(X) A*X+X*A'+Q; >> X = fsolve(LyEqn, eye(n)); X1 = -0.8574 -0.6776 1.0713 -0.6776 2.7742 -2.6581 1.0713 -2.6581 0.7611

% Matrix equation to be solved: AX + XAT + Q = 0 % Solve the Lyapunov using fsolve

>> Xx = lyap(A,Q); Xx =

% Solve the Lyapunov using the control toolbox

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

86 14

2.9.6

-0.8574 -0.6776 1.0713

-0.6776 2.7742 -2.6581

S UMMARY

1.0713 -2.6581 0.7611

OF STABILITY ANALYSIS

The stability of a linear continuous time transfer function is determined by the sign of the real part of the poles. For the transfer function to be stable, all the poles must lie strictly in the left-handplane. In the discrete case, the stability is determined by the magnitude of the possibly complex poles. To be stable, the discrete poles must lie within the unit circle. (See Fig. 6.3.) The key difference between the stability of discrete and continuous transfer functions is that the sample time plays an important role. Generally as one increases the sample time, the discrete system tend to instability. ℑ

stability boundary

ℑ

1 Stable entire LHP

Unstable

Unstable 0

ℜ

−1

Stable 1 0

ℜ

unit circle

Continuous s-plane

Discrete z-plane

Figure 2.28: Regions of stability for the poles of continuous (left) and discrete (right) systems To establish the stability of the transfer functions, one need not solve for the roots of the denominator polynomial since exact algebraic methods such as the Routh Array, Jury blocks and Lyapunov methods exist. However with the current computer aided tools such as M ATLAB, the task of reliably extracting the roots of high order polynomials is not considered the hurdle it once was. The Lyapunov method is also applicable for nonlinear systems. See Ogata’s comments, [150, p250].

2.10

S UMMARY

This chapter briefly developed the tools needed to analyse discrete dynamic systems. While most physical plants are continuous systems, most control is performed on a digital computer, and therefore the discrete description is far more natural for computer implementation or simulation. Converting a continuous differential equation to a discrete differential equation can be done either by using a backward difference approximation such as Euler’s scheme, or by using z transforms. The z transform is the discrete equivalent to the Laplace transform in the continuous domain. A vector/matrix approach to systems modelling and control has the following characteristics:

2.10. SUMMARY

87

1. We can convert any linear time invariant differential equation into the state space form x˙ = Ax + Bu. 2. Once we have selected a sampling time, T , we can convert from the continuous time domain to the discrete form equivalent; xk+1 = Φxk + ∆uk . 3. The stability of both the discrete and continuous time systems is determined by the eigenvalues of the A or Φ matrix. 4. We can transform the mathematical description of the process to dynamically similar descriptions by adjusting our base coordinates, which may make certain computations easier. Stability is one of the most important concepts for the control engineer. Continuous linear systems are stable if all the poles lie in the left hand side of the complex plane, ie they have negative real parts. For discrete linear systems, the poles must lie inside the unit circle. No easy check can be made for nonlinear systems, although the method due to Lyapunov can possibly be used, or one can approximate the nonlinear as a linear system and check the stability of the latter. When we deal with discrete systems, we must sample the continuous function. Sampling can introduce problems in that we may miss interesting information if we don’t sample fast enough. The sampling theorem tells us how fast we must sample to reconstruct particular frequencies. Over sampling is expensive, and could possibly introduce unwanted artifacts such as RHP zeros etc.

88

CHAPTER 2. FROM DIFFERENTIAL TO DIFFERENCE EQUATIONS

C HAPTER 3

M ODELLING

DYNAMIC SYSTEMS

WITH DIFFERENTIAL EQUATIONS The ability to solve so many problems with a small arsenal of mathematical methods gave rise to a very optimistic “world view” which in the 18th Century came to be called the “Age of Reason”. The essential point is that the world was felt to be predictable. Hans Mark

3.1

D YNAMIC SYSTEM MODELS

Dynamic system models are groups of equations that involve time derivative terms that attempt to reproduce behaviour that we observe around us. I admire the sense of optimism in the quote given above by Hans Mark: if we know the governing equations, and the initial conditions, then the future is assured. We now know that the 18th century thinkers were misguided in this sense, but nevertheless engineers are a conservative lot, and still today with similar tools, they aim to predict the known universe. It is important to realise that with proven process models, control system design becomes systematic, hence the importance of modelling. Physically dynamic systems are those that change with time. Examples include vibrating structures (bridges and buildings in earthquakes), bodies in motion (satellites in orbit, fingers on a typewriter), changing compositions (chemical reactions, nuclear decay) and even human characteristics (attitudes to religion or communism throughout a generation, or attitudes to the neighbouring country throughout a football match) and of course, there are many other examples. The aim of many modellers is to predict the future. For many of these types of questions, it is infeasible to experiment to destruction, but it is feasible to construct a model and test this. Today, it is easiest to construct a simulation model and test this on a computer. The types of models referred to in this context are mathematically based simulation models. Black-box or heuristic models, where we just fit any old curve to the experimental data, and Whitebox or fundamental models, where the curves are fitted to well established physical laws represent two possible extremes in modelling philosophy. In practice, most engineering models lie somewhere in the grey zone, known appropriately as ‘grey-box’ models where we combine our partial prior knowledge with black-box components to fit any residuals. Computer tools for grey-box modelling are developed in [32]. 89

Modelling

90

3.1.1

S TEADY

STATE AND DYNAMIC MODELS

Steady state models only involve algebraic equations. As such they are more useful for design rather than control tasks. However they can be used for control to give broad guidelines about the operating characteristics of the process. One example is Bristol’s relative gain array (§ 3.2.6) which uses steady state data to analyse for multivariable interaction. But it is the dynamic model that is of most importance to the control engineer. Dynamic models involve differential equations. Solving systems involving differential equations is much harder than for algebraic equations, but is essential for the study and application of control. A good reference for techniques of modelling dynamic chemical engineering applications is [198].

3.2

A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS

An economist is an expert who will know tomorrow why the things he predicted yesterday didn’t happen today. Laurence J. Peter

This section reviews the basic steps for developing dynamic process models and gives a few examples of common dynamic models. Modelling is particularly important in the capital-intensive world of chemical manufacture. Perhaps there is no better example of the benefits of modelling than in distillation. One industry magazine1 reported that distillation accounted for around 3% of the total world energy consumption. Clearly this provides a large economic incentive to improve operation. When modelling chemical engineering unit operations, we can consider (in increasing complexity): 1. Lumped parameters systems such as well-mixed reactors or evaporators 2. Staged processes such as distillation columns, multiple effect evaporators, floatation. 3. Distributed parameter systems such as packed columns, poorly mixed reactors, dispersion in lakes & ponds. An evaporator is an example of a lumped parameter system and is described in §3.2.3 while a rigorous distillation column model described on page 102 is an example of a staged process. At present we will now investigate only lumped parameter systems and staged processes which involve only ordinary differential equations, as opposed to partial differential equations, albeit a large number of them. Problems which involve algebraic equations coupled to the ordinary differential equations are briefly considered in section 3.4.2.

A collection of industrial process models is available in the nonlinear model library from www.hedengren.net/resear There are many text books devoted to general modelling for engineers, notably [35, 114] for general issues, [45, 197] for solution techniques and [131, 198] for chemical engineering applications. I recommend the following stages in the model building process: 1. Draw a diagram of the system with a boundary and labelling the input/output flows that cross the boundary. 1

The Chemical Engineer, 21st October, 1999, p16

3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS

91

2. Decide on what are state variables, what are parameters, manipulated variables, uncontrolled disturbances etc. 3. Write down any governing equations that may be relevant such as conservation of mass and energy. 4. Massage these equations in a standard form suitable for simulation. The following sections will describe models for a flow system into and out of a tank with external heating, a double integrator such as an inverted pendulum or satellite control, a forced circulation evaporator, and a binary distillation column.

3.2.1

S IMPLE

MODELS The best material model of a cat is another, or preferably the same, cat. Arturo Rosenblueth, Philosophy of Science, 1945

P ENDULUMS Pendulums provide an easy and fruitful target to model. They can easily be built, are visually impressive, and the governing differential equations derived from the physics are simple but nonlinear. Furthermore, if you invert the pendulum such as in a rocket, the system is unstable, and needs an overlaying control scheme.

Torque, Tc θ

,l gth len θ mg mg

Pendulum (stable)

Inverted Pendulum (unstable)

Figure 3.1: A stable and unstable pendulum Fig. 3.1 shows the two possible orientations. For the classical stable case, the equation of motion for the pendulum is d2 θ ml2 2 + mgl sin θ = Tc (3.1) dt where θ is the angle of inclination, and Tc is the applied torque. We can try to solve Eqn. 3.1 analytically using dsolve from the symbolic toolbox,

Modelling

92

>> syms m l g positive >> syms Tc theta >> dsolve('m*lˆ2*D2theta + m*g*l*sin(theta) = Tc') % Eqn. 3.1 4

Warning: Explicit solution could not be found; implicit solution returned.

but the symbolic engine fails to find an analytical solution. Eqn. 3.1 is a second order nonlinear differential equation, but by defining a new state variable vector, x, as def

x1 = θ

(3.2)

dθ def x2 = x˙ 1 = dt

(3.3)

then we note that x˙ 2 = x ¨1 , so the single second order differential equation can now be written as two coupled first order equations x˙ 1 = x2 Tc g x˙ 2 = − sin x1 + l ml2

(3.4) (3.5)

Linearising Eqn. 3.5 is trivial since sin θ ≈ θ for small angles. Consequently, both linear and nonlinear expressions are compared below # # # " # " " " 0 0 0 1 0 x1 + x+ , x˙ = −g (3.6) x˙ = −g Tc Tc sin(x1 ) 0 0 2 2 l l ml } ml } {z {z | | nonlinear

linear

while the numerical results are compared on page 125.

In the inverted case, the equation for x˙ 2 is almost the same, g Tc x˙ 2 = + sin x1 + l ml2

(3.7)

but the sign change is enough to place the linearised pole in the right-hand plane.

A DOUBLE

INTEGRATOR

The double integrator is a simple linear example from classical mechanics. It can describe, amongst other things, a satellite attitude control system, J

d2 θ =u dt2

(3.8)

where J is the moment of inertia, θ is the attitude angle, u is the control torque (produced by small attitude rockets mounted on the satellite’s side). We can convert the second order system in Eqn. 3.8 to two first order systems by again defining a new state vector     x1 def θ = (3.9) x= x2 θ˙ Substituting this definition into Eqn. 3.8, we get J x˙ 2 = u and x˙ 1 = x2 , or in a matrix-vector form       0 1 x˙ 1 0 (3.10) = x 1 u 0 0 x˙ 2 J

3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS

93

A LIQUID LEVEL TANK Buffer storage vessels such as shown in Fig. 3.2 are commonly used in chemical plants between various pieces of equipment such as reactors and separators to dampen out fluctuations in production flows. Flow in, Fin Tank with cross sectional area, A

ˆ measured height, h

∆P true height, h

flow out, Fout valve Figure 3.2: Simple buffer tank Our system boundary is essentially the figure boundary, so we have one input, the input flow rate, Fin , and one output, the output flow rate, Fout . What interests us is the amount of fluid in the tank, M , which physically must always be less than the full tank Mmax , and greater than 0. The governing relation is the conservation of mass, dM = Fin − Fout (3.11) dt If the tank has a constant cross sectional area, then the amount of water in the tank M is proportional to the height of water h, since most fluids are assumed incompressible. In addition the mass M is proportional to the volume, M = ρV = ρAh. Rewriting Eqn. 3.11 in terms of height dh Fin − Fout = dt ρA

(3.12)

since the area and density are constant. If the liquid passes though a valve at the bottom of the tank, the flowrate out will be a function of height. For many processes, the flow is proportional to the square root of the pressure drop, (Bernoulli’s equation) √ √ Fout ∝ ∆P = k h This square root relation introduces a mild nonlinearity into the model. The “tank” level is not actually measured in the tank itself, but in a level leg connected to the tank. The level in this level leg lags behind the true tank level owing to the restriction valve on the interconnecting line. This makes the system slightly more difficult to control owing to the added measurement dynamics. We assume, that the level leg can be modelled using a first order dynamic equation, with a gain of 1, and a time constant of about 2 seconds. I can estimate this by simply watching the apparatus. Thus ˆ ˆ dh h−h = (3.13) dt 2

Modelling

94

The level signal is a current, I, between 4 and 20 mA. This current is algebraically related (by some function f (·) although often approximately linearly) to the level in the level leg. Thus the variable that is actually measured is: ˆ I = f (h) (3.14) The current I is called the output or measured variable. In summary, the full model equations are; √ Fin − k h dh = (3.15) dt ρA ˆ ˆ dh h−h = (3.16) dt 2 ˆ I = f (h) (3.17) The input variable (which we may have some hope of changing) is the input flow rate, Fin , and the exit valve position which changes k. The dependent (state) variables are the actual tank level, ˆ The output variable is the current, I. Constant h, and the measurable level in the level leg, h. parameters are density, and tank cross-sectional area. In Table 3.1 which summarises this nomenclature, I have written vectors in a bold face, and scalars in italics. The input variables can be further separated into variables that can be easily changed or changed on demand such as the outlet valve position (manipulated variables), and inputs that change outside our control (disturbance variables). In this case the inlet flowrate. Table 3.1: Standard nomenclature used in modelling dynamic systems

type independent states inputs outputs parameters

A STIRRED TANK

symbol t x u y θ

variable time, t ˆ h, h Fin , k I ρ, A

HEATER

Suppose an electrical heater is added causing an increase in water temperature. Now in addition to the mass balance, we have an energy balance. The energy balance in words is “the rate of temperature rise of the contents in the tank is proportional to the amount of heat that the heater is supplying plus the amount of heat that is arriving in the incoming flow minus the heat lost with the out flowing stream minus any other heat losses”. We note that for water the heat capacity cp , is almost constant over the limited temperature range we are considering, and that the enthalpy of water in the tank is defined as H = M cp ∆T

(3.18)

where ∆T is the temperature difference between the actual tank temperature T and some reference temperature, Tref . Writing an energy balance gives M cp

dT = Fin cp (Tin − Tref ) − Fout cp ∆T + Q − q dt

(3.19)

where Q is the heater power input (kW) and q is the heat loss (kW). These two equations (Eqn 3.12 and 3.19) are coupled. This means that a change in mass in the tank M, will affect the temperature in the tank T (but not the other way around). This is called one-way coupling.

3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS

95

For most purposes, this model would be quite adequate. But you may decide that the heat loss q to the surroundings is not constant and the variation significant. A more realistic approximation of the heat loss would be to assume that it is proportional to the temperature difference between the vessel and the ambient (room) temperature Troom . Thus q = k(T − Troom ). Note that the heat loss now can be both positive or negative. You may also decide that the heat capacity cp , and density ρ of water are a function of temperature. Now you replace these constants with functions as cp (T ) and ρ(T ). These functions are tabulated in the steam tables2 . The more complete model would then be dh Fin − Fout = dt ρ(T )A dT Aρ(T ) = Fin cp (T ) (Tin − T ref ) − Fout cp (T )∆T + Q − k (T − Troom ) dt

(3.20)

“C AVEAT E MPTOR ” Finally we should always state under what conditions the model is valid and invalid. For this example we should note that the vessel temperature should not vary outside the range 0 < T < 100◦C since the enthalpy equation does not account for the latent heat of freezing or boiling. We should also note that h, Fin , Fout , M, A, k, cp , ρ are all assumed positive. As a summary and a check on the completeness of the model, it is wise to list all the variables, and state whether the variable is a dependent or independent variable, whether it is a dynamic or constant variable, or whether it is a parameter. The degree of freedom for a well posed problem should be zero. That is, the number of unknowns should equal the number of equations. Incomplete models, and/or bad assumptions can give very misleading and bad results. A great example highlighting the dangers of extrapolation from obviously poor models is the biannual UK government forecast of Britain’s economic performance. This performance or growth, is defined as the change in the Gross Domestic Product (GDP) which is the total amount of goods and services produced in the country that year. Figure 3.3 plots the actual growth over the last few years (solid line descending), compared with the forecasts given by the Chancellor of the Exchequer, (dotted lines optimistically ascending). I obtained this information from a New Scientist article provocatively titled Why the Chancellor is always wrong, [51]. Looking at this performance, it is easy to be cynical about governments using enormous resources to model things that are essentially so complicated as to be unmodellable, and then produce results that are politically advantageous.

3.2.2

A CONTINUOUSLY- STIRRED

TANK REACTOR

Tank reactors are a common unit operation in chemical processing. A model presented in [84] considers a simple A → B reaction with a cooling jacket to adjust the temperature, and hence the reaction rate as shown in Fig. 3.4. The reactor model has two states, the concentration of compound A, given the symbol Ca measured in mol/m3 and the temperature of the reaction vessel liquid, T measured in K. The manipulated variables are the cooling jacket water temperature, Tc , the temperature of the feed Tf and 2 Steam

tables for M ATLAB are available from http://www.kat.lth.se/edu/kat061/MfileLib.html

Modelling

96 119

Forecasts

Growth GDP change

118 117 116 115 114 113 112

Figure 3.3: The UK growth based on the GDP comparing the actual growth (solid) versus Treasury forecasts (dashed) from 1989–1992.

Actual

111 1989

1990

1991 Year

1992

1993

Stirrer Feed, Tf , Caf

Coolant input, Tc

A→B

Coolant outflow

Product, T, Ca Figure 3.4: A continually stirred tank reactor (CSTR) reactor the concentration of the feed, Caf .   q −E dT Ca = (Caf − Ca ) − k0 exp dt V RT   UA q m∆H −E dCa Ca + k0 exp (Tc − T ) = (Tf − T ) + dt V ρCp RT V ρCp

(3.21) (3.22)

The values for the states and inputs at steady-state are def

xss =



T Ca



= ss



324.5 0.8772



,



   Tc 300 def uss =  Tf  =  350  Caf ss 1

(3.23)

The values of the model parameters such as ρ, Cp etc. are given in Table 3.2.

Note: At a jacket temperature of Tc = 305K, the reactor model has an oscillatory response. The oscillations are characterized by reaction run-away with a temperature spike. When the concen-

3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS

97

Table 3.2: Parameters of the CSTR model

name Volumetric Flowrate Volume of CSTR Density of A-B Mixture Heat capacity of A-B Mixture Heat of reaction for A→B Pre-exponential factor Overall Heat Transfer Coefficient × Area

variable q = 100 V = 100 ρ = 1000 Cp = .239 m∆H = 5 · 104 E/R = 8750 k0 = 7.2 · 1010 U A = 5 · 104

unit m3 /sec m3 kg/m3 J/kg-K J/mol K 1/sec W/K

tration drops to a low value, the reactor cools until the concentration builds and there is another run-away reaction.

3.2.3

A FORCED

CIRCULATION EVAPORATOR

Evaporators such as illustrated in Fig. 3.5 are used to concentrate fruit juices, caustic soda, alumina and many other mixtures of a solvent and a solute. The incentive for modelling such unit operations is that the model can provide insight towards better operating procedures or future design alternatives. A relatively simple dynamic model of a forced circulation evaporator is developed in [147, chapter 2] is reproduced here as a test process for a wide variety of control techniques of interest to chemical engineers. Other similar evaporator models have been reported in the literature; [66] documents research over a number of years at the University of Alberta on a pilot scale double effect evaporator, and [42] describe a linearised pilot scale double effect evaporator used in multivariable optimal selftuning control studies. The nonlinear evaporator model can be linearised into a state space form with the variables described in Table 3.3. The parameters of the linearised state space model are given in Appendix D.1. Table 3.3: The important variables in the forced circulation evaporator from Fig. 3.5.

type State

Manipulated

Disturbance

name level steam pressure product conc product flow steam pressure c/w flow circulating flow feed flow feed composition feed temperature c/w temperature

variable L2 P2 x2 F2 P100 F200 F3 F1 x1 T1 T200

unit m kPa % kg/min kPa kg/min kg/min kg/min % ◦ C ◦ C

range 0–2 unspecified 0–100 0-10 100–350 0–300 30-70 5-15 0-10 20-60 15-35

The distinguishing characteristics of this evaporator system from a control point are that the detailed plant model is nonlinear, the model is non-square (the number of manipulated variables, u, do not equal the number of state variables, x), and that one of the states, level, is an integrator. The dynamic model is both observable and controllable, which means that by measuring the outputs only, we can at least in theory, control all the important state variables by changing the inputs.

Modelling

98 cooling water flow, F200 condensor Vapour

Cooling water T200 condensate L

Pressure, P2

level, L2

P separator

Steam Steam pressure, P100

Evaporator Condensate Feed F1 , x1 , T1

F3

circulation pump

composition, x2 A Product

Product flow, F2 Figure 3.5: A forced circulation evaporator from [147, p7] showing state (black), manipulated (blue) and disturbance variables (green). See also Table 3.3. Problem 3.1 1. Look at the development of the nonlinear model given in [147, chapter 2]. What extensions to the model would you suggest? Particularly consider the assumption of constant mass (M = 50kg) in the evaporator circuit and the observation that the level in the separator changes. Do you think this change will make a significant difference? 2. Describe in detail what tests you would perform experimentally to verify this model if you had an actual evaporator available. 3. Construct a M ATLAB simulation of the nonlinear evaporator. The relevant equations can be found in [147, Chpt 2].

3.2.4

A

BINARY DISTILLATION COLUMN MODEL

Distillation columns, which are used to separate mixtures of different vapour pressures into almost pure components, are an important chemical unit operation. The columns are expensive to manufacture, and the running costs are high owing to the high heating requirement. Hence there is much incentive to model them with the aim to operate them efficiently. Schematics of two simple columns are given in Figures 3.6 and ??. Wood and Berry experimentally modelled a 9 inch diameter, 8-tray binary distillation column in [205] that separated methanol from water as shown in Fig. 3.6. The transfer function model they derived by step testing a real column has been extensively studied, although some authors, such

3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS

99

as [187], point out that the practical impact of all these studies has been, in all honesty, minimal. Other distillation column models that are not obtained experimentally are called rigorous models and are based on fundamental physical and chemical relations. A more recent distillation column model, developed by Shell and used as a test model is discussed in [139].

c/w Reflux, R Distillate, D, xD

Feed, F, xF Distillation column

Steam, S Reboiler Bottoms, B, xB Figure 3.6: Schematic of a distillation column Columns such as given in Fig. 3.6 typically have at least 5 control valves, but because the hydrodynamics and the pressure loops are much faster than the composition dynamics, we can use the bottoms exit valve to control the level at the bottom of the column, use the distillate exit valve to control the level in the separator, and the condenser cooling water valve to control the column pressure. This leaves the two remaining valves (steam reboiler and reflux) to control the top and bottoms composition. The Wood-Berry model is written as a matrix of Laplace transforms in deviation variables 

−18.9e−3s 12.8e−s  16.7s + 1 21s + 1 y=  6.6e−7s −19.4e−3s 10.9s + 1 14.4s + 1 {z | G(s)



 3.8e−8s     u +  14.9s + 1  d   4.9e−3.4s  13.2s + 1 } 

(3.24)

where the inputs u1 and u2 are the reflux and reboiler steam flowrates (in lb/min) respectively, the outputs y1 and y2 are the weight % of ethanol in the distillate and bottoms, and the disturbance variable, d is the feed flowrate. The time scale is in minutes. It is evident from the transfer function model structure in Eqn. 3.24 that the plant will be interacting and that all the transfer functions have some time delay associated with them, but the off-diagonal terms have the slightly larger time delays. Both these characteristics will make controlling the column difficult. In M ATLAB, we can construct such a matrix of transfer functions (ignoring the feed) with a matrix of deadtimes as follows:

Modelling

100

5

10

>> G = tf({12.8, -18.9; 6.6 -19.4}, ... { [16.7 1],[21 1]; [10.9 1],[14.4 1]}, ... 'ioDelayMatrix',[1,3; 7,3], ... 'InputName',{'Reflux','Steam'}, ... 'OutputName',{'Distillate','bottoms'}) Transfer function from input "Reflux" to output... 12.8 Distillate: exp(-1*s) * ---------16.7 s + 1

bottoms:

6.6 exp(-7*s) * ---------10.9 s + 1

15

Transfer function from input "Steam" to output... -18.9 Distillate: exp(-3*s) * -------21 s + 1 20

bottoms:

-19.4 exp(-3*s) * ---------14.4 s + 1

Once we have formulated the transfer function model G, we can perform the usual types of analysis such as step tests etc as shown in Fig. 3.7. An alternative implementation of the column model in S IMULINK is shown in Fig. 3.8.

Step Response From: Reflux

From: Steam

Figure 3.7: Step responses of the four transfer functions that make up the Wood-Berry binary distillation column model in Eqn. 3.24.

To: Distillate

0 −10 −20

To: bottoms

Amplitude

10

0

−10

−20

0

50

100

0 Time (sec)

50

100

A 3-input/3 output matrix of transfer functions model of an 19 plate ethanol/water distillation

3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS

reflux

reflux

distillate

steam

bottoms

101

distillate

Wood Berry Column

steam

bottoms

(a) Overall column mask. 1 reflux

12.8

1

16.7s+1 Transport Delay

Transfer Fcn

distillate

−18.9 21s+1 Transfer Fcn1

Transport Delay1

6.6 10.9s+1 Transfer Fcn2

2 steam

Transport Delay2

−19.4 14.1s+1 Transfer Fcn3

Transport Delay3

2 bottoms

(b) Inside the Wood Berry column mask. Compare this with Eqn. 3.24. Figure 3.8: Wood-Berry column model implemented in S IMULINK column model with a variable side stream draw off from [153] is  −0.61e−3.5s −0.0049e−s 0.66e−2.6s  6.7s + 1 8.64s + 1 9.06s + 1    y1  1.11e−6.5s −3s −2.36e −0.012e−1.2s  y2  =    3.25s + 1 5s + 1 7.09s + 1 y3  −9.4s  −34.68e−9.2s 46.2e 0.87(11.61s + 1)e−2.6s 8.15s + 1 10.9s + 1 (3.89s + 1)(18.8s + 1)   −1.2s −0.0011(26.32s + 1)e−2.66s 0.14e  6.2s + 1 (3.89s + 1)(14.63s + 1)      −0.0032(19.62s + 1)e−3.44s  d1  0.53e−10.5s +    d2 6.9s + 1 (7.29s + 1)(8.94s + 1)    −11.54e−0.6s  −2.6s 0.32e 7.01s + 1



    u1   u2    u3  

(3.25)

7.76s + 1

which provides an alternative model for testing multivariable control schemes. In this model the three outputs are the overhead ethanol mole fraction, the side stream ethanol mole fraction, and the temperature on tray 19, the three inputs are the reflux flow rate, the side stream product rate and the reboiler steam pressure, and the two disturbances are the feed flow rate and feed temperature. This system is sometimes known as the OLMR after the initials of the authors.

Modelling

102

Problem 3.2 1. Assuming no disturbances (d = 0), what are the steady state gains of the Wood-Berry column model? (Eqn 3.24). Use the final value theorem. 2. Sketch the response for y1 and y2 for; (a) a change in reflux flow of +0.02 (b) A change in the reflux flow of −0.02 and a change in reboiler steam flow of +0.025 simultaneously. 3. Modify the S IMULINK simulation to incorporate the feed dynamics.

3.2.5

R IGOROUS

TRAY- BY- TRAY DISTILLATION COLUMN MODELS

Distillation column models are important to chemical engineers involved in the operation and maintenance of these expensive and complicated units. While the behaviour of the actual multicomponent tower is very complicated, models that assume ideal binary systems are often good approximations for many columns. We will deal in mole fractions of the more volatile component, x for liquid, y for vapour and develop a column model following [132, p69]. A generic simple binary component column model of a distillation column such as shown in Fig. ?? assumes: 1. Equal molar overflow applies (heats of vapourisation of both components are equal, and mixing and sensible heats are negligible.) 2. Liquid level on every tray remains above weir height. 3. Relative volatility and the heat of vapourisation are constant. In fact we assume a constant relative volatility, α. This simplifies the vapour-liquid equilibria (VLE) model to yn =

αxn 1 + (α − 1)xn

on tray n with typically α ≈ 1.2–2. 4. Vapour phase holdup is negligible and the feed is a saturated liquid at the bubble point. The N trays are numbered from the bottom to top, (tray 0 is reboiler and tray N + 1 is the condenser). We will develop separate model equations for the following parts of the column, namely: Condenser is a total condenser, where the reflux is a liquid and the reflux drum is perfectly mixed. General tray has liquid flows from above, and vapour flows from below. It is assumed to be perfectly mixed with a variable liquid hold up, but no vapour hold up as it is assumed very fast. Feed tray Same as a ‘general’ tray, but with an extra (liquid) feed term. Top & bottom trays Same as a ‘general’ tray, but with one of the liquid (top) or vapour (bottom) flows missing, but recycle/reflux added. Reboiler in a perfectly mixed thermo-siphon reboiler with hold up Mb .

3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS

103

c/w condensor condensate collector tray #N

Feed, F, xF

Distillate, D, xD Reflux, R, xD

feed tray, NF

tray #1

reboiler, V Bottoms, B, xB

Figure 3.9: A binary distillation tower The total mass balance in the condenser and reflux drum is dMd =V −R−D dt and the component balance on the top tray dMd xd (3.26) = V yNT − (R + D)xD dt For the general nth tray, that is trays #2 through N − 1, excepting the feed tray, the total mass balance is dMn = Ln+1 − Ln + Vn−1 − Vn | {z } dt = Ln+1 − Ln

≈0

(3.27)

and the component balance dMn xn = Ln+1 xn+1 − Ln xn + V yn−1 − V yn dt For the special case of the feed nF th tray, dMnF = LnF +1 − LnF + F, mass balance dt dMnF xnF = LnF +1 xnF +1 − LnF xnF + F xF + V ynF −1 − V ynF , component balance dt and for the reboiler and column base dMB = L1 − V − B dt dMB xB = L1 x1 − BxB − V xB dt

(3.28)

Modelling

104 In summary the number of variables for the distillation column model are:

VARIABLES

Tray compositions, xn , yn Tray liquid flows, Ln Tray liquid holdups, Mn Reflux comp., flow & hold up, xD , R, D, MD Base comp., flow & hold up, xB , yB , V, B, MB Total # of equations

2NT NT NT 4 5 4NT + 9

and the number of equations are:

E QUATIONS

Tray component balance, Tray mass balance Equilibrium (tray + base) hydraulic Reflux comp. & flow Base comp & flow Total # of equations

NT NT NT + 1 NT 2 2 4NT + 7

Which leaves two degrees of freedom. From a control point of view we normally fix the boildup ˙ and the reflux flow rate (or ratio) with some sort of controller, rate, Q, R = f (xD ),

V ∝ Q˙ = f (xB )

Our dynamic model of a binary distillation column is relatively large with two inputs (R, V ), two outputs (xB , xD ) and 4N states. Since a typical column has about 20 trays, we will have around n = 44 states. which means 44 ordinary differential equations However the (linearised) Jacobian for this system while a large 44 × 44 matrix is sparse. In our case the percentage of non-zero elements or sparsity is ≈ 9%.   .. ∂ x˙ ∂ x˙ .  ∂x ∂u  |{z} |{z} 44×44

44×2

The structure of the A and B matrices are, using the spy command, given in Fig. 3.10.

There are many other examples of distillation column models around (e.g. [142, pp 459]) This model has about 40 trays, and assumes a binary mixture at constant pressure and constant relative volatility.

S IMULATION OF

THE DISTILLATION COLUMN

The simple nonlinear dynamic simulation of the binary distillation column model can be used in a number of ways including investigating the openloop response, interactions and quantify the extent of the nonlinearities. It can be used to develop simple linear approximate transfer function models, or we could pose “What if?” type questions such as quantifying the response given feed disturbances. An openloop simulation of a distillation column gives some idea of the dominant time constants and the possible nonlinearities. Fig. 3.11 shows an example of one such simulation where we step change: the reflux from R = 128 to R = 128 + 0.2, and the reboiler from V = 178 to V = 178 + 0.2. From Fig. 3.11 we can note that the open loop step results are overdamped, and that steady-state gains are very similar in magnitude. Furthermore the response looks very like a 2 × 2 matrix of

3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS

105

0 5

ODE equations

10 15 20 25 30 35 40 45

0

10

20 nz = 25

30

40

Figure 3.10: The incidence of the Jacobian (blue) and B matrix (red) for the ideal binary distillation column model. Over 90% of the elements in the matrix are zero.

1

Distill conc

0.995 0.99 0.985 0.98 0.975

0.1

base conc

0.08 0.06 0.04 Step change in Reflux Step change in Vapour

0.02 0

0

5

10

15 time (min)

20

25

30

Figure 3.11: Open loop responses of the 20-tray binary distillation column for a step change in reflux, and a step change in vapour boil-off rate.

Modelling

106 second order overdamped transfer functions,    xD (s) G11 = xB (s) G21

G12 G22



R(s) V (s)



So it is natural to wonder at this point if it is possible to approximate this response with a loworder model rather than requiring the entire 44 states and associated nonlinear differential equations.

C ONTROLLED

SIMULATION OF A DISTILLATION COLUMN

A closed loop simulation of the 20 tray binary distillation column with a feed disturbance from xF = 0.5 to xF = 0.54 at t = 10 minutes is given in Fig. 3.12. Note that while we have plotted the liquid concentrations on all the 20 trays, we are really only interested in the very top (distillate) and bottom (reboiler) trends. However simply to obtain the top and bottom trends, we are forced to calculate all the tray concentrations in this rigorous model. We can look more closely in Fig. 3.13 Binary distillation column model 1 0.9

Liquid concentration

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

R

135 130 125 190 V

Figure 3.12: A closed loop simulation of a binary distillation column given a feed disturbance. The upper plot shows the concentration on all the 20 trays, the lower plots shows the manipulated variables (reflux and boil-up) response.

180 170

0

20

40 60 time (min)

80

100

at the distillate and base compositions to see if they really are in control, namely at x⋆b = 0.02, x⋆d = 0.98. Distillation columns are well known to interact, and these interactions cause difficulties in tuning. We will simulate in Fig. 3.14 a step change in the distillate setpoint from x⋆D = 0.98 to x⋆D = 0.985 at t = 10 min and a step change in bottoms concentration at 150 minutes. The interactions are evident in the base composition transients owing to the changing distillate composition and visa versa. These interactions can be minimised by either tightly tuning one of the loops so consequently leaving the other ‘loose’, or by using a steady-state or dynamic decoupler, or even a multivariable controller.

3.2. A COLLECTION OF ILLUSTRATIVE DYNAMIC MODELS

107

Distillate

0.99 0.985 0.98

Bottoms

0.975 0.04 0.03 0.02 0.01

0

20

40 60 time [mins]

80

100

Figure 3.13: Detail of the distillate and base concentrations from a closed loop simulation of a binary distillation column given a feed disturbance. (See also Fig. 3.12.)

Distillate xD

1

0.99

0.98

Base, xB

0.04 0.03 0.02 0.01

reboiler

Reflux

0 160 140 120 200 180 160

3.2.6

0

50

100

150 time (min)

I NTERACTION

200

AND THE

250

300

Figure 3.14: Distillation interactions are evident when we step change the distillate and bottoms setpoints independently.

R ELATIVE G AIN A RRAY

The Wood-Berry distillation column model is an example of a multivariable, interacting process. This interaction is evident from the presence of the non-zero off diagonal terms in Eqn. 3.24. A quantitative measure of the extent of interaction in a multivariable process is the Relative Gain Array (RGA) due to Bristol, [37]. The RGA is only applicable to square systems (the number of manipulated variables equal the number of controlled variables), and is a steady-state measure. The RGA, given the symbol Λ, is an (n × n) matrix where each element is the ratio of the open loop gain divided by the closed loop gain. Ideally, if you are the sort who do not like interactions, you would like Λ to be diagonally dominant. Since all the rows and all the columns must sum to 1.0, then a system with no interaction will be such that Λ = I. The open loop steady-state gain is easy to determine. Taking the Wood-Berry model, Eqn. 3.24, as an example, the final steady-state as a function of the manipulated variable u, is   12.8 −18.9 yss = Gss u = u (3.29) 6.6 −19.4 The ijth element of Gss is the open loop gain between yi and uj . The Gss matrix can be evaluated experimentally or formed by applying the final value theorem to Eqn 3.24. Now the closed loop gain is a function of the open loop gain, so only the open loop gains are needed to evaluate the

Modelling

108

RGA. Mathematically the relative gain array, Λ, is formed by multiplying together Gss and G−⊤ ss elementwise,3
⊤ (3.30) Λ = Gss ⊗ G−1 ss where the special symbol ⊗ means to take the Hardamard product (also known as the Schur product), or simply the elementwise product of two equally dimensioned matrices as opposed to the normal matrix multiplication. In M ATLAB, the evaluation of the relative gain array Λ is easy. 1

>>Gss = [12.8, -18.9; 6.6, -19.4] % Steady-state gain from Eqn. 3.29 >>L = Gss.*inv(Gss)' % See Eqn. 3.30. Don’t forget the dot-times (.*)

which should return something like Λ=



2.0094 −1.0094 −1.0094 2.0094



Note that all the columns and rows sum to 1.0. We would expect this system to exhibit severe interactions, although the reverse pairing would be worse. The usefulness of the RGA is in choosing which manipulated variable should go with which control variable in a decentralised control scheme. We juggle the manipulated/control variable parings until Λ most approaches the identity matrix. This is an important and sensitive topic in process control and is discussed in more detail in [193, pp494–503], and in [181, p457]. The most frequently discussed drawback of the relative gain array, is that the technique only addresses the steady state behaviour, and ignores the dynamics. This can lead to poor manipulated/output pairing in some circumstances where the dynamic interaction is particularly strong. The next section further illustrates this point.

T HE

DYNAMIC RELATIVE GAIN ARRAY

As mentioned above, the RGA is only a steady-state interaction indicator. However we could use the same idea to generate an interaction matrix, but this time consider the elements of the transfer function matrices as a function of frequency by substituting s = jω. This now means that the dynamic relative gain array, Λ(ω), is a matrix where the elements are functions of ω as opposed to being constants. Consider the (2 × 2) system from [183]  2.5e−5s    (15s + 1)(2s + 1) y1 =  y2 1 3s + 1 {z | G(s)

 5   4s + 1   u1 −4e−5s  u2 20s + 1 }

(3.31)

which has as its distinguishing feature significant time delays on the diagonal elements. We could compute the dynamic relative gain array matrix using the definition Λ(s) = G(s) ⊗ G(s)−T

(3.32)

then substituting jω for s, and perhaps using the symbolic toolbox to help us with the possibly unwieldy algebra. 3 Note

details.

 ⊤ that this does not imply that Λ = Gss G−1 , i.e. without the Hardamard product. See [181, p456] for further ss

3.3. CURVE FITTING EXPERIMENTAL DATA

109

Listing 3.1: Computing the dynamic relative gain array analytically >>syms s >>syms w real 3

>>G=[2.5*exp(-5*s)/(15*s+1)/(2*s+1), 5/(4*s+1); ... 1/(3*s+1), -4*exp(-5*s)/(20*s+1)] % Plant from Eqn. 3.31

8

13

>>RGA = G.*inv(G') % Λ(s), Eqn. 3.32 >>DRGA = subs(RGA,'s',1j*w) % Substitute s = jω >> abs(subs(DRGA,w,1)) ans = 0.0379 0.9793 0.9793 0.0379

% Magnitude of the DRGA matrix at ω = 1 rad/s

You can see that from the numerical values of the elements of Λ at ω = 1 rad/s that this system is not diagonally dominant at this important frequency. Fig. 3.15 validates this observation. An alternative numerical way to generate the elements of the DRGA matrix as a function of frequency is to compute the Bode diagram for the multivariable system, extract the current gains, and then form the RGA from Eqn. 3.30. Listing 3.2: Computing the dynamic relative gain array numerically as a function of ω. See also Listing 3.1. 2

7

12

G = tf({2.5, 5; 1 -4}, ... {conv([15 1],[2,1]),[4 1]; [3 1],[20 1]}, ... 'ioDelayMatrix',[5,0; 0,5]) % Plant from Eqn. 3.31 [¬,¬,K] = zpkdata(G); sK =sign(K); % get sign of the gains [Mag,Ph,w] = bode(G); % Compute Bode diagram DRGA = NaN(size(Mag)); for i=1:length(w) K = Mag(:,:,i).*sK; DRGA(:,:,i) = K.*(inv(K))'; end

% Gain, including signs % Λ(ω) = K ⊗ K−T

OnDiag = squeeze(DRGA(1,1,:)); OffDiag = squeeze(DRGA(1,2,:)); semilogx(w, OnDiag,'-',w,OffDiag,'--'); % See Fig. 3.15

The trends of the diagonal and off-diagonal elements of Λ are plotting in Fig. 3.15. What is interesting about this example is that if we only consider the steady-state case, ω = 0, then Λ is diagonally dominant, and our pairing looks suitable. However what we should be really concentrating on are the values around the corner frequency at ω ≈ 0.1 where now the off-diagonal terms start to dominate. For comparison, Fig. 3.16 shows the dynamic RGA for the (3×3) OLMR distillation column model from Eqn. 3.25. In this case the off-diagonal elements do not dominate at any frequency.

3.3

C URVE FITTING EXPERIMENTAL DATA

In most cases of industrial importance, the models are never completely white-box, meaning that there will always be some parameters that need to be fitted, or at least fine-tuned to experimental

Modelling

110 1 λ

1,1

λ

0.8

1,2

0.6 0.4 0.2

Figure 3.15: The diagonal and off-diagonal elements of the RGA matrix as a function of frequency.

0 −3 10

2.5

−2

10

1

10

11

λ

22

1.5 elements of Λ(ω)

0

10

λ

2

Diagonal elements

λ33

1 0.5 λ

0

23

λ

13

−0.5 −1

Figure 3.16: Comparing the diagonal and off-diagonal elements of the (3 × 3) RGA matrix from Eqn. 3.25.

−1

10 frequency

Off−diagonal elements

λ

−1.5 −3 10

12

−2

10

−1

10 frequency

0

1

10

10

data. Generally the following three steps are necessary: 1. Collect N experimental independent xi , and dependent yi data points. 2. Select a model structure, f (θ, x) with a vector of parameters θ that are to be estimated. 3. Search for the parameters that optimise some sort performance index such as to maximise the “goodness of fit”, or equivalently minimise the sum of squared residuals. In this, and the following sections, we will colour the parameters to be estimated in green to distinguish them clearly from the experimental (x, y) data pairs. A common performance index or loss function is to select the adjustable parameters so that the sum of difference squared between the raw data and your model predictions is minimised. This is the classical Gaussian least-squares minimisation problem. Formally we can write the performance index as, N X 2 (yi − yˆi ) (3.33) J = i=1

where the ith model prediction, yˆi is a function of the model, parameters, θ, and input data, yˆi = f (θ, xi )

(3.34)

We want to find the “best” set of parameters, or in other words the set of θ that minimises J in Eqn. 3.33, ) (N X 2 ⋆ (3.35) (yi − yˆi ) θ = arg min θ i=1

3.3. CURVE FITTING EXPERIMENTAL DATA

111

The term “arg min” part of Eqn. 3.35 can be read as “the argument who minimises . . . ” since we are not particularly interested in the actual value of the performance function at the optimum, J , but rather the value of the parameters at the optimum, θ⋆ . This least-squared estimation is an intuitive performance index, and has certain attractable analytical properties. The German mathematician Gauss is credited with popularising this approach, and indeed using it for analysing data taken while surveying Germany and making astronomical observations. Note however other possible objectives rather than Eqn. 3.33 are possible, such as minimising the sum of the absolute value of the deviations, or minimising the single maximum deviation. Both these latter approaches have seen a resurgence of interest of the last few years since the computer tools have enabled investigators to by-pass the difficulty in analysis. There are many ways to search for the parameters, and any statistical text book will cover these. However the least squares approach is popular because it is simple, requires few prior assumptions, and typically gives good results.

3.3.1

L EAST- SQUARES

POLYNOMIAL REGRESSION

A popular way to fit smooth curves to experimental data is to find a polynomial that passes through the cloud of experimental data. This is known as polynomial regression. The nth order polynomial to be regressed is yˆ = θn xn + θn−1 xn−1 + · · · + θ1 x + θ0

(3.36)

where we try different values of the (n + 1) parameters in the vector θ until the calculated dependent variable yˆ is close to the actual measured value y. Mathematically the vector of parameters θ is obtained by solving a least squares optimisation problem. Given a set of parameters, we can predict the ith model observation using   θn  θn−1     n    xn−1 · · · x 1 ·  ...  yˆi = x    θ1  θ0

or written in compact matrix notation

(3.37)

yˆi = xTi θ

where xi is the independent (column) data vector for the ith observation. If all n observations are stacked vertically together, we obtain the matrix system   n   x1 x1n−1 · · · x1 1 yˆ1  yˆ2   xn2 x1n−1 · · · x2 1      (3.38)  ..  =  .. .. .. .. ..  θ  .   . . . . .  yˆN

|

xnN

n−1 xN

· · · xN {z X

1

}

or y = Xθ in a more compact matrix notation. The matrix of comprised of the stacked rows of measured independent data, X, in Eqn. 3.38 is called the Vandermonde matrix or data matrix and can be easily constructed in M ATLAB using the vander command, although it is well known that this matrix can become very illconditioned. We can search for the parameters in a number of ways, but finding the parameters that minimise the sum of squared errors is a common and easy approach. In matrix notation, the error vector is defined as def ǫ = y − Xθ (3.39)

Modelling

112

and in this case our scalar objective function that we want to minimise (sometimes called the cost function), is the sum of squared errors, J = ǫ⊤ ǫ = (y − Xθ)⊤ (y − Xθ)

= y⊤ y − θ⊤ X⊤ y − y⊤ Xθ + θ⊤ X⊤ Xθ

We want to choose θ such that J is minimised, ie a stationary point, thus we can set the partial derivatives of J with respect to the parameters to the zero row vector,4 ∂J =0 ∂θ  T = − XT y − yT X − 2θT XT X

(3.40)

= −2yT X + 2θT X⊤ X

(3.41)

which we can now solve for the unknown vector θ as  −1 θ = X⊤ X X⊤ y | {z }

(3.42)

pseudo-inverse

As a consequence of the fact that we carefully chose our model structure in Eqn. 3.36 to be linear in the parameters, θ, then the solution given by Eqn. 3.42 is analytic and therefore very reliable, and straight forward to implement as opposed to nonlinear regression which require iterative solution techniques.

This method of fitting a polynomial through experimental data is called polynomial least-squares regression. In general, the number of measurements must be greater (or equal) to the number of parameters. Even with that proviso, the data matrix X⊤ X can get very ill-conditioned, and hence it becomes hard to invert in a satisfactorily manner. This problem occurs more often when high order polynomials are used or when you are trying to over-parameterise the problem. One solution to this problem is given in section 3.3.2.  −1 The matrix X⊤ X X⊤ is called the left pseudo-inverse of X, and is sometimes denoted X+ . The pseudo inverse is a generalised inverse applicable for even non-square matrices and is dis −1 cussed in [152, p928]. M ATLAB can compute X⊤ X X⊤ with the pseudo inverse command, pinv, enabling the parameters to be evaluated simply by typing theta = pinv(X)*y, although it is more efficient to simply use the backslash command, theta = X\y.

A N EXAMPLE

OF POLYNOMIAL CURVE FITTING USING LEAST- SQUARES

Tabulated below is the density of air as a function of temperature. We wish to fit a smooth quadratic curve to this data. Temperature [◦ C] Air density [kg/m3 ]

−100 1.98

−50 1.53

0 1.30

60 1.067

100 0.946

160 0.815

250 0.675

350 0.566

To compute the three model parameters from the air density data we can run the following m-file. Listing 3.3: Curve fitting using polynomial least-squares 4 See also section 9.3.1. Ogata, [150, p938], also gives some helpful rules when using matrix-vector differentials, but uses a slightly different nomenclature standard.

3.3. CURVE FITTING EXPERIMENTAL DATA

113

2.5

Air density, [kg/m3]

Raw Data fitted curve 2

1.5

1

0.5 −200

−100

0

100

200

Temperature [°C]

300

400

Figure 3.17: The density of air as a function of temperature. Experimental data, •, and a fitted quadratic curve. (See also Fig. 3.18 following for a higher-order polynomial fit.)

T= [-100 -50 0 60 100, 160 250 350]'; % Temperature, T rho= [1.98, 1.53, 1.30, 1.067 0.946, 0.815, 0.675, 0.566]'; % Air density, ρ

5

X =vander(T);X =X(:,end-2:end); theta = X\rho;

% Keep only last 3 columns of the Vandermonde matrix % Solve for θ, where ρˆ = θ1 T 2 + θ2 T + θ3

Ti = linspace(-130,400); % validate with a plot rho_pred = polyval(theta,Ti); plot(T,rho,'o',Ti,rho_pred,'r-') % See Fig. 3.17.

The resulting curve is compared with the experimental data in Fig. 3.17. In the example above, we constructed the Vandemonde data matrix explicitly, and solved for the parameters using the pseudo-inverse. In practice however, we would normally simply use the built-in M ATLAB command polyfit which essentially does the same thing.

3.3.2

I MPROVEMENTS

TO THE LEAST SQUARES ESTIMATION ALGORITHM

There are many extensions to this simple multi-linear regression algorithm that try to avoid the poor numerical properties of the scheme given above. If you consider pure computational speed, then solving a set of linear equations is about twice as fast as inverting a matrix. Therefore instead of Eqn. 3.42, the equivalent X⊤ Xθ = X⊤ y (3.43) is the preferred scheme to calculate θ. M ATLAB ’ S backslash operator, \, used in the example above follows this scheme internally. We write it as if we expect an inverse, but it actually solves a system of linear equations. A numerical technique that uses singular value decomposition (SVD) gives better accuracy (for the same number of bits to represent the data), than just applying Eqn 3.42, [161, 178]. Singular values stem from the property that we can decompose any matrix T into the product of two orthogonal matrices (U, V) and a diagonal matrix Σ, T = UΣV⊤

(3.44)

where due to the orthogonality, UU⊤ = I = U⊤ U

and

VV⊤ = I = V⊤ V

(3.45)

Modelling

114

The diagonal matrix Σ consists of the square roots of the eigenvalues of T⊤ T which are called the singular values of T, and the number that differ significantly from zero is the rank of T. We can make one further modification to Eqn 3.42 by adding a weighting matrix W. This makes the solution more general, but often in practice one sets W = I. Starting with Eqn 3.43 including the weighting matrix W−1 = G⊤ G, X⊤ W−1 Xθ = X⊤ W−1 y ⊤

⊤

⊤

⊤

X G GXθ = X G Gy def

(3.46) (3.47)

def

Now we define T = GX and z = Gy gives T⊤ Tθ = T⊤ z

(3.48)

Now we take the singular value decomposition (SVD) of T in Eqn 3.48 giving    VΣU⊤ UΣV⊤ θ = VΣU⊤ z VΣ2 V⊤ θ =

(3.49) (3.50)

Multiplying both sides by V⊤ and noting that Σ is diagonal we get, Σ2 V⊤ θ = ΣU⊤ z ⊤

VV | {z } θ = VΣ

−1

(3.51) ⊤

U z

(3.52)

identity

−1 ⊤ θ = VΣ | {z U } z

(3.53)

pseudo inverse

The inverse of Σ is simply the inverse of each of the individual diagonal (and non-zero) elements, since it is diagonal. The key point here is that we never in the algorithm needed to calculate the possibly ill-conditioned matrix T⊤ T. Consequently we should always use Eqn. 3.53 in preference to Eqn. 3.42 due to the more robust numerical properties as shown in Listing 3.4. Listing 3.4: Polynomial least-squares using singular value decomposition. This routine follows from, and provides an alternative to, Listing 3.3. 1

[U,S,V] = svd(X,0); % Use the ‘economy sized’ SVD, UΣV⊤ = X theta2 = V*(S\U')*rho % θ = VΣ−1 U⊤ , Eqn. 3.53.

It does not take much for the troublesome XT X matrix to get ill-conditioned. Consider the temperature/density data for air from Fig. 3.17, but in this case we will use temperature in degrees Kelvin (as opposed to Celsius), and we will take data over slightly larger range of temperatures. This is a reasonable experimental request, and so we should expect to be able to fit a polynomial to this new data in much the same way we did in Listing 3.3. However as shown in Fig. 3.18, we cannot reliably fit a fifth-order polynomial using standard least-squares to this data set, although we can reliably fit such a 5th-order polynomial using the SVD strategy of Eqn. 3.53. Note that M ATLAB’s polyfit uses the reliable SVD strategy internally. A further refined regression technique is termed partial least squares (PLS), and is summarised in [74]. Typically these schemes decompose the data matrix X, into other matrices that are ordered in some manner that roughly corresponds to information. The matrices with the least information (and typically most noise) are omitted, and the subsequent regression uses only the remaining information. The example on page 112 used the least squares method for estimating the parameters of an algebraic equation. However the procedure for estimating a dynamic equation remains the same. This will be demonstrated later in §6.7.

3.3. CURVE FITTING EXPERIMENTAL DATA

115

Polynomial order = 5 3.5 3

Standard least−squares

Air density, [kg/m3]

θ = (X ′ X)−1 X ′ y

Raw Data standard LS SVD fitted

2.5 LS using SVD

2

θ = V Σ−1 U T y 1.5 1 0.5 0 100

200

300 400 500 Temperature [degrees Kelvin]

600

700

Figure 3.18: Comparing various ways to fit a 5th order polynomial to the density of air as a function of temperature in degrees Kelvin. (See also Fig. 3.17.)

Problem 3.3 Accuracy and numerical stability are very important when we are dealing with computed solutions. Suppose we wish to invert   1 1 A= 1 1+ǫ where ǫ is a small quantity (such as nearly the machine eps) 1. What is the (algebraic) inverse of A? 2. Obviously if ǫ = 0 we will have trouble inverting since A is singular, but what does the pseudo-inverse, A+ , converge to as ǫ → 0?

3.3.3

I DENTIFICATION

OF NONLINEAR MODELS

If the model equation is nonlinear in the parameters, then the solution procedure to find the optimum parameters requires a nonlinear optimiser rather than the relatively robust explicit relation given by Eqn. 3.42 or equivalent. Nonlinear optimisers are usually based on iterative schemes additionally often requiring good initial parameter estimates and even then may quite possibly fail to converge to a sensible solution. There are many algorithms for nonlinear optimisation problems including exhaustive search, the simplex method due to Nelder-Mead, and gradient methods. M ATLAB provides a simple unconstrained optimiser, fminsearch, which uses the Nelder-Mead simplex algorithm. A collection of more robust algorithms and algorithms to optimise constrained systems possibly with integer constraints is the O PTI toolbox5 . The following biochemical example illustrates the solution of a nonlinear algebraic optimisation problem. Many biological reactions are of the Michalis-Menton form where the cell number y(t) is given at time t by the relation αt y(t) = (3.54) βt + 1 In this model, time is the independent variable, y(t) is the observed variable, and the two parameters are   α def θ= β 5 The

O PTI toolbox is available from www.i2c2.aut.ac.nz

Modelling

116 Suppose we have some experimental data for a particular reaction given as time, t cell count, y

0 0

0.2 1.2

0.5 2.1

1 2.43

1.5 2.52

2 2.58

2.5 2.62

where we wish to estimate the parameters α and β in Eqn. 3.54 using nonlinear optimisation techniques. Note that the model equation (3.54) is nonlinear in the parameters. While it is not possible to write the equation in the linear form y = f (t) · θ as done in §3.3, it is possible to linearise the equation (Lineweaver-Burke plot) by transforming the equation. However this will also transform and bias the errors around the parameters. This is not good practice as it introduces bias in the estimates, but does give good starting estimates for the nonlinear estimator. Transforming Eqn. 3.54 we get y α y= − β βt Thus plotting y/t against y should result in a straight line with an intercept of α/β and a slope of −1/β. 3.5

Assuming we have the experimental data stored as column vectors [t,y] in M ATLAB, we could plot

3

cell count, y

t = [0, 0.2, 0.5:0.5:2.5]'; y = [0,1.2,2.1,2.43,2.52,2.58,2.62]'; % ignore divide-by-zero plot(y./t,y,'o-')

3

2.5

2

1.5

1

0

1

2

3 y/t

4

5

6

which should given an approximately straight line (ignoring the first and possibly the second points). To find the slope and intercept, we use the polyfit function to fit a line and we get a slope = −0.2686 and intercept = 2.98. This corresponds to α = 11.1 and β = 3.723. Now we can refine these estimates using the fminsearch Nelder-Mead nonlinear optimiser. We must first construct a a small objection function file which evaluates the sum of the squared error for a given set of trial parameters and the experimental data. This is succinctly coded as an anonymous function in Listing 3.5 following. Since we require the experimental data variables y and t in the objective function, but we are not optimising with respect to them, we must pass these variables as additional parameters to the optimiser. Listing 3.5: Curve fitting using a generic nonlinear optimiser 1

6

% Compute sum of square errors given trial θ and (t, y) data. sse= @(theta,t,y) sum((y-theta(1)*t./(theta(2)*t+1)).ˆ2); optns = optimset('Diagnostics','on','Tolx',1e-5); theta = fminsearch(sse,[11.1 3.723]',optns,t,y) % Polish the estimates. plot(t,y,'o',t, theta(1)*t./(theta(2)*t+1)) % Check final fit in Fig. 3.19.

Listing 3.5 returns the refined estimates for θ as     ˆ = α = 12.05 θ 4.10 β and a comparison of both the experimental data •, and the model’s predictions is given in Fig. 3.19.

3.3. CURVE FITTING EXPERIMENTAL DATA

117

α/β

3

cell count

2.5 2 1.5 1 0.5 0

Exptal data Best fit curve 0

0.5

C URVE FITTING

1

1.5 2 time

USING THE

2.5

3

3.5

Figure 3.19: The fitted (dashed) and experimental, •, data for a bio-chemical reaction. The asymptotic final cell count, α/β, is given by the dashed horizontal line.

O PTI OPTIMISATION

TOOLBOX

The M ATLAB O PTIMISATION toolbox contains more sophisticated routines specifically intended for least-squares curve fitting. Rather than write an objective function to compute the sum of squares and then subsequently call a generic optimiser as we did in Listing 3.5, we can solve the problem in a much more direct manner. The opti_lsqcurvefit is the equivalent routine in the O PTI toolbox that solves least-squares regression problems. Listing 3.6: Curve fitting using the O PTI optimisation toolbox. (Compare with Listing 3.5.)

4

>> theta0 = [11,3]'; % Initial guess θ0 >> F = @(x,t) x(1)*t./(x(2)*t+1); % f (θ, t) = θ1 t/(θ2 t + 1) >> >> theta = opti_lsqcurvefit(F,theta0,t,y) theta = 12.0440 4.1035

Note that in Listing 3.6 we encapsulate the function in an anonymous function which is then passed to the least-squares curve fit routine, opti_lsqcurvefit.

H IGHER

DIMENSIONAL MODEL FITTING

Searching for parameters where we have made two independent measurements is just like searching when we have made one independent measurement, except that to plot the results we must resort to contour or three-dimensional plots. This makes the visualisation a little more difficult, but changes nothing in the general technique. We will aim to fit a simple four parameter, two variable function to model the compressibility of water. Water, contrary to what we taught in school, is compressible, although this is only noticeable at very high pressures. If we look up the physical properties for compressed water in steam tables,6 we will find something like table 3.4. Fig. 3.20(a) graphically illustrates the strong temperature influence on the density compared with pressure. Note how the missing data represented by NaNs in M ATLAB is ignored in the plot. Our model relates the density of water as a function of temperature and pressure. The proposed 6 Rogers

& Mayhew, Thermodynamic and Transport properties of Fluids, 3rd Edition, (1980), p11

Modelling

118 Table 3.4: Density of compressed water, (ρ × 10−3 kg/m3 ) Temperature, ◦ C 200 250 300 0.8711 0.8065 0.7158 0.8795 0.8183 0.7391 0.8969 0.8425 0.7770 0.9242 0.8772 0.8244

Pressure (bar) 0.01 1.0050 1.0111 1.0235 1.0460

100 221.2 500 1000

100 0.9634 0.9690 0.9804 1.0000

350 — 0.6120 0.6930 0.7610

374.15 — — 0.6410 0.7299

density (kg/m3) 1100 1000

1000

900

950

850

800

1000

775

900 Pressure (bar)

True density kg/m

3

1100

900 850 800 800

700 600 0

1000 100 200

500

Temperature, C

600 500

750

400

700

300

700 700

200

650

300 °

700

1e+003 1e+003

1e+003

925 925

850

775

100

400

0

Pressure (bar)

0

50

(a) True density of compressed water as a function of temperature and pressure.

100

150 200 250 Temperature (deg C)

300

350

400

(b) Model fit

Figure 3.20: A 2D model for the density of compressed water. In Fig. 3.20(b), the • mark experimental data points used to construct the 2-D model given as contour lines. model structure is

  b c ρ = P exp a + + 3 T T k

(3.55)

where the model parameters def

θ= need to be determined.



a

b

c

k

⊤

(3.56)

Again the approach is to minimise numerically the sum of the squared errors using an optimiser such as fminsearch. We can check the results of the optimiser using a contour plot with the experimental data from Table 3.4 superimposed. The script file in Listing 3.7 calls the minimiser which in turn calls the anonymous function J_rhowat which given the experimental data and proposed parameters returns the sum of squared errors. This particular problem is tricky, since it is difficult to know appropriate starting guesses for θ, and the missing data must eliminated before the optimisation. Before embarking on the full nonlinear minimisation, I first try a linear fitting to obtain good starting estimates for θ. I also scale the parameters so that the optimiser deals with numbers around unity. Listing 3.7: Fitting water density as a function of temperature and pressure rhowat = @(a,P,T) P.ˆa(1).*exp(a(2) + ... 1e2*a(3)./T + 1e7*a(4)./T.ˆ3) % Assumed model ρ = P k exp a + J_rhowat = @(a,P,T,rho) ...

b T

+

c T3




3.3. CURVE FITTING EXPERIMENTAL DATA norm(reshape(rho,[],1)-reshape(rhowat(a,P,T),[],1)); % SSE J =

4

9

14

119 P

(ρ − ρˆ)2

T = [0.01, 100, 200, 250,300,350, 374.15]'; % Temp [deg C] P = [100 221.2 500 1000]'; % Pressure [Bar] rhof = 1.0e3*[1.00250, 0.9634, 0.8711, 0.8065, 0.7158, NaN, NaN; ... 1.0111, 0.9690, 0.8795, 0.8183, 0.7391, 0.6120, NaN; ... 1.0235, 0.9804, 0.8969, 0.8425, 0.7770, 0.6930, 0.6410; ... 1.0460, 1.0000, 0.9242, 0.8772, 0.8244, 0.7610, 0.7299]; % density kg/m3 [TT,PP] = meshgrid(T+273,P); Tv = TT(:); Pv = PP(:); % vectorise A = [ones(size(Tv)), 1.0e2 ./Tv, 1.0e7 ./Tv.ˆ3, log(Pv)]; % Scaled data matrix idx = isnan(rhof(:)); % find missing data points rhofv = rhof(:); rhofv(idx) = []; % remove bad points Tv(idx) = []; Pv(idx) = []; A(idx,:) = []; theta = A\log(rhofv); % first (linear) estimate

19

% Do nonlinear fit theta_opt = fminsearch(@(theta) J_rhowat(theta,Pv,Tv,rhofv),theta);

24

[Ti,Pi] = meshgrid([0.01:10:370]+273,[100:100:1000]); % Compare fit with data rho_est = rhowat(theta_opt,Pi,Ti);

Fig. 3.20(b) compares contour plots of the density of water as a function of pressure and temperature derived from the experimental data and the fitted model. The • shows the location of the experimental data points. The solid contour lines give the predicted density of water compared with the contours derived from the experimental data (dashed lines). The optimum parameters found above are     5.5383 a  b   5.8133 · 102     (3.57) θ⋆ =   c  =  −1.8726 · 107  0.0305 k

3.3.4

T HE

CONFIDENCE OF THE OPTIMISED PARAMETERS

Finding the optimum model parameters to fit the data is only part of the task. A potentially more difficult objective is to try to establish how precise these optimum parameters are, or what the confidence regions of the parameters are. simply expressing the uncertainty of your model as parameter values ± some interval is a good first approximation, but it does neglect the interaction of the other parameters, the so-called correlation effect. The following data was taken from [86, p198], but with some scaling, modifications and corrections. Himmelblau, [86, p198], gives some reaction rate data, y, as a function of pressure, x, as:

Pressure, x rate, y

2 0.68

3 0.858

3.5 0.939

4 0.999

5 1.130

5.5 1.162

6 1.190

and proposes a two parameter nonlinear model of the form yˆ(x) =

θ1 x 1 + θ2 x

(3.58)

In this regression problem we have n = 2 parameters, and m = 7 experimental observations.

Modelling

120

Now suppose that we use a nonlinear optimiser to search for the parameters θ such that the sum of squared error is minimised, ! n X ⋆ 2 (3.59) (yi − yˆi ) θ = arg min θ i=1 If we do this optimisation using say the same technique as described in §3.3.3 calling lsqcurvefit with an initial guess of say θ T = [1, 1], Listing 3.8: Finding the optimum parameter for a nonlinear reaction rate model 1

6

x = [2,3,3.5,4,5,5.5,6]'; % Pressure, x y = [0.68,0.858,0.939,0.999,1.130,1.162,1.190]'; % Reaction rate, y(x) % y(x) =

Rxn_rate = @(p,x) p(1)*x./(1+p(2)*x); theta0 = [1,1]'; % initial guess theta = lsqcurvefit(Rxn_rate,theta0,x,y);

θ1 x 1+θ2 x

% Find optimum θ using lsqcurvefit

then we find that the optimum parameters are approximately   0.5154 θ⋆ = 0.2628

(3.60)

and the raw data and model predictions are compared in Fig. 3.23. Experimental data (o) & model fit 1.3 Data model

1.2 1.1

rate

1 0.9 0.8 0.7 0.6

Figure 3.21: Fitting experimental raw data (•), to a nonlinear model.

0.5

1

2

3

4 pressure

5

6

7

The fit in Fig. 3.21 looks reasonable, but it would be prudent to be able to statistically defend the quality of the fit. To do that, we need to establish the individual confidence limits of the parameters, and to do that, we need to know the following: 1. The n optimum parameters θ obtained from the m experimental observations. (For the nonlinear case, this can be found using a numerical optimiser as shown above in Listing 3.8.) 2. A linearised (m × n) data matrix, also known as a Jacobian, X, centered about the optimised parameters at each observation defined as def

Xi,j =

∂yi ∂θj

(3.61)

Thus each of the m rows of X are the partial derivatives of that particular observation with respect to each of the n parameters. Note that the dimensions of X should be taller than wide.

3.3. CURVE FITTING EXPERIMENTAL DATA

121

It is possible that the numerical optimiser can export X at the solution, or we could compute it using either an analytical expression or finite differences. 3. We can now form the (scaled) covariance matrix def

P= from the data matrix.



−1 X⊤ X

(3.62)

Being a covariance matrix, P must be symmetric positive definite although in practice this requirement may not always hold owing to poor numerical conditioning. The diagonal elements are the variances we will use for the individual confidence limits. When using M ATLAB, it is better to look at the singular values, or check the rank of X⊤ X before doing the inversion. 4. An estimate of the measurement noise sY¯i . This may be known from prior knowledge, or simply approximated by s p ǫ⊤ ǫ (3.63) sY¯i ≈ s2r = m−n where the ǫ⊤ ǫ is the sum of squared residuals, and the positive integer ν = (m−n) is known as the degree of freedom.

5. Finally we also need to decide on a suitable confidence interval, typically 90%, 95% or 99%. One this is decided, we can compute the value of the inverse cumulative t-distribution at ν degrees of freedom using appropriate statistical tables, or the simple statistical functions given in Listing 3.9. Listing 3.9: Routines for the cumulative probability distribution, pt, and the inverse CDF, qt, for the t-distribution.

4

pt = @(x,v) (x≥0).*(1-0.5*betainc(v./(v+x.ˆ2),v/2,0.5)) + ... (x>G_est= pem(Zdata,'na',2,'nb',2,'nc',2,'nk',1);% Estimate A(q −1 ), B(q −1 ) & C(q −1 ) >>G_est= armax(Zdata,'na',2,'nb',2,'nc',2,'nk',1)% Alternative to using pem.m 4

9

>>present(G_est) % present results G_est = Discrete-time ARMAX model: A(z)y(t) = B(z)u(t) + C(z)e(t) A(z) = 1 - 1.416 zˆ-1 + 0.514 zˆ-2 B(z) = 1.209 zˆ-1 + 0.2935 zˆ-2 C(z) = 1 + 0.7703 zˆ-1 - 0.1181 zˆ-2

14

>>yp = sim(G_est,[u e]); % form prediction >>plot([y yp]) % check prediction performance in Fig. 6.59.

Fig. 6.59 compares the true and predicted output data. In the left-hand case, the fit is not perfect even though we used the correct model structure which is partly due to the iterative nature of the nonlinear regression, but mostly the poor fit is due to the fact that we used only 100 input/output samples. On the other hand, if we used 10,000 samples for the fitting as in the right-hand case, we will estimate a far superior model. It is interesting to note just how many input/output samples we need to estimate a relatively modest sized system of only 6 parameters. Using only 100 points for estimation

Using 10000 points for estimation

15

−2 −4

Output & estimated output

10

−6 −8

5

−10 0

−12 −14

−5

−16 −18

−10

−15

0

20

40 60 sample time

Actual Estimated

−20

Estimated 80

100

−22 9900

9920

9940 9960 sample time

9980

10000

Figure 6.59: A validation plot showing the actual plant output compared to the model predictions of an ARMAX model using series of different lengths. Note that to obtain good results, we need a long data series. Listing 6.22 repeats this estimation, but this time does it recursively in M ATLAB.

6.10. ONLINE ESTIMATING OF NOISE MODELS

311

Listing 6.22: Recursively identify an ARMAX process. 1

A = [1 -0.4 0.5]; B = [1.2, 0.3]; C = [1, 0.8 -0.1]; % True plant to be estimated na = length(A); nb = length(B); nc = length(C); % Order of the polynomials+1 d = na-nb; % deadtime

6

11

Npts = 5e4; u = randn(Npts,1); a = randn(Npts,1); y = zeros(Npts,1); err = y; thetaAll = [];

% dummy

np = na+nb+nc - 2; % # of parameters to be estimated theta = randn(np,1); P = 1e6*eye(np); % initial guess of θ lambda = 1; % forgetting factor, λ ≤ 1 for i=na:Npts; y(i)= -A(2:end)*y(i-1:-1:i-na+1)+ B*u(i-d:-1:i-nb+1-d)+ C*a(i:-1:i-nc+1);

16

phi = [-y(i-1:-1:i-na+1)', u(i-d:-1:i-nb+1-d)', err(i-1:-1:i-nc+1)']'; err(i,1) = y(i) - phi'*theta; % estimated noise sequence

21

26

K = P*phi/(lambda + phi'*P*phi); % update gain P = (P-(P*phi*phi'*P)/(lambda + phi'*P*phi))/lambda; % covariance update theta = theta + K*err(i,1); % update θ thetaAll(i,:) = theta'; end Aest = [1, thetaAll(end, 1:na-1)]; Best = thetaAll(end, na:na+nb-1); Cest = [1, thetaAll(end,na+nb:end)]; % See Fig. 6.60.

As shown in Fig. 6.60, the estimates do eventually converge to the correct values without bias, and furthermore, the step response of the final estimate compares favourably with the true step response.

6.10.2

R ECURSIVE

IDENTIFICATION USING THE

SI

TOOLBOX

The System Identification toolbox can also calculate the parameter estimates recursively. This essentially duplicates the material presented in §6.7.1. However in this case there are many more options to choose from regarding type of method, numerical control, starting point etc. The toolbox has a number of recursive algorithms (See [126, p1–64]) but they all work in the same way. For most demonstration purposes, we will recursively estimate the data pseudo online. This means that we will first generate all the data, and then estimate the parameters. However we will make sure that when we are processing element i, the data ahead in time, elements i + 1 . . ., are not available. This pseudo online approach is quite common in simulation applications. The Toolbox command that duplicates the recursive least squares estimation of a ARMAX process with a forgetting factor is rarmax(z,nn,’ff’,0.95).

6.10.3

S IMPLIFIED RLS

ALGORITHMS

The standard recursive least squares (RLS) algorithm such as presented in §6.7.1 requires two update equations;– one for the update gain, and one for the covariance matrix. But for implementation in a small computer, there is some motivation to simplify the algorithm, even at the ˚ om expense of the quality of parameter estimates. Astr ¨ and Wittenmark, [18, p69-70] describe one

CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

312

RELS: λ=0.9990 2

Model at k=4e4 5

0

1.5

Estimated Actual −5 0

1

5

10

15

20

25

(seconds)

Parameter estimates

c b22 0.5 a b3 3

b c31

0

−0.5 Model at k=2e4

−1

3

a2

2

−1.5

1

Estimated Actual

0 0

−2

5

10

15

20

25

(seconds)

0

1

2

3 sample time [k]

4

5 4

x 10

Figure 6.60: Recursive extended least-squares estimation where there is an abrupt model change at time k = 2 · 104. The inserted figures compare a step response of the true plant with the current estimate. such method known as Kaczmarz’s projection algorithm. The input/output relation is assumed yk = ϕk θk where θ is the parameter vector and ϕ is the row data vector, and the correction to the estimated parameter vector is ˆk = θ ˆ k−1 + α ϕ⊤ θ k ˆ k which gives the full updating formula in our now familiar where α is chosen such that yk = ϕk θ form as   T ˆk = θ ˆ k−1 + ϕ ˆ θ (6.79) y − ϕ θ k k k−1 ϕk ϕT

Note that in [18], there is some nomenclature change, and that the data vector is defined as a column vector, where here I have assumed it is a row vector. This update scheme (Eqn 6.79) is sometimes modified to avoid potential problems when the parameter vector equals zero.

The script in listing 6.23 first generates data from an ARX plant, and then the data is processed pseudo-online to obtain the estimated parameters. Listing 6.23: Kaczmarz’s algorithm for identification u = randn(50,1); 3

8

% random input

A = [1 -1.25 0.8]; B = 0.3; % G = 0.3/(1 − 1.25q −1 + 0.8q −2 ) G = idpoly(A,B,1); y = sim(G,u); theta = randn(3,1);% Initial parameter estimate, θ0 , is random Param = []; % estimated parameters so far for i=4:length(u); phi= [-y(i-1:-1:i-2)', u(i)]'; % regressor vector, ϕk

6.10. ONLINE ESTIMATING OF NOISE MODELS

313

  ˆk ← θ ˆ k−1 + ϕTk yt − ϕT θ ˆ k−1 theta=theta+phi*(y(i)-phi'*theta)/(phi'*phi); % θ k ϕk ϕ Param= [Param; theta']; % collect parameters end % 13

Param_x = ones(size(Param(:,1)))*([A(2:3) ,B(1)]); % true parameters plot([Param_x, Param]) % have they converged ?

Input & output

The input/output data (top) and parameters from this data are shown in Fig. 6.61. The estimated parameters converge relatively quickly to the true parameters, although not as quickly as in the full recursive least-squares algorithm. 5

0

−5 1 a Estimated & true parameters

2

0.5 b

0

0

−0.5

−1 a1 −1.5

0

10

20 30 sample time

40

50

Figure 6.61: The performance of a simplified recursive least squares algorithm, Kaczmarz’s algorithm. Upper: The input/output data. Lower: The estimated parameters (solid) eventually converge to the true parameters (dashed).

Problem 6.3 The following questions use input/output data from the collection of ‘classic’ time series data available from the time series data library maintained by Rob Hyndman at: www-personal.buseco.monash.edu.au/˜hyndman/TSDL/ or alternatively from the DAISY: D ATABASE FOR THE I DENTIFICATION OF S YSTEMS collection maintained by De Moor, Department of Electrical Engineering, ESAT/SISTA, K.U.Leuven, Belgium at www.esat.kuleuven.ac.be/sista/daisy/. These collections are useful to test and verify new identification algorithms. Original sources for this data include [34] amongst others. 1. The data given in /industry/sfbw2.dat shows the relationship in a paper making machine between the input stock flow (gal/min) and basis weight (lb/3300 ft2 ), a quality control parameter of the finished paper. Construct a suitable model of this time series. Ref: [157, pp500–501]. 2. This problem is detailed and suitable for a semester group project. An alternative suite of programs to aid in model development and parameter identification is described in [34, Part V]. The collection of seven programmes closely follows the methodology in the text, and are outlined in pseudo-code. Construct a T IME S ERIES A NALYSIS toolbox with graphics based around these programs. Test your toolbox on one of the time series tabulated in the text. A particularly relevant series is the furnace data that shows the percentage of carbon dioxide output from gas furnace at

CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

314

9 seconds intervals as a function of methane input, (ft3 /min), [34, Series J, pp532–533]. This input/output data is reproduced in Fig. 6.62, although the input methane flowrate is occasionally given as negative indicating that the series is probably scaled. The original reference footnoted that the unscaled data lay between 0.6 and 0.04 ft3 /min. 65

% CO2

60 55 50

Figure 6.62: The percent CO2 in the exhaust gas from a furnace as a function of the methane input (scaled). Data from [34, p532–533].

6.11

Methane flow

45 4 2 0 −2 −4

0

500

1000

1500

2000

2500

time [s]

C LOSED LOOP IDENTIFICATION

The identification techniques described this far assume that the input signal can be chosen freely and that this signal has persistent excitation. However if you are trying to identify while controlling, you have lost the freedom to choose an arbitrary input since naturally, the input is now constrained to give good control rather than good estimation. Under good control, the input will probably be no longer exciting nor inject enough energy into the process so that there is sufficient information available for the identification. Closed loop identification occurs when the process is so important, or hazardous, that one does not have the luxury of opening the control loop simply for identification. Closed loop identification is also necessary, (by definition), in many adaptive control algorithms. If we use a low order controller, then the columns of the data matrix X in Eqn. 6.22 become linearly dependent. This is easy to see if we use a proportional controller, ut = kyt , then this in turn creates columns in X that only differ by the constant controller gain k. In reality, any noise will destroy this exact dependency, but the numerical inversion may still be a computationally difficult task. One way to ensure that u(t) is still persistently exciting the process is to add a small dither or perturbation signal on to the input u. This noisy signal is usually of such a small amplitude, that the control is not seriously effected, but the identification can still proceed. In some cases there may be enough natural plant disturbances to make it unnecessary to add noise to u. With this added noise, added either deliberately or otherwise, the estimation procedure is the same as described in § 6.7. The above arguments seems to suggest that closed loop identification is suboptimal, and that if possible, one should always try to do the identification in open loop, design a suitable controller, then close the loop. This avoids the possible ill-conditioning, and the biasing in the estimated parameters. However Landau in a series of publications culminating in [112] argues that in fact closed loop control, with suitable algorithms, actually develops better models since the identification is constrained to the frequencies of interest for good closed loop control. It all depends in the final analysis on whether we want a good model of the plant for say design in which case we do

6.12. SUMMARY

315

the identification in open loop, or if we want good control in which case closed loop identification may be better. A very readable summary of closed loop identification is given in [93, pp517–518], [18, p82] and especially [203]. Identification in the closed loop is further discussed in chapter 7.

6.11.1

C LOSED

LOOP

RLS

IN

S IMULINK

Implementing recursive identification using S IMULINK is not as straight forward as writing raw M ATLAB code because of the difficulty in updating the parameters inside a transfer function block. Under normal operation these are considered constant parameters within the filter block, so it is not a problem. Fig. 6.63 shows the closed loop estimation using the same RLS blocks as those used in Fig. 6.47 on page 295. In this example the identified parameters are not used in the controller in any sort of adaptive manner, for that we need an adaptive controller which is the subject of the following chapter.

theta

5

phi

5

P

y(k)

phi generator

Plant i/o data

y

u

uy

theta

phi1

5

5 5

[5x5]

theta Parameters

P Covariance

RLS

3

u

Signal Generator

PID

Gd

PID Controller

Unknown plant

y

y

Figure 6.63: Closed loop estimation using RLS in S IMULINK. See also Fig. 6.47.

6.12

S UMMARY

System identification is where we try to regress or fit parameters of a given model skeleton or structure to fit input/output data from a dynamic process. Ideally we would obtain not only the values of the parameters in question, but also an indication of the “goodness of fit”, and the appropriateness of the model structure to which the parameters are fitted. Identification methodologies usually rely on a ‘trial & error’ approach, where different model structures are fitted, and the results compared. The S YSTEM I DENTIFICATION T OOLBOX within M ATLAB, [126], is a good source of tools for experimentation in this area, because the model fitting and analysis can be done so quickly and competently. Identification can also be attempted online, where the current parameter estimate is improved or updated as new process data becomes available in an efficient way without the need to store all the previous historical data. We do this by re-writing the least-squares update in a recursive algorithm. Updating the parameters in this way enables one to follow time-varying models or even nonlinear models more accurately. This philosophy of online estimation of the process model is a central component of an adaptive control scheme which is introduced in chapter 7. However there are two key problems with the vanilla recursive least-squares algorithm which

316

CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

become especially apparent when we start to combine identification with adaptive control. The first is the need to prevent the covariance matrix P from being illconditioned. This can, in part be solved by ensuring persistent excitability in input. The second problem is that given non-white noise or coloured noise, the RLS scheme will deliver biased estimates. However, there are many extensions to the standard schemes that address these and other identification issues. Succinct summaries of system identification with some good examples are given in [23, pp422–431] and [150, p856], while dedicated texts include [125, 190, 203].

6.12. SUMMARY

317

P ROBLEMS Problem 6.4 An identification benchmark problem from [108]. 1. Simulate the second-order model with external disturbance vt , yt =

2 X

ai yt−i +

i=1

2 X

bi ut−i +

2 X i=1

i=0

di vt−i + e(t), e(t) ∼ N (0, σ 2 )

(6.80)

where the parameter values are: a1 0.98

a2 −0.9

b0 0.5

b1 −0.25

b2 0.1

d1 0.8

d2 0.2

σ 0.1

The input ut is normally distributed discrete white noise, and the external disturbance is to be simulated as a rectangular signal alternating periodically between the values +1 and −1 at t = 100, 200, . . . Run the simulation for 600 time steps, and at t = 300, change a1 to −0.98. 2. This is a challenging identification problem, because the rarely varying external disturbance signal gives little information about the parameters d. Identify the parameters in Eqn. 6.80 ˆ 1|0 = 0, P1|0 = 50I and using an exponential forgetting factor, λ = 0.8. perhaps starting with θ Problem 6.5 1. Run through the demos contained in the S YSTEM I DENTIFICATION toolbox for M ATLAB. 2. Investigate the different ways to simulate a discrete plant with a disturbing input. Use the model structure A(q −1 )yt = B(q −1 )ut + C(q −1 )et Choose a representative stable system, say something like A(q −1 ) = 1 − 0.8q −1 ,

B(q −1 ) = 0.5,

C(q −1 ) = 1 + 0.2q −1

and choose some sensible inputs for ut and et . Simulate the following and explain any differences you find. (a) Write your own finite difference equation in M ATLAB, and compare this using filter. (b) Use the transfer function object, tf, in MISO (multiple input/single output) mode. Hint: Use cell arrays as >>G = tf( {B ,C} , {A,A},T); % discrete version >>G.variable = ’q’;

(c) Use idpoly and idmodel/sim from the S YSTEM I DENTIFICATION toolbox. (d) Build a discrete S IMULINK model Hint: You may find problems due to the differing lengths of the polynomials. Either ensure all polynomials are the same length, or swap from z −1 to z, or introduce delays. 3. Repeat problem 2 with an unstable model (A is unstable.) 4. What is the difference in S IMULINK when you place the A polynomial together with the B and C polynomials? Particularly pay attention to the case where A is unstable. 5. Identification of plants that include coloured noise inputs is considerably more complex than just using arx. The following illustrates this. (a) Create a ‘truth’ plant with polynomials, A, B, C. (Use idpoly).

318

CHAPTER 6. IDENTIFICATION OF PROCESS MODELS (b) Simulate this plant using idmodel/sim for the following two cases: i. A noise-free version: i.e. random u(t) and no e(t) ii. A version with coloured noise: i.e. random u(t), e(t). (c) Estimate an ARX model (i.e. only A, B polynomials) using both data sets. ‘Present’ the model with uncertainty bounds on the estimated parameters using present. What was the problem with the coloured noise version and how can we avoid it? (d) Estimate the A, B and C polynomials using an extended-LS scheme such as armax or pem. Has the identification improved significantly? How did the computational time change from using to arx to armax?

6. The D AISY (Data Base for the Identification of Systems), [137], located at www.esat.kuleuven.ac.be/sista/d and contains a number of simulated and experimental multivariable data sets. Down load one of these data sets, preferably one of the real industrial data trends, and make an identification. 7. RLS with modified forgetting factors Construct a simulation to estimate the parameters in a dynamic model (as in tident.m) to test various extensions to the forgetting factor idea. Create a simulation with output noise, and step change the model parameters part way through the run. (For more information on these various extensions, see [203, pp140–160].) (a) Use a start-up forgetting factor of the form    −t λ1 (t) = λ0 + (1 − λ0 ) 1 − exp τf

(6.81)

where λ0 is the initial forgetting factor, (say 0.9–0.95), and τf is the time constant that determines how fast λ(t) approaches 1.0. Note that Eqn. 6.81 can be re-written in recursive form as λ1 (t) = αλ1 (t − 1) + (1 − α) where α = e−1/τf and λ1 (0) = λ0 . (b) Combine with the start-up forgetting factor, an adaptive forgetting factor of the form   ǫ2 (t) τf λ2 (t) = 1− (6.82) τf − 1 τf sf (t) where sf (t) is a weighted average of past values of the squared error ǫ2 . [203, p159] suggest using the filter sf (t) =

τf − 1 ǫ2 sf (t − 1) + τf τf

Combine both the start-up and the adaptive forgetting factor to get a varying forgetting factor λ(t) = λ1 (t)λ2 (t). (c) A directional forgetting factor tries to update in only those directions in parameter space where there is information available. The covariance update with directional forgetting is # " xt x⊤ t Pt−1 (6.83) Pt = Pt−1 I − −1 rt−1 + x⊤ t Pt−1 xt where the directional forgetting factor r(t) is rt = λ′ −

1 − λ′ x⊤ t+1 Pt xt+1

and the scalar λ′ is like the original fixed forgetting factor.

(6.84)

6.12. SUMMARY

319

What ever happened to elegant solutions? 8. Compare the operation count of the RLS scheme with square root updates with the normal update. Construct a sufficiently ill-conditioned example where you can demonstrate a significant difference. Summary of the RLS square root update scheme for y = ϕT θ. Define the square root of P as def

P = SST Update St by following f t = STt−1 ϕt βt = 1 + f T f 1 √ αt = βt + βt Lt = St−1 f t St = St−1 − αt Lt f Tt Update the parameter estimates using θt+1 = θ t +


Lt yt − ϕT θt βt

9. Modify the square root RLS scheme to incorporate a forgetting factor. 10. Construct an RLS update algorithm in S IMULINK, perhaps using the matrix/vector blocks in the DSP blockset. Use the following update equations: " # ϕk+1 ϕTk+1 Pk Pk ϕk+1 , Pk+1 = Pk I − Kk+1 = 1 + ϕTk+1 Pk ϕk+1 1 + ϕTk+1 Pk ϕk+1
θk+1 = θk + Kk+1 yk+1 − ϕTk+1 θk

Note that as of Matlab R12, there is considerable support for vector/matrix manipulations in raw S IMULINK.

CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

320

11. Construct a square root update algorithm (from Problem 8) in S IMULINK, perhaps using the matrix/vector blocks in the DSP blockset. 12. Wellstead & Zarrop, [203, p103], develop an expression for the bias as a function of the model parameters for a simple 1-parameter model |a| < 1,

yt = ayt−1 + et + cet−1 ,

|c| < 1

where et is white noise with variance σe2 , as a ˆ−a≈

c(1 − a2 ) 1 + c2 + 2ac

(a) Validate with simulation the expression for the size of the bias, a − a ˆ. (b) The size of the bias should not be a function of the size of the variance of et . Is this a surprising result? Verify this by simulation. What happens if you drop the variance of the noise, σe2 , to zero? Do you still observe any bias? Should you? 13. Starting with the S IMULINK demo rlsest which demonstrates an adaptive pole-placement of a discrete plant do the following tasks. (a) Modify the rlsest simulation to use a continuous plant of your choice and, for the moment, a fixed (non-adaptive) PID controller. (I.e. disconnect the closed-loop adaption part of the controller.) Use Fig. 6.64 as a guide to what you should construct.

theta To Workspace1 RLS Estimator parameters Recursive least squares Parameter Estimator phi Clock1

t time yr To Workspace

State Generator

ZOH

ZOH1

Mux spt/plant

Setpoint

+ −

PID PID Controller

u[n]

3(s+0.3) (s+2)(s+1)(s+0.4) plant

y[n] + + Sum1

distrurbance

Figure 6.64: A modified version of rlsest in Simulink. Explain in detail how the state generator works and how the RLS estimator works. Modify the blocks so that you can identify a different number of A terms from the B terms, (such as say two B(q −1 ) and three A(q −1 ) polynomials terms). (b) Run the estimation routine under closed-loop conditions. What do you observe especially as you change the noise level? (To be realistic, you should add at least one more noise source, where ?)

6.12. SUMMARY

321

(c) Modify the RLS estimator s-function to export the covariance matrix as well as the updated parameters. Do these confirm your findings from part 13b? (d) Modify your RLS scheme to better handle noise and the closed loop conditions. You may like to use some simple heuristics based on the trace of the covariance matrix, or use a directional and/or variable forgetting factor. 14. Read the M ATLAB Technical support note #1811 How Do I Create Time Varying Matrix Gain and State-space Blocks as C MEX S-functions?. Download the relevant files and implement an adaptive controller. 15. Model identification in state-space with state information. The English mathematician Richardson has proposed the following simple model for an arms race between two countries: xk+1 = axk + byk + f yk+1 = cxk + dyk + g

(6.85) (6.86)

where xk and yk are the expenditures on arms of the two nations and a, b, c, d, f and g are constants. The following data have been compiled by the Swedish International Peace Research Institute, (SIPRI yearbook 1982). Millions of U.S. dollars at 1979 prices and 1979 exchange rates

year 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981

Iran 2891 3982 8801 11230 12178 9867 9165 5080 4040

Iraq 909 1123 2210 2247 2204 2303 2179 2675

NATO 216478 211146 212267 210525 205717 212009 215988 218561 225411 233957

WTO 112893 115020 117169 119612 121461 123561 125498 127185 129000 131595

(a) Determine the parameters of the model and investigate the stability of this model. (b) Determine the estimate of the parameters based on 3 consecutive years. Look at the variability of the estimates. (c) Create a recursive estimate of the parameters. Start with 1975 with the initial values determined from 1972–74.

322

CHAPTER 6. IDENTIFICATION OF PROCESS MODELS

C HAPTER 7

A DAPTIVE C ONTROL Adopt, adapt and improve . . . Motto of the round table. John Cleese whilst robbing a lingerie boutique.

7.1

W HY ADAPT ?

Since no one yet has found the “holy grail” of controllers that will work for all conceivable occasions, we must not only select an appropriate class of controller, but also be faced with the unpleasant task of tuning it to operate satisfactorily in our particular environment. This job is typically done only sporadically in the life of any given control loop, but in many cases owing to equipment or operating modifications, the dynamics of the plant have changed so as to render to controller sub-optimal. Ideally the necessary re-tuning could be automated, and this is the idea behind an adaptive controller. In a control context, adaption is where we adjust the controller as the process conditions change. The problem is that for tight control we essentially want to have a high feedback gain, giving a rapid response and good correction with minimal offset. However if the process dynamics subsequently change due to environmental changes for example, the high feedback gain may create an unstable controller. To prevent this problem, we adapt the controller design as the process conditions change, to achieve a consistently high performance. Adaptive control was first used in the aerospace industry around the late 1950s where automatic pilots adapted the control scheme depending on external factors such as altitude, speed, payload and so forth. One classic example was the adaptive flight control system (AFCS) for the X-15 in the late 1960s described in [8]. The adaptive control system was developed by Honeywell and was fundamentally a gain scheduler that adapted the control loop gains during the flights that ranged from sea level to space at 108 km. Unfortunately the crash of X-15-3 and loss of pilot Michael Adams in 1967 somewhat terminated the enthusiasm of adaptive flight control, and the research program was terminated shortly afterwards. In the past couple of decades however, adaptive control has spread to the processing industries with several commercial industrial adaptive controllers now available. Some applications where adaptive control is now used are: Aeronautical control As the behaviour of high performance fighter aircraft depends on altitude, 323

CHAPTER 7. ADAPTIVE CONTROL

324

payload, speed etc, an adaptive control is vital for the flight to be feasible. Tanker steering The dynamics of oil supertankers change drastically depending on the depth of water, especially in relatively shallow and constricted water-ways. Chemical reactions Chemical processes that involve catalytic reactions will vary as the catalyst denatures over time. Often coupled to this, is that the heat transfer will drop as the tubes in the heat exchanger gradually foul. Medical The anesthetist must control the drug delivery to the patient undergoing an operation, avoiding the onsets of either death or consciousness.1 Some further examples of adaptive control are given in a survey paper in [182]. Adaptive control is suitable:

Adaptive control is not suitable if:

• if the process has slowly varying parameters

• PID control is already satisfactory

◦ if clean measurements are available

• The system output is essentially constant

• for batch processes or startup/shut down of continuous processes

• The system is essentially linear and time invariant

• for widely changing operating conditions (such as in pH controlled processes)

• If the system is nonlinear, but easy to model

7.1.1

T HE

ADAPTION SCHEME

When faced with tuning a new control loop, the human operator would probably do the following steps: 1. Disturb the plant and collect the input/output data. 2. Use this data to estimate the plant dynamics. 3. Using the fitted model, an appropriate controller can be designed. 4. Update the controller with the new parameters and/or structure and test it. If the scheme outlined above is automated, (or you find an operator who is cheap and does not easily get bored), we can return to step 1, and repeat the entire process at regular intervals as shown in Fig. 7.1. This is particularly advantageous in situations where process conditions can change over time. The process and feedback controller are as normal, but we have added a new module in software (dashed box) consisting of an on-line identification stage to extract a process model M(θ), and given this model information, a controller design stage to find suitable controller tuning constants. Of course there will be some constraints if we were to regularly tune our controllers. For example, it is advisable to do the the estimation continuously in a recursive manner, and not perturb the plant very much. We could adapt the three tuning constants of a standard PID controller, or we could choose a controller structure that is more flexible such as pole-placement. The adaption module could operate continuously or it could be configured to adapt only “on demand”. In the latter case, once the tuning parameters have converged, the adaption component of the controller turns itself off automatically, and the process is now controlled by a constant-tuning parameter controller such as a standard PID controller. 1 One

particularly ambitious application is described in [54, p429].

7.2. GAIN SCHEDULING

325

Adaptive module

Control design ✛

Identify M(θ) ✯

❄ r(t) setpoint

+ ✲+

✲

✻−

controller

❨ Measure input & output

u(t)

y(t) ✲

input

Plant, S

✲ output

Figure 7.1: The structure of an indirect adaptive controller

7.1.2

C LASSIFICATION

OF ADAPTIVE CONTROLLERS

As one may expect, there are many different types of adaptive control, all possessing the same general components, and an almost equal number of classification schemes. To get an overview of all these similar, but subtly different controllers, we can classify them according to what we adapt; Parametric adaption Here we continually update the parameters of the process model, or even controller, or Signal adaption where we vary a supplementary signal to be added directly to the output of the static controller. Alternatively we can classify them as to how we choose a performance measure, Static A static performance measure is one such as overall efficiency, or maximising production. Dynamic A dynamic performance measure is to optimise the shape of a response to a step test. Functional of states and inputs This more general performance index is discussed more generally in chapter 9.

7.2

G AIN SCHEDULING

The simplest type of adaptive control is gain scheduling. In this scheme, one tries to maintain a constant overall loop gain, K by changing the controller gain, Kc in response to a changing process gain Kp . Thus the product Kc Kp is always constant. An example where gain scheduling is used is in the control of a liquid level tank that has a varying cross sectional area. Suppose we are trying to control the level in a spherical tank by adjusting a

CHAPTER 7. ADAPTIVE CONTROL

326

valve on the outlet. The exiting flow is proportional to the square root of height of liquid above the valve, which in turn is a nonlinear function of the volume in the tank. Fig. 7.2 shows how this gain varies as a function of material in the tank, illustrating why this is a challenging control problem. No single constant controller setting is optimal for both operating in the middle and the extremes of the tank.

Plant gain K ∝ h1/2

1.5

p

1

0.5 V

max

Figure 7.2: The plant gain, Kp is proportional to the square toot of the hydraulic head which in turn is a nonlinear function of volume in the tank. Consequently the controller gain, Kc , should be the inverse of the plant gain.

Controller gain K

c

0 8 6 4 2 0

0

1

2 3 Volume

4 Vmax

5

A way around this problem is to make the controller parameter settings a function of the level (the measured variable, y). Here the controller first measures the level, then it decides what controller constants (Kc , τi , τd ) to use based on the level, and then finally takes corrective action if necessary. Because the controller constants change with position, the system is nonlinear and theoretical analysis of the control system is more complicated. To do this in practice one must: 1. Determine the open loop gain at the desired operating point and several points either side of the desired operating point. 2. Establish good control settings at each of these three operating points. (This can be done by any of the usual tuning methods such as Ziegler-Nichols, Cohen-Coon etc) 3. Generate a simple function (usually piecewise linear) that relates the controller settings to the measured variable (level). The gain scheduling described above is a form of open-loop or feedforward adaption. The parameters are changed according to a pre-set programme. If the tank is in fact different from the tank that was used to calculate the parameter adaption functions such as having a △ cross section, then the controller will not function properly. This is because the controller is not comparing its own system model with the true system model at any time. However despite this drawback, gain scheduling is an appropriate way to control things like tanks of varying cross sectional area. It is unlikely that the tank’s cross sectional area will change significantly with time, so the instrument engineer can be quite confident that the process once tuned, will remain in tune.

7.3. THE IMPORTANCE OF IDENTIFICATION

7.3

327

T HE IMPORTANCE OF IDENTIFICATION

The adaptive controller (with feedback adaption) must be aware of the process, or more accurately it must be aware of changes in process behaviour — hence the importance of system identification covered in the previous chapter. Central to the concept of controllers that adapt, is an identification algorithm that constantly monitors the process trying to establish if any significant changes have occurred. In the gain schedular given in §7.2, the identification component was trivial, it was simply the level reading. The purists will argue that the gain schedular is not a true adaptive controller, because if the dynamics of the system change (such as the cross section of the tank suddenly varies), the controller is unaware of this change, and cannot take appropriate corrective action. There are two popular adaptive control techniques used currently in industry. The first approach is to use a controller with auto-tuning. Normally these controllers are operating in just the same manner as a traditional PID controller, but if the operator requests, the controller can be tuned automatically. This is sometimes called “tuning on demand”. Once the auto-tuning part is complete, the tuning procedure switches itself off and the controller reverts back to the traditional PID mode. These types of “adaptive controllers” are used where adaptive control is required, but the loop may be too sensitive or critical to have constant adaptation. The second approach to adaptive control is called true adaptive control. This is where the controller will adapt itself to the changing process without operator demand. Unlike the auto-tuners, the “true adaptive” control remains continuously identifying the process and updating the controller parameters. Many use variants of the least-squares identification that was described in §6.7. Industrially auto-tuners have been successful and are now accepted in the processing industries. Most of the major distributed computer control companies provide an adaptive control in the form of a self-tuner as part of their product (Satt Control, Bailey, ABB, Leeds-Northrup, Foxboro). ˚ om Astr ¨ and H¨agglund describe some of these auto-tuning commercial products in chapter 5 of [16, 105-132]. However the true adaptive control has not been accepted as much as the self-tuner. This is due to the fact that true adaptive control requires a higher level of expertise than self-tuning regulators. In addition, because the adaptive control is continuous, it requires a much larger safety network to guarantee correct operation. Even though the adaptive controller does not require “tuning”, the correct operation still requires estimates of values before the estimator can correctly function. These types of parameters are things such as expected model order, sampling period and expected dead time. If these values are significantly different from the true values, then poor adaption and hence control will result. Section 7.4 will consider self-tuning regulators.

7.3.1

P OLYNOMIAL

MANIPULATIONS

The controller design strategies in this section will use the so-called polynomial approach. In this design technique, we will need to be able to add and multiply polynomials, of possibly differing lengths. The two routines given in Appendix B on page 507 add and multiply polynomials and are used in the following controller design scripts in this chapter. An alternative programming methodology is to create a polynomial object and then overload the plus and times operators.

CHAPTER 7. ADAPTIVE CONTROL

328

7.4

S ELF TUNING REGULATORS (STR S )

A better adaptive controller than a gain schedular is one where the controller parameters are continually updated as the process characteristics change with time in an undetermined manner. These are called self tuning regulators and an early application was in the pulp and paper industry controlling a paper machine cited in [10]. Further development culminating in the self tuning regulator is described in the classic reference, [13]. Self tuning regulators are more flexible than gain scheduling controllers, which to be effective, the relation between the input and output must be known beforehand and is assumed constant. A textbook covering many aspects of the design and use of self tuning regulators is [203]. The basic elements of a self tuning regulator are: 1. Identify the model using input-output data. Usually the model is estimated online by using a recursive least squares parameter estimation. 2. A relation between the model parameters estimated in step 1 and the controller parameters. 3. The control algorithm. Many different types of control algorithms exist, but many are functions that try to adjust the controller parameters, given the continually changing process parameters in such a way that the closed loop response is constant. Because the controller is being continuously updated to the new varying conditions, these control algorithms can be quite vigorous. Step 1, the identification phase, was covered in chapter 6. Steps 2 and 3 are the controller tuning and the controller structure sections and are discussed further in this chapter. Suppose we have previously identified our plant under test using an ARX model of the form A(q)yk = B(q)uk + ek

(7.1)

then it is natural to ask what should the control law be? The simplest control law is a direct model inversion. We choose u such that y = ek which is the best possible result given the unknown noise. This results in the control law A(q) uk = yk (7.2) B(q) which is called minimum variance control, [10, 13, 15]. Expanding this control law gives the manipulated input at sample time k as uk =

1 (y ∗ + a1 yk−1 + a2 yk−2 + · · · + an yk−n − b1 uk−1 − · · · − bm uk−m ) b0 k

(7.3)

where yk∗ is the setpoint. Clearly for this type of controller to be realisable, the b0 term in Eqn. 7.3 must be non-zero, or in other words, the plant must have no dead time. With that constraint, this means that we can calculate the present control action u that will drive y to the desired y ∗ . Even without deadtime however, this type of control law will exhibit extremely vigorous action in an attempt to bring the process back to the setpoint in a single sample time. While the simulated results look very impressive, (see for example Fig. 7.3), in practice we will need to alter the structure of the control law, relax it somewhat, to account for imperfect modelling and manipulated variable constraints. These problems are partly circumvented in §7.6.4 and summarised in §7.7.

7.4. SELF TUNING REGULATORS (STRS)

7.4.1

S IMPLE

329

MINIMUM VARIANCE CONTROL

The algorithm to construct an adaptive minimum variance controller consists of two parts: the identification of the model parameters, θ, in this instance using recursive least-squares, and the implementation of the subsequent minimum variance control law. Algorithm 7.1 Simple minimum variance control for plants with no delay

1. Measure the current output yt . 2. Form the new input/output data matrix X. 3. Calculate the prediction yˆt from ϕ and θ t−1 . 4. Apply the control law, Eqn. 7.3, to obtain the current input, ut , using the estimated parameters. 5. Update the estimated parameters θk using a recursive least-squares algorithm such as Algorithm 6.2 or equivalent. A simple minimum variance controller is demonstrated in Listing 7.1 To test the simulation, we will construct a siumulation where we abruptly swap between two different plants: 0.2 + 0.05q −1 , 1 − 0.78q −1 − 0.65q −2 + 0.2q −3 0.3 + 0.15q −1 , G2 (q −1 ) = 1 − 0.68q −1 − 0.55q −2 + 0.1q −3

G1 (q −1 ) =

for 0 ≤ t < 100

(7.4)

for t > 100

(7.5)

For this simulation we will assume no model/plant structural mismatch and we will use the recursive estimation with the forgetting factor option. Listing 7.1: Simple minimum variance control where the plant has no time delay t = [0:200]'; yspt = square(t/18); y=zeros(size(yspt)); u=zeros(size(y)); nr=0.02; 5

10

15

% Create time & setpoint % noise or dither sigma

n1 = [0.2 0.05]; d1 = [1 -0.78 -0.65 0.2]; % system #1, Eqn. 7.4 n2 = [0.3 0.15]; d2 = [1 -0.68 -0.55 0.1]; % system #2, Eqn. 7.5, for t > 100 thetatrue = [ones(t(1:100))*[d1 n1]; ones(t(101:length(t)))*[d2 n2]]; na = 4;nb = 2; dead=0; nt=na+nb; % Estimated model parameter orders, na , nb & nk . lam = 0.95; P=1000*eye(nt); % Identification parameters: λ, P0 thetaest=rand(nt,1); % random start guess for θ 0 Param=rand(na,nt);Pv = ones(na,1)*diag(P)'; for i=na+1:length(yspt) theta = thetatrue(i,:)'; thetar=[thetaest(1:na); thetaest(na+2:nt)]; b0 = thetaest(na+1); yv=y(i-1:-1:i-na); uv=u(i-1:-1:i-nb+1); u(i)=(yspt(i)-[yv; uv]'*thetar)/b0;

% Actual plant, G ˆ % Estimated plant, G % old output & input data % Min. Var. control law

20

c = [yv', u(i:-1:i-nb+1)']; y(i) = c*theta(2:end) + 0.01*randn(1)*nr;

% Now do identification % true process with noise

CHAPTER 7. ADAPTIVE CONTROL

330

25

[thetaest,P] = rls(y(i),c',thetaest,P,lam); % RLS with forgetting Param = [Param; thetaest']; Pv = [Pv;diag(P)']; end % for loop

Initially we will set the forgetting factor λ to 0.99, and we will introduce a small amount of noise so the estimation remains reasonable even under closed loop conditions, nr=0.02. We will aim to follow the trajectory given as the dashed line in the upper figure of Fig. 7.3. Our minimum variance controller will both try to estimate the true plant and control the process to the setpoint, y ⋆ . The lower trend in Fig. 7.3 shows the parameter values for the estimated model. Minimum variance control output & setpoint

4 G2 (q −1 )

G1 (q −1 )

2

0

−2 20

input

10 0 −10 −20

λ = 0.99

θ

1 0 −1

0

50

100

150 time

200

250

300

Figure 7.3: Simple minimum variance control with an abrupt plant change at t = 100. (See also Fig. 7.4.) In this unrealistically ideal situation with no model/plant mismatch, the controlled response looks practically perfect, despite the plant change at t = 100 (after of course the converging period). Fig. 7.4 shows an enlarged portion of Fig. 7.3 around the time of a setpoint change. Note how the response reaches the setpoint in one sample time, and stays there. This is as expected when using minimum variance control since we have calculated the input to force y exactly equal to the desired output y ⋆ in one shot. In practice this is only possible for delay-free plants with no model-plant mismatch. Of course under industrial conditions (as opposed to simulation), we can only view the controlled result, and perhaps view the trends of the estimated parameters, we don’t know if they have converged to the true plant parameters. However if the estimates converge at all to steady values, and the diagonal elements of the covariance matrix are at reasonable values, (these are application dependent), then one may be confident, but never totally sure, that a reasonable model has been identified.

7.5. ADAPTIVE POLE-PLACEMENT

331

Minimum variance control

output & setpoint

1 0.5 0 −0.5 −1

input

10 0 −10 −20 35

7.5

40 time

45

Figure 7.4: A zoomed portion of Fig. 7.3 showing the near-perfect response around a setpoint change when using simple minimum variance control on a plant with no delay.

A DAPTIVE POLE - PLACEMENT

Clearly the problem with the simple minimum variance control is the excessive manipulated variable demands imposed by the controller in an attempt to reach the setpoint in one sample time. While our aim is to increase the speed of the open-loop response, we also need to reduce the sensitivity to external disturbances. Both these aims should be addressed despite changes in the process. Fig. 7.5 shows a generalised control structure given in shift polynomial form known as a two degree of freedom polynomial controller. In this section we are going to modify our colour scheme slightly. As shown in the figure, the controller polynomials will be couloured blue hues, and the plant to be controlled, red hues. The B polynomial, because it operates on the blue input is coloured purple, i.e. between blue and red.

Controller pre-compensator

r(t) setpoint

H(q −1 )

Plant + −

1 F (q −1 )

u(t) B(q −1 ) A(q −1 ) b

y(t)

G(q −1 )

Figure 7.5: Two degree of freedom pole-placement control structure If the controller polynomials G = H = 1, then we have the classic controlled feedback loop, otherwise we have a slightly more general configuration known as a two-degree of freedom configuration since the controller (dashed box in Fig. 7.5) has two inputs. Our aim, as controller designers, is to select polynomials F , G, H given process polynomials B, A such that our performance aims

CHAPTER 7. ADAPTIVE CONTROL

332 are met. The closed loop transfer function of Fig. 7.5 is y(t) =

HB r(t) F A + GB

where clearly the stability is given by the denominator, F A + GB. Note that this polynomial equation consists of two terms, each with a blue term multiplied by a red term. This fact will be important later when we derive an algorithm to solve for the unknown blue polynomials. Rather than demand the one-step transient response used in the minimum variance case, we can relax this criterion, and ask only that the controlled response follow some pre-defined trajectory. Suppose that we restrict the class of dynamic systems to the form, Ay(t) = Bu(t − 1)

(7.6)

where we have assumed a one sample time delay between u and y. This is to leave us enough time to perform any controller calculations necessary. We will use the controller structure given in Fig. 7.5, F u(t) = Hr(t) − Gy(t) (7.7)

where the polynomials F, G, H are of the form:

F = 1 + f1 q −1 + f2 q −2 + · · · + fnf q −nf

G = g0 + g1 q −1 + g2 q −2 + · · · + gng q −ng

H = h0 + h1 q −1 + h2 q −2 + · · · + hnh q −nh With the computational delay, the closed-loop is (F A + q −1 BG)y(t) = q −1 BHr(t)

(7.8)

and we desire the closed-loop poles to be assigned to specified locations specified in the polynomial T , T = 1 + t1 q −1 + t2 q −2 + · · · + tnt q −nt (7.9) Assigning the closed-loop poles to given locations is called pole-placement or pole assignment. (In practice, the closed loop polynomial T will be a design requirement given to us perhaps by the customer.)

Note that some authors such as [22] use a slightly different standard nomenclature convention for the controller polynomials. Rather than use the variable names F, G, H in the control law Eqn. 7.7, the variable names R, S, T are used giving a control law of Ru(t) = T r(t) − Sy(t).

7.5.1

T HE D IOPHANTINE

EQUATION AND THE CLOSED LOOP

To assign our poles specified in T , we must select our controller design polynomials F, G such that the denominator of the actual closed-loop is what we desire, F A + q −1 BG = T

(7.10)

where I have coloured the design polynomial red indicating that it is to drive the output to the setpoint. The polynomial equation Eqn. 7.10 is called the Diophantine equation, and may have many solutions. To obtain a unique solution, A, B must be coprime (no common factors), and we should select the order of the polynomials such that nf = nb ng = na − 1, (na 6= 0) nt ≤ na + nb − nc

7.5. ADAPTIVE POLE-PLACEMENT

333

One way to establish the coefficients of the design polynomials F, G and thus solve the Diophantine equation is by expanding Eqn. 7.10 and equating coefficients in q −i giving a system of linear equations.    1    a1    a  2   ..  .   a  na    0   .  ..   .  .  .  0 |

0

···

1 a1

..

.

0 .. . .. . .. .

b0 b1 .. .

···

0

..

b0

.

b1

. a2 . . bnb .. . . . 1 0 bnb . . . 0 ··· ana . . a1 .. .. .. . a2 . · · · · · · 0 .. .

0    ..  f1  .    f2 ..   .   .  . ..   .  .    fnf   0    g0   b0   g1    b1    .. .  ..   .  gng  bnb | {z } θc



                   =                 | }

t1 − a 1 t2 − a 2 .. .

tnt − ant −ant+1 .. . −ana 0 .. . {z b

                

(7.11)

}

· · · · · · ana 0 · · · · · · {z A The matrix A in Eqn. 7.11 has a special structure, and is termed a Sylvester matrix and will be invertible if polynomials A, B are co-prime. Solving Eqn. 7.11 θ c = A−1 b gives us the coefficients of our controller polynomials, F , G. Now the dynamics of the closed-loop requirement are satisfied, BH r(t − 1) y(t) = T but not the steady-state gain, which ideally should be 1. We could either set H to be the scalar, T (7.12) H= B q=1 which forces the steady-state gain to 1, or preferably cancel the zeros H=

1 B

(7.13)

which improves the transient response in addition to keeping the steady-state gain of unity. Cancelling the zeros should only be attempted if all the zeros of B are well inside the unit circle. Section 7.6.1 describes what to do if the polynomial B does has poorly damped or unstable zeros. This scheme has now achieved our aim of obtaining a consistent dynamic response irrespective of the actual process under consideration. Since we cannot expect to know the exact model B, A ˆ Aˆ in the controller design. we will use an estimated model, B,

7.5.2

S OLVING

THE

D IOPHANTINE

EQUATION IN

M ATLAB

Since solving the Diophantine equation is an important part of the adaptive pole-placement regulator, the following functions will do just that. Listing 7.2 solves for the polynomials F and G such that F A + BG = T . If a single time delay is introduced into the B polynomial such as in Eqn. 7.10, then one needs simply to pad the B polynomial with a single zero. Note that the expression AF + BG is optionally computed using the polyadd routine given in Listing B.1, and this polynomial should be close to the T polynomial specified.

CHAPTER 7. ADAPTIVE CONTROL

334

Listing 7.2: A Diophantine routine to solve F A + BG = T for the polynomials F and G.

5

function [F,G,Tcheck] = dioph(A,B,T); % [F,G] = dioph(A,B,T); % Solves the Diophantine equation by setting up the Sylvester matrix, % AF + BG = T and subsequently solves for polynomials F & G. % See [19, p175] (follows nomenclature of except R=F, S=G). if abs(A(1)-1) > 1.0e-8; error('Error: Polynomial A not monic A(1) = %g\n',A(1)); end % if

10

15

20

na = length(A)-1; nb = length(B)-1; % Order of polynomials A(q −1 ), B(q −1 ) nf = nb; ng = na-1; n = nf+ng+1; n = nf+ng+1; % Required orders for F & G D = zeros(n,n); % construct Sylvester matrix for i=1:nf D(i:i+na,i) = A'; % fill in columns end % for i for i=1:ng+1 D(i:i+nb,nf+i) = B'; % fill in columns end % for i rhs = [A(2:na+1), zeros(size(na+1:n))]'; % right-hand side rhs = [T(2:length(T)), zeros(size(length(T):n))]' - rhs; FG= D\rhs; % Find solution for polynomials F & G F= FG(1:nf)'; G= FG(nf+1:n)';% Monic polynomial F has leading 1 dropped.

25

if nargout >2 % then check that AF + BG = T as required Tcheck = polyadd(conv(A,[1,F]), conv(B,G)); end return

A N ALTERNATIVE D IOPHANTINE

SCHEME

An alternative, and somewhat more elegant, scheme for solving the Diophantine equation is given in Listing 7.3 which uses matrix convolution to construct the Sylvester matrix. Listing 7.3: Alternative Diophantine routine to solve F A + BG = T for the polynomials F and G. Compare with Listing 7.2. 1

6

11

function [F,G,Tcheck] = dioph_mtx(A,B,T); % Alternative version for polynomial Diophantine equation solving % Solves AF + BG = T for F & G. da = length(A)-1; db=length(B)-1; T = [T, zeros(1,da+db-length(T)+1)]; dt=length(T)-1; dg =da-1; df=dt-da ; B = [ zeros(1, df-db+1), B];

% da = deg(A) etc. % pad with zeros % Convert to full length % assuming for RAB to be square % pad with leading zeros

Rac = [convmtx(A,df+1);convmtx(B, dg+1)]; % Construct RAB = [RA ; RB ] FG = T/Rac; % [F, G] = T R−1 AB F = FG(1:df+1); G = FG(df+2:df+dg+2);% Note F & G are row vectors.

16

Tcheck = polyadd(conv(A,F), conv(B,G)); % Verify solution return

7.5. ADAPTIVE POLE-PLACEMENT

335

The alternative scheme in Listing 7.3 is slightly different from the one presented earlier in that this time the F polynomial contains the leading unity coefficient. A third method to construct the Sylvester matrix is to use the lower triangle of a Toeplitz matrix in M ATLAB say with the command tril(toeplitz(1:5)).

E XAMPLE OF

A

D IOPHANTINE

PROBLEM

Suppose we only want to place one pole, T = 1 + t1 q −1 , given a na = 3, nb = 2 model. The recommended orders of the controller polynomials are nf = nb = 2 ng = na − 1 = 2

nt = 1 ≤ na + nb − nc = 5 and expanding the closed-loop equation Eqn. 7.10, (1 + f1 q −1 + f2 q −2 )(1 + a1 q −1 + a2 q −2 + a3 q −3 )+ q −1 (b0 + b1 q −1 + b2 q −2 )(g0 + g1 q −1 + g2 q −2 ) = 1 + t1 q −1 which by equating the coefficients in q −i , i = 1, . . . , 5 gives the linear system following Eqn. 7.11      1 0 b0 0 0 f1 t1 − a 1  a1 1 b1 b0 0   f2   −a2        a2 a1 b2 b1 b0   g0  =  −a3        a3 a2 0 b 2 b 1   g 1    0 0 a3 0 0 b 2 g2 0 A N EXAMPLE OF

SOLVING THE

D IOPHANTINE

EQUATION

In this example we start with some known polynomials, and then try to reconstruct them using the Diophantine routines presented previously. Suppose we start with some given plant polynomials, say A(q −1 ) = 1 + 2q −1 + 3q −2 + 3q −3 ,

B(q −1 ) = 5 + 6q −1 + 7q −2

where the orders are na = 3 and nb = 2. This means we should choose orders nf = nb = 2, and ng = na − 1 = 2, so let us invent some controller polynomials F (q −1 ) = 1 + 8q −1 + 9q −2 ,

G(q −1 ) = 10 + 11q −1 + 12q −2

noting the A and F are monic. We can now compute the closed loop, AF + BG using convolution and polynomial addition as shown in Listing 7.4 Listing 7.4: Constructing polynomials for the Diophantine equation example >> A = [1 2 3 4]; B = [5 6 7]; >> F = [1 8 9]; G = [10 11 12]; 3

% Note nf = nb & ng = na − 1.

>> T = polyadd(conv(A,F),conv(B,G)) % Compute T (q −1 ) = A(q −1 )F (q −1 ) + B(q −1 )G(q −1 ) T = 1 60 143 242 208 120

CHAPTER 7. ADAPTIVE CONTROL

336

which gives T (q −1 ) = 1 + 60q −1 + 143q −2 + 242q −3 + 208q −4 + 120q −5 . We are now ready to essentially do the reverse. Given the polynomials A(q −1 ), B(q −1 ), and T (q −1 ), we wish to reconstruct F (q −1 ) and G(q −1 ) using the Diophantine routines given in Listings 7.2 and 7.3. Listing 7.5: Solving the Diophantine equation using polynomials generated from Listing 7.4.

4

9

14

>> [F1,G1,Tcheck1] = dioph(A,B,T) F1 = 8.0000 9.0000 G1 = 10.0000 11.0000 12.0000 Tcheck1 = 1 60 143 242 208 120

% Use Listing 7.2

>> [F2,G2,Tcheck2] = dioph_mtx(A,B,T) % Use Listing 7.3 F2 = 1 8 9 G2 = 10.0000 11.0000 12.0000 Tcheck2 = 1 60 143 242 208 120

Note that in both cases we manage to compute the polynomials F (q −1 ) and G(q −1 ) which reconstruct back to the correct T (q −1 ), although the routine in Listing 7.2 drops the leading 1 in the monic F polynomial.

7.5.3

A DAPTIVE

POLE - PLACEMENT WITH IDENTIFICATION

We have now all the components required for an adaptive controller. We will use an RLS module for the plant identification, and then subsequently solve the Diophantine equation to obtain a required closed loop response. Algorithm 7.2 describes the algorithm behind this adaptive controller. Algorithm 7.2 Adapting pole-placement algorithm We will follow Fig. 7.6 which is a specific case of the general adaption scheme given in Fig. 7.1. 1. At each sample time t, collect the new system output, y(t). 2. Update the polynomial model estimates A, B using recursive least-squares on Ay(t) = Bu(t − 1) + e(t) 3. Synthesize the controller polynomials, F, G by solving the identity, F A + q −1 BG = T where T is the desired closed-loop response. (You could use the dioph routine given in Listing 7.2.) 4. Construct H using either Eqn. 7.12 or Eqn. 7.13. 5. Apply control step F u(t) = −Gy(t) + Hr(t) 6. Wait out the remainder of the sample time & return to step 1.

7.5. ADAPTIVE POLE-PLACEMENT

337

Pole placement controller design

Identify Plant Aˆ

Performance requirements Diophantine

T

structure

RLS ˆ B

H H(q −1 )

r(t)

F

G

B(q −1 ) u(t) A(q −1 )

1 F (q −1 )

+ −

setpoint

Plant b

G(q −1 ) 2 DOF Controller Figure 7.6: Adaptive pole-placement control structure with RLS identification. See also Fig. 7.5. A N ADAPTIVE CONTROLLER

WITH

RLS IDENTIFICATION

EXAMPLE

In this example will attempt to control three different plants in the configuration as shown in Fig. 7.7. The open loop step tests of the plants are compared in Fig. 7.8. Of the three plants, G1 is stable and well behaved, G2 has open-loop unstable poles, and G3 has unstable zeros.

Plant alternatives }| { z setpoint r

✲ ✲

Gc

✲

G1

✲

G2

✲

current active plant ❄ switch

✲ y

✻ Adaptive controller

✲

G3

current inactive plant models

Figure 7.7: Control of multiple plants with an adapting controller. We desire the same closed loop response irrespective of the choice of plant. In practice we must estimate the process model and if the plant changes, we redesign our controller such that we obtain the same closed loop response. The customer for this application demands a slight overshoot for the controlled response as shown in Fig. 7.9. We can translate this

y(t)

CHAPTER 7. ADAPTIVE CONTROL

338 Open−loop plant step responses

Pole−Zero Map

6000 1.5

G1

5000

G1 (q−1 ) 3000

2000

0.5 0 −0.5

G3 (q

1000

0

2

G3 Imaginary Axis

output

4000

G

1

G2 (q−1 )

−1

)

−1 −1.5

0

20

40

60 time

80

100

120

−1

−0.5

0 0.5 Real Axis

1

1.5

Figure 7.8: Step responses of the 3 discrete open-loop plants (one of which is unstable), and the corresponding pole-zero maps. Note that G3 has zeros outside the unit circle. requirement for the denominator of the closed-loop transfer function to be T = 1 − 1.4685q −1 + 0.6703q −2

Desired closed loop response 1.4 1.2

output

1 0.8

1 T(q −1 )

0.6 0.4 0.2

Figure 7.9: The desired closed loop response, 1/T .

0

0

5

10

15

20

25

30

35

time

Listing 7.6 and Fig. 7.10 show the impressive results for this combined identification and adaptive control simulation when using the three plants from Fig. 7.8. Note that the controller design computation is done in the dioph.m routine given in Listing 7.2 while the identification is done using the rls routine. Here I assume a model structure where na = 5, nb = 3, and I add a small amount of noise, σ = 0.005, to keep the estimation quality good. A forgetting factor of λ = 0.95 was used since the model changes were known a priori to be rapid. Listing 7.6: Adaptive pole-placement control with 3 different plants Gx=struct('b',[3 0.2 0.5],'a',poly([0.95 0.9 0.85])); % 3 different plants Gx(2) = struct('b',[2,2,1],'a',poly([1.1 0.7 0.65])); Gx(3) = struct('b',[0.5 0.2 1],'a',poly([0.6 0.95 0.75])); 5

na = 3; nb = 3; % order of polynomials, na , nb a = randn(1,na); b = randn(1,nb); % model estimates

7.5. ADAPTIVE POLE-PLACEMENT

10

np = length([a,b]); P = 1.0e4*eye(np); lambda = 0.95; param_est = [a,b]';

339 % # of parameters to be estimated % Large initial co-variance, P0 = 104 I % Forgetting factor, 0 < λ < 1

% Design a reasonable CLTF response with τ = 2 and ζ = 0.4 tau=2; zeta = 0.4; Grc = tf(1,[tauˆ2 2*tau*zeta 1]); Grd = c2d(Grc,1); T = cell2mat(Grd.den); 15

dt = 1; t = dt*[0:700]'; r = square(t/10); % sample time & setpoint y = zeros(size(t)); u = y; noise = 0.005*randn(size(t)); Param = zeros(length(t),np); trP = zeros(size(t));% plot holder tG2 = 150; tG3 = tG2+200; k=1; % Swap over times for the different plants 20

for i=10:length(t) if t(i) == tG2, k=2; end % Swap over to model #2 if t(i) == tG3, k=3; end % Swap over to model #3 25

30

35

40

45

% Plant response y(i) = Gx(k).b*u(i-1:-1:i-length(Gx(k).b)) - ... Gx(k).a(2:end)*y(i-1:-1:i-length(Gx(k).a)+1); % Model identification x = [-y(i-1:-1:i-na)', u(i-1:-1:i-length(b))']; % shift register [param_est,P] = rls(y(i),x',param_est,P,lambda); % update parameters, Listing 6.18. ˆ B ˆ a=[1 param_est(1:na)']; b=param_est(na+1:na+nb)';% extract estimates: A, Param(i,:) = param_est'; trP(i) = trace(P); % Controller design [F,G] = dioph(a,b,T); % Solve Diophantine polynms, Listing 7.2. %[F,G] = dioph_mtx(a,b,T); F(1)=[]; % Alternative Diophantine, Listing 7.3. H = sum(T)/sum(b); % Get steady-state correct, H = T /B|q=1 . % Control law, F u = −Gy + Hr u(i) = H*r(i) - G*y(i:-1:i-length(G)+1) - F*u(i-1:-1:i-length(F)); end % for i %% Check the results G = tf(sum(T),T,1); % desired closed loop yref = lsim(G,r);

Fig. 7.10 shows the impressive results of this adaptive controller. When the plant changes, unbeknown to the controller, the system identification updates the new model, and the controller consequently updates to deliver a consistent response. If the pole-placement controller is doing its job, then the desired closed loop response should match the actual output of the plant. These two trajectories are compared in Fig. 7.11 which shows an enlarged portion of Fig. 7.10. Here it can be seen that apart from the short transients when the plant does change, and the identification routine is struggling to keep up, the adaptive pole-placement controller is satisfying the requirement to follow the reference trajectory.

7.5.4

A DAPTIVE

POLE - PLACEMENT IN

S IMULINK

The time-varying filter blocks make it relatively easy to implement an adaptive pole-placement controller in S IMULINK as showed in Fig. 7.12. The controller design is executed in the orange ‘RLS & APP’ sub-block which is shown in Fig. 7.13. This sub-block contains two m-file scripts

CHAPTER 7. ADAPTIVE CONTROL

output & setpoint

340

2 1 0 −1 −2 −3 −4

input

0.5 0 −0.5 3 2 b 2 b a0 3 b a1

1 θ

0

2

a

−1

1

trace(P)

−2

0

50

100

150

200 250 300 sample time

350

400

450

Figure 7.10: Adaptive pole-placement with RLS identification. After a period for the estimated parameters to converge (lower trends), the controlled output response (upper trend) is again reasonably consistent despite model changes. Results from Listing 7.6. 1.8

y

1.6

Reference, y*

1.4 1.2 1 0.8 0.6 0

50

100

150

200

250 time

300

350

400

450

500

Figure 7.11: An enlarged portion of Fig. 7.10 showing the difference between the actual plant output y(t), and the reference output, y ∗ (t).

7.6. PRACTICAL ADAPTIVE POLE-PLACEMENT

341

that first do the RLS model identification update, and then do the pole-placement design based on the currently identified model. The various memory blocks and the initial condition blocks are needed to ensure that we do not have any algebraic loops when we start the simulation.

Figure 7.12: Adaptive pole-placement with identification in S IMULINK. See also Fig. 7.13.

Figure 7.13: Internals of the Adaptive pole-placement design routine from Fig. 7.12.

7.6

P RACTICAL ADAPTIVE POLE - PLACEMENT

Some problems still remain before the adaptive pole-placement algorithm is a useful and practical control alternative. These problems include:

CHAPTER 7. ADAPTIVE CONTROL

342

• Poor estimation performance while the system is in good control, and vice-versa, (poor control during the periods suitable for good estimation). (This is known as the dichotomy of adaptive control, or dual control). • Non-minimum phase systems possessing unstable zeros in B which must not be cancelled by the controller. • Dead time where at least b0 = 0. We must try to solve all these practical problems before we can recommend this type of controller.

A N EXAMPLE

OF BURSTING

If our controller does its job perfectly well, then the output remains at the setpoint. Under these circumstances we do not have any identification potential left, and the estimated model starts to drift. The model will continue to drift until the controller deteriorates so much that there is a period of good excitation (poor control), that the model re-converges again. This Fig. 7.14 shows the experimental results from using an adaptive controller, (in this case an adaptive LQR regulator) on the blackbox over a substantial period of time. The controller sampling at T = 1 second was left to run over night for 10 hours (36,000 data points), and for a 3 hour period in the middle we had no setpoint changes, and presumably no significant disturbances. During this time, part of which is shown in Fig. 7.14, the adaptive controller oscillated into, and out of, good control. This is sometimes known as “bursting”. Adaptive LQR of the blackbox ∆ t=1.00 (sec) 0.4

output & setpoint

0.3 0.2 0.1 0 −0.1 −0.2 −0.3 2.5

3

3.5

4

3.5

4

RLS: λ=0.995 1

Input

0.5

0

−0.5

−1 2.5

3 time (hours)

Figure 7.14: Bursting phenomena exhibited by the black box when using an adaptive controller. With no setpoint changes or external disturbances, the adaptive controller tends to oscillate between periods of good control and bad estimation and the reverse.

7.6. PRACTICAL ADAPTIVE POLE-PLACEMENT

7.6.1

D EALING

343

WITH NON - MINIMUM PHASE SYSTEMS

If we use the steady-state version for H, def

H =

T B q=1

as proposed in Eqn. 7.12, then we do not cancel any zeros, and we avoid the problem of inadvertently inverting unstable model zeros. However we can achieve a better transient response if we cancel the zeros with the pre-compensator H, H=

1 B

giving the closed loop transfer function, y(t) =

1 HB r(t − 1) = r(t − 1) T T

which looks fine until we investigate closer what is likely to happen if B has zeros outside the unit circle. Remember that the polynomial B is a function of the plant that we are trying to control and hence an output of the estimation scheme. We have no influence on the properties of B; we get what we are given. In fact, we will have problems not only if the zeros are unstable, but even if they are stable but poorly damped. Fig. 7.15 shows the adaptive pole-placement response of a plant with stable, but ◦ poorly damped zeros when using H = 1/B. In fact the zeros are 0.95e±j140 indicating that they are close to, but just inside the unit circle. While the output response looks fine, it is the ringing behaviour of the input that should cause concern. The results in Fig. 7.15 were obtained by using the modified controller given in Listing 7.7 in Listing 7.6. Listing 7.7: The pole-placement control law when H = 1/B if

4

any(abs(roots(b))>1) % Use H = T /B|q=1 if any zeros of B are unstable.

H = polyval(T,1)/polyval(b,1); % Simple scalar pre-filter, H = T /B|q=1 . u(i) = H*r(i) - G*y(i:-1:i-ng) - F*u(i-1:-1:i-nf); else % Alternatively, use dynamic compensation, H = 1/B. v(i) = 1/b(1)*(-b(2:end)*v(i-1:-1:i-nb+1)+sum(T)*r(i)); u(i) = v(i) - G*y(i:-1:i-ng) - F*u(i-1:-1:i-nf); % control action end

Suppose we have estimated a B polynomial possessing an unstable zero at say q −1 = +2/3, or B = 1 − 1.5q −1 and we desire T = 1 − 0.7q −1 . Then the closed-loop transfer function is y(t) =

q −1 (1 − 1.5q −1 ) BH Hr(t) r(t − 1) = F A + q −1 BH 1 − 0.7q −1

Now suppose we use a dynamic pre-compensator, H = 1/B, H=

1 q(1 − 1.5q −1 )

CHAPTER 7. ADAPTIVE CONTROL

344 3

yref y* y

output & setpoint

2 1 0 −1 −2 2 1.5 1

input

0.5 0 −0.5 −1 −1.5 −2

0

50

100

150

200

250 300 sample time

350

400

450

500

Figure 7.15: Adaptive pole-placement with a stable, but poorly damped, B polynomial. The poorly damped zeros cause the ringing behaviour in the input. which has a time-advance term, q +1 , meaning that the control law is a non-causal function of the setpoint. In other words, we need to know r(t + 1) at time t. In fact since we need to know only future values of setpoint, and not output, this type of control law is easily implemented. The main drawback stems, however, not from the acausality, but from the ill-advised cancellation. In all practical cases, we will experience some small numerical round-off error such as where the pole is represented in a finite-word length computer as 1.50001 rather than exactly 1.5. Consequently the pole will not completely cancel the zero, y(t) =

q −1 (1 − 1.5q −1 ) q × r(t) 1 − 0.7q −1 (1 − 1.50001q −1) | {z } small error

leaving an unstable controller.

The remedy is that if we discover unstable modes in B, we simply do not cancel them, although we continue to cancel the stable modes. We achieve this by separating the B polynomial into two groups of factors: the stable roots, B + , (which we will colour ‘green for good’) and the unstable roots, B − , (which we will colour ‘red for bad’) def

B = B+B−

(7.14)

and then set H = 1/B + only. A simple M ATLAB routine to do this factorisation is given in Listing 7.8 in section 7.6.2.

7.6. PRACTICAL ADAPTIVE POLE-PLACEMENT

345

So given the model and control law, Ay(t) = Bu(t − 1)

F u(t) = −Gy(t) + Hr(t) we modify the pole assignment to AF + q −1 BG = T B +

(7.15)

so closed-loop transfer function is y(t) =

HB − HB r(t − 1) = r(t − 1) T B+ T

(7.16)

We solve the modified pole-assignment problem, Eqn. 7.15, by noting that B + must be a factor of F, F = B + F1 so we solve the now reduced Diophantine equation AF1 + q −1 B − G = T in the normal manner. We should always check for unstable zeros in the B polynomial when using online estimation because we have no control over what the RLS routine will deliver. Even in a situation where we know, or reasonably suspect a stable B, we may stumble across an unstable B while getting to the “correct” values. As an example, we can apply the adaptive pole-placement controller where we cancel the stable B polynomial zeros, but not the unstable ones, using the three plants from page 337. Fig. 7.16 shows the result of of this controller where we note that the performance of plant #3 is also acceptable exhibiting no ringing. For diagnostic purposes, we can also plot the instantaneous poles and zeros of the identified model, and perhaps display this information as a movie, or perhaps as given in Fig. 7.16, shown with decreasing intensity of colour for the older data. This will show how the poles and zeros migrate, and when zeros, or even poles, leap outside the unit circle. Fig. 7.16 also shows the magnitude of the estimated poles and zeros which clearly indicate when they exceed 1 and jump outside the unit circle.

7.6.2

S EPARATING

STABLE AND UNSTABLE FACTORS

When applying pole-placement in the case with unstable zeros, we need to be able to factor B(q) into stable and unstable factors. Listing 7.8 does this factorisation, and additionally it also rejects the stable, but poorly damped factors given a user-specified damping ratio, ζ. (If you want simply stable/unstable, use ζ = 0.) Fig. 7.18 illustrates the region for discrete well-damped poles where ‘well-damped’ is taken to mean damping ratios ζ > 0.15. This algorithm is adapted from [150, p238]. Listing 7.8: Factorising an arbitrary polynomial B(q) into stable, B + (q), and unstable and poorly damped, B − (q), factors such that B = B + B − and B + is defined as monic. 2

function [Bp,Bm] = factorBz(B,zeta); % Factorise B(q) = B + (q)B − (q) where B + are factors inside the unit circle and B − (q) are outside.

CHAPTER 7. ADAPTIVE CONTROL

346

3 G1 (q −1 )

output & setpoint

2

G2 (q −1 )

Unstable zeros: G3 (q −1 )

1 0 −1 −2 0.2

input

0.1 0 −0.1 −0.2 4

θ

λ=0.99 2

a

0

b b 2 0 b 1 a

2

3

−2

a

1

−4 5

tr(P)

10

0

10

0

500

1000

1500

sample time def

Figure 7.16: Adaptive pole-placement control with H = 1/B and an unstable B polynomial in Plant #3. Note that the controller does not exhibit any ‘ringing’, even with a plant with unstable zeros.

7.6. PRACTICAL ADAPTIVE POLE-PLACEMENT

347

Pole−Zero Map

1.5 z1, z2

1.5

1

1

p1

|poles & zeros|

Imaginary Axis

0.5

0

p2 p3 0.5

−0.5

−1

−1.5

−1

−0.5

0 0.5 Real Axis

1

1.5

0

0

500

1000 sample time

1500

Figure 7.17: The migration of poles and zeros in the complex plane (left hand plot) and the time trend of the magnitude of the poles and zeros (right-hand plot) from the simulation presented in Fig. 7.16. % The optional parameter, ζ defines the cutoff. Use ζ = 0 for inside/outside circle. % Note B + (q) or Bp is defined as monic.

7

12

17

22

if ¬(exist('zeta') == 1) zeta = 0.15; % default reasonable cut-off is ζ = 0.15. end zeta = max(min(zeta,1),0); % Ensure ζ ∈ [0, 1]. tol = 1e-6; r = roots(B); % Now separate into poorly damped & good poles mag_r = abs(r); rw = angle(r)/2/pi; % See equations in [150, p238]. mz = exp(-2*pi*zeta/sqrt(1-zetaˆ2)*rw); % magnitude of z idx = mag_r < mz; Bp = poly(r(idx)); % Set B + (q) to contain all the zeros inside unit circle Bm = poly(r(¬idx))*B(1); % All the others if norm(conv(Bp,Bm)-B) > tol % Check B = B + B − warning('Problem with factorisation') end % if return

7.6.3

E XPERIMENTAL

ADAPTIVE POLE - PLACEMENT

Actually running an adaptive pole-placement scheme on a real plant, even a well behaved one in the laboratory is considerably more difficult than the simulation examples presented so far. Our desire is to test an adaptive pole-placement controller on the black-box to demonstrate the following: 1. To follow a specified desired trajectory (as quantified by the T polynomial),

CHAPTER 7. ADAPTIVE CONTROL

348

Figure 7.18: The boundary of the shaded area is the loci of constant damping ratio with ζ = 0.15. Inside this region, the poles are considered well damped. (Figure plotted with zgrid.) 2. under both the ‘NORMAL’ and ‘SLOW’ positions on the black box, and 3. to run stably for at least 10 minutes following both disturbances and servo changes. The first design decision is to choose a desired response (closed loop poles) and suitable sampling interval. We shall select a second-order prototype response with a time constant of τ = 2.1 seconds and some overshoot, ζ = 0.6. With a sampling time of T = 0.6 seconds, the desired discrete closed loop transfer function is G⋆cl (s) =

1 2 4.41s + 2.52s + 1

→

G⋆cl (q −1 ) =

0.03628 + 0.03236q −1 1 − 1.641q −1 + 0.7097q −2

from which follows that T = 1 − 1.641q −1 + 0.7097q −2. Fig. 7.19 shows the results from one experiment where the characteristics of the plant are changed at t = 128 by manually switching over from 7 cascaded low-pass filters to 9. This increases the effective time constant of the plant. It is important to verify that the output from the blackbox did in fact follow the intended trajectory once the model parameters have converged. An enlarged portion of Fig. 7.19 is repeated in Fig. 7.20 comparing the intended trajectory, y ⋆ , (dashed line) with the actual trajectory, y. This shows that the response converged to the desired trajectory even after the plant subsequently changed. Fig. 7.19 shows clearly that the shape of the input must vary considerably to force the same output if the plant itself changes.

7.6.4

M INIMUM

VARIANCE CONTROL WITH DEAD TIME

The minimum variance control formulation of §7.4.1 is not very useful not the least due to its inability to handle any dead time and unstable process zeros. Even without the dead time, practical experience has shown that this control is too vigorous and tends to be over-demanding causing excessive manipulator variable fatigue. Moreover when using discrete time models, delay between input and output is much more common, and typically the deadtime is at least one sample period. The best we can do in these circumstances is to make the best prediction of the output we can for d samples times ahead, and use this prediction in our control law as done above. This introduces the general concept of minimum variance control which combines prediction with control. Further details are given in [23, pp367–379]. Suppose we start with the ARMAX canonical model written in the forward shift operator q, A(q)yk = B(q)uk + C(q)ǫk

(7.17)

7.6. PRACTICAL ADAPTIVE POLE-PLACEMENT

349

Adaptive pole−placement with RLS

output & setpoint

0.4 Normal

0.3

Slow

0.2 0.1 0 −0.1 −0.2

∆ t=0.6, 1

input

0.5 0 −0.5 −1 λ=0.99, na=4, nb=3 2

parameters

1 0 −1 −2

5

tr(P)

10

4

10

3

10

0

50

100

150

200 time (sec)

250

300

350

400

Figure 7.19: Adaptive pole-placement of the black-box showing consistent output response behaviour despite the plant change when the switch was manually activated at t = 128 seconds. Plot (A): black-box output (solid), intended trajectory (dashed) and setpoint point (dotted). (Refer also to Fig. 7.20 for an enlarged version.) Plot (B): Controller input to the plant, Plot (C): Plant parameter estimates, and Plot (D): Trace of the covariance matrix.

CHAPTER 7. ADAPTIVE CONTROL

350 0.28

Normal

0.26

actual y setpoint intended y*

Slow

output & setpoint

0.24

Figure 7.20: A comparison of the intended trajectory, y ⋆ (dashed line) with the actual trajectory y (solid line) for the black box from Fig. 7.19. Even after the plant change, the actual response gradually converges to the desired trajectory.

0.22

0.2

0.18

0.16 60

80

100

120

140

160 180 time (sec)

200

220

240

260

280

and we assume that A and C are monic. We can always divide out the coefficients such that A is monic, and we can choose the variance of the white noise term so that C is monic. For this simplified development we will also need to assume that all the zeros of both C and B are inside the unit circle. The pole excess of the system d, is the difference in the order of the A and B polynomials and is equivalent to the system time delay. The output d steps ahead in the future is dependent on the current and past control actions, u, which are known, and the current, past and future disturbances, the latter which are not known. To avoid using the unknown disturbance values we need to make an optimal prediction.

O PTIMAL

PREDICTION

If we have a process driven by white noise ǫ, zk =

C(q) ǫk A(q)

(7.18)

then the minimum variance predictor over d steps is zˆk+d|k = where G(q) is defined from

qG(q) zk C(q)

q d−1 C(q) = A(q)F (q) + G(q)

(7.19)

(7.20)

We can calculate the quotient F and remainder G polynomials in Eqn. 7.20 from polynomial division in M ATLAB using the deconv function, or alternatively, one could calculate them by equating the coefficients in Eqn. 7.20.

M INIMUM

VARIANCE CONTROL LAW

Once we can make an optimal prediction using the formulation of Eqn. 7.20, we can predict the output d samples times into the future and then calculate the control action required to equate this with the setpoint. This will give the desired minimum variance control law as, uk = −

G(q) yk B(q)F (q)

(7.21)

7.6. PRACTICAL ADAPTIVE POLE-PLACEMENT

351

The closed loop formulation by combining Eqn. 7.17 and Eqn. 7.21 is G yk + C ǫk BF AF yk = −Gyk + CF ǫk CF ǫk = F ǫk yk = AF + G Ayk = −B

(7.22)

where the last equation is simplified using Eqn. 7.20. Eqn. 7.22 indicates that the closed loop response is a moving average process of order d − 1. This provides us with a useful check in that the covariance of the error will vanish for delays larger than d − 1. Listing 7.9 designs the F (q) and G(q) polynomials required in the control law, Eqn. 7.21, by using deconvolution. Note this routine uses stripleadzeros to strip any leading zeros from the B(q) polynomial from Listing B.3 given in Appendix B. Listing 7.9: Minimum variance control design 2

7

12

17

function [F,G,BF] = minvar_design(A,B,C) % Design a minimum variance controller for plant B(q)/A(q) with noise polynomial C(q). % Control law, Eqn. 7.21, is uk = −G(q)/(B(q)F (q))yk B = stripleadzeros(B); % Remove any leading zeros from the B polynomial n = length(A); % order of plant if length(C) 6= n C = [C,zeros(1,1:n-length(C))]; % Force C(q) the same length as A(q) end % if d0 = length(A)- length(B); % Time delay is na − nb = d0 qd0 = [1,zeros(1,d0-1)]; qd0C = conv(qd0,C); [F,G] = deconv(qd0C,A);

% Construct q d0 −1 % Construct q d0 −1 C(q)

BF = conv(B,F); return

% Construct B(q)F (q)

U NSTABLE PROCESS ZEROS As in the adaptive pole-placement case outlined in section 7.6.1, if we have a non-minimum phase plant with unstable zeros, then the resulting controller given by Eqn. 7.21 will be unstable since essentially we are attempting to cancel right-hand plane zeros with right-hand plane poles. Again the solution is to factor B into two factors, one containing all the stable modes, B + , and one containing all the unstable modes, B − , and using only B + in the control law. (See Listing 7.8 to do this.) The F and G polynomials are obtained from solving the Diophantine equation which is a generalisation of Eqn. 7.20. Refer to [23, pp380-385] for further details. Moving average example Consider the following plant system with a noise term, (q − 0.9)(q − 0.8)(q − 0.2)yk = −2(q − 1.25)(q − 0.4)uk + (q 3 − 0.7q 2 + 0.14q − 0.2) ek noting the unstable plant zero at q = 1.25.

CHAPTER 7. ADAPTIVE CONTROL

352

2

Plant = zpk([1.25 0.4],[0.9 0.8 0.2],-2,1); % Plant poles & zeros Ptf = tf(Plant); B = stripleadzeros(Ptf.num{:}); A = Ptf.den{:}; C = [1 -0.7 0.14 -0.2]; % Noise polynomial var_e = 1; % Variance of noise

The step response and pole-zero plot in Fig. 7.21 illustrates the potential difficulties with this plant. Pole−Zero Map

Step Response 20

1

Figure 7.21: A nonminimum phase plant with an unstable zero which causes an inverse step response.

Amplitude

Imaginary Axis

15 0.5

0

10

5 −0.5 0 −1 −1

−0.5

0 0.5 Real Axis

1

1.5

−5

0

20

40 Time (seconds)

60

80

We can design a moving average controller for this NMP plant as follows. B = stripleadzeros(B); [Bp,Bm] = factorBz(B); 4

% Remove the leading zeros, (if any) % Factorise B(q) into well-damped and unstable zeros

n = length(A); % order of plant C = [C,zeros(1,n-length(C))]; % Back-pad with zeros to get same length as A(q) (if necessary) d = length(A)- length(Bp); % # of process zeros to cancel

9

14

19

% Now solve Diophantine eqn AR + BS = q d−1 CB + to compute control law qd = [1,zeros(1,d-1)]; % Construct q d−1 qdCBp = mconv(qd,C,Bp); % Evaluate q d−1 CB + using (multiple) convolution [R,S] = dioph_mtx(A,B,qdCBp) % Listing 7.3. [R1, R1rem] = deconv(R,Bp) % Check that R/B + has no remainder G = S; BF = R; F=R1; % relabel B = [zeros(1,length(A)-length(B)), B]; % Make B(q) same length as A to prevent AE loops sim('sminvarctrl',[0 1e5]) % Now run the Simulink simulation variance = var(ey); varR1 = var_e*sum(R1.*R1); disp([variance, varR1]) % Note that var{y} ≈ var{(F e)}

Note that this controller does not attempt to cancel all two process zeros since that would result in an unstable controller giving rise to unbounded internal signals. However instead we cancel the one stable process zero giving rise to a moving average controller. In this example shown in Fig. 7.22, F (q) = q + 5.0567 which indicates that the expected covariance of the output is σy2 = (12 + 5.062 ) = 26.6 which, provided we take enough samples as shown in Fig. 7.22(b), it is.

7.7. SUMMARY OF ADAPTIVE CONTROL

353

Random Number

F(z)

ey

1 To Workspace

F C

C(z) 1

B(z)

1

1

A(z)

Add

B

Scope

A

Gain G(z)

−1

den(z) Min var ctrlr

(a) A minimum variance moving average controller implementation in S IMULINK . 30 25

Variance

20

variance of output

15 10

variance of noise

5 0 3 10

4

5

10 10 Experiment duration

6

10

(b) The variance of the output should converge to σy2 = 26.6.

Figure 7.22: Minimum variance moving average control. If the controller is designed correctly, then the output variance should be approximately equal to the variance of the noise signal filtered through F , which in this case it is, provided we take enough samples.

7.7

S UMMARY OF ADAPTIVE CONTROL

Adaptive control is where the controller adjusts in response to changing external conditions. This is suitable for plants that are time varying, severely nonlinear or perhaps just unknown at the controller design stage. The general adaptive controller has two parts; an identification part and a controller design part although in some implementations these two components are joined together. The main styles of adaptive control are: ◦ Where the controller parameters are some static function of the output. The simplest example is gain scheduling. This has the advantage that it is easy to implement, but it is really only open loop adaption, and may not perform as expected if the conditions change in an unpredictable manner. ◦ Where the classical PID controller is augmented with a “self-tuning” option. This can be based on the Ziegler-Nichols design, using a relay feedback to extract the ultimate frequency and gain.

354

CHAPTER 7. ADAPTIVE CONTROL

• Where the parameters of the plant transfer function model are estimated online using the techniques described in chapter 6, and the controller is designed to control this model. If the controller is approximately the inverse of the process model, this leads to minimum variance control, but is usually too severe, and even tends to instability for many actual applications. A better alternative is adaptive pole-placement, where reasonable closed loop poles are specified by the control engineer, and the controller is varied to provide this performance as the identified model changes. As with all high performance control schemes, there are some open issues with adaptive controllers. For the simple schemes presented here we should be aware of the following issues: 1. The structure (order and amount of deadtime) of the plant must be known beforehand, and typically is assumed constant. 2. Minimum variance control, while simple to design, attempts to force the process back to the desired operating point in one sample time. This results in overly severe control action. 3. There are potential numerical problems when implementing recursive estimators, particularly in fixed point or with short word lengths. 4. If dead time is present, and usually “dead time” of at least one sample time is always present, then the controller will be unrealisable since future values of y are needed to calculate current values of u. In this case, a sub-optimal controller is used where the dead time is simply ignored, or the future output is predicted. However the real problem of adaptive control is that when the plant is in control, there is little or no information going to the estimator. Hence the controller parameters begin to oscillate wildly. This is not normally noticed by the operator, because the process is in control. However as soon as a disturbance enters the plant, the controller will take extra action attempting to correct for the upset, but now based on the wildly inaccurate plant model. During this time of major transients however, the widely changing process conditions provide good information to the estimator which soon converges to good process parameters, which in turn subsequently generate appropriate controller parameters. The adaptive controller then brings the process back under control and the loop starts all over again. This results in alternating periods of very good control and short periods of terrible control. One solution to this problem is to turn off the estimator whenever the system is at the setpoint. In summary when applying adaptive control, one wants: • A constant output for good control, but • a varying input and output for good estimation (which is required for good control)! This conflict of interest is sometimes called the dichotomy of adaptive control.

7.7. SUMMARY OF ADAPTIVE CONTROL

355

P ROBLEMS Problem 7.1 1. Construct a S IMULINK diagram to simulate the general linear controller with 2-degrees of freedom controller. That is, with the R, S and T controller polynomials. Follow [19, Fig3.2, p93]. Hint: Use the discrete filter blocks in z −1 , since these are the only blocks that you can construct S(z − )/1 in. 2. Simulate the controlled response in problem 1 using the polynomials from [19, Ex3.1], repeated here B(q) 0.1065q + 0.0902 = 2 A(q) q − 1.6065q + 0.6065 and R(q) = q + 0.8467,

S(q) = 2.6850q − 1.0319,

T (q) = 1.6530q

3. Verify that the closed loop is the same as the expected model, BT Bm = AR + BS Am in the example 3.1, [19, Ex3.1, p97–98]. Hint: You may like to use the M ATLAB symbolic toolbox. 4. Construct an m-file to do the minimum degree pole-placement design. (a) Do an ‘All zeros cancelled’ version (b) Do an ‘No zeros cancelled’ version (c) Do an general version, perhaps with some automated decider which zeros to cancel. 5. For the minimum variance controller we need to be able to compute the variance of the output of a FIR filter, refer [19, chapter 4]. (a) What is the theoretical output variance of a FIR filter with input variance of σe2 ? i. In S IMULINK you have the choice of the DSP format z −1 or the transfer function version z. What would be the natural choice for q? ii. Test your answer to part (1) by constructing a simulation in S IMULINK. (b) Approximately how many samples do you need to obtain 2, 3 and 4 decimals accuracy in your simulation? (c) Using the hist function look at the distribution of both the input, e, and output, y. Is the output Gaussian and how could you validate that statistically? 6. For stochastic control and for regulatory control topics we want to be able to elegantly simulate models of the form A(q)y(t) = B(q)u(t) + C(q)e(t) (7.23) where A, B, C are polynomials in the forward shift operator q. Note that the deadtime is defined as deg(A)-deg(B). (a) For the standard model in Eqn. 7.23 what are the constraints on the polynomials A, B, C? i. Choose sensible polynomials for A, B, C with at least one sample of deadtime. Create a model object from the System Identification toolbox using the idpoly command.

356

CHAPTER 7. ADAPTIVE CONTROL ii. Simulate this model either using idmodel/sim or the idsim command. What is the difference between these two commands? Choose a square wave for u(t) and say random noise for e(t). (b) Create a S IMULINK model and compare the simulated results with above. Use the same random input. (c) Write a raw M ATLAB script file to run the simulation and compare with the two alternatives above. (They should give identical results.)

C HAPTER 8

M ULTIVARIABLE

CONTROLLER

DESIGN As our engineering endeavors became more sophisticated, so did our control problems. The oil crises of the mid 1970s gave impetus for process engineers to find ways to recycle energy and materials. Consequently the plants became much harder to control because they were now tightly coupled to other plants. As the plants became multivariable, it made sense to consider multivariable controllers. If we have a dynamic system with say 3 states and 3 inputs, then assuming that our system is controllable, we can feedback the three states directly to the three inputs, state 1 to input 1, state 2 to input 2 etc. This is termed de-centralised control, and our first dilemma is which state should be paired with which input. We then have the problem of tuning the three loops, especially if there are some interactions. Alternatively we could feedback each state to all the inputs, which is termed centralised control. In matrix form, the two alternative control laws look like     ⋆ 0 0 ⋆ • • u =  0 ⋆ 0 x or u =  • ⋆ •  x 0 0 ⋆ • • ⋆ {z } {z } | | centralised

de-centralised

Exploiting the off-diagonals of the gain matrix tends to be more efficient and is often what is meant by multivariable control. While centralised or multivariable control structures typically give better results for similar complexity, they are harder to design and tune since we have more controller parameters (9 as opposed to 3) to select. Not surprisingly, decentralised control remains popular in industry, none the least for the following reasons given by [89]: 1. Decentralised control is easier to design and implement than a full multivariable controller, and also requires fewer tuning parameters. 2. They are easier for the operators to understand and possibly retune. 3. Decentralised controllers are more tolerant of manipulated or measurement variable failure. 4. Startup and shutdown operating policies are easier with decentralised controllers, since the control loops can be gradually brought into service one by one. Despite these application style advantages of decentralised controllers, it is useful to study what is possible with a full multivariable controller. This is the purpose of this chapter. 357

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

358

Finally a one-step optimal controller that is applicable for nonlinear processes called GMC (Generic Model Control) is presented in §8.5. Exact feedback nonlinearisation is a generalisation of the GMC controller follows in §8.6.

8.1

C ONTROLLABILITY AND OBSERVABILITY

This section will introduce in a descriptive manner the concept of controllability, observability and other “abilities”. For a more formal treatment see for example [152, §9–4, p699-713] or [150, p627 §6–2]. The criteria of controllability is given first in §8.1.1 followed by the related property of observability (which is the dual of controllability) in §8.1.2. These conditions are necessary for some of the multi-variable controller and estimator design that follow. Methods to compute the controllability matrix, and establishing the condition are given in §8.1.3.

8.1.1

C ONTROLLABILITY

When designing control systems, we implicitly assume that we can, at least in principle influence the system to respond in some desired manner. However, it is conceivable, perhaps due to some structural deficiency, that we cannot manipulated independently all the states of the system. The controllability criteria tells us if this is the case. Formally defined, a system is controllable if it is possible to transfer from any arbitrary initial state x0 to any other desired state x⋆ in a finite time by using only an uncompensated control signal u. Note that while the transformation is allowed to take place over more than one sample time, it must be achieved in a finite time. In addition, even if the above controllability requirement is satisfied, you may not be able to keep the state x at the desired position, the guarantee is only that x could pass through x⋆ at some time in the future. Nevertheless, it is instructive to know if a given ˚ om system is controllable or not. Some care is needed with these terms. Astr ¨ and Wittenmark, [23, p127], define reachability as I have defined controllability above, and then define controllability as a restricted version where x⋆ = 0. These two definitions are equivalent if A or Φ is invertible. First consider a continuous dynamic system that has a diagonal A matrix and an arbitrary B matrix,   ∗ 0 ··· 0  0 ∗ ··· 0    x˙ =   x + Bu . .  0 0 . 0  0 0 ··· ∗

where an ∗ indicates the presence of a non-zero value. Now suppose if u(t) = 0, then the steady state is xss = 0. Since A is a diagonal matrix, there is no interaction between the states. This means that if one particular state xi , is 0 at some time t, then it will always remain at zero since u= 0. Now consider the full system (u 6= 0), but suppose that the ith row of B are all zeros such as the second row in the following control matrix, 

∗  0 B=  0 ∗

∗ 0 ∗ ∗

0 0 0 0

 0 0   ∗  ∗

Then if xi happens to be zero, it will remain at zero for all time irrespective of the value of the manipulated variable u. Quantitatively this is due to the following two points:

8.1. CONTROLLABILITY AND OBSERVABILITY

359

1. No manipulated variable, uj , can modify the value of xi because all the coefficients in the B matrix for that particular state are zero. 2. No other states can influence xi because in this special case the system matrix A is diagonal. Hence if xi is zero, it will stay at zero. The fact that at least one particular state is independent of the manipulated vector means that the system is uncontrollable. We cannot influence (and hence control) all the states by only changing the manipulated variables. In summary, a system will be uncontrollable if the system matrix A is diagonal, and any one or more rows of B are all zero. This is sometimes known as Gilbert’s criteria for controllability. If the matrix A is not diagonal, then it may be transformed to a diagonal form (or Jordon block form), and the above criteria applied. We can easily transform the system to a block diagonal form using the canon function in M ATLAB. However a more general way to establish the controllability of the n × n system is to first construct the n × (m ∗ n) controllability matrix C, h i def C = B ... AB ... A2 B ... · · · ... An−1 B (8.1)

and then compute the rank of this matrix. If the rank of C is n, then all n system modes are controllable. This is proved in problem [150, A-6-6 p753]. The discrete controllability matrix is defined in a similar manner; h i def C = ∆ ... Φ∆ ... Φ2 ∆ ... · · · ... Φn−1 ∆ (8.2) and also must be of rank n for the discrete system to be controllable.

An often less restrictive controllability requirement is output controllability. This is where we wish only to control the outputs y, rather than the full state vector x. The output controllability matrix C y is defined as h i def C y = CB ... CAB ... CA2 B ... · · · ... CAn−1 B (8.3)

Output controllability is given in Example A-9-11 Ogata, [152, p756], and [150, p636].

The rank of a matrix is defined as the number of independent rows (or columns) in a matrix, and is a delicate numerical computation using a finite precision computer. The presence of a non-zero determinant is not a good indication of rank because the value of the determinant scale so poorly. By the same argument, a very small determinant such as say 10−6 is also not a reliable indication of dependence. Numerically the rank is computed reliably by counting the number of non-zero singular values, or the number of singular values above some reasonable threshold value. See also §8.1.3. The controllability matrix for the submarine system in §2.8.2 is   0 −0.095 0.0192 h i 0.0192 −0.0048  C = B ... AB ... A2 B =  −0.095 0.072 −0.0283 0.0098 which has a rank of 3. Hence the submarine system is controllable.

8.1.2

O BSERVABILITY

Observability is a related property to controllability. A system is observable if one can evaluate the state position at some initial point, x0 , by recording only the output yt over a finite time interval. In other words, the system is observable if all the states eventually “effect” the outputs.

360

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

Note that the observation is allowed to take place over more than one sample time. Once we have back-calculated x0 , we can use the history of the manipulated variable ut to forward calculate what the current state vector xt is. It is clear that this condition of observability is important in the field of estimation. As a trivial example, suppose we had a 3 state system and we have three outputs which are related to the states by   1 0 0 y = Cx =  0 1 0  x 0 0 1

Here the outputs (measurements) are identically equal to the states (C= I). It is clear that we can evaluate all the states simply by measuring the outputs and inverting the measurement matrix x = C−1 y = Iy = y since C−1 = I in this trivial example. This system is therefore observable. Suppose however we measure only two outputs where C is now the non-square matrix   1 0 0 y = Cx = x 0 0 1 Now, obviously, if we recorded only one output vector (2 measurements), we could not hope to reconstruct the full 3 state vector. But the observability property does not require us to estimate x in one sample time, only in a finite number of sample times. Hence it may be possible for a system with fewer outputs than states to be still observable.

To get a feel for the observability of a system, we will use a similar line of argument to that used for the controllability. Suppose we have a diagonal system and an arbitrary output matrix   ∗ 0 ··· 0  0 ∗ ··· 0    (8.4) x˙ =  x  0 0 ... 0  0

y = Cx

0 ··· ∗

(8.5)

Note that any system can be converted into the above form, and for observability criteria we are not interested in the manipulated variable effects. Now if any of the columns of the C matrix are all zeros, then this is equivalent to saying that that a particular state will never influence any of the outputs, and since the system matrix is diagonal, that same state is never influenced by any of the other states. Therefore none of the outputs will ever influence that state variable. Note the symmetry of this criterion to the controllability criteria. Formally for a linear continuous time system described by Eqn. 2.40, the n-state system is observable if the np × n observability matrix O is of rank n where     C C  CA   CΦ      2 2 def  def    CA O=  or in the discrete case O =  CΦ (8.6) ,      .. ..     . . CAn−1 CΦn−1 Some authors, perhaps to save space, define O as a row concatenation of matrices rather than stacking them in a column as done above.

Problem 8.1 1. What are the dimensions of the observability and controllability matrices? What is the maximum rank they can achieve? 2. Establish C and O for the Newell & Lee evaporator.

8.1. CONTROLLABILITY AND OBSERVABILITY

361

3. Examples B-9-13, B-9-10, B-9-11, [152, p771] 4. Examples B-6-6, B-6-7, [150, p805] 5. [156] gives some alternative mathematical definitions for controllability and observability: A system is controllable if and only if rank(B, A − λi I) = n, for i = 1, · · · , n

(8.7)

and observable if and only if rank(C⊤ , A⊤ − λi I) = n, for i = 1, · · · , n

(8.8)

where λ are the eigenvalues of A. Construct a M ATLAB m-file implementation using these algorithms. Test your routines on various systems, perhaps using the rmodel command.

8.1.3

C OMPUTING

CONTROLLABILITY AND OBSERVABILITY

The M ATLAB functions ctrb and obsv create the controllability and observability matrices respectively, and the rank function can be used to establish the rank of them. While the concept of observability and controllability applies to all types of dynamic systems, Eqns 8.1 and 8.6 are restricted to linear time invariant systems. Read problem A-6-1 [150, p745] for a discussion on the equivalent term “reachable”. The algorithms given above for computing controllability or observability, (equations 8.1 and 8.6 or the discrete equivalents), seem quite straight forward, but are in fact often beset by subtle numerical errors. There exist other equivalent algorithms, but many of these are also susceptible to similar numerical problems. Issues in reliable numerical techniques relevant to control are discussed in a series of articles collected by [158]. An example quoted in [156] illustrates common problems to which these simple algorithms can fall prey. Part of the problem lies in the reliable computation of the rank of a matrix which is best done by computing singular values. Consider computing the controllability of the dynamic system     1 µ 1 A= , B= 0 1 µ where µ is a ‘small’ number. We can do this symbolically with

4

9

>> syms mu positive >> A = [1 mu; 0 1]; B = [1; mu]; >> C = [B, A*B] % Compute controllability matrix, C C = [ 1, muˆ2 + 1] [ mu, mu] >> det(C) ans = -muˆ3

which tells us that the controllability matrix is   1 1 + µ2 C= µ µ

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

362

and has a nonzero determinant of |C| = −µ3 indicating that the system is technically controllable given a nonzero µ. However in √ this example we are particularly interested in small, (but not minute) values of µ, say where |µ| < ǫ, where ǫ is the machine precision. Note that this effectively means that 1 + µ2 = 1 whereas 1 + µ > 1 when executed on the computer. Consequently the controllability matrix is now   √ 1 1 C= , for |µ| < ǫ µ µ which indicates the system is uncontrollable.

We can try this numerically for values near the expected limit, µ = 1

6

√ ǫ,

√ >> mu = sqrt(eps(1)) %µ≈ ǫ mu = 1.4901e-08 >> A = [1 mu; 0 1]; B = [1; mu]; >> det(ctrb(A,B)) ans = -3.3087e-24

where fortuitously in this case we get a non-zero determinant of equal to −µ3 as hoped. However for (slightly) smaller values of µ,

3

√ >> mu = sqrt(eps(1))/10 % µ < ǫ mu = 1.4901e-09 >> A = [1 mu; 0 1]; B = [1; mu]; >> det(ctrb(A,B)) ans = 0

due to the roundoff, this tells us the system is uncontrollable. Problem 8.2 Write a M ATLAB m-file to compute the exponential of a matrix using the Taylor series formula, ∞ X 1 k eAt = A k! k=0

Test your algorithm on

A=



−49 24 −64 31

and comment on the expected accuracy of your result.



Problem 8.3 1. When using good (in the numerical reliability sense) software, we tend to forget one fundamental folklore of numerical computation. This folklore, repeated in [158, p2], states

If an algorithm is amenable to “easy” hand calculation, then it is probably a poor method if implemented in the finite floating point arithmetic of a digital computer. With that warning in mind, calculate the singular values of   1 1 A= µ 0  0 µ

8.1. CONTROLLABILITY AND OBSERVABILITY

363

√ where µ is a real number such that |µ| < ǫ where ǫ is the machine precision. Calculate the singular values both analytically, and using the relationship between singular values and the eigenvalues of A⊤ A.

8.1.4

S TATE

RECONSTRUCTION

If a system is observable, we should in principle, be able to reconstruct the states based only on the current, and perhaps historical outputs. So, supposing we have collected the following input/output data, time (k) 0 1 2

u(k) 4 5 –

y(k) −4 4 36

x1 (k) ? ? ?

x2 (k) ? ? ?

we should be able to reconstruct the states assuming a model of the form     0 1 0 xk+1 = xk + uk −3 2 1   yk = 4 0 x k

Note that we cannot simply invert the measurements back to the states, partly because the measurement matrix is not square.

However we can see that y(k) = 4x1 (k) from the measurement relation, so we can immediately fill in some of the blank spaces in the above table, in the x1 (k) column; x1 (0) = −1, x1 (1) = 1, x1 (2) = 9. From the model relation, we can see that x1 (k + 1) = x2 (k), so we can set x2 (0) = x1 (1) = 1. Now we have established the initial condition  ⊤ x(0) = −1 1

and with the subsequent manipulated variable history u(t), we can calculate x(t) simply using the recursive formula, Eqn. 2.41. This technique of estimation is sometimes called state reconstruction and assumes a perfect model and no measurement noise. Furthermore, the estimation reconstruction takes, at most, n input-output sample pairs. As a check, we can construct the observability matrix,        4 0  C 4     O= = 0 1 = CΦ 4 0 × 0 −3 2

0 4



which clearly has a rank of 2 indicating that the state reconstruction is in fact possible.

We can generalise this state reconstruction scheme outlined above, following [20, p315], for any observable discrete plant xk+1 = Φxk + ∆uk yk = Cxk where we wish to reconstruct the current state, xk , given past values of input and output, uk , uk−1 , . . . and yk , yk−1 , . . . Our basic approach will be to use the outputs from the immediate past n samples, to develop linear equations for the n unknown states at the point n samples in the past. We will then use the

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

364

knowledge of the past inputs to roll this state forwards to the current sample k. For the present development, we will just consider SISO systems. Listing the immediate previous n values finishing at the current sample k gives yk−n+1 = Cxk−n+1 yk−n+2 = CΦxk−n+1 + C∆uk−n+1 yk−n+3 = CΦ2 xk−n+1 + CΦ∆uk−n+1 + C∆uk−n+2 .. . yk = CΦn−1 xk−n+1 + CΦn−2 ∆uk−n+1 + · · · + C∆uk−1

(8.9)

To make things clearer, we can stack the collected input and output values in a vector,     yk−n+1 uk−n+1  yk−n+2     uk−n+2      .. Yk =  , U =    .. k−1 .     .  yk−1  uk−1 yk

(8.10)

and we note that the input vector collection Uk−1 is one element shorter than the output vector collection Y k . Now we can write Eqn. 8.9 more succinctly as Y k = Oxk−n+1 + Wu Uk−1 with matrices defined as     O=  

C CΦ CΦ2 .. .

CΦn−1



   ,  



   Wu =   

0 C∆ CΦ∆ .. .

CΦn−2 ∆

(8.11)

0 0 C∆ .. .

CΦn−3 ∆

··· ··· ··· .. .

0 0 0 .. .

· · · C∆

and we may recognise O is the observability matrix defined in Eqn. 8.6.

      

(8.12)

Solving Eqn. 8.11 for the states (in the past at sample time k − n + 1) gives xk−n+1 = O −1 Yk − O −1 Wu Uk−1

(8.13)

provided O is non-singular. This is why we require the observability condition to be met for state reconstruction. The minor problem with Eqn. 8.13 is that it delivers the state n sample times in the past, rather than the current state, xk , which is of more immediate interest. However, it is trivial to use the plant model and knowledge of the past inputs to ‘roll the states onwards’ to current time k. Repeated forward use of the model gives xk = Φn−1 xk−n+1 + Φn−2 ∆uk−n+1 + Φn−3 ∆uk−n+2 + · · · + ∆uk−1 The entire procedure for state reconstruction is summarised in the following algorithm. Algorithm 8.1 State reconstruction assuming a perfect observable model and no noise. Given xk+1 = Φxk + ∆uk ,

yk = Cxk

then the current state, xk , is given by the linear equation xk = Ay Y k + Bu Uk−1

8.1. CONTROLLABILITY AND OBSERVABILITY

365

where the matrices are Ay = Φn−1 O +  Bu = Φn−2 ∆

Φn−3 ∆ · · · ∆



− Φn−1 O + Wu

and Y k , Uk−1 are defined in Eqn. 8.10 and O, Wu are defined in Eqn. 8.12. We should note: • To reconstruct the state (at the current time k), we need to log past input and output data, and the minimum number of data pairs required is n, where n is the number of states. Any more will gives a least-squares fit. • Since we invert O, this matrix must not be singular. (Alternatively we require that the system must be observable.) For the multivariable case we should use the pseudo-inverse, O + , as written above. • We use in the algorithm, but do not invert, the controllability matrix. A simple state reconstructor following Algorithm 8.1 is given in Listing 8.1. Listing 8.1: A simple state reconstructor following Algorithm 8.1. function x_est = state_extr(G,Yk,Uk) % Back extract past state from outputs & inputs (assuming observable) 3

Phi = G.a; Del = G.b; C = G.c; error(ABCDCHK(Phi,Del,C)) % bullet proofing [p,n] = size(C); [n,m] = size(Del); 8

13

18

Wo = obsv(G); % same as: O = obsv(Phi,C) Ay = Phiˆ(n-1)/Wo; % similar to Φn−1 O −1 S = C*Del; for i=1:n-2 S = [S; C*Phiˆi*Del]; % Building Wu end % for zC = zeros(p,m); % or zeros(size(C*Del)); Wu1 = [zC;S]; Wu = Wu1; for i=n-3:-1:0 Wu1 = [zC; Wu1]; Wu1(end-p+1:end,:) = []; Wu = [Wu, Wu1]; end % for

23

Co = Del; % Reversed controllability matrix for i=1:n-2 Co = [Phiˆi*Del,Co]; end % for

28

Bu = Co - Ay*Wu; x_est = Ay*Yk + Bu*Uk; return

Fig. 8.1 illustrates the results of the state reconstruction algorithm in the case where there is no model/plant mismatch, but there is noise on both the input and output. Consequently, given that the algorithm has the wrong u(t) and y(t) trends, then the reconstructed states (• in Fig. 8.1) are slightly different from the true states. In summary, this reconstruction algorithm takes current and historical outputs and inputs, and, provided the system is observable, then we can reconstruct the current states. This strategy is

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

366

0.3

Output

0.2 0.1 0

actual y measured y

−0.1 4 actual u measured u

u

2 0 −2 2

x

1

est

x

1

1 0 −1 2 x

2

est

2

1 x

Figure 8.1: Reconstructing states. Since the state reconstructor is given an incorrect input and a noisy output, then it is no surprise that the estimated states, •, are slightly different from the true states (solid lines in the lower two trends).

0 −1

0

1

2

3

4

5

6

7

time [s]

important because it then allows us to use state feedback in the cases where we only measure the outputs. In the presence of process or measurement noise, and/or model/plant mismatch this reconstruction will be in error, although in many cases this may not be significant. An improved version of this algorithm that takes into account the statistics of the expected noise and plant errors is known as state estimation and is described in section 9.5.

8.2

S TATE SPACE POLE - PLACEMENT CONTROLLER DESIGN

This section describes how to design a controller to control a given process, using state space techniques in the discrete domain. When a process engineer asks the control engineer to design a control system, the first question that the control engineer should ask back, is ‘what sort of performance is required?’ In this way, we start with some sort of closed loop specification, which, for example, may be that we demand that the system return to setpoint after a disturbance with no offset, or rise to a new setpoint in a reasonable time, perhaps one open-loop time constant. If the loop is a critical one, then we may wish to tighten this requirement by specifying a smaller time to reach control, or conversely if we have a non-critical loop we could be satisfied with a more relaxed settling time requirement. It is our job as control engineers to translate these vague verbal requirements into mathematical constraints, and if possible, then design a controller that meets these constraints. When we talk about a response, we are essentially selecting the poles (or eigenvalues) of the closed

8.2. STATE SPACE POLE-PLACEMENT CONTROLLER DESIGN

367

loop transfer function, and to some lesser degree, also the zeros. Specifying these eigenvalues by changing the controller parameters is called controller design by pole placement. In order to design a controller by pole placement, we must know two things: 1. A dynamic model of the process, and 2. what type of response we want (i.e. what the closed loop eigenvalues should be). However choosing appropriate closed loop poles is non-trivial. Morari in [138] reminds us that “poles are a highly inadequate measure of closed loop performance”, and he continues to assert that because of the lack of guidance to the control designer, “pole placement techniques have been abandoned by almost everybody today”. Notwithstanding this opinion, it is still instructive for us to understand how our closed loop response is affected by different choices of eigenvalues, and how this forms the basis for more acceptable tuning methods such as linear quadratic controllers discussed in the following chapter. Suppose we have a single input–multiple output (SIMO) system such as xk+1 = Φxk + ∆uk

(8.14)

where u is a scalar input and we wish to control all n states by using a proportional feedback control law such as u = −Kx (8.15) where K is called the state feedback gain matrix. (Actually in this SIMO case, the gain “matrix” collapses to a row vector). Substituting Eqn 8.15 into Eqn 8.14 gives the closed loop response as xk+1 = (Φ − ∆K) xk

(8.16)

Now we desire (Φ−∆K) to have the n eigenvalues at arbitrary positions µ1 , µ2 , · · · , µn To achieve this, we must check that arbitrary pole placement is achievable. In other words, the system must be controllable. This means that the controllability matrix Co , must be of rank n. If this condition is satisfied, we can design the feedback gain matrix K, by using Ackermann’s formula K=



0

0 ··· 0

1

h

. ∆ ..

Φ∆

.. .

. · · · ..

Φn−1 ∆

i−1

α(Φ)

(8.17)

where α is the matrix characteristic equation of the desired closed loop transfer function. Note that the matrix to be inverted in Eqn 8.17 is the discrete controllability matrix Co , hence the need for full rank. A quick way to calculate α(Φ) is to use poly and polyvalm. For instance we can verify the Cayley-Hamilton theorem, (any matrix satisfies its own characteristic equation), elegantly using 1

>> A = rand(3) % take any test matrix A = 0.0535 0.0077 0.4175 0.5297 0.3834 0.6868 0.6711 0.0668 0.5890

6

11

>> polyvalm(poly(A),A) % Cayley-Hamilton theorem: α(A) should = 0 ans = 1.0e-016 * 0.1388 0.0119 0.1407 0.1325 0 0.7412 -0.0496 0.0073 0.5551

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

368

Algorithm 8.2 SIMO state-space pole-placement Given a continuous system model, x˙ = Ax + Bu, or discrete version, xk+1 = Φxk + ∆uk , and design requirements for the closed loop poles, µ, then 1. Check the controllability of the system, ctrb(A, B), or ctrb(Φ, ∆). If not all the states are controllable, then terminate the algorithm, otherwise carry on. 2. Compute the characteristic polynomial, α, and substitute either matrix A or Φ. 3. Compute the row-vector state feedback gain using Eqn. 8.17. Example Pole placement of the double integrator system. In this example, we wish to design a controller for the discrete double integrator system sampled at T = 1 that has closed loop eigenvalues at µ = 0.5 ± 0.5i. In this case we will assume a zeroth-order hold on the discretisation.

giving matrices   2 −0.5 Φ= , 2 0

Gc = tf(1,[1 0 0]); % Continuous system G = ss(c2d(Gc,1)); % Sampled at Ts = 1

∆=



1 0



To use Ackermann formula, Eqn. 8.17, we need to both compute the controllability matrix and the matrix characteristic equation α. def

Note that as α(z) = (z − µ1 )(z − µ2 ) = z 2 − z + 0.5, then   1 −0.5 α(Φ) = Φ2 − Φ + 0.5I = 2 −0.5 and that the controllability matrix C= has full rank.

3



1 2 0 2



% Design requirements for µ

>> mu = [0.5+0.5i; 0.5-0.5i]; >> alpha = polyvalm(poly(mu),G.a) >> Cb = ctrb(G.a,G.b)

Finally the feedback gain is computed by >> K = [0 1]/Cb*alpha % giving K =



1

−0.25



We can easily check that the closed loop poles are as required with

4

>> eig(G.a-G.b*K) % Check that µ = eig(Φ − ∆K) ans = 0.5000 + 0.5000i 0.5000 - 0.5000i

Rather than constructing Ackermann’s equation by hand, M ATLAB can design a pole placement controller using the acker routine. At heart this routine simply computes Eqn. 8.17 as

8.2. STATE SPACE POLE-PLACEMENT CONTROLLER DESIGN

1

k = ctrb(a,b)\polyvalm(real(poly(p)),a); k = k(n,:);

369

% Form gains using Ackerman’s formula

or, as noted in the next section, acker is very ill-conditioned, so we could use the preferred place function. The latter function gives an estimate of the precision of the closed loop eigenvalues, and an optional error message if necessary. (See also Fig. 8.3.)

3

[K,prec] = place(G.a,G.b,mu) K = 1.0000 -0.2500 prec = 15

Eqn 8.15 assumes that the setpoint is at x=0. (We can always transform our state variables by subtracting the setpoint so that this condition is satisfied, but this becomes inefficient if we are likely to change the setpoint regularly.) In these cases, it is better to modify our control law from Eqn 8.15 to u = K(r − x) (8.18) where r is a vector of the state setpoints or reference states, now assumed non-zero. Consequently the closed loop response is now modified from Eqn 8.16 to xk+1 = (Φ − ∆K) xk + ∆Krk

8.2.1

P OLES

(8.19)

AND WHERE TO PLACE THEM

While pole-placement seems an attractive controller design strategy, the problem lies in deciding where the closed loop poles out to be. Listing 8.2 designs a pole-placement controller for the blackbox, G(s) = 0.98/(2s + 1)(s + 1), with discrete closed loop poles specified, somewhat arbitrarily, at µT = [0.9, 0.8]. Listing 8.2: Pole-placement control of a well-behaved system >> Gc = tf(0.98,conv([2 1],[1,1])) % Blackbox model >> G = ss(c2d(Gc,0.1)) % Convert to discrete state-space with sample time Ts = 0.1 >> [Phi,Delta,C,D] = ssdata(G); 5

>> lambda = [0.9 0.8]'; % Desired closed loop discrete poles µT = [0.9, 0.8] >> K = place(Phi,Delta,lambda)% Compute controller gain K K = 2.4971 -2.2513

Now we can check the closed loop performance from a non-zero initial condition with Gcl = ss(Phi-Delta*K,zeros(size(Delta)),eye(size(Phi)),0,Ts); 2

x0 = [1; -2]; initial(Gcl,x0)

% start position % See Fig. 8.2

The results of a controlled disturbance recovery is given in the left hand plot of Fig. 8.2. However an obvious question at this point is where exactly should I place the poles. The three trends in

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

370 T

T

States x1 & x2

µ = [0.9, 0.8]

µ = 0.6e

6

6

6

4

4

4

2

2

2

0

0

0

−2

−2

100

100

0

0

0

input

−2 100

−100

0

2 1 2.2

ℑ

0.5

2.7

0 −0.5

1.7 1.2 0.2 0.4 0.7 0.6 0.8 0.2 0.2

2.7

0

2 1 2.2

0.5

2.7

0

1

−1 −1

4

−100

1.7 1.2 0.2 0.4 0.7 0.6 0.8 0.2 0.2

2.7

2 1 2.2

0.5

2.7

0

1

−1 −1

4

1.7 1.2 0.2 0.4 0.7 0.6 0.8 0.2 0.2

2.7 0.7 2.2

1.7 1.2

0 ℜ

0

−0.5

0.7 2.2

1.7 1.2

0 ℜ

−100

−0.5

0.7 2.2

−1 −1

4

°

± j80

T

µ = [−0.9, −0.8]

1.7 1.2

0 ℜ

1

Figure 8.2: Pole-placement of the black box for three different pole locations. From left to right: ◦ (a) µT = [0.9, 0.8], (b) µT = [−0.9, −0.8], and (c) µ = 0.6e±j80 . Fig. 8.2 illustrate the difference. A conservative choice is as given above is where we place the poles in right-hand side of the origin, on on the real axis close to the +1 point. In this case I chose µT = [0.9, 0.8]. The second choice is the mirror image of those poles µT = [−0.9, −0.8], and the third choice is a pair of complex conjugates at 0.66 ±80◦ . Clearly there is a trade off between the speed of the disturbance rejection, the amount of input energy, and the degree of oscillation. If we can quantify these various competing effects, then we can design controllers systematically. This is the subject of optimal control given in the next chapter.

T HE

POLE - PLACEMENT DESIGN PROBLEM IS ILL - CONDITIONED

The design of controllers using pole-placement is inherently numerically ill-conditioned, especially for large systems. Ackermann’s algorithm requires double precision for systems larger than about 10th order, and outright fails for systems larger than about 15th order. Fig. 8.3 illustrates this degradation in accuracy by plotting an estimate of the number of accurate decimal places in the solution using place for 1500 randomly generated discrete models from orders 3 to 15. Note that once the order is above 9 or so, we should only expect about 6 decimal places of accuracy.

8.2. STATE SPACE POLE-PLACEMENT CONTROLLER DESIGN

371

20

precision

15

10

5

pole−place Acker

0

−5

1

2

3

4

5

6

7

8 9 System order

10

11

12

13

14

15

Figure 8.3: The loss of accuracy using pole-placement algorithms place and acker as a function of system order for 500 randomly chosen models. It is clear that both routines struggle for systems of order above about 6.

8.2.2

D EADBEAT

CONTROL

If the system is controllable such that any arbitrary poles are allowable, one is naturally inclined to extrapolate the selection to poles that will give a near perfect response? Clearly a perfect response is one when we are always at the setpoint. If a step change in reference value occurs, then we instantaneously apply a corrective measure to instantaneously push the system to the new desired point. A system that rises immediately to the new setpoint has a closed loop time constant of τ = 0 or alternatively a continuous pole at s = −∞. This corresponds to a discrete pole at e−∞T or the origin. If we decide to design this admittedly rather “drastic” controller, we would select all the desired closed loop poles at µ = 0. Such a controller is feasible in the discrete domain and is called a deadbeat controller. The design of a deadbeat controller is the same as for an arbitrary pole placement controller. However it is better to use the acker function rather than the place function, because the place function will not allow you to specify more multiple eigenvalues than inputs. This restriction does not apply for the Ackermann algorithm. Suppose we design a deadbeat controller for the continuous plant     −0.3 0 0 3 0 −0.5 2  x +  −2  u x˙ =  0 −2 −0.5 5

(8.20)

To design a deadbeat controller, we specify the desired discrete closed loop eigenvalues as µ = 0 and select a sample time of T = 4, we obtain   K = 3.23 · 10−2 −3.69 · 10−4 1.98 · 10−2 The closed loop transition matrix is



 0.075 0.003 −0.139 0.095  Φcl = Φ − ∆K =  −0.063 −0.019 −0.060 −0.133 −0.056

and we could also compute that all the elements in Φ3cl are in the order of 10−19 and all the eigenvalues of eig(Φ − ∆K) are in the order of 10−7 as required. Such a matrix where An−1 6= 0, but An = 0 is called a Nilpotent matrix.

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

372

Fig. 8.4 shows the impressive controlled response of the plant (Eqn. 8.20) under deadbeat control. The top half of the diagram shows the states which converge from the initial condition, ( x0 =  ⊤ −2 −3 1 ), to the setpoint x=0 in only three sample times (3T = 12 s). It takes three sample times for the process to be at the setpoint, because it is a third order system. Deadbeat controller 4 3 2

states

1 0 −1 −2 −3

Figure 8.4: The states (top) and the manipulated variable (bottom) for the deadbeat controlled response. The states after the third sample time, t ≥ 12, remain exactly at the setpoint at the sample times.

input

−4 0.04 0.02 0 0

5 10 15 time (s), sample time T=4

20

The deadbeat controlled response looks almost perfect in Fig. 8.4, at least after the 3 samples required to get top the setpoint. We can investigate this by repeating the simulation at a smaller time interval (say T = 0.05 s) but with the same input as before, to see what happens between the sample intervals. (This is how I evaluated the smooth state curve in Fig 8.4 between the states at the controller sample time denoted with a • in Listing 8.3.) Note that the states are oscillating between the samples, but quickly die out after 10 seconds. Listing 8.3: A deadbeat controller simulation 1

x0= [-2 -3 1]'; dt=4; tf=dt*5; t=[0:dt:tf]'; % initial condition & time vector A= [-0.3,0,0;0,-0.5,2;0,-2,-0.5];B= [3;-2;5]; % Continuous plant, Eqn. 8.20 Gc= ss(A,B,eye(3),0); Gd= c2d(Gc,dt,'zoh'); % Convert to a discrete model

6

11

16

21

mu= [0;0;0]; % Desired closed loop eigenvalues, µ = 0 K= acker(Gd.a,Gd.b,mu); % construct deadbeat controller Gd_cl = Gd; Gd_cl.a = Gd.a-Gd.b*K; % closed loop [y,t,x]= lsim(Gd_cl,zeros(size(t)),[],x0); U = [-K*x']'; % back extract input % Reduce sample time at same conditions to see inter-sample ripple dt = dt/100; t2 = [0:dt:tf]'; % new time scale, Ts⋆ = Ts /100 [ts,Us] = stairs(t,U); % create new input ts = ts + [1:length(ts)]'/1e10; ts(1) = 0; U2 = interp1(ts,Us,t2); % approx new input [y2,t2,x2]= lsim(Gc,U2,t2,x0);% continuous plant simulation subplot(2,1,1);plot(t,x,'o',t2,x2); % Refer Fig. 8.4 subplot(2,1,2);plot(ts,Us);

8.2. STATE SPACE POLE-PLACEMENT CONTROLLER DESIGN

373

Naturally under practical industrial conditions, the plant will not be brought under control so rapidly owing to either model–plant mismatch, or to manipulated variable constraints. This type of controller can be made more robust by specifying a larger sample time, which is in fact the only tuning constant possible in a deadbeat controller. Ogata gives some comments and cautions about deadbeat control in [150, pp670-71]. Problem 8.4 Create a system with eigenvalues of your choice; say v=



−0.2 −0.3 −0.1

T

and design a deadbeat controller. Simulate this using the dlsim function. (Try a sample time of ≈ 7.) Investigate the same controller, but at a higher sampling rate and plot the intersample behaviour of the states. Investigate the effect on the gain when using different sample rates.

8.2.3

P OLE - PLACEMENT

FOR TRACKING CONTROL

The pole-placement controller developed in Algorithm 8.2 is only a proportional controller so will suffer from the problem of offset given setpoint changes as graphically demonstrated in the upper plot in Fig. 8.6. Such controllers are suitable for disturbance rejection because they return to a setpoint of zero after a disturbance, but they are not much good when we wish to track an arbitrary setpoint. To include tracking characteristics in our pole-placement controller, we will need to introduce extra states, z defined as the integral of the normal states Z def z= x dt (8.21) which can be augmented to the original system as        x˙ A 0 x B = + u. z˙ I 0 z 0

(8.22)

The feedback control law is now u=

Kp

Ki






x z



(8.23)

where Kp are the gains for the proportional controller and Ki are the gains for the integral controller. Now the pole-placement controller is designed for the new system with n additional integral states in the same manner as for the proportional state only case. In M ATLAB we can formulate the new augmented system as Ai = [A zeros(size(A)); eye(size(A)) zeros(size(A))] Bi = [B; zeros(size(B))]

This modification improves the system type, (i.e. the number of poles at the origin), enabling our controller to track inputs. [110, §10–14,p802] gives further details. We shall see later in section 9.4.6 how we can design optimal pole-placement servo controllers known as linear quadratic regulators.

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

374 SISO SERVO INTEGRAL

OUTPUT CONTROL

If we want to track setpoint changes (servo-control) for a SISO system, we need only to add one extra integral state. The augmented system is x˙ = Ax + Bu z˙ = r − y y = Cx

where x is the vector of original states, u, y and r are the scalar input, output and setpoint respectively. We simply concatenate the new integral state to the existing, def

¯= x

h

x1

x2

.. .

· · · xn

z

i⊤

¯ x now has so that z is relabelled as xn+1 . The feedback gain row-vector in the control law u = −K¯ an additional element h i .. ¯ = K k1 k2 · · · kn . kn+1 and the augmented closed-loop system is now


d¯ x ¯ −B ¯K ¯ x ¯+ = A dt



0 1



r

where the augmented matrices are:

¯ = A



 A 0 , −C 0 {z } |

¯ = B

(n+1)×(n+1)



 B 0 | {z }

(n+1)×1

To implement, the augmented system must be controllable, which is a tougher requirement than just ctrb(A, B). A block diagram of a pole-placement controller with a single integral state is given in Fig. 8.5, after [110, p802]. The wide lines in the diagram are vector paths.

integral state

r

✲ + x˙ n+1✲ − ✻

R

xn+1 ✲

u −kn+1 ✲ + ✲ − ✻

x˙ R B ✲+ ✲ ✻

x

✲ C

A ✛ K ✛

full state feedback

Figure 8.5: State feedback control system with an integral output state. We can compare the effect of adding an integral state to remove the offset given the SISO system     4 3 2 x˙ = x+ u −1 6 1   y= 1 0 x

✲ y

8.3. ESTIMATING THE UNMEASURED STATES

375

where we want to control x1 to follow reference r(t) using pole-placement. I will place the closedloop eigenvalues for the original system at   −6 + 2i λ= −6 − 2i and with the extra integral state, I will add another pole at λ3 = −10. Fig. 8.6 compares the response of both cases. Clearly the version with the integral state that tracks the setpoint with no steady-state error is preferable. 4 3

output & setpoint

2

Integral state

1 0 −1 setpoint

−2

−4

8.3

standard integral states

Proportional only

−3 0

1

2

3

4

5 time [s]

6

7

8

9

10

Figure 8.6: Adding an integral state removes the offset for an LQR (or pole-placement) controller.

E STIMATING THE UNMEASURED STATES

The pole-placement control design strategies as developed in § 8.2 utilises all the states in the control law u = −Kx so implicitly assumes that all the states are measurable. In many cases this is not practically possible, so the only alternative then is to estimate the states. The states can be reconstructed using the outputs, the inputs, and the model equations. In fact we need not restrict ourselves to just estimating states, we could also estimate derived quality control variables, or even estimate the outputs themselves. The quantities that we estimate are known as observers, and offer cost and reliability advantages over actual sensors in many applications. The simplest state estimator is where we use a model and subject it to the same input u that is ˆ is the estimate for the true state x. applied to the true process. The resultant x ˆ k+1 = Φˆ x xk + ∆uk

(8.24)

If we know the true model, Φ,∆, and the manipulated variable history uk , then this estimator will ˆ 0 . Since we have no feedback information work provided we select the correct initial condition x as to how well we are doing, this estimator is called an “open loop” estimator. If the conditions stated above are not met, and they will rarely be satisfied in practice, then the estimator will provided estimates that will probably get worse and worse. The error of the estimate is ˆk ǫk = xk − x

(8.25)

Ideally, we would like to force this error vector to zero. This is actually still a control problem, although while we are not controlling the plant, we are controlling the error of the estimate scheme. We introduce a feed back error scheme such as ˆ k+1 = Φˆ x xk + ∆uk + L (yk − Cˆ xk )

(8.26)

which is called a prediction error observer since the prediction is one sample period ahead of the observation. In the single output case, L is a (n × 1) column vector and is called the observer gain

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

376

matrix and is a design parameter. The L stems from the fact that it is a Luenberger observer and we need to carefully distinguish it from the controller gain, K. Following our colour convention, we will sometimes colour the estimator, L red because it involves states, and the feedback gain, K blue because it is part of the controller. Now the error model is ǫk+1 = (Φ − LC) ǫk

(8.27)

We must select the matrix L such that the observer system Φ − LC is stable and has eigenvalues (observer poles) that are suitably fast. This means that even if we were to get the initial condition wrong for x0 , we can still use the estimator since any initial error will eventually be forced to zero. Typically we would choose the poles of the observer system (Eqn. 8.27) much faster than the openloop system. To be able to solve for L, the system must be observable. For many chemical process applications, the sampling rate is relatively slow, since the processes are generally quite slow, so the basic estimator of Eqn 8.26 can be modified so that the prediction of xk+1 uses the current measurement yk+1 rather than the previous measurement yk as is the case in Eqn 8.26. This is only possible if the computational time is short, compared with the sample time, but if this is the case, then the sample period is better utilised. This type of formulation is termed a current estimator and both are described in [71, pp147–8] and in terms of the Kalman filter in [150, pp873–75]. The current estimator is ¯ k+1 = Φˆ x xk + ∆uk ˆ k+1 = x ¯ k+1 + L yk+1 − C¯ x xk+1

(8.28)


(8.29)

In practice for most systems, the two forms (current and prediction estimators) are almost equivalent. The design of the observer gain L by pole-placement is similar to the controller gain K given in section 8.2, but in the observer case we have the closed loop expression eig(Φ − LC) rather than eig(Φ − ∆K) However we note that eig(Φ − LC) = eig(ΦT − CT LT ) since eig(A) = eig(AT ), so we can use the M ATLAB functions place or acker for the observer problem by using the dual property L = place(ΦT , CT , λ)T Table 8.1 shows the equivalent relationships between the matrices used for observation and regulation and make the relevant substitution. Table 8.1: The relationship between regulation and estimation

Control Φ B K Co eig(Φ − ∆K)

Estimation Φ⊤ C⊤ L⊤ Ob eig(Φ − LC)

8.4. COMBINING ESTIMATION AND STATE FEEDBACK

8.4

377

C OMBINING ESTIMATION AND STATE FEEDBACK

We can combine the state feedback control from section 8.2 with the state estimation from section 8.3 into a single algorithm. This is useful because now we can use the estimator to provide the states for our state feedback controller. For linear plants, the design of the state feedback controller is independent from the design of the observer. The block diagram shown in Fig. 8.7, which is adapted from [20, p438], shows the configuration of the estimator, L, and state-feedback controller, K.

✲ ∆

✲+ ✻

✲ q −1

✲ C

✲+

Φ ✛

Plant ǫ

L ✛ input, u

✲ ∆

❄ ˆ ✲ + ✲ q −1 x ✻

output, y

✲ −C

✲+✛

Φ ✛

Estimator

−K ✛

State feedback

Figure 8.7: Simultaneous control and state estimation Remember that in this example, we have taken the perhaps unreasonable assumption that we have model plant mis-match and we assume that the plant is linear, but we are faced with the problem that we do not know the initial conditions of the states x0 . The equations are summarised below. The true plant dynamics are xk+1 = Φxk + ∆uk

(8.30)

yk = Cxk

(8.31)

and the measurement is but we do not know this true state vector x, so we estimate it with ˆ k+1 = Φˆ x xk + ∆uk + L (yk − Cˆ xk )

(8.32)

so consequently the scalar control law uk = −Kˆ xk

(8.33)

ˆ , not the actual state, x. Combining Eqns 8.33 can only use what we know, that is the estimate, x and 8.30 gives xk+1 = Φxk − ∆Kˆ xk (8.34)

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

378

and combining Eqns 8.32 to 8.33 gives the closed loop ˆ k+1 = (Φ − LC − ∆K) x ˆ k + LCxk x

(8.35)

Concatenating Eqns 8.30 and 8.35 together gives the homogeneous equation      xk+1 xk Φ −∆K = zk+1 = ˆ k+1 ˆk x x LC Φ − ∆K − LC | {z } Φcl-ob

(8.36)

I will call the transition matrix for Eqn 8.36 the closed-loop/observer transition matrix (Φcl-ob ). Ogata gives a slightly different formulation of the combined observer–state feedback control system as Eqn 6-161 in [150, p714]. Ogata also recommends choosing the observer poles 4–5 times faster than the system response.

E XAMPLE OF

A COMBINED STATE - FEEDBACK AND OBSERVER

Suppose we are given a continuous plant G(s) =

0.98 , (3s + 1)(2s + 1)

T = 0.1

which we will sample at T = 0.1. The design specifications are that the estimator poles are to be relatively fast compared to the controller poles     0.8 0.92 def def and λctrl = λest = 0.85 0.93 We can design the controller and estimator gains as follows Listing 8.4: Pole placement for controllers and estimators G = tf(0.98,conv([2 1],[3 1])); % Blackbox plant Gss = ss(G); % convert to state-space 3

Gss1 = Gss; % Now create a special version with full outputs Gss1.c = eye(2); Gss1.d = [0;0]; T = 0.1; Gssz = c2d(Gss,T);

% Convert to discrete state-space at Ts = 0.1

p = [0.92 0.93]'; K = place(Gssz.a, Gssz.b, p);

% Design controller gain, K, such that . . . % λctrl = eig(Φ − ∆K)

8

13

pe = [0.80 0.85]'; % Design estimator gain, L, such that . . . L = place(Gssz.a', Gssz.c', pe)'; % λest = eig(Φ − LC) sort(eig(Gssz.a-L*Gssz.c)) - sort(pe) % Test results should be zero

which gives controller, K, and observer, L, gains as K=



1.3848 1.6683



and

L=



0.2146 0.4109



Fig. 8.8(a) gives the S IMULINK configuration for the controller which closely resembles the block diagram given in Fig. 8.7. To test the estimation and control, we will simulate a trend where the actual and estimated starting conditions are different, namely     1 −1.6 ˆ0 = x0 = , which is not the same as x 1 2

8.5. GENERIC MODEL CONTROL

379

The trends shown in Fig. 8.8(b) show the (discrete) estimated states converging rapidly to the true continuous states, and that both converge, although slightly less rapidly, to the setpoint.

8.5

G ENERIC MODEL CONTROL

Control of multivariable nonlinear processes such as x˙ = f (x, u, d, θ, t)

(8.37)

is in general, much harder than that for linear processes. Previously we discussed the simple state feedback control law; u = −K(x∗ −x) where x∗ is the desired state vector or setpoint. We showed that we could augment this simple proportional only law to incorporate integral action to ensure zero offset to get a control law such as 8.23. Now suppose that we wish to control not x, but the ˙ ⋆ . We now wish to find u(t) derivative of x, dx/dt, to follow some desired derivative trajectory (x) such that x˙ = r(t) where r(t) is some reference function that we can specify and where dx∗ /dt is given by Eqn 8.37. For any controller, when the process is away from the desired state, x∗ , we want to force the rate of change of the states, x˙ to be such that it is moving in the direction of the desired setpoint, and in addition, we would like the closed loop to have zero offset. The control law Z

˙ ∗ = K1 (x∗ − x) + K2 (x)

t

0

(x∗ − x) dτ

(8.38)

˙ ∗ = x, ˙ so substitutcombines these two weighted ideas. Ignoring any model-plant mistmatch, (x) ing in the plant dynamics Eqn. 8.37 to Eqn. 8.38, we get the control law Z

∗

f (x, u, t) − K1 (x − x) − K2

t 0

(x∗ − x) dτ = 0

(8.39)

Solving Eqn. 8.39 for u is the control law and this scheme is called “Generic Model Control” or GMC and was developed by Lee and Sullivan in [116]. Note that solving Eqn 8.39 for u is in effect solving a system of m nonlinear algebraic equations. This must be done at every time step. Actually, the solution procedure is not so difficult since the previous solution ut−1 can be used as the initial estimate for ut .

8.5.1

T HE

TUNING PARAMETERS

The GMC control law (Eqn 8.39) imbeds a model of the process, and two matrix tuning parameters, K1 and K2 . Selecting different tuning parameters changes the type of response. Assuming we have a perfect model, the control law (Eqn 8.39) is x˙ = K1 (x∗ − x) + K2

Z

0

t

(x∗ − x) dτ

(8.40)

If we differentiate once the control law Eqn 8.40, we get ¨ = K1 (x˙ ∗ − x) ˙ + K2 (x∗ − x) x ¨ + K1 x˙ + K2 x = K1 x˙ ∗ + K2 x∗ x

(8.41)

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

380

states

yv

4

To Workspace

4 4

Mux1 2

states

K*u

2

2

2

input output

ctrl Gain

plant BB

output

C model 2

K*u

estimator Gain1

K* u 2

K*u

2

1

2

Delta

2

z

2

Unit Delay K*u

Phi

2

(a) S IMULINK configuration 1 x1 (t) 0.5 x1

0 −0.5 −1 ˆ1 (t) −1.5 x actual estimated

−2 ˆ2 (t) 2 x

x

2

1.5 1 x2 (t) 0.5 0 0

1

2

3 time

4

5

6

7

(b) The estimated states converge to the true states which in turn converge to the setpoint.

Figure 8.8: Simultaneous control and estimation

8.5. GENERIC MODEL CONTROL

381

1.6 increasing ζ

1.4

2

1.2 1.5 response

response

1 0.8 0.6

0.5

ξ=0.2 ξ=0.5 ξ=1 ξ=3

0.4 0.2 0

1

0

2

4 6 Normalised time, τ=1

8

0 0

10 0.5

5

1

10 shape factor ξ

(a) Various choices for shape factor. ξ = 0.2, 0.5, 1 and ξ = 3.

1.5

0

normalised time

(b) A 3-dimensional plot of the same relation.

Figure 8.9: The normalised (τ = 1) response curve for a perfect GMC controller for various choices of shape factor ξ. and taking Laplace transforms of 8.41 gives s2 x + sK1 x + K2 x = sK1 x∗ + K2 x∗ K1 s + K2 x(s) = 2 x∗ (s) s + K1 s + K2 2τ ξs + 1 = 2 2 τ s + 2τ ξs + 1 where

1 τ=√ , k2

k1 ξ= √ 2 k2

(8.42)

(8.43)

or alternatively 2ξ 1 (8.44) , k2 = 2 τ τ Note that Eqn 8.42 is a modified second-order transfer function. The time domain solution to Eqn 8.42 for a step change in setpoint, X ∗ (s), is given by    √ √ p
−ξt 1−ξ 2 t 1−ξ 2 t 2 2 2 1−ξ −e τ + ξ 1 − ξ sin 1 + ξ cos τ τ , ξ 6= 1 (8.45) x(t) = 2 1−ξ k1 =

For the limiting case where ξ = 1, the solution to Eqn. 8.42 simplifies to just   −t t x(t) = 1 + e τ −1 τ

(8.46)

Fig. 8.9 shows the response for a perfectly GMC controlled process for different ξ. Note that as ξ → ∞, the output response approaches a step response at t = 0, and as ξ → 0, the output tends to x(t) = sin(2πt) + 1. This improvement comes, naturally, at the expense of decreased robustness. Fig. 8.10 shows the comparison between the overshoot, decay ratio and the ξ tuning value. These values are easily obtained from Fig. 8.9 or analytically starting with Eqn. 8.45. Note that for ξ > 1, there is no oscillation in the response, and that a ξ ≈ 0.22 gives a quarter decay ratio.

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

382

overshoot decay ratio

overshoot & decay ratio

1

Figure 8.10: The relation between the shape factor ξ, the maximum overshoot (solid) and decay ratio (dashed) for the perfectly controlled GMC response when τ = 1

0.8 0.6 0.4 0.2 0 0

0.2

0.4 0.6 0.8 shape factor ξ

1

1.2

Algorithm 8.3 Designing the GMC controller To choose appropriate values for K1 and K2 , do the following: 1. For each state to be controlled, choose an appropriate shape from Fig. 8.9 for the response. In other words, choose a ξ value. For tight control with little overshoot, choose a shape corresponding to ξ ≈ 3 or higher , or if the loop is not so important, choose a more conservative response such as ξ ≈ 1. 2. Specify the time scale by choosing an appropriate value for τ . Often it is convenient to specify the rise time. For distillation column control, you may choose a cross over time of 30 minutes, or for a flow controller you may wish to cross the setpoint after only 5 seconds. The time scale is application dependent. 3. From the specified ξ and τ , evaluate the scalars k1 and k2 using Eqn. 8.44. 4. The diagonal matrices K1 and K2 are formed by collecting the k scalars into diagonal n × n matrices.

8.5.2

GMC

CONTROL OF A LINEAR MODEL

Since GMC can be applied to a nonlinear process, one would expect it to work especially well given a linear process x˙ = Ax + Bu. Now the GMC control law simplifies from the general Eqn. 8.39 to   Z t (x∗ − x) dτ (8.47) u = B−1 K1 x∗ − (K1 + A)x + K2 0

−1

noting that B must exist and hence at least should be square. This means that we must have the same number of manipulated variables as state variables. It is easy to test such a controller using a stable random model using rmodel. Hopefully our randomly generated control matrix B is inevitable. (If it is not, just re-run the script file.) To tune the response, I will select a slightly different shape factor and time constant for each state, namely ξ1 = 0.7, τ1 = 1.5 ξ2 = 0.8, τ2 = 2.6

8.5. GENERIC MODEL CONTROL

383

Running the script file in Listing 8.5 gives results much like Fig. 8.11.

Listing 8.5: GMC on a Linear Process dt = 0.5; n = 2;% sample time & # of states m = n; p = 1; % ensure square system & 1 output

5

[A,B,C,D]=rmodel(n,p,m); % generate random continuous model D = zeros(size(D)); C = eye(size(C)); % not interesting [Phi,Del] = c2dm(A,B,C,D,dt,'zoh'); % discretise

10

x0 = round(3*randn(n,1)); % nice starting position t = [0:dt:80]'; % simulation lifetime Xspt = [square(t/8), 0.75*square(t/8*0.7)+3]; % setpoint

15

% GMC tuning rules zeta=[0.7 0.8]'; tau=[1.5 2.6]'; % Shape factor ζ, & time constant τ , for each state. K1 = 2*diag(zeta./tau); K2 = diag(1.0./tau./tau); % Diagonal K1 , K2 . Ie = zeros(size(x0)); % reset integrator

20

x= x0; X = []; Uc = []; for i=1:length(t) % start simulation err = (Xspt(i,:)-x); % current error (x∗ − x). Ie = Ie + err*dt; % update integral Rt rhs = K1*err + K2*Ie - A*x; % K1 x∗ − (K1 + A)x + K2 0 (x∗ − x) dτ u = B\rhs; % Solve (linear) control law, Eqn. 8.47. x = Phi*x + Del*u; % Do 1-step integration

25

X = [X; x']; end % for

Uc = [Uc;u']; % collect input for plotting

plot(t,[Xspt, X],t,Uc,'--')

% Refer Fig. 8.11.

We would expect the GMC controller to perform well, but what should really convince us is if the actual response y(t) follows the analytical & desired trajectory, y ⋆ (t) since GMC is really a model reference controller. These two curves are compared in Fig. 8.12 which shows an enlarged portion of Fig. 8.11. The differences between the actual response and the specified trajectory are due to sampling discretisation. This demonstration of the GMC controlling a linear process does not do the algorithm justice, since the controller really is intended for nonlinear processes. This is demonstrated in the following section.

8.5.3

GMC

APPLIED TO A NONLINEAR PLANT

Section 8.5.2 did not show the true power of the GMC algorithm since the plant was linear and any linear controller such as an LMR would work. However this section introduces a nonlinear

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

384

Linear GMC 5 x2

Outputs & setpoints

4 3 2 1 0 −1

x

1

−2 −3 20

inputs

0

Figure 8.11: The response of a randomly generated linear process controlled by GMC. See also Fig. 8.12.

u2

−20 −40 −60 u

1

−80

0

10

20

30

40 time

50

60

70

80

2

Figure 8.12: Comparing the actual response, y, to a setpoint change with the expected response, y ⋆ . This response is a zoomed part of the response given in Fig. 8.11.

output & reference output

1.5 1 0.5 0 −0.5

Actual y *

Reference y setpoint

−1 48

50

52

54 time

56

58

60

62

model of a continuously stirred tank reactor, CSTR, from [123]. The CSTR model equations are Fa Ta Uar Fb Tb (Fa + Fb )T ∆H T˙ = r− (T − Tc ) + − − V V V ρCp ρV Cp Fa Cai (Fa + Fb )Ca C˙a = − −r V V Uar Fc (Tci − Tc ) + (T − Tc ) T˙c = Vc ρVc Cp

(8.48) (8.49) (8.50)

where the reaction kinetics are second order E

r = Ke− RT Ca2

(8.51)

8.5. GENERIC MODEL CONTROL

385

Table 8.2: The state, manipulated and disturbance variables and parameter values for the Litchfield nonlinear CSTR, [123]. (a) State, input and disturbance variables

variable T Ca Tc Tci Cai Fb Ta Fa Tb Fc

description reactor temperature concentration of A temperature of jacket temperature of cooling inlet inlet concentration of A inlet flow of stream B inlet temp. of stream A inlet flow of stream A inlet temp. of stream B cooling flow

units K kgmol m−3 K K kgmol m−3 m3 s−1 50◦ C (333K) 3.335 · 10−6 m3 s−1 50◦ C (333K) 3 · 10−4 m3 s−1

(b) Model parameters

parameter V Vc ρ Cp

value 0.0045 0.0074 1000 4.187

units m3 m3 kg/m3 kJ/kg/K

parameter K E/R ∆H Uar

value 16.6 · 106 5.6452 · 103 −74.106 · 103 0.175

units m3 /kmol/s K kJ/kg kW/K

The state, manipulated and disturbance variables as explained in Table 8.2 are defined as1 def

x=



T

Ca

Tc

T

,

def

u=



Tci

Cai

Fb

The model parameters are also given in Table 8.2.

T

,

def

d=



Ta

Fa

Tb

Fc

T

(8.52)

The plant dynamics defined by Eqns 8.48 to 8.50 are nonlinear owing to the reaction rate r, being an exponential function of the temperature T . This type of nonlinearity is generally considered significant.

D ESIGNING

THE

GMC

CONTROLLER

First we note that have n states and n manipulated variables. Thus the B matrix, or at least a linearised version of the nonlinear control function will be square. We also note that the GMC algorithm assumes that the process states (x) are measured, hence the measurement relation must be C = I. The GMC control law, Eqn. 8.39, for this case is      Fa Uar ∆H (T − Tc ) 0 0 TbV−T Tci V (Ta − T ) − ρCp r − ρC p
  0 Fa − Ca   Cai  +  − FaVCa + r −  V V Fc Fb 0 0 − FVc Tc c + ρVUcarCp (T − Tc ) Vc Z t K1 (x∗ − x) − K2 (x∗ − x) dτ = 0 0

Now you will notice that the control law is a system of algebraic equations unknown in u. For this case, the control law is linear in u because I had the freedom of choosing the manipulated variables carefully. If, for example, I had chosen Fa rather than Fb as one of the manipulated variables, then 1 The original reference had only one manipulated variable T . This was changed for this simulation so the GMC is ci now a true multivariable controller.

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

386

the system would be control nonlinear, and hence more difficult to solve. But in this case, the system is control linear, the GMC control law is a linear system of algebraic equations, and that the 3×3 matrix to be inverted is in a lower diagonal form already making the computation straight forward.

S IMULATING GMC

WITH A NONLINEAR PROCESS

We will use a GMC controller to drive the states to follow a setpoint as follows:   ⊤  0 ≤ t < 5 min.   300 0.03 290  ⊤ ∗ x = 310 0.03 290 5 ≤ t < 10 min.  ⊤   310 0.025 295 t > 10 min.

 ⊤ starting from an initial condition of x0 = 290 0.035 280 and with a relatively coarse sample time of T = 10 seconds. We set the shape factor ξ = 0.5, and the time constant for the response 1 1 to τ = 20 seconds. This gives k1 = 20 and k2 = 400 . The disturbance variables (d) are assumed constant for this simulation and are taken from Table 8.2. The state derivative x˙ as a function of x, u, d, θ, t is evaluated using Eqns 8.48 to 8.50 and is given in listing 8.7 which is called by the integrator ode45 in listing 8.6. Listing 8.6: GMC for a batch reactor 1

6

11

16

21

26

31

% GMC CSTR simulation % Ref: Litchfield, Campbell & Locke Trans I Chem E Vol 57 1979 x0 = [290, 0.035, 280]'; % dt = 10; t = [0:dt:15*60]'; % Xspt = [300+10*square(t/150,70), 295-5*square(t/150,30)]; % d = [333 3.335e-6 333 3e-4]'; % Ta = d(1); Fa = d(2); Tb = d(3);

initial condition sample time (seconds) 0.03+0.05*square(t/150), ... setpoint disturbance vector constant Fc = d(4);

k1 = 1/20; k2=1/400; % GMC tuning constants K1 = eye(3)*k1; K2 = eye(3)*k2; %Parameters for the CSTR V = 0.0045; Vc = 0.0074; rho = 1000.0; K = 16.6e6; ER = 5.6452e3; dH = -74.106e3; Cp = 4.187; %kJ/Kg.K Uar = 0.175; % kW/K rCp = rho*Cp ; x=x0;Ie=zeros(size(x)); X=[]; U=[]; for i=1:length(t) % start simulation xspt = Xspt(i,:)'; % current setpoint T = x(1); Ca = x(2); Tc = x(3); % state variables Ie = Ie+(xspt-x)*dt; % update integral error r = K*exp(-ER/T)*Ca*Ca; % reaction rate R = [Fa/V*(Ta-T)-dH*r/rCp-Uar*(T-Tc)/rCp;-(Fa*Ca/V+r); ... -Fc*Tc/Vc+Uar/rCp/Vc*(T-Tc)]; A = [0 0 (Tb-T)/V; 0 Fa/V -Ca/V; Fc/Vc 0 0]; u = A\(K1*(xspt-x)+K2*Ie - R); % solve for u GMC control law % u = max(zeros(size(u)),u); % ensure feasible [ts,xv] = ode45('fcstr',[0,dt/2,dt],x,[],u, ...

8.5. GENERIC MODEL CONTROL

387

V, Vc, K, ER, dH, rCp, Uar, Ta, Fa, Tb ,Fc); % integrate nl ODEs x = xv(3,:)'; % keep only last state 36

X(i,:) = x'; U(i,:) = u'; end %for plot(t,[X,U]); % will need to rescale

Listing 8.7: The dynamic equations of a batch reactor 1

6

11

function x_dot = fcstr(t,x,flag,u, ... V, Vc, K, ER, dH, rCp, Uar, Ta, Fa, Tb ,Fc); % CSTR dynamic model, called from gmccstr.m T = x(1); Ca = x(2); Tc = x(3); % states Tci = u(1); % Inputs, Coolant temp in Kelvin Cai = u(2); Fb = u(3); r = K*exp(-ER/T)*Caˆ2; % reaction rate kmol/m3/s x_dot(1,1)=(Fa*Ta+Fb*Tb-(Fa+Fb)*T)/V - dH*r/rCp - Uar/V/rCp*(T-Tc); x_dot(2,1)=(Fa*Cai - (Fa+Fb)*Ca)/V - r ; x_dot(3,1)=Fc*(Tci-Tc)/Vc+Uar*(T-Tc)/rCp/Vc; return % fcstr.m

Physically the manipulated variables are constrained to be positive, and unless these constraints are added explicitly to the GMC optimisation control law, negative values for some of the inputs will occur. In the above example I simply clamp the input when these violations occur. If you removed the u=max(zeros(size(u)),u); line from the control law, you will have the unconstrained version that ignores the physically realistic constraints which is the case shown in the simulations given in Fig. 8.13. Fig 8.13 shows the response of the CSTR controlled by the GMC. The upper plots show the state trajectories and the lower plots give the manipulated variables used. Note that the controlled response is good, but not perfect as we can still see some interaction between the states. The reaction concentration Ca particularly seems strongly coupled to the other states, and this coupling is not completely removed by the GMC controller. We could improve on this response both by decreasing the sample time, or tightening the controller design specifications. Just for fun, we can plot a three dimensional plot of the trajectory of the state variables from the CSTR example under GMC control. Fig. 8.14 shows the actual state trajectory (solid line) converging to the 4 distinct setpoints, (∗), in state space. Note that the independent variable time is removed in this illustration.

A LTERNATIVE MANIPULATED VARIABLES CHOICES Manipulating the inlet concentration of A or the inlet temperature of the cooling fluid is not very practical. It is much easier to manipulate flow rates rather than temperatures of concentrations. Therefore in this situation, a better manipulated variable vector would be u = [Fa , Fb , Fc ]⊤ rather than that used above. Note however that if we had chosen Fa , Fc and Tci as u, we would have a control problem that is not only nonlinear, but one that is also degenerate. If you construct the GMC control law of Eqn. 8.39, you will get a structure such as 

⋆ 0  ⋆ 0 0 ⋆

  Z 0 Fa 0   Fc  = g(x) + K1 (x⋆ − x) + K2 (x⋆ − x) ⋆ Tci

(8.53)

388

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

Temp T,T (deg C)

40 T 30

T

C

c

20 10 0.4

A

0.35 Conc C

0.3 0.25 C

A

40 30

Tci

Conc C , Flow F * 5e4

Ci

Temp T (deg C)

0.2

10 30 20 10

C

0

F

ai

Ai

b

Figure 8.13: The nonlinear CSTR controlled by a GMC controller where ξ = 0.5 and τ = 20 seconds and sample time T = 10 seconds. Upper: The state trajectories (temperatures T and Tc , and reaction concentration Ca ). Lower: The manipulated variable trajectories Tci and Cai and 5 · 104 Fb .

20

−10

b

0

5

10

15

time (min)

Temperature Tc

35 30 25 20 15 10 0.5 0.4

Figure 8.14: A phase plot of T, Tc and Ca for the GMC controlled CSTR example. The state trajectory is the solid line, and the setpoints are ∗.

40

0.3

35 30

0.2 Conc CA

25 0.1

20 15

Temperature T

8.6. EXACT FEEDBACK LINEARISATION

389

where we know everything on the right hand side of Eqn. 8.53, (it is a constant for the purposes of this calculation) and wish to solve for u. Unfortunately, however, we can see from the structure matrix on the left hand-side of Eqn. 8.53, that Fa appears alone in two equations, and the remaining two manipulated variables appear together in one equation. Thus this nonlinear algebraic system is degenerate since in general we cannot expect a single unknown Fa to satisfy two independent conditions. Note that even though we do not have an overall degree of freedom problem, (we have three unknowns and three equations), we do have a sub-structure that has a degree of freedom problem. Problem 8.5 1. Re-run this simulation and test the disturbance rejection properties of the GMC controller. You will need to choose some suitable disturbance profile. 2. Re-run the simulation for different choices of GMC tuning constants. For each simulation, verify that the response is the linear response that you actually specified in the GMC design. (You may need to decrease the sampling interval to T ≈ 2 seconds to get good results.) Try for example; (a) τ = 20, ξ = 1.5 (b) τ = 5, ξ = 0.5 (c) any other suitable choice.

8.6

E XACT FEEDBACK LINEARISATION

For certain nonlinear systems, we can derive controllers such that the closed loop input/output behaviour is exactly linear. This is quite different from our traditional approach of approximating the linearities with a Taylor series approximation and then designing a linear controller based on the approximate model. Designing exact feedback linearisation uses nonlinear results from the field of differential geometry, which, to say the least, is somewhat abstract to most engineers. Lie algebra for nonlinear systems will replace matrix algebra for linear systems, and Lie brackets and Lie derivatives will be extensions to normal matrix operations. A good tutorial of this topic with applications of interest to the process industry, and one from where most of this material was taken, is [104] and the follow up, [105]. A review of nonlinear control for chemical processes which includes a section on exact feedback linearisation is [30] and the classic text book for this and other topics in nonlinear control is [189]. Extensions for when the process is not affine in input, and for systems where the relative degree is greater than unity are discussed in [82].

8.6.1

T HE

NONLINEAR SYSTEM

We will only deal with SISO nonlinear systems of the form: x˙ = f (x) + g(x)u

(8.54)

y = h(x)

(8.55)

where f , g are vector functions, or vector fields, h(x) is a scalar field and the input u, and output y are scalars. Note that the system described by Eqn 8.54 is written in affine form, that is the manipulated variable enters linearly into the system. It is always possible to transform an arbitray system into affine form by introducing a new variable. For example, given the non-affine system x˙ = f (x, u)

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

390

we can define a new input, ν, which is the derivative of our original input, u˙ = ν, so now our augmented system becomes       x˙ f (x, u) 0 = + ν (8.56) u˙ 0 1 which is now in affine form. Whether creating a new variable ν which requires differentiating the input is practical, or even feasible under industrial conditions is a different matter.

8.6.2

T HE

INPUT / OUTPUT FEEDBACK LINEARISATION CONTROL LAW

Given the nonlinear system in Eqns 8.54 and 8.55, we will try to construct a control law of the form u = p(x) + q(x) y ⋆ (8.57) where y ⋆ is the desired setpoint. The closed loop system obtained by substituting Eqn 8.57 into Eqns 8.54 and 8.55 is x˙ = f (x) + g(x)p(x) + g(x)q(x) y ⋆ y = h(x)

(8.58) (8.59)

and is linear from reference variable y ⋆ to actual output y. The dashed box in Fig. 8.15 contains the nonlinearity, but as far as we are concerned, viewing the system from the outside, it may as well be a linear black box. Reference input y ⋆ (t)

Πbc

⊕

u(t)

Πbc

⊕

x˙

x(t)

output

1 s

h(·)

y(t)

f (·) q(x)

p(x) g(·)

Nonlinear plant Figure 8.15: The configuration of an input/output feedback linearisation control law Note that for a controller in this configuration, the internal states will still be nonlinear. Notwithstanding, once we have a linear dynamic relation from reference, y ⋆ (t), to outputs, y(t), it is much easier to design output-based controllers to control this now linear system, rather than the original nonlinear system. It is also clear from Fig. 8.15, that the controller requires state information to be implemented. The control design problem is to select a desired linear response, perhaps by specifying the time constants of the desired closed loop, and then to construct the p(x) and q(x) functions in the control law, Eqn. 8.57. These are given by the following relations: −(Lrf h(x) + β1 Lr−1 h(x) + · · · + βr−1 Lf h(x) + βr h(x)) f Lg Lr−1 h(x) f 1 q(x) = Lg Lr−1 h(x) f

p(x) =

(8.60) (8.61)

8.6. EXACT FEEDBACK LINEARISATION

391

where the Lie derivative, Lf h(x), is simply the directional derivative of the scalar function h(x) in the direction of the vector f (x). Here, the Lie derivative is calculated by n

Lf h(x) = f (x)⊤ ·

∂h(x) ∂h(x) X fi (x) · = ∂x ∂xi i=1

(8.62)

One can apply the Lie derivative operator repeatedly, L2f = Lf Lf either to the same function, or to different functions as in Lg L2f . The relative order of a system such as Eqns 8.54–8.55 is the smallest integer r such that Lg Lr−1 h(x) 6= 0 f

(8.63)

An alternative interpretation of relative order is that r is the smallest order of derivatives of y that depends explicitly on u. For linear systems, the relative order is simply the difference between the number of poles and the number of zeros of the system, i.e. na − nb . Finally, the βi in Eqn 8.60 are the user-chosen tuning constants that determine how our output will respond to the command y ⋆ . They are defined by dr−1 y dy dr y + β + · · · + βr−1 + βr y = y ⋆ (8.64) 1 dtr dtr−1 dt and typically it is convenient to choose Eqn. 8.64 as some sort of stable low-pass filter, such as a Butterworth filter. Algorithm 8.4 Exact feedback linearisation design procedure:

1. We are given a single-input/single-output nonlinear model (which may need to be transformed into affine form), Eqn 8.54–8.55. 2. Find the relative order, r, of the system using Eqn. 8.63. This will govern the order of the desired response differential equation chosen in the next step. (If r = 1, perhaps a GMC controller is easier to develop.) 3. We choose a desired linear response, which we consider reasonable, in the form of Eqn 8.64, that is, we need to choose r time constants or equivalent parameters. 4. We create our exact linearising nonlinear controller, Eqn 8.57 using the relations Eqns 8.60 and 8.61 by constructing Lie derivatives. (A symbolic manipulator such as M APLE may be useful here.) 5. Closing the loop creates a linear input/output system, which can be controlled by a further controller sitting around this system, designed by any standard linear controller design technique such as LMR or pole placement.(However this is hardly necessary since we have the freedom to choose any reasonable response in step 3.)

C AUTIONS

AND RESTRICTIONS

This technique is not applicable if the “zero dynamics” are unstable. This is analogous to the linear case where we are not allowed to cancel right-hand plane zeros with unstable poles. Technically, it works, but in practice, the slightest model mis-match will cause an unstable pole which in turn will create an unstable controller. This then excludes process systems with dead time. Clearly also owing to the nature of the cancellation of the process nonlinearities, exact feedback linearisation is very sensitive to modelling errors. The technique demands state feedback, but only delivers output linearisation. Full state linearisation techniques do exist, but exist only under very restrictive conditions.

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

392

8.6.3

E XACT

FEEDBACK EXAMPLE

Building an exact feedback controller is relatively easy for well behaved smooth nonlinear systems. The only calculus operation required is differentiation, and this poses few problems for automated computer tools. This example shows how to build such a controller using a simple program written for a symbolic manipulator. With this program, we can subsequently change the system, and regenerate the new nonlinear control law. This is a departure from all the other simulations presented so far, since here we can change the entire structure of the process in our program, not just the order or the values of the parameters, and subsequently re-run the simulation. Suppose we have a simple 3-state nonlinear SISO system, 

     x˙1 sin(x2 ) + (x2 + 1)x3 0  x˙2  =  x51 + x3 + 0 u x˙3 x21 1 {z } | {z } | g (x ) f (x )   y= 1 0 0 x | {z } h(x)

(8.65)

(8.66)

which is already in the form of Eqn. 8.54–Eqn. 8.55 or affine form, and we wish to construct an exact feedback linearised system following Eqn. 8.57. To do this we will follow Algorithm 8.4. Since we will need to compute some Lie derivatives analytically, Lf h(x) = f (x)⊤ ·

∂h(x) ∂x

to establish the relative order r, it may help to write a small procedure to do this automatically. Listing 8.8 gives an example using the Symbolic toolbox.

Listing 8.8: Find the Lie derivative for a symbolic system 2

function L = lie_d(F,h,x) % Find the Lie derivative Lf h(x) % Note jacobian returns a row vector L = F'*jacobian(h,x)'; % f (x)⊤ · return

∂h(x) ∂x

First we define the actual nonlinear functions f (x), h(x), and g(x) for our particular dynamic problem. syms x1 x2 x3 ystar real % Specify symbolic variables as real or will have conjugate problems x = [x1 x2 x3]';

5

F = [sin(x2)+(x2+1)*x3; x1ˆ5+x3; x1ˆ2]; % Eqn. 8.65 h = [1 0 0]*x g = [0 0 1]'

8.6. EXACT FEEDBACK LINEARISATION

393

The Lie derivatives are easy to construct, Lf h(x) = f (x)⊤ · =



∂h(x) ∂x

sin(x2 ) + (x2 + 1)x3 , x51 + x3 , x21

= sin(x2 ) + (x2 + 1)x3





 1 × 0  0

and check in M ATLAB using the routine giving in Listing 8.8 >> lie_d(F,h,x) ans = sin(x2)+(x2+1)*x3

To find the relative degree, we simply repeatedly call the function in Listing 8.8 starting at r = 1 and incrementing r until Lg Lr−1 h(x) 6= 0 at which point we have established the relative order. f Listing 8.9: Establish relative degree, r (ignore degree 0 possibility) 2

Lfn = h % Dummy start LgLfnh = lie_d(g,h,x) % Lg L1f h(x) case. r = 1; % # relative degree counter

7

while LgLfnh == 0 r =r+1; % increment relative degree counter, r Lfn = lie_d(F,Lfn,x) % need to build-up LgLfnh = lie_d(g,Lfn,x) % stop when non-zero end % while

Now that we know the relative degree r = 2, we can design our desired response filter Listing 8.10: Design Butterworth filter of order r. 1

[z,p,K] = buttap(r) % Low-pass Butterworth filter analogue prototype beta = real(poly(p)); %[B,beta] = butter(r,2,'s') % Alternative: but must scale with B(end).

Now we can construct the controller components p(x) and q(x) to form the closed loop. Listing 8.11: Symbolically create the closed loop expression 2

7

qx = 1/LgLfnh; % Controller components q(x), p(x) , Eqn. 8.60, Eqn. 8.61. px = beta(r+1)*h; Lfn = h; for i=r-1:-1:0 Lfn = lie_deriv(f,Lfn,x); px = px+ beta(i+1)*Lfn; end px = -px*qx; xdot = f+g*(px+qx*ystar) % Closed loop, Eqn. 8.58, hopefully linear.

In practice, we are only practically interested in the control law, Eqn. 8.57,

CHAPTER 8. MULTIVARIABLE CONTROLLER DESIGN

394

y u

y*

Signal Generator

x

3 y 3

multiply

3

Scope

states

3

Nonlinear Plant

y

1./(u(2)+1)

1 Out1

q(x)

p(x)

u

px 3

3

butter yref

Analog Filter Design

(a) S IMULINK configuration. Compare with Fig. 8.15. 1.5 1 actual y

y=x1

0.5 0 −0.5

desired trajectory

−1 −1.5

−10

x 10 2

error between trajectory and actual

ε

1 0 −1

0

10

20

30

40

50 time

60

70

80

90

100

(b) The closed loop response of the controlled nonlinear system compared to the desired linear response

Figure 8.16: This example of exact feedback linearisation shows that there is no practical difference between the output of the nonlinear system and the desired linear filter.

>> (px+qx*ystar) % Control law, u = p(x) + q(x)y ⋆ ans = (-4*x1-2*2ˆ(1/2)*(sin(x2)+(x2+1)*x3)-(x1ˆ5+x3) *(cos(x2)+x3)-x1ˆ2*(x2+1))/(x2+1)+1/(x2+1)*ystar

which could, in principle, now be implemented on the nonlinear plant. A good test for this controller is to see if the closed loop response is indeed indistinguishable from the specified linear filter. As demonstrated in Fig. 8.16, there is no discernable difference indicating that the nonlinear controller design worked as intended in this instance.

Problem 8.6 Create an exact nonlinear feedback controller for a penicillin fermentation process.

8.6. EXACT FEEDBACK LINEARISATION

395

The system dynamics are: dX = µ(S, X)X − DX dt dS = −σ(S, X)X + D(Sf − S) dt y=X

(8.67) (8.68) (8.69)

where X, S are the cell mass and glucose concentration, D is the dilution rate and manipulated variable, and Sf is the feed glucose concentration, considered constant in this example. The empirical growth rate expressions are: µx S Kx X + S σ(S, X) = µ(S, X)/Yxs + π(S)/Yps + m µp S π(S) = Kp S(1 + S/Ki ) µ(S, X) =

(8.70) (8.71) (8.72)

and for simulation purposes, use parameters values as µx µp Yxs Ki

1.5 3.45 0.45 0.89

Kx Kp Yps m

2.34 1.56 0.78 0.34

and manipulated, disturbance and initial conditions as

manipulated disturbance initial conditions

D Sf X0 S0

0> x0 = [0.2, 0.1]'; % Start guess for Kc and τi . >> x = fminunc(@foptslink_pi,x0) % See function in Listing 9.3. 4

9

Optimization terminated successfully: Current search direction is a descent direction, and magnitude of directional derivative in search direction less than 2*options.TolFun x = 66.5402 25.6549

The optimisation routine found that a gain of Kc = 66.5 and an integral time of τi = 25.7 minimised our integral-time squared error performance index.

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

406

9.2.4

AN

OPTIMAL BATCH REACTOR TEMPERATURE POLICY

This example illustrates how to establish an optimal input profile u(t) for a given dynamic system and initial conditions. In this section we are going to approximate the smooth profile with a small number of constant variables and we are going to use parametric optimisation to obtain a slightly sub-optimal solution. Later in section 9.3.4 we will repeat this optimisation problem, but solve for the continuous profile using functional optimisation. Suppose we have a batch reactor as shown in Fig. 9.7 in which two chemical reactions are carried out in series, k1 k2 (9.5) A → B → C The first reaction produces the desired product, B, which is what we wish to maximise after the two hour reaction duration, but the second reaction consumes our valuable product, and produces a poisonous byproduct C, which naturally we wish to minimise.

A

Stop reaction after 2 hours ❄ ✲ product (at end), x(tf ) ✲ condensate (cold)

steam heating, u(t) heating coil

Figure 9.7: Production of a valuable chemical in a batch reactor. In this case, the only manipulated variable is the reaction temperature, u(t), which we can adjust over time by varying the heating input. The concentrations of A and B are the two states: def

x = [ca , cb ]T . As the temperature varies, so does the reaction rates since they are governed by the Arrhenius rate constants (k1 , k2 ), which in turn changes the ratio of desired product to by-product. The plant dynamics are given by the reaction kinetics and initial conditions are dx1 = −2e−6/u x1 , dt dx2 = 2e−6/u x1 − 5e−11/u x2 , dt

x1 (t = 0) = 0.9 x2 (t = 0) = 0.1

Our objective for the operation of this reactor is to maximise the amount of B after 2 hours, by adjusting the temperature over time as the reaction progresses, or J (u) = x2 (tf = 2) +

Z

tf =2

0 dt

(9.6)

0

The reason for the seemingly superfluous greyed out integral in the performance objective above will become evident when we look at the standard optimal control objective function in the next section. If we decide to operate at a constant temperature for the entire 2 hour batch, then Fig. 9.8(a) shows us which temperature is the one to maximise the amount of B at the end of the reaction.

9.2. PARAMETRIC OPTIMISATION

407

Alternatively, if we decide to run the batch reactor for 1 hour at one temperature, and the second hour for another then we would expect the optimum to improve. Fig. 9.8(b) shows the three dimensional response surface for this case.

(a) Optimisation of a single constant temperature

(b) The response surface of the objective function when using 2 temperatures

(c) The response volume of the objective function when using 3 temperatures

Figure 9.8: Visualising the objective function using one, two, and three different temperatures throughout the 2 hour batch reaction. Compare with Fig. 9.9. We could in principle continue this discretisation of the input profile. Fig. 9.8(c) shows the situation when we decide to use three temperatures equally spaced in time (40 mins each) across the 2 hour duration of the batch. In this case the objective function becomes a response volume so we must use volume visualisation techniques to see where the optimum lies. Fig. 9.9 compares the optimum trajectories for the case where we select 1,2,3,5 and 7 equally spaced temperatures divisions. Also given is the limiting case where the number of temperature divisions tend to ∞. The analytical computation for this latter case is given in section 9.3.4.

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

408

7

analytical T* profile

6 T constant

5 4 7 6

2 divisions

5 4 7 6

3 divisions

5 4 7 6

5 divisions

5 4

Figure 9.9: The optimum temperature profiles for cases where we have different numbers of temperature divisions. Note how as the number of allowable temperature divisions increase, the discretised trajectory converges to the smooth analytical optimum profile.

9.3

Temperature

7 6 7 divisions

5 4 0

0.5

1 Time [hrs]

1.5

2

T HE GENERAL OPTIMAL CONTROL PROBLEM

The optimal control problem has the following characteristics: 1. A scalar performance index or cost function, J (u) that is to be minimised (or maximised) by adjusting the manipulated variables u(t). 2. Constraints that must be either: (a) continuously satisfied, that is the constraint must be satisfied for all time, or (b) satisfied only at the end of the optimisation problem, a terminal constraint. These constraints can either be hard (they must never be violated under any conditions, or soft (they can be violated under special conditions).

9.3. THE GENERAL OPTIMAL CONTROL PROBLEM

409

3. The optimisation horizon is the length of time over which the optimisation is allowed to take place. Sometimes the horizon is infinite, but in practical cases the horizon has some upper limit over which an optimal solution is sought to minimise the objective function. The optimal control problems can be written as a standard mathematical optimisation problem, thus many algorithms and therefore computer programmes exist to give a solution. Two common solution procedures are Pontryagin’s maximum principle and dynamic programming. [64, pp16– 20] or [53] covers these aspects in more detail and a simple introduction to Pontryagin’s maximum principle is given in [173, Chpt 13] and [169, pp 84–105]. The optimal control problem as described above is quite general, and the solution will be different for each case. Section 9.3.2 develops the equations to solve the general optimal control problem using variational calculus and Section 9.2.4 demonstrates this general approach for a simple nonlinear batch reactor application. The accompanying M ATLAB simulation clearly shows that the computation required is involved, and motivates the development of less general linear optimal control designs. This results in the Linear Quadratic Regulator (LQR) described in subsequently Section 9.4. There are many other texts that describe the development of optimal controllers. Ogata has already been mentioned, and you should also consult [100, p226] for general linear optimal control, [164, 168, 169] for chemical engineering applications and [40] is considered a classic text in the optimisation field. A short summary available on the web is [47]. A survey really highlighting the paucity of applications of linear quadratic control, (which is a simplified optimal controller for linear systems), in the process industries is [97].

9.3.1

A REVISION

OF MULTIVARIABLE CALCULUS

This section reviews some basic theory of the derivatives of scalar and vector functions with respect to a vector. We used this information when developing the least-squares solution to polynomial fits in section 3.3.1. In the following, we will follow the convention that the derivative of a scalar function V with respect to a vector x,   dV dV dV dV = (9.7) ··· dx dx1 dx2 dxn is a row vector. (I have deliberately chosen the symbol V for this scalar since one can immediately recognise this as a ‘voltage-like’ variable, since voltage is a scalar quantity.) The Jacobian, J, is a matrix of derivatives of an m vector function f with respect to an n vector x, is and (m × n) matrix   df1 df1 df1 ···  dx1 dx2 dxn     df2 df2 df2    · · ·  dx dx2 dxn  1  J=  .   .   .     df dfm dfm  m ··· dx1 dx2 dxn which can also be thought of as a series of individual gradients stacked on top of each other. Using these definitions, we can derive some common derivatives. The derivative of a (constant) matrix times a vector is d(Ax) =A (9.8) dx

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

410 and makes intuitive sense. The derivative of a quadratic form is

  d(xT Ax) = xT AT + A dx = 2xT A if A is symmetric

(9.9) (9.10)

and can be proved by using the chain rule. Note that some authors define the gradient as a column vector hence their corresponding derivatives are transposed from those given above.1

9.3.2

T HE

OPTIMAL CONTROL FORMULATION

To develop our optimal controller, we need to define, as listed in section 9.3, an objective function, a feasible search space, and list our constraints. The Optimal Control Problem, or OCP, is to minimise the scalar performance criteria Z tf L(x, u, t) dt (9.11) J (u(t)) = φ (x(tf )) + 0

over a time horizon from 0 to a final time tf , where our system dynamics follow the general nonlinear dynamic model, x˙ = f (x, u, t), x(t = 0) = x0 (9.12) by choosing an appropriate input trajectory u(t). This performance criteria J (u) to be minimised is quite general and is usually related to process economic considerations and is often called a cost function. Technically Eqn. 9.11 is termed a functional since it maps the function u(t) to the scalar J . The objective function, repeated here, J (u(t)) = φ (x(tf )) + | {z } Mayer problem

|

Z

tf

|0

L(x, u, t) dt {z }

(9.13)

Lagrange problem

{z

Bolza problem

}

is quite general and consists of two terms. The first term (or Mayer problem) is called the termination criteria, and is only concerned about the penalty of the final state. The second integral term is concerned with the cost incurred getting to the final state and is typically some sort of ISE or IAE type term. The scalar integrand, L(x, u, t), is called the Lagrangian. For example if L(x, u, t) = x⊤ Qx where the matrix Q is a positive definite matrix, then this term computes the squared deviations in state variables and is used when we are concerned with the cost of deviations in state variables. Alternatively if L(x, u, t) = 1 1 Note that Ogata, [150, Appendix A] has a very good summary of matrix operations including differentiation, although he defines the gradient as a column vector. Hence he uses the following:

∂Ax = A⊤ ∂x ∂x⊤ Ax = 2Ax ∂x

if A is symmetric.

9.3. THE GENERAL OPTIMAL CONTROL PROBLEM

411

then we are concerned sorely with the total time for the optimisation (which we want to minimise). If only the terminal criterion, φ, is present, we have a Mayer problem, if we are only concerned with the integral term, we have a Lagrange problem, and if we have both terms, then we have what is known as a Bolza problem. We can convert between all three types of problems by introducing extra state variables, see [53, p71] for more hints how to do this.

T HE

AUGMENTED COST FUNCTION

We solve this constrained optimisation problem by converting it to an unconstrained optimisation problem using generalised Lagrange multipliers. These are a vector of introduced variables λ, that are pre-multiplied to each of the dynamic constraint equations and then added to the original objective function. Converting Eqns 9.11 and 9.12 to an unconstrained optimisation problem gives Z tf   ˙ dt L(x, u, t) + λ⊤ (f (x, u) − x) (9.14) J =φ+ 0

where the dimension of the introduced variables, dim(λ) = dim(x) = n. Since we have introduced one λ for each state, these Lagrange multipliers are sometimes known as co-state variables, or adjoint states. It is convenient to simplify the notation by introducing the scalar Hamiltonian def

H = L + λ⊤ f (x, u)

(9.15)

so now our augmented objective function, Eqn. 9.14 is more concisely written as Z tf   H − λT x˙ dt J = φ+

(9.16)

0

˙ in the term λT x˙ in Eqn. 9.16 by integrating by parts. Recall We can eliminate state derivative, x, that the scalar integration by parts is Z Z uv ′ = uv − u′ v so if we set u = λT and v ′ = x˙ so consequently v = x(t), then Z tf tf Z tf λT x˙ = λT (t) x(t) − λ˙T (t)x(t) dt 0 0 0   Z T = λ (tf ) x(tf ) − λT (0) x(0) −

tf

λ˙T (t)x(t) dt

(9.17)

0

Substituting the right-hand expression in Eqn. 9.17 for the λT x˙ term in Eqn. 9.16 gives Z tf  0  ⊤ ⊤ H + λ˙ x dt J = φ + λ x + tf

(9.18)

0

Using variational calculus, we get the Euler-Lagrange equations which define the dynamics of the co-state variables T T   ∂φ (x(tf )) ∂H with final conditions λ(tf ) = (9.19) λ˙ = − ∂x ∂x where we emphasise that the co-state dynamics are specified by terminal boundary conditions at final time t = tf , as opposed to the initial conditions at t = 0 given for the state dynamics in Eqn. 9.12.

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

412

Finally, the optimum control input profile is given by extremetising the Hamiltonian, or equivalently solving for u such that ∂H =0 (9.20) ∂u Solving these two equations, and applying the manipulated variable to our dynamic system (Eqn. 9.12) will result in optimal operation. In many practical cases, the manipulated variable may be constrained between an upper and lower limit. In this case, we would not blindly use the stationary point of the Hamiltonian (Eqn. 9.20) to extract the manipulated variable, but we would simply minimise H, now conscious of the constrained manipulated variable. In summary, the Optimal Control Problem, or OCP, is given in Algorithm 9.1. Algorithm 9.1 The Optimal Control Problem The solution procedure of the optimum control problem (OCP) is as follows: 1. Given a dynamic model with initial conditions x˙ = f (x, u, t),

x(t = 0) = x0

(9.21)

and a scalar performance index which we wish to extremetise over a finite time from t = 0 to final time tf , Z tf L(x, u, t) dt (9.22) J (u) = φ (x(tf )) + 0

2. then we can form the Hamiltonian def

H(x, u, λ) = L + λT f

(9.23)

where the n introduced co-state dynamics are given by λ˙ = −



∂H ∂x

T

with final conditions λ(tf ) =



∂φ(x(tf )) ∂x

T

(9.24)

3. and the optimum input, u⋆ (t), is given by extremetising H, or by solving ∂H =0 ∂u

(9.25)

for u(t). Section 9.3.4 solves the optimal control problem for some simple examples. including the optimal temperature profile of a batch reactor from section 9.2.4. For the batch reactor we have only a termination criteria in our objective function, and no state or manipulated variable energy cost. This makes the calculation somewhat easier.

9.3.3

T HE

TWO - POINT BOUNDARY PROBLEM

Equations 9.12, 9.19–9.20 need to be solved in order to find the desired u(t), the optimal control policy at every sampling instance. This, however, is not a trivial task. Normally we would expect to know the initial conditions for the states so in principle we could integrate Eqn. 9.12 from the

9.3. THE GENERAL OPTIMAL CONTROL PROBLEM

413

known x(0) for one sample interval, excepting that we do not know the correct control variable (u) to use. However, we can solve for the instantaneous optimum u by solving Eqn. 9.20. The only problem now is that we do not know the correct co-state (λ) variables to use in Eqn. 9.20. So far the only things we know about the co-state variables are the dynamics (Eqn 9.19), and the final conditions, but unfortunately not the initial conditions. These types of ODE problems are known as Two Point Boundary Value problems. If we have n state dynamic equations, then the number of state and co-state dynamic equations is 2n. We know n initial state conditions, but that still leaves us n initial conditions short for the costates. (To solve the system of ODEs, we must know as many initial conditions as DEs). However we do know the final values of the costates, and this information supplies the missing n conditions. Methods to solve two point boundary value problems are described in [161, Chp16] and [198, p176] and are in general much harder to solve than initial value ODE problems, and often require an iterative approach. The most intuitive way to solve two point boundary value ODEs is by the shooting method or as it is sometimes termed, the boundary condition iteration, or BCI [168, p237]. This technique is analogous to the artillery gunner who knows both his own and the enemy’s position, but does not know the correct angle to point the gun so that the shell lands on the enemy. Typically an iterative approach is used where the gun is pointed at different angles with an observer who watches where the shells land, and then communicates with the gunner what corrections to make. However, the iteration loop outside the integration is not the only complication. In many physical cases, numerically integrating the state equations backwards or the co-state equations forwards is unstable, where integrating in the opposite direction causes no problems. An alternative method known as Control Vector Iteration (CVI) avoids some of the failings of the BCI method. A comparison of these two techniques is given in [95]. Perhaps the most robust scheme is to use collocation which is the strategy used by the M ATLAB routine bvp4c designed for boundary value problems.

9.3.4

O PTIMAL

CONTROL EXAMPLES

The following examples illustrate two optimal control problems. In both cases we follow Algorithm 9.1 and use the S YMBOLIC toolbox to assist in developing the necessary equations.

R AYLEIGH ’ S

OPTIMAL CONTROL PROBLEM

We wish to minimise J (u) =

Z

2.5

0


x21 + u2 dt

(9.26)

subject to the state dynamics and state initial conditions

x1 (t = 0) = −5

x˙ 1 = x2 , x˙ 2 = −x1 + 2 −

0.1x22




x2 (t = 0) = −5

x2 + 4u,

Above we can see that L = x21 + u2 and φ = 0. For this problem it is convenient to use the Symbolic toolbox to develop the necessary equations. First we define the state dynamics, the objective function, and form the Hamiltonian. 1

>> syms u lam1 lam2 x1 x2 real

% define symbolic variables def

def

>> x = [x1 x2]'; lambda = [lam1 lam2]'; % x = [x1 , x2 ]T , λ = [λ1 , λ2 ]T

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

414

6

>> f = [x(2); ... -x(1)+(2-0.1*x(2)ˆ2)*x(2) + 4*u];% State dynamics x˙ = f (x, u) R >> phi = 0; L = x(1)ˆ2 + uˆ2; % Terms in the objective function, φ + L dt >> H = L + lambda'*f % Hamiltonian, H = L + λT f H = lam1*x2 + uˆ2 + x1ˆ2 - lam2*(x1 - 4*u + x2*(x2ˆ2/10 - 2))

The above symbolic manipulation shows us that the Hamiltonian is H = L + λT f =

x21

  2  x2 + u + λ1 x2 − λ2 x1 − 4u + x2 −2 10 2

Given the Hamiltonian, we can construct the co-state dynamics, and the terminal conditions from Eqn. 9.24. 1

6

>> lam_dot = -jacobian(H,x)' % Eqn. 9.24. Note: Jacobian is a row vector lam_dot = lam2 - 2*x1 lam2*((3*x2ˆ2)/10 - 2) - lam1 >> lam_final = jacobian(phi*x(1),x)' lam_final = 0 0

Or from λ˙ = − (∂H/∂x)T and λ(tf ) = (∂φ/∂x)T we get 

λ˙1 λ˙2



=

"

 λ22 − 2x1 3x λ1 102 − 2 − λ1

#

,



λ1 (tf ) λ2 (tf )



=



0 0



The optimal control input is given by solving Eqn. 9.25 for u(t), or

2

>> uopt = solve(diff(H,u),u) % Eqn. 9.25 uopt = (-2)*lam2

showing us that u⋆ (t) = −2λ2 (t) Now all that is left to do is to form and solve our two-point boundary value problem. We can substitute the expression for the optimum u(t) found above in the state dynamics, and append the co-state dynamics

2

7

>> x_dot = subs(f,u,uopt); >> [x_dot; lam_dot] % State & co-state dynamics ans = x2 - 8*lam2 - x1 - x2*(x2ˆ2/10 - 2) lam2 - 2*x1 lam2*((3*x2ˆ2)/10 - 2) - lam1

9.3. THE GENERAL OPTIMAL CONTROL PROBLEM

415

So in summary, to solve the optimal control problem, we need to solve the following boundary value ODE problem,         x2
x˙1 −5 x1 (0) 2    x2 (0)   −5   x˙2   −x1 + 2 − 0.1x2 x2 − 8λ2        , λ − 2x  λ1 (tf )  =  0   λ˙1  =  2 1     2 3x 0 λ2 (tf ) λ1 102 − 2 − λ1 λ˙2

using the two-point boundary value ODE routine bvp4c. We need additionally need to specify a suitable grid for the collocation points to give an initial estimate trajectory for the state and costate profiles over the integration interval. In this case I will simply supply constant trajectories.

3

8

13

zdot = @(t,z) ... [z(2); ... -8*z(4) - z(1) - z(2)*(z(2)ˆ2/10 - 2); ... z(4) - 2*z(1) ;... z(4)*(3/10*z(2)ˆ2-2) - z(3)]; % State & co-state dynamics % Set up boundary conditions x0 = [-5;-5]; lam_final = [0;0]; % Initial state conditions & terminal co-state conditions BCres = @(a,b) [a(1:2)- x0; ... b(3:4)- double(lam_final)]; % Boundary residuals solinit = bvpinit(linspace(0,2.5,10),[-1 -1 -0.2 -0.5]) sol = bvp4c(zdot,BCres,solinit); % Now solve TPBVP

The optimum profile is given in Fig. 9.10 where we can quickly check that the boundary conditions are satisfied. We can also visually check that the optimum input u⋆ (t) = −2λ2 (t) as required. The circles plotted show the position of the collocation points and we should check that they capture the general trend of the profile.

A N OPTIMAL

BATCH REACTOR POLICY REVISITED

In the optimum profile determination for the batch reactor example in section 9.2.4, we approximated the continuous temperature profile with a finite series of constant zeroth-order holds. In this example, we will compute the exact continuous curve. Recall that the plant dynamics are given by the reaction kinetics and initial conditions are dx1 = −2e−6/u x1 , starting from x1 (t = 0) = 0.9 dt dx2 = 2e−6/u x1 − 5e−11/u x2 , starting from x2 (t = 0) = 0.1 dt

(9.27)

and the objective function to be maximised after a batch reaction of total time tf = 2 hours is Z t=2 0 dt J = x2 (t = 2) + 0

which indicates that the Lagrangian L(x, u, t) = 0 and φ(x(tf )) = x2 (tf ). The Hamiltonian function, H = L + λT f , in this instance is     −2e−6/u x1 H = 0 + λ1 λ2 2e−6/u x1 − 5e−11/u x2   = −2λ1 e−6/u x1 + λ2 2e−6/u x1 − 5e−11/u x2

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

416 6

x (t) 2

4

x(t)

2 0 −2 x (t)

−4

1

−6 10 u(t)

5 0 −5 5

Figure 9.10: The optimum control policy for the Rayleigh problem. The upper two trends are the states, x(t), the middle is the optimal control policy u⋆ (t), and the lower two trends are the corresponding co-states, λ(t).

λ(t)

0 λ (0) 2

−5

λ1

λ (0)

λ

2

1

−10

0

0.5

1

1.5

2

2.5

time

The costates vary with time following λ˙ = −(∂H/∂x)T and finish with dλ1 = 2e−6/u (λ1 − λ2 ), ending at λ1 (t = 2) = 0 dt dλ2 = 5e−11/u λ2 , ending at λ2 (t = 2) = 1 dt

(9.28)

since φ = x2 . Now the optimum temperature trajectory is the one that maximises H or the solution to dH/du = 0, which in this case can solved analytically to give 5

u⋆ (t) = ln



55λ2 x2 12x1 (λ2 −λ1 )

(9.29)



We can replicate this development of constructing of the costate dynamics and optimal input with the help of a symbolic manipulator as shown in Listing 9.4 below. Listing 9.4: Analytically computing the co-state dynamics and optimum input trajectory as a function of states and co-states >> syms u lam1 lam2 x1 x2 real 2

7

% define symbolic variables def

def

>> x = [x1 x2]'; lambda = [lam1 lam2]'; % x = [x1 , x2 ]T , λ = [λ1 , λ2 ]T >> f = [-2*exp(-6/u)*x(1); ... 2*exp(-6/u)*x(1) - 5*exp(-11/u)*x(2)]; >> phi = x(2); L = 0; >> H = L + lambda'*f; % H = L + λT f >> lam_dot = -jacobian(H,x)'; % −∂H/∂x

9.3. THE GENERAL OPTIMAL CONTROL PROBLEM

12

17

417

lam_dot = (2*lam1)/exp(6/u) - (2*lam2)/exp(6/u) (5*lam2)/exp(11/u) >> lam_final = jacobian(phi,x)'% λtfinal = ∂φ/∂x lam_final = 0 1 >> uopt = solve(diff(H,u),u) uopt = 5/log(-(55*lam2*x2)/(12*lam1*x1 - 12*lam2*x1))

Once again we can solve the two-point boundary value problem succinctly using bvp4c as demonstrated in Listing 9.5. We need to supply the state and co-state dynamic equations, Eqn. 9.27 and Eqn. 9.28, the four required boundary conditions, the initial x1 (0), x2 (0) and the final λ1 (2), λ2 (2), and the optimum input trajectory as a function of states and co-states. Listing 9.5: Solving the reaction profile boundary value problem using the boundary value problem solver, bvp4c.m. u = @(x) 5/log(55*x(4)*x(2)/12/x(1)/(x(4)-x(3))) % Optimum input trajectory, Eqn. 9.29 2

7

12

% State and co-state dynamics, Eqn. 9.27 & Eqn. 9.28. zdot = @(t,x) ... [-2*exp(-6/u(x))*x(1); 2*exp(-6/u(x))*x(1) - 5*exp(-11/u(x))*x(2); 2*exp(-6/u(x))*(x(3)-x(4)); 5*exp(-11/u(x))*x(4) ]; % Boundary residuals x1 (0) = 0.9, x2 (0) = 0.1 and λ1 (2) = 0, λ2 (2) = 1 BCres = @(a,b) [a(1:2)-[0.9; 0.1]; ... b(3:4)-[0;1]]; tf = 2; % Final time tfinal = 2 solinit = bvpinit(linspace(0,tf,10),[0.6,0.2,0.4 0.6]) sol = bvp4c(zdot,BCres,solinit); % Now solve TPBVP

17

xint = linspace(0,tf,1e3)'; Sxint = deval(sol,xint); % interpolate solution plot(sol.x, sol.y,'ro',xint,[Sxint(1,:)',Sxint(2,:)'],'-') % See Fig. 9.11

The optimum input temperature policy is shown in Fig. 9.11 with the resultant state and costate trajectories. The amount of desired product B that is formed after 2 hours using this policy is 0.438 and we can be confident that no other temperature policy will improve on that result. In both the Rayleigh and the batch reactor optimal control examples we could extract analytical expressions for the optimal control policy as an explicit function of states and co-states, u⋆ (t) = g(x, λ, t) which made the subsequent boundary-value problem easier to solve. However in many practical cases, the solution of Eqn. 9.25 to give u⋆ (t) may not be possible to find in closed form. In these cases, we may need to embed an algebraic equation solver such as fsolve, or an optimiser such as fminunc.m inside the BVP solution routine. However the problem to be solved now is a differential algebraic equation or DAE which introduces considerable computational problems.

9.3.5

P ROBLEMS

WITH A SPECIFIED TARGET SET

In some optimal control problems we wish not only to establish the optimum profile for u(t) starting from a given initial condition, x(0), but we might also require some (or all) of our states

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

418

Optimal batch reactor operation 1

Conc; Ca, Cb

0.8 C

a

0.6

Cb(2)=0.438

0.4 Cb

0.2

Temp

0 8 6 4

Figure 9.11: The concentrations, the optimum temperature trajectory, and co-states for the batch reactor as evaluated using an optimum control policy.

Co−states, λ

2 1.5 λ2

1 0.5

λ1

0 −0.5

0

0.5

1 time (hours)

1.5

2

to satisfy some arbitrary condition at the final time, ψ (xf , tf ) = 0

(9.30)

These problems are known as OCP problems with a target or manifold set. Further details and solution strategies for these sorts of problems are given in [47, 164]. The main difference here is that we need to introduce a new vector of Lagrange multipliers, ν, the same dimension as the number of target state constraints in Eqn. 9.30, and these get added to the modified objective functional. A simplified, but very common situation is where we wish some of our states to hit a specified target, and perhaps leave the remainder of our states free. In this case the boundary conditions simplify to: 1. if xi (tf ) is fixed, then we use that as a boundary constraint, i.e. xi (tf ) = xif , 2. if xi (tf ) is free, then we use λi (tf ) = ∂φ/∂xi (tf ) as usual for the boundary constraint. Note that together, these two conditions supply the additional n boundary conditions we need to solve the two-point boundary value problem of dimension 2n.

9.3. THE GENERAL OPTIMAL CONTROL PROBLEM

419

Suppose we wish to find the optimum profile u(t) over the interval t ∈ [0, 1] to minimise Z 1 J = u2 dt 0

subject to the dynamics

x˙ = x2 sin(x) + u,

x(0) = 0

but where we also require the state x to hit a ‘target’ at the final time x(1) = 0.5 In this case we have one state variable and one associated co-state variable, so we need two boundary conditions. One is the initial state condition, and the other is the specified state target. This means that for this simple problem we do not need to specify any boundary conditions for the costate. The rest of the OCP problem is much the same as that given in Algorithm 9.1. 1

6

11

syms x u lam

real

% define symbolic variables, u(t), x(t), λ(t)

tf = 1; xtarget = 0.5;

% final time % target constraint

f = [xˆ2.*sin(x) + u]; phi = 0; L = uˆ2;

% State dynamics x˙ = f (x, u) % Terms in the objective function, φ, L

H = L + lam'*f

% Hamiltonian, H = L + λf

lam_dot = -jacobian(H,x)'; % ensure column vector uopt = solve(diff(H,u),u)

Running the above code gives the optimum input as u⋆ = −λ/2 and the the two-point boundary problem to solve as     x˙ x2 sin(x) − λ2
(9.31) = −λ x2 cos(x) + 2x sin(x) λ˙ in two variables. The two boundary conditions are x(0) = 0,

3

8

and

x(1) = 0.5

% Define state & co-state dynamics, Eqn. 9.31. ztmp = @(t,z1,z2) ... [z1ˆ2*sin(z1) - z2/2; -z2*(z1ˆ2*cos(z1) + 2*z1*sin(z1))]; zdot = @(t,z) ztmp(t,z(1),z(2)); x0 = [0]; BCres = @(a,b) [a(1)- x0; ... b(1)- xtarget]; % Boundary residuals: Initial & terminal state conditions xlam_init = @(t) [0.5*t; -1];

13

18

solinit = bvpinit(linspace(0,tf,10),xlam_init) sol = bvp4c(zdot,BCres,solinit); % Now solve TPBVP t = sol.x; z = sol.y'; % Extract solution x = z(:,1); lam = z(:,2); u = -lam(:,1)/2; % Optimum input: u⋆ = −λ/2

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

420

Fig. 9.12 shows the result of this optimal control problem. We should note that the state, x, did indeed manage to hit the specified target at t = 1. Finally we can establish the optimum profile for u(t). 0.8 0.6 States

target

0.4 0.2 0 0.5

u

*

0.45 0.4 0.35 −0.6 co−states, λ

−0.7 −0.8 −0.9 −1

Figure 9.12: Optimal control with a target where x(1) must equal 0.5.

−1.1

0

0.2

0.4

0.6

0.8

1

time

Problem 9.1 (These problems were taken in part from [53, p76].) 1. Show that to determine the optimal input trajectory u(t) given the one dimensional system, x˙ = x + u,

x(0) = 1

such that

Z
1 1 2 x + u2 dt J (u) = 2 0 is minimised reduces to solving the two point boundary value problem,       x˙ 1 −1 x x(0) = 1 = , −1 −1 λ λ(1) = 0 λ˙

Can you find an analytical solution for this problem? 2. Construct the two point boundary problem as above, (but do not solve it) for the two state, one input system       0 1 0 1 x˙ = x+ u, x(0) = 0 0 1 0
R 1 with the cost functional J (u) = 12 0 x⊤ x + u2 dt

9.4. LINEAR QUADRATIC CONTROL

9.4

421

L INEAR QUADRATIC CONTROL

The general optimal controller from section 9.3 has some nice properties, not the least that the trajectory is optimal. However there are some obvious disadvantages, some of which are substantial. Quite apart from the sheer complexity of the problem formulation and the subsequent numerical solution of the two-point boundary value problem, there is the fact that the final result is a manipulated variable trajectory which is essentially an open loop controller. This is fine provided we always start from the same conditions, and follow down the prescribed path, but what happens if due to some unexpected disturbance we fall off this prescribed path? Do we still follow the pre-computed trajectory for u(t), or must we re-compute a new optimal profile for u(t)? Ideally we would like our optimal controller to have some element of feedback, which after all is a characteristic of all our other practical controllers. Furthermore if we restrict our attention to just linear plant models, and use just a quadratic performance objective, then the subsequent problem setup, and implementation, is made considerably easier. These types of optimal controllers designed to control linear plants with quadratic performance are known as Linear Quadratic Regulators, or LQR.

9.4.1

C ONTINUOUS

LINEAR QUADRATIC REGULATORS

The linear quadratic control problem is to choose a manipulated variable trajectory u(t), that minimises a combination of termination error and the integral of the squared state and manipulated variable error over a time horizon from t = 0 to some specified (although possibly infinite) final time t = T , Z
1 T T 1 x Qx + uT Ru dt (9.32) J = x(T )Sx(T ) + 2 2 0 given the constraint of linear plant dynamics

x˙ = Ax + Bu,

x(t = 0) = x0

(9.33)

As we assume linear plant dynamics, and the performance objective, Eqn. 9.32, is of a quadratic form, this type of controller is known as a continuous Linear Quadratic Regulator or LQR. The weighting matrices Q and S are positive semi-definite (and typically diagonal) matrices, while R must be a strictly positive definite matrix. Note that any real positive diagonal matrix is also positive definite. The expressions xT Qx and uT Ru are called quadratic forms and 2 2 are sometimes compactly written as ||x||Q and ||u||R respectively. Using the terminology introduced in the general optimal control problem given in section 9.3.2 Eqn. 9.11 we have the following in the specific case of an LQR f = Ax + Bu, x(t = 0) = x0 1 ⊤ 1 ⊤ L(x, u, t) = x Q x + u R u 2 2 1 ⊤ φ = x (T ) S x(T ) 2

(9.34) (9.35) (9.36)

for the state dynamics constraint, the Lagrangian or ‘getting there’ cost, and the final cost term respectively. For many applications, there is no explicit termination cost, since all the cost of state deviations is accounted for in the energy term, thus often simply the matrix S = 0. In addition, the upper

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

422

limit of the time horizon is often set to T = ∞, in which case we are obviously now no longer concerned with the termination cost term. The Hamiltonian in the LQR case (following the definition from Eqn. 9.15) is H = L + λT f =

1 1 T x Qx + uT Ru + λT (Ax + Bu) 2 2

(9.37)

Now applying the Euler-Lagrange equations (Eqns 9.19 and 9.20) using the differentiation rules from section 9.3.1 gives dλ =− dt



∂H ∂x

T

 T   ∂ 12 x⊤ Qx + λ⊤ Ax  = − ∂x

 T = − x T Q + λT A

= −Qx − A⊤ λ

(9.38)

since Q is symmetric and we have made use of the matrix identity that (AB)T = BT AT . Similarly using Eqn. 9.20 gives ∂H = 0 = uT R + λ T B ∂u

(9.39)

or RT u = −BT λ (again using the above matrix identity) which implies that the optimal control law is u = −R−1 B⊤ λ (9.40) Eqn. 9.40 defines the optimal u as a function of the co-states, λ. Note that now our problem has 2n unknown “states”: the original n x states, and the n introduced co-states, λ. We can write this in a compact form using Eqns 9.40, 9.34 and 9.38 #  "   A −BR−1 B⊤ x˙ x = (9.41) λ λ˙ −Q −A⊤ with mixed boundary conditions x(0) = x0 ,

λ(T ) = Sx(T )

and

From Lyapunov’s theorem, we know that the costates are related to the states via λ(t) = P(t)x(t)

(9.42)

where P(t) is a time varying matrix. Differentiating Eqn. 9.42 using the product rule gives ˙ + Px˙ λ˙ = Px

(9.43)

Now we can substitute the control law based on the costates (Eqn 9.40) into the original linear dynamic model (Eqn 9.34) and further substitute the relation for the states from the costates (Eqn. 9.42) giving x˙ = Ax + Bu   = Ax + B −R−1 B⊤ λ −1

= Ax − BR

⊤

B Px

(9.44) (9.45) (9.46)

Now substituting Eqn. 9.46 into Eqn. 9.43 and equating with the Euler-Lagrange equation Eqn. 9.38 gives   ˙ + P Ax − BR−1 B⊤ Px λ˙ = −Qx − A⊤ λ = Px (9.47)

9.4. LINEAR QUADRATIC CONTROL

423

Rearranging, and cancelling x from each term gives the matrix differential equation ˙ = −PA − A⊤ P + PBR−1 B⊤ P − Q P

(9.48)

which is called the differential matrix Riccati equation. The boundary requirement or termination value, from Eqn. 9.19 is P(t = T ) = S (9.49) and which can be solved backwards in time using the original dynamic model matrices A, B, and the controller design matrices Q, R. While we need to know the model and design matrices for this solution, we need not know the initial condition for the states. This is fortunate, since it means that we can solve the matrix Riccati equation offline once, store the series of P(t), and apply this solution to our problem irrespective of initial condition. We only need to repeat this computation if the plant model or design criteria change. In summary, we now have developed an optimal state feedback controller of the proportional form u = −K(t)x (9.50) where the time varying matrix feedback gain is (from Eqn 9.40) defined as K(t) = R−1 B⊤ P(t)

(9.51)

where the time-varying P matrix is given by Eqn. 9.48 ending with Eqn. 9.49. Since we have assumed that the setpoint is at the origin in Eqn. 9.50, we have a regulatory problem, rather than a servo or tracking problem. Servo controllers are discussed in §8.2.3.

9.4.2

A NALYTICAL

SOLUTION TO THE

LQR

PROBLEM

The problem with implementing the full time-varying optimal controller is that we must solve the nonlinear differential matrix Riccati equation, Eqn. 9.48. This is a complicated computational task and furthermore we must store offline the resultant gain matrices at various times throughout the interval of interest. However there is an analytical solution to this problem which is described in [62, §2.4] and [164, §5.2]. The downside is that it requires us to compute matrix exponentials, and if we want to know P(t) as opposed to just the costates, we must do this computation every sample time. We start by solving for both states and costates in Eqn. 9.41, repeated here      x˙ x(0) = known x A −BR−1 B⊤ , = λ(T ) = known λ λ˙ −Q −A⊤ {z } | H

(9.52)

where the matrix H is known as the Hamiltonian matrix. This approach has the attraction that it is a linear, homogeneous differential equation with an analytical solution. Starting from any known point at t = 0, the states and co-states at any future time are given by     x(0) x(t) H t (9.53) =e λ(0) λ(t) However the outstanding problem is that we do not know λ(0) so we cannot get started! def

If we define eHt = Φ(t), then we can re-write Eqn. 9.53 as     x(t) x(0) = Φ(t) λ(t) λ(0)

(9.54)

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

424

and remember that with M ATLAB it is straight forward to compute Φ using matrix exponentials since it is a function of known, constant matrices. We then partition the matrix Φ into four equal sized n × n blocks as   Φ11 (t) Φ12 (t) Φ(t) = Φ21 (t) Φ22 (t) which means we can express Eqn. 9.54 as x(t) = Φ11 (t)x(0) + Φ12 (t)λ(0)

(9.55)

λ(t) = Φ21 (t)x(0) + Φ22 (t)λ(0)

(9.56)

Employing the Riccati transformation, λ(t) = P(t)x(t), changes Eqn. 9.56 to P(t)x(t) = Φ21 (t)x(0) + Φ22 (t)λ(0) and multiplying Eqn. 9.55 by P(t) and equating to the above gives P(t)Φ11 (t)x(0) + P(t)Φ12 (t)λ(0) = Φ21 (t)x(0) + Φ22 (t)λ(0) Solving for the mystery λ(0) gives λ(0) = [Φ22 (t) − P(t)Φ12 (t)]−1 [P(t)Φ11 (t) − Φ21 (t)] x(0) Now at the final point, t = T , we know that P(T ) = S from the transversality conditions, so λ(0) is given by −1 λ(0) = [Φ22 (T ) − SΦ12 (T )] [SΦ11 (T ) − Φ21 (T )] x(0) (9.57) which is something that we can compute since all the terms are known. Now having established λ(0), we can use Eqn. 9.53 to compute analytically both states, x(t), and costates, λ(t), at any time t. Consequently, we can compute P(t) at any intermediate time between 0 and T using P(t) = [Φ22 (T − t) − SΦ12 (T − t)]

−1

[SΦ11 (T − t) − Φ21 (T − t)]

(9.58)

In many applications, it suffices to ignore the costates and just consider P(0) which is given directly by −1 P(0) = [Φ22 (T ) − SΦ12 (T )] [SΦ11 (T ) − Φ21 (T )] (9.59) We will investigate further this important simplification in section 9.4.3.

A N LQR

TIME - VARYING EXAMPLE

Suppose we wish to control the continuous plant     −2 −4 4 x˙ = x+ u, 3 −1 6

x(0) =



−2 1



using an optimal controller of the form u = −K(t)x to drive the process from some specified initial condition to zero. For this application, we decide that the manipulated variable deviations are more costly than the state deviations,     1 0 0 0 Q= , R = 2, S= 0 1 0 0 Listing 9.6 first defines the plant, then solves for the initial costates, λ(0), using Eqn. 9.57, and then finally simulates the augmented state and costate system given by Eqn. 9.52.

9.4. LINEAR QUADRATIC CONTROL

425

Listing 9.6: Computes the full time-evolving LQR solution 2

7

A = [-2 -4; 3 -1]; B=[4,6]'; x0 = [-2,1]'; % Plant: x˙ = Ax+Bu & initial state, x(t = 0) n=length(A); % Problem dimension Q = eye(2); R=2; S=zeros(size(A)); % Design criteria: Q, R and S T = 0.5; % final time, T H = [A, -B/R*B'; -Q, -A']; Phi = expm(H*T);

% Create Hamiltonian using Eqn. 9.52 def % Φ(t) = eHt

Phi11 = Phi(1:n,1:n); Phi12 = Phi(1:n,n+1:end); % Construct Φ11 (t), Φ12 (t), . . . Phi21 = Phi(n+1:end,1:n); Phi22 = Phi(n+1:end,n+1:end); Lam_0 = (Phi22 - S*Phi12)\(S*Phi11 - Phi21)*x0; % costate initial condition, Eqn. 9.57 12

z0 = [x0;Lam_0]; % augment states & costates G = ss(H,zeros(2*n,1),eye(2*n),0); initial(G,z0,T) % simulate it

The results of this simulation are given in Fig. 9.13 which also compares the near identical results from a slightly sub-optimal controller which uses simply the steady state solution to the Riccati equation discussed next in section 9.4.3. Note that the performance drop when using the considerably simpler steady-state controller is small, and is only noticeable when the total time horizon is relatively short. An alternative solution procedure to the analytical solution given in Listing 9.6 is to numerically integrate the continuous time Riccati equation, Eqn. 9.48, backwards using say the workhorse ode45 with ‘initial’ condition Eqn. 9.49. The trick that the function in Listing 9.7 does is to collapse the matrix Eqn. 9.48 to a vector form that the ODE function can then integrate. The negative sign on the penultimate line is because we are in fact integrating backwards. Listing 9.7: The continuous time differential Riccati equation. This routine is called from Listing 9.8. function dPdt = mRiccati(t, P, A, B, Q) % Solve the continuous matrix differential Riccati equation % Solves dP/dt = −AT P − PA + PBBT P − Q 5

P = reshape(P, size(A)); % Convert from (n2 × 1) vector to n × n matrix dPdt = -A.'*P - P*A + P*B*B.'*P - Q; % Eqn. 9.48 dPdt = -dPdt(:); % Convert back from n × n to (n2 × 1) return

We call the function containing the matrix differential equation in Listing 9.7 via the ode45 integrator as shown in Listing 9.8. Note that we can express BR−1 B efficiently as ZZT where Z = B(chol(R)−1 ). Listing 9.8: Solves the continuous time differential Riccati equation using a numerical ODE integrator. 2

7

A = [-2 -4; 3 -1]; B=[4,6]'; % Plant dynamics of interest Q = eye(2); R=2; S=zeros(size(A)); % Weighting matrices, Q, R T = 1; % Final time for the finite time horizon P0 = S; % “Initial” condition for P [t,P] = ode45(@(t,P) mRiccati(t,P,A,B/(chol(R)),Q),[0 T],P0(:)); % Call listing 9.7 P = flipud(P); % reverse time order of P(t)

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

426

Long horizon: Tfinal= 2

Short horizon: Tfinal=0.5

2

2 x

states

2

0

0 x

−2

−2

1

−4

−4

0.5

0.5 u (t)

0

u

∞

u(t) u∞(t)

0

−0.5

−0.5

1

1

u(t)

K

K

2

0.5

0.5

K

1

0

0

0.5

0.5 P22 P

P

11

0

P12

−0.5

0 −0.5

0.5

0.5

λ

λ

2

0 −0.5

0 λ1

0

0.5

1 time [s]

1.5

2

−0.5

0

0.1

0.2 0.3 time [s]

0.4

0.5

Figure 9.13: Comparing the full time-varying LQR control with a steady-state sub-optimal controller for a long horizon (left) and a short horizon (right). The upper two plots trend the two states and one input for the steady-state case and for the time varying case. The lower three plots trend the time-varying elements of the matrices K(t), P(t), and the co-state vector λ(t) for the full time-varying optimal case.

9.4. LINEAR QUADRATIC CONTROL

427

Now we have generated a trajectory for the four elements of the P(t) matrix as a function of time. We could then pre-compute the time varying gains, K(t), and then store them ready to be used in our closed loop simulation. Compared to the analytical solution, this numerical integration scheme requires that we integrate n2 equations as opposed to 2n when we combine the co-states. Furthermore we must store the trajectory of pre-computed time-varying gains. In practice of course, this is storage requirement rapidly becomes unwieldy and, as we shall see in the next section, rarely worth the effort.

9.4.3

T HE

STEADY- STATE SOLUTION TO THE MATRIX

R ICCATI

EQUATION

In the optimal controller scheme presented above, the gain K(t), is a continuously time varying matrix, as opposed to a more convenient constant matrix. A time varying controller gain requires excessive computer resources even if we did discretise, since we need to store an (m × n) gain matrix at every sample time in the control computer over the horizon of interest. We can make an approximation to our already approximate optimal controller by using only the steady-state gain derived from the steady-state solution of the matrix differential Riccati equation ˙ = 0. With this approximation, the differential Eqn. 9.48 is simplified to the algebraic when P matrix equation, 0 = A⊤ P∞ + P∞ A − P∞ BR−1 B⊤ P∞ + Q (9.60)

where the steady-state solution, P∞ , is now a constant gain matrix which is considerably more convenient to implement. Eqn. 9.60 is known as the continuous-time algebraic Riccati equation, or CARE. The now slightly sub-optimal steady-state controller gain is still obtained from Eqn. 9.51, K∞ = R−1 B⊤ P∞

(9.61)

and is also a constant gain matrix. Whether the performance deterioration due to this approximation is significant depends on the plant, and on the time horizon of interest. However as evident in Fig. 9.13, the differences in performance are negligible for all but the shortest time horizons. Even for the case where T = 2 which is still a relatively brief optimisation horizon, the gains are essentially constant for more than two thirds of the horizon, and this feature is generally true for all plants, and optimisation scenarios provided the horizon is reasonably long. We can solve for the elements of the matrix P∞ in Eqn. 9.60 using one of several strategies. One direct way is to by equating the individual elements in each of the matrices and thus reducing the single matrix equation to a set of algebraic equations. However unlike the Lyapunov equation, Eqn. 2.103, which we could collapse to a system of linear equations using vectorisation and Kronecker products as described in section 2.9.5, this time the steady-state Riccati equation includes a quadratic term of the form P∞ BP∞ which means our resultant system of equations will not be linear. However an iterative scheme based on this idea is given on page 431. Another strategy is to compute the analytical expression for P(T ), Eqn. 9.59, with a large time horizon T . Unfortunately Eqn. 9.59 is numerically very ill-conditioned when T is large, let alone ∞, and it is difficult to know a priori exactly what time horizon is sufficiently ‘large’. A final strategy given later in Listing 9.12 is only applicable for the discrete version of the optimal controller. The control toolbox in M ATLAB provides a reliable algebraic Riccati equation routine, care, that solves Eqn. 9.60 for P∞ in a numerically reliable manner2 , and the ric routine verifies the computed P from care (or equivalent) by calculating the residual of Eqn. 9.60. The linear quadratic 2 An

older toolbox routine, are.m, essentially also does this calculation.

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

428

regulator lqr routine finds the controller gain directly which is slightly more convenient than the two-step procedure solving the algebraic Riccati equation explicitly. All three routines are demonstrated in Listing 9.9.

A STEADY- STATE CONTINUOUS LQR

EXAMPLE

Simulating a steady-state optimal state-feedback controller is trivial in M ATLAB. We can try a steady-state version of the optimal controller for the example given previously on page 424. A script to solve the controller design problem using the toolbox functions are, care or lqr, and to simulate the results in M ATLAB is given in Listings 9.9 and 9.10. Listing 9.9: Calculate the continuous optimal steady-state controller gain. >>A = [-2, -4; 3 -1]; B = [4; 6]; % Plant under consideration >>Q = eye(2); R = 2; % Controller weighting matrices, Q, R. 3

8

13

18

>>P = are(A, B/R*B', Q) % Solve for X such that AT X + XA − XBX + Q = 0 P = 0.1627 -0.0469 -0.0469 0.2514 >>K = R\B'*P % Now compute gain from Eqn. 9.61, K∞ = R−1 B⊤ P∞ . K = 0.1847 0.6603 >>[Kerr,Perr] = ric(A,B,Q,R,K,P) Kerr = 0 0 Perr = 1.0e-014 * 0.0999 0.2720 0.1471 0.1887

% Check results

% Residuals should ≈ 0 which they do

>> [P2,L,G]=care(A,B/chol(R),Q);% Solve for X such that AT X + XA − XBBT X + Q = 0 >> K2 = chol(R)\G;

% Compute controller gain given that G = BT X

23

>>[K2,P2,E] = lqr(A,B,Q,R) >> K2 K2 = 0.1847 0.6603

% Gain from lqr should ≈ gain from care.

This completes the controller design problem, the rest is just a simple closed-loop simulation as shown in Listing 9.10. Listing 9.10: Closed loop simulation using an optimal steady-state controller gain. C = eye(size(A)); D = zeros(2,1); Gcl = ss(A-B*K,zeros(size(B)),C,D); % Hopefully stable optimal closed loop 3

T = 1; x0 = [-2,1]'; [Y,t,X]=initial(Gcl,x0,T); U = -X*K'; 8

% final time & initial state, x0 % back-calculate input, u(t), for plotting

subplot(2,1,1);plot(t,X); ylabel('states'); % Refer Fig. 9.14. subplot(2,1,2);plot(t,U); ylabel('input'); xlabel('time (s)')

9.4. LINEAR QUADRATIC CONTROL

429

The state and input trajectories of this optimal controller given a non-zero initial condition are given in the Fig. 9.14. 1 x (t) 2

states

0 −1 −2

x (t) 1

−3 0.4

input

0.2 0 −0.2 −0.4

0

0.2

0.4

0.6

0.8

1

time (s)

Figure 9.14: A steady-state continuous LQR controller.

The difference between the fully optimal time varying case and the simplification when using the steady-state solution to the Riccati equation is given in Fig. 9.13 which uses the same example from page 428. For short optimisation intervals there is a small difference between the steady-state solution and the full evolving optimal solution as shown in the right-hand figure of Fig. 9.13. However as the final time increases, there is no discernible difference in the trajectories of the states nor in the input in the left-hand figure of Fig. 9.13. This goes along way to explain why control engineers hardly ever bother with the full optimal solution. The lower two plots in Fig. 9.13 trend the time varying P(t) and K(t) matrices.

M ULTIPLE SOLUTIONS

TO THE ALGEBRAIC

R ICCATI

EQUATION

As mentioned in section 9.4.3, being a quadratic, there are multiple solutions to the continuous algebraic Riccati equation, Eqn. 9.60. We can investigate all these solutions using the symbolic toolbox for small sized problems by equating the coefficients and solving the resultant system of quadratic equations. Note that there are in fact only (n2 + n)/2 unknown elements in P as opposed to n2 due to the symmetry. Suppose we wish to design an LQR for the simple SISO system G(s) = 1/(s(s + 1)), with equal state and input weighting which is a problem initially discussed in [5]. 1

G = tf(1,[1 1 0]) % Plant [A,B,C,D] = ssdata(ss(G)) % actually transposed from [5], but no matter . . . Q = eye(2); R = 1;

Matlab choses a state-state representation as   −1 0 A= , 1 0

B=



1 0



Now inserting the plant and tuning matrices into the continuous algebraic Riccati equation, Eqn. 9.60, gives the nonlinear matrix system 0 = A⊤ P + PA − PBR−1 B⊤ P + Q     −1 1 p11 p12 p11 = + 0 0 p12 p22 p12

p12 p22



−1 1 0 0



−



p11 p12

p12 p22



1 0

0 0



p11 p12

p12 p22



+



1 0

0 1



CHAPTER 9. CLASSICAL OPTIMAL CONTROL

430

noting of course that since P is symmetric, p12 = p21 . From this point onwards, the algebraic manipulation gets messy, so we will use the Symbolic toolbox to help us.

2

7

>>n = 2; % Dimension of problem >>P = sym('P',[n,n]); % Create symbolic P matrix >>P = triu(P) + triu(P,1).' % Force P to be symmetric >>Res = A'*P+P*A - P*B/R*B'*P+Q % Continuous algebraic Riccati equation, Eqn. 9.60. Res = [ - P1_1ˆ2 - 2*P1_1 + 2*P1_2 + 1, P2_2 - P1_2 - P1_1*P1_2] [ P2_2 - P1_2 - P1_1*P1_2, 1 - P1_2ˆ2]

This final line shows us that we have 4 conditions to be satisfied, (although two of them are the same), 0 = −p11 2 − 2p11 + 2p12 + 1 0 = p22 − p12 − p11 p12

0 = 1 − p12 2

giving us enough equations to solve for the three unknown elements in P. We can see immediately that p212 = 1 giving us two solutions: p12 = ±1. We could continue this substitution by hand, but it might be faster at least to use the analytical solve command:

3

>> Res = Res(find(triu(ones(size(Res))))) % Drop redundant equations . . . >> S = solve(Res) % and solve S = P1_1: [3x1 sym] P1_2: [3x1 sym] P2_2: [3x1 sym]

The analytical solve routine returned three solutions for the three unknowns, namely:

4

>> [S.P1_1 S.P1_2 S.P2_2] ans = [ 1, 1, 2] [ -1, -1, 0] [ -3, 1, -2]

which tells us that we have three distinct P matrix solutions,       1 1 −1 −1 −3 1 Pa = , Pb = , Pc = 1 2 −1 0 1 −2 but only one of which is positive definite     0.38 −1.6 eig(Pa ) = , eig(Pb ) = , 2.62 1.6

eig(Pc ) =



−3.6 −1.3



and therefore stabilising. Practically we can extend this approach to solving 3rd (with 8 solutions) and 4th order (with 15 solutions) systems, but anything larger seems to stall. Of course, we could have used the standard numerical M ATLAB command and avoided all the spurious non-stabilising solutions: [P,E,K] = care(A,B,Q,R) % standard way

9.4. LINEAR QUADRATIC CONTROL S OLVING THE

ALGEBRAIC

R ICCATI

431

EQUATION USING

K RONECKER PRODUCTS

Notwithstanding that the algebraic Riccati equation of Eqn. 9.60 is nonlinear, we can devise an iterative scheme to solve for P∞ in a manner similar to that for the Lyapunov equation using vectorisation and Kronecker products described in section 2.9.5. Collapsing Eqn. 9.60 to vectorised form gives   − I ⊗ AT + AT ⊗ I − I ⊗ P∞ BR−1 BT vec(P∞ ) = vec(Q)

(9.62)

which we now must solve iteratively due to the unknown P∞ in the coefficient matrix in Eqn. 9.62. Listing 9.11 uses this solution strategy which works (most of the time) for modest sized problems, but is not numerically competitive with, nor as reliable as, the ARE scheme in M ATLAB. Listing 9.11: Solving the algebraic Riccati equation for P∞ using Kronecker products and vectorisation given matrices A, B, Q and R.

4

9

n = size(A,1); I = eye(size(A)); % Identity, I BRB = B/R*B'; % Constant matrix, BR−1 BT D = kron(I,A') + kron(A',I); % vec(AT P + PA) = (I ⊗ AT + AT ⊗ I)vec(P) P = I; % Start with P∞ = I tol = 1e-8; dP = tol*1e2; % Suggested tolerance, ≈ 10−5 while norm(dP) > tol Pnew = reshape(-(D-kron(I,P*BRB))\Q(:),n,n); % Solve Eqn. 9.62 iteratively for P∞ dP = Pnew - P; P = Pnew; end

In a similar manner to solving the Lyapunov equation using fsolve on page 85, we could also construct the algebraic Riccati matrix equation to be solved and submit that to the general nonlinear algebraic equation solver fsolve. This works in so far as it delivers a solution, but the solution may not be a stabilising solution, or in other words while the matrix P∞ does satisfy the original matrix equation, Eqn. 9.60, it may not be symmetric positive definite, and hence lead to a stable closed loop system, as shown in the example given on page 429. This is due to the fact that the matrix Riccati equation has multiple solutions and general purpose numerical schemes such as fsolve will return the first solution it comes across as oppose to the stabilising one that we are interested in. For this reason it is better to use the dedicated Riccati solving routines than the general purpose fsolve.

9.4.4

T HE

DISCRETE

LQR

In the discrete domain, the optimisation problem analogous to Eqn. 9.32 is to search for the N future optimal control moves, u0 , u1 , · · · , uN −1 to minimise J =

N −1
1 1 X T xN SxN + xk Qxk + uTk Ruk 2 2

(9.63)

k=0

where the continuous plant dynamics, Eqn. 2.40, are now replaced by the discrete equivalent xk+1 = Φxk + ∆uk

(9.64)

and the state feedback control law, Eqn. 9.50, is as before. Now the discrete matrix difference Riccati equation analogous to Eqn. 9.48 is  −1 ∆⊤ Pk+1 Φ (9.65) Pk = Q + Φ⊤ Pk+1 Φ − Φ⊤ Pk+1 ∆ R + ∆⊤ Pk+1 ∆

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

432 with final boundary condition

PN = S

(9.66)

 −1 Kk = R + ∆⊤ Pk+1 ∆ ∆⊤ Pk+1 Φ

(9.67)

Kk = R−1 ∆T Φ−T (Pk − Q)

(9.68)

and the feedback gain K at time t = kT is

An alternative form to Eqn. 9.67 is

which uses the current Pk rather than the future version, but has the disadvantage that Φ must be invertible. Note that the optimal feedback gain K, is now an (m × n) matrix rather than a row vector as in the SIMO pole placement case described in chapter 8. It is possible to compute the value of the performance index J , of Eqn. 9.63 in terms of the initial states and the initial P0 matrix, 1 J = xT0 P0 x0 (9.69) 2 although this precise value of J has probably little practical worth in most industrial control applications.

U SING THE

SLIGHTLY SUB - OPTIMAL DISCRETE STEADY- STATE GAIN

Clearly the discrete optimal gain given by Eqn. 9.67 is again time varying as it was in Eqn. 9.51 for the continuous case, but this can be computed offline since it is only dependent on the constant plant matrices Φ and ∆, the weighting matrices Q and R, and the time varying Pk . However, once again as in the continuous case given in §9.4.3, we can use the considerably simpler steady-state approximation to Eqn. 9.65 with little reduction in optimality. We solve for the steady state covariance,  −1 P∞ = Q + Φ⊤ P∞ Φ − Φ⊤ P∞ ∆ R + ∆⊤ P∞ ∆ ∆⊤ P∞ Φ

(9.70)

by starting with P∞ = 0 and stepping, (rather than integrating), backwards in time until Pk approximately equals Pk+1 . This is known as the discrete-time algebraic Riccati equation, or DARE. It follows that the resultant steady-state (and slightly sub-optimal) gain K∞ given by Eqn. 9.67

will now also be constant.

S OLVING THE

 −1 K∞ = R + ∆⊤ P∞ ∆ ∆⊤ P∞ Φ

DISCRETE ALGEBRAIC

R ICCATI

EQUATION

A simple way to solve the discrete algebraic Riccati equation implemented in the listing 9.12 is to iterate around Eqn. 9.70 starting from a zero initial condition. Listing 9.12: Calculate the discrete optimal steady-state gain by ‘iterating until exhaustion’. Note it is preferable for numerical reasons to use lqr for this computation. function [K,P] = dlqr_ss(Phi,Delta,Q,R)

9.4. LINEAR QUADRATIC CONTROL

433

u(t) ∆

+

z −1

b

y(t)

C

Φ Plant

−K

Exported states, x(t)

LQR gain

Figure 9.15: A block diagram of LQR control.

4

9

% Steady-state discrete LQR by unwinding the Riccati difference eqn, Eqn. 9.65. % Routine not as reliable as dlqr.m P = zeros(size(Phi)); % Start with P = 0 at t = ∞ tol = 1e-5; % default tolerance P0 = P+tol*1e3; while norm(P0-P,'fro') > tol P0=P; P = Q + Phi'*P*Phi - Phi'*P*Delta*((Delta'*P*Delta+R)\Delta')*P*Phi; end % while
−1 T K = (Delta'*P*Delta + R)\Delta'*P*Phi; % K∞ = R + ∆T P∞ ∆ ∆ P∞ Φ end % end function dlqr_ss

However, the backward iteration calculation technique given in dlqr_ss in listing 9.12 is both somewhat crude and computationally dangerous. For this reason it is advisable to use the functionally equivalent, but more robust dlqr (or lqr) provided in the C ONTROL T OOLBOX. We could also use dare if we only wanted to solve for the P∞ matrix. C OMPARING

THE CONTINUOUS AND DISCRETE STEADY- STATE

LQR

Listing 9.13 compares a continuous LQR regulator for the system given on page 424 with a discrete LQR sampled at a relatively coarse rate of Ts = 0.25s. Note that in both the discrete and continuous cases we design the optimal gain using the same lqr function. M ATLAB can tell which algorithm to use from the supplied model characteristics. Listing 9.13: Comparing the continuous and discrete LQR controllers. 2

A = [-2, -4; 3 -1]; B = [4; 6]; % Continuous plant, x˙ = Ax + Bu C = eye(size(A)); D = zeros(2,1); G = ss(A,B,C,D) Q = eye(2); R = 2; % Controller weighting matrices Q = I2 , R = 2 K = lqr(G,Q,R); % Optimal gain, K = [k1 , k2 ] Gcl = ss(A-B*K,zeros(size(B)),C,D); % Continuous closed loop

7

T = 1; x0 = [-2,1]'; % final time & initial condition [Y,t,X]=initial(Gcl,x0,T); U = -X*K'; % Simulate closed loop & back-calculate input

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

434

12

% Discrete version of the LQR controller Ts = 0.25; % Rather coarse sampling time, Ts , to exaggerate differences Gd = c2d(ss(A,B,C,D),Ts); % Discrete plant, xk+1 = Φxk + Buk , with Ts = 0.25 Kd = lqr(Gd,Q,R); % Design discrete LQR gain Gcl_d = ss(Gd.A-Gd.b*Kd,zeros(size(B)),C,D,Ts);

17

[Yd,td,Xd]=initial(Gcl_d,x0,T); Ud = -Xd*Kd'; % Simulate & back-calculate input subplot(2,1,1); plot(t,X,td,Xd,'o'); % Refer Fig. 9.16 subplot(2,1,2); plot(t,U,td,Ud);

Even though the heavily discretised LQR controller has quite a different control input trajectory as evident in the lower figure of Fig. 9.16, the difference between the discrete output, ◦, and the continuous LQR controller, (solid line), is noticeable in that the solid line does not pass exactly through the circles, but the difference is very marginal. 1 x2(t)

states

0

−1

−2

x1(t)

−3 0.4 continuous input

Figure 9.16: Comparing discrete, ◦, and continuous, , LQR controllers. Note that even with a relatively coarse sample time, the controlled response in both cases is very similar, (but not identical), at the sampled points.

input

0.2 0 discrete input, T =0.25 s

−0.2 −0.4

0

0.2

0.4

0.6

0.8

1

time (s)

Problem 9.2 1. Load in the Newell & Lee evaporator matrices (refer Appendix D.1) and design an LQR with equal weighting on the Q and R matrices by typing Q = eye(A); R = eye(B); and using the dlmr function with a sample time of 1.0. This feedback matrix should be the same as the gain matrix given in [147, p55]3 . Comment on any differences in solution you obtain. 2. Repeat the LQR design using the supplied dlqr function rather than the dlqr_ss function. Using dlqr however, the gain elements are now slightly different. Is this difference important and how would you justify your answer? 3. Use either dlqr_ss or dlqr to generate feedback gains for the limiting cases when either (or both) Q or R are zero. What difficulties do you encounter, and how do you solve them? 3 Newell & Lee define the control law in a slightly different way; u = Kx omitting the negative sign. This means that the K on p55 should be the negative of that evaluated by M ATLAB.

9.4. LINEAR QUADRATIC CONTROL C OMPUTING

435

THE PERFORMANCE OF A STEADY- STATE DISCRETE

LQR

CONTROLLER

As mentioned previously, in most cases we are uninterested in the exact value of the objective function since we are far more interested in the control law in order to minimise it. However it is interesting to numerically validate the actual performance with the analytical expectations. In the steady-state case, we are uninterested in the termination part of the performance index of Eqn. 9.63 leaving just J = =

∞


1X T xk Qxk + uTk Ruk 2

(9.71)

k=0

1 T x P∞ x0 2 0

(9.72)

reinforcing the fact that while the controller gain is not dependent on the initial condition, the value of the performance index is. Suppose we had a plant xk+1 =



0 0 −0.5 1



xk +



1 0



uk , starting from x0 =



2 2



with controller design matrices Q = diag[1, 0.5], and R = 1, and we wanted to compute the performance index J comparing the numerical summation given by Eqn. 9.71 against the analytical expression given by Eqn. 9.72.

4

>> Phi = [0 0; -0.5, 1]; Del = [1 0]'; % Plant matrices Φ, ∆ and controller design Q, R >> Q = diag([1 0.5]); r = 1; >> [K,P,E] = dlqr(Phi,Del,Q,r) % Solve the algebraic Riccati equation for K∞ & P∞ . K = 0.2207 -0.4414 P = 1.5664 -1.1328 -1.1328 2.7656

Now that we have the steady-state Riccati equation solution we can compute the performance directly , J .

2

>> x0 = [2,2]'; % Initial condition x0 >> J = 0.5*x0'*P*x0 % Performance J = 12 xT0 P∞ x0 , Eqn. 9.72. J = 4.1328

To obtain an independent assessment of this, we could ‘simulate until exhaustion’ and numerically compute Eqn. 9.71. Since we have simulated the closed loop, we need to ‘back-extract’ the input by using the control law and the states. 1

Gcl = ss(Phi-Del*K,0*Del,eye(2),0, 1); % Form closed loop (Φ − ∆K) [y,t,x] = initial(Gcl,x0); % Let M ATLAB establish a suitable on final time, k∞ u = -x*K'; % Back extract control input from closed loop simulation, u = −Kx

Now M ATLAB has returned a vector of input and states, and we need toP compute the summation given in Eqn. 9.71. One elegant way to calculate expressions such as xT Qx given the states stacked in rows is to use the M ATLAB construction sum(sum(x.*x*Q’)). This has the advantage that it can be expressed succinctly in one line of code and it avoids an explicit loop.

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

436

2


P T >> uRu = u'*u*r; % k uk Ruk , but only works for scalar r >> J_numerical = 0.5*(sum(sum(x.*x*Q'))+uRu) % Performance J_numerical = 4.1328

1 2

P

k

xTk Qxk + uTk Ruk




In this case the numerical value of J is very close to the analytical value. VARYING

THE

Q AND R WEIGHTING

MATRICES

The Q and R weighting matrices in the objective function Eqn. 9.32 reflect the relative importance of the cost of state deviations with respect to manipulated variable deviations and are essentially the tuning parameters for this control algorithm. If we increase the Q matrix, then the optimum controller tries hard to minimise state deviations and is prepared to demand very energetic control moves to do it. Likewise, if we increase R, the weighting on the deviations from u, then excessive manipulated variables moves are discouraged, so consequently the quality of the state control deteriorates. If we are only really interested in output deviations, yT y, as opposed to state deviations, then Q is often set equal to CT C. This follows simply from the output relation y = Cx. Fig. 9.17 and the listing below demonstrate the design and implementation of a discrete LQR for a stable, interacting arbitrary plant. We will start from a non-zero initial condition and we subject the system to a small external disturbance at sample time k = 20. In this listing, the state deviations are considered more important than manipulated deviations. 1

6

11

16

% Create an simple, stable, interacting, 3*2 plant Phi = [0.8 0.1 0.1; -0.1 0.7 0.2; 0 0.6 0.1]; Del = [1,2; -2,1; 0,-3]; C = [1,0,0]; D=[0,0]; t=[0:40]'; dist = 20; % sample vector & disturbance point Q = eye(size(Phi)); [n,nb] = size(Del); R=eye(nb,nb);

% tuning constants

K = dlqr(Phi,Del,100*Q,R); % Design LQR controller with Q = 100I, R = I Gd = ss(Phi-Del*K,Del,C,D,1); % closed loop system U = zeros(size([t,t]));U(dist,:) = [1,0.2]; % not much + disturbance x0 = [-1,2,0]'; % given initial condition [Y,T,X3] = lsim(Gd,U,t,x0); % do closed loop simulation U3 = X3*K'; % extract control vector from x stairs(t,[X3,U3]); P T P T % Compute performance indices: x Qx, & u Ru perfx3 = sum(sum(X3.*(Q*X3')')), perfu3 = sum(sum(U3.*(R*U3')'))

Fig. 9.17 compares two responses. The left controlled response is when we consider state deviations more important than manipulated variations, i.e. the weighting factor for the states is 100 times that for the manipulated variables. In the right plot, the opposite scenario is simulated. Clearly in Fig. 9.17P the left controlled response returns a relatively low ISE (integral squared error) on the states, xT Qx = 12.3, but at the cost of a high ISE on the manipulated variables, P T u Ru = 1.57. Conversely the right controlled response shows the situation, where perhaps owing to manipulated variable constraints, we achieve a much lower ISE on the manipulated variables, but at a cost of doubling the state deviation penalty. As Fig. 9.17 shows, the design of an optimal LQR requires the control designer to weigh the relative costs of state and manipulated deviations. By changing the ratio of these weights, different

9.4. LINEAR QUADRATIC CONTROL

437 Input movements are deemed costly: q=1,r=100

1.5

1.5

1

1 State Q=1

2

0.5 0 −0.5

−0.5

−1.5

−2

−2

1

1

0.5

0.5

0 Σu’Ru=1.571 −0.5

Σx’Qx=27.4

x1(t)

−1

−1.5

input R=1

x3(t)

0

Σx’Qx=12.3

−1

−1

x2(t)

0.5

input R=100

State Q=100

State deviations are deemed costly: q=100,r=1 2

0 Σu’Ru=0.059 −0.5

0

10

20 Sample number

30

40

−1

0

10

20 Sample number

30

40

Figure 9.17: Varying the state and manipulated weights of a discrete linear quadratic regulator where there is a disturbance introduced at sample time k = 20. When state deviations are costly (q is large compared to r), the states rapidly return to setpoint at the expense of a large manipulated movement. controlled responses can be achieved. As we did in §8.2, we can modify the simulation given above to account for non-zero setpoints, x⋆ , by using the control law, u = K(x⋆ − x) rather than Eqn 9.50, so simulating the closed loop xk+1 = (Φ − ∆K)xk + ∆Kx⋆k thus enabling response to setpoint changes.

9.4.5

A NUMERICAL

VALIDATION OF THE OPTIMALITY OF

LQR

If the LQR is indeed an optimal controller, its performance must be superior than any arbitrarily designed pole-placement controller such as those tested in section 8.2.1. Listing 9.14 designs an optimal controller for the blackbox plant with equal state and input weights. Listing 9.14: An LQR controller for the blackbox >> G = ss(tf(0.98,conv([2 1],[1,1])),0.1) % Blackbox plant with sample time Ts = 0.1 3

8

>> [n,m] = size(G.b); % system dimensions  1 0 >> Q = eye(n); % state weight, Q = 0 1 >> R = eye(m); % Input weight, R = r = 1 >> K = dlqr(G.a,G.b,Q,R) % Compute optimal controller gain K. K =

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

438 4.7373

-3.5942

Now it would be interesting to compare this supposedly optimal controller with a pole-placement controller with poles arbitrarily placed at say µT = [0.5 ± 0.5i]. Listing 9.15: Comparing an LQR controller from Listing 9.14 with a pole-placement controller 1

6

11

>> pc = [0.5-0.5i; 0.5+0.5i]; % Desired closed loop poles, µ = [0.5 ± 0.5i] >> Kp = acker(G.a,G.b,pc); % Pole-placement controller gain Kp = 13.6971 -5.7713 >> Gcl = G; Gcl.b = 0*G.b; Gcl.c = eye(n); >> Gcl.a = G.a-G.b*K; % Closed loop system with LQR, (Φ − ∆K) >> Gclpp = Gcl; Gclpp.a = G.a-G.b*Kp; % Closed loop system with pole-placement >> x0 = [-1,2]'; % Simulate disturbance rejection >> initial(Gcl,Gclpp,x0) % Refer Fig. 9.18.

The trajectory of the states for the two schemes are given in Fig. 9.18. However what is surprising is that the pole-placement response looks superficially considerably better than the somewhat sluggish, though supposedly optimal, LQR response. How could this be? Surely the optimally designed controller should be better than any ad-hoc designed controller? Comparing LQR & Pole−placement 2 pole−placement

x1

0 −2 −4

LQR

−6 2 0 x2

Figure 9.18: Comparing the state trajectories for both pole-placement and LQR. Note that the optimal LQR is considerably more sluggish to return to the setpoint than the poleplacement controller.

−2 dlqr Pole−placement λ =0.5±0.5i

−4 −6

0

0.5

1

1.5

2

2.5

3

What Fig. 9.18 has not shown us is the input trajectory. This is also important when comparing controlled responses, although typically perhaps not quite as important as the state variable trajectories. We need to consider the input trajectory for at least two reasons. Excessive deviations in input will almost certainly reach the input saturation constraints that are present in any real control system. Secondly, excessive movement in the input can cause unreasonable wear in the actuators. Finally in the case where the input magnitude is related to cost, say fuel cost in an engine, then the larger the magnitude, the more expensive the operating cost of the corrective action will be. Fig. 9.19 reproduces the same problem, but this time includes the input trajectory. In this instance, the cost of the pole-placement controller is J = 1288 units, while for the LQR it is under half at 553. Obviously different pole locations will give different performances, and it is up to us as control designers to choose the most appropriate pole location for the particular control application at

9.4. LINEAR QUADRATIC CONTROL

439

Performance: LQR = 553, Pole−place = 1288 2 pole−placement states

0 −2 LQR

−4 −6 10

input

0 −10 dlqr Pole−placement λ =0.5±0.5i

−20 −30

0

0.5

1

1.5

2

2.5

3

Figure 9.19: Comparing poleplacement and LQR, but this time including the input trajectories. Compared to Fig. 9.18, the input trajectory for the optimal LQR controller is considerably less active.

hand. Using M ATLAB, it is easy to do an ‘exhaustive search’ over a reasonable domain to find the best pole location. For real systems (that is systems with real coefficients), the desired poles must appear in conjugate pairs. That means that we only need to consider one half of the unit circle. Furthermore, we should only consider the 1st quadrant, because in the 2nd quadrant, the response will be overly oscillatory. Naturally it goes without saying that in the discrete domain we must be inside the unit circle. I have chosen a set of trial poles lying on a polar grid from 0 degrees round to 90◦ , and from a magnitude of about 0.1 to 0.9. These (unequally spaced) trial pole locations are shown in Fig. 9.20. 1

0.6π/T 0.8

0.5π/T

0.4π/T 0.1 0.3π/T 0.2 0.3 0.4 0.2π/T 0.5 0.6 0.7 0.1π/T 0.8 0.9

0.7π/T

0.6

0.8π/T

0.4

0.9π/T ℑ{λ}

0.2

1π/T 1π/T

0

−0.2

0.9π/T

0.1π/T

−0.4

0.8π/T

−0.6

0.2π/T 0.7π/T

−0.8

0.3π/T 0.6π/T

−1

−1

−0.8

−0.6

−0.4

−0.2

0.5π/T 0 ℜ{λ}

0.4π/T 0.2

0.4

0.6

0.8

1

Figure 9.20: We will test a pole-placement controller at each of the pole locations specified by a +. (See also Fig. 9.21 following.)

For each of the pole locations specified in Fig. 9.20, I will: 1. Design a pole placement controller, then 2. simulate the response for a reasonably long time which I assume will approximate infinity, and then P∞ 3. compute the performance criteria, J = i=0 xT Qx + uT Ru.

440

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

As you can see from the resultant contour plot in Fig. 9.21, the performance of our previously arbitrarily chosen pole location of µT = [0.5 ± 0.5i] is reasonable, but not great, and clearly not optimal. From the plot, the best performance is located at a pole location of approximately µ⋆ ≈ [0.75 ± 0.15i]T marked as a ✷ in Fig. 9.21. The question is: can we find this location without doing an exhaustive search, or indeed any type of numerical optimisation search strategy?

Figure 9.21: The performance of the closed loop for all the possible pole locations in the first quadrant. When µT = [0.5 ± 0.5i], the weighted performance is approximately 1288, while the optimum performance, ✷, is considerably better at around 550 at pole location µ⋆ ≈ [0.78 ± 0.17i]T . When we look at the closed loop poles of the LQR controller, Listing 9.16: Computing the closed loop poles from the optimal LQR controller from Listing 9.14.

4

9

>> [K,S,E] = dlqr(G.a,G.b,Q,R) K = 4.7373 -3.5942 % Controller gain (as before). S = 90.4087 -65.2391 -65.2391 50.4963 E = 0.7800 + 0.1664i % Optimum closed loop poles, µ⋆ . 0.7800 - 0.1664i

then we see that in fact the computed closed loop poles are intriguingly close to the optimum found numerically in Fig. 9.21. It turns out that the LQR routine did indeed find the optimum controller without going to the computational trouble of an exhaustive search. Just remember of course, that this is not a proof of the optimality of the LQR, it is just a numerical validation.

9.4. LINEAR QUADRATIC CONTROL

441

Nonetheless, you should find it a convincing argument. Note of course, if we change the dynamic system, or the controller design weights, the of course the shape of the performance surface in Fig. 9.21, and the corresponding optimum will also change. Problem 9.3 1. Simulate the controlled response of the evaporator for about 500 time steps (T = 0.5 time units) with the initial conditions x0 = for the 4 cases:



1

−4.3 2.2

⊤

(a) Q = R = I (b) Q = 103 R = I (c) 103 Q = R = I (d) any other combination of your choice. In each case, sketch the results of the input u, and the states x with time and calculate the ISE for the states, manipulated variables and the weighted sum. What conclusions can you deduce from the relative importance of Q and R? 2. Compare the performance of the full multivariable LQR controller with that of a controller comprised of just the diagonal elements of the controller gain. What does this say about the process interactions?

9.4.6

T RACKING LQR

CONTROL

Since the LQR is essentially just an optimal version of pole-placement, for the tracking (or servo) control problem we can simply follow the approach taken in section 8.2.3 where we introduced auxiliary states which were defined as the integral of the outputs. The M ATLAB routine lqi designs LQR controllers for servo applications. As an example, let us design a continuous servo LQR controller for the SISO system G(s) =

20(s + 2) (s + 3)(s + 5)(s + 7)

Clearly this system has m = 1 input, p = 1 output and n = 3 internal states. We desire that the output follows without offset some reference command signal. To achieve this we are going to design an optimal LQR controller with weighting matrices Qa (of dimension of the augmented states, n + p,) and R of dimension m as shown in Listing 9.17. Listing 9.17: Tracking LQR controller with integral states 1

6

>> Gc = ss(zpk([-2],[-3 -5 -7],20)); % Plant to be controlled >> [A,B,C,D] = ssdata(G); >> [n,m] = size(B); p = size(C,1); % Dimensions state-space model >> Qa = eye(n+p); R = eye(m); >> Ka = lqi(G,Qa,R) Ka = -0.7832 0.3879 0.4898

% Choose equal weighting for now % Design augmented gain [K, Ki ] -1.0000

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

442

This final augmented gain, is comprised of two parts: the first n elements are associated with the original plant states, and the final p elements are associated with the outputs we wish to control with zero offset. An implementation in S IMULINK for this application showing the signal dimensions is given in Fig. 9.22.

D1

Band-Limited White Noise

Cont

1 s

Cont 3

Add2 y Integrator

4 Cont 4

-Ka* u

3 Cont

B* u

u Cont

Ka feedback

3 Cont 3

3 3 Add

B

1 s

3 Cont x

3

C* u

Cont y

Terminator

Integrator

C

A

3 Cont

A* u

3

Figure 9.22: An LQR servo controller for a SISO plant with 3 states implemented in S IMULINK. See also Fig. 9.23 for the results of this simulation. If we are really only interested in controlling the outputs to their setpoint as opposed to all the states to zero, then we should indicate that desire by the appropriate design of the (augmented) controller weighting matrix Qa . In this case if I wish to have fast control at the expense of input movement, I would choose something like   0 0 0 0   Qa =  0 0 0 0  , R = 1 0 0 0 0 0 0 0 10

remembering that Qa need only be semidefinite, but R must be strictly positive definite. Fig. 9.23 shows the result for this servo controlled example for three different tuning ratios. Note that as Q gets large relative to R, then the output trends faster to the setpoint at the expense of a more vigorous input. 0.6 Q=1,R=1 Q=5,R=1 Q=1,R=20

y

0.4

0.2

0 Q=1,R=1 Q=5,R=1 Q=1,R=20

−0.2 1

u

0.5

0

Figure 9.23: LQI servo control for 3 different Q/R tuning ratios.

−0.5

0

20

40

60

80

100 time (s)

120

140

160

180

200

In experimental, rather than simulated applications, since we do not measure the states directly, we can use a plant model to produce state estimates based on the measured outputs using the simple state reconstructor from section 8.1.4, or we could use a state estimator which is the subject of the following section, section 9.5. In the experimental optimal regulator example shown in Fig. 9.24, at each sample time we:

9.4. LINEAR QUADRATIC CONTROL

443

1. update an estimate of a simple 3 state model online using recursive least-squares, 2. which we use to reconstruct the states given past input and output data, 3. and in parallel recompute the optimal steady state LQR gains given that the model has been updated. 4. Using the new LQR gains and the reconstructed states, we apply a state feedback control law. In this application the sample time was relatively long, T = 1.5 seconds, and the RLS forgetting factor was λ = 0.995. No feedback control was used for the first 20 sample times in order to given sufficient time for the estimation algorithm to deliver a reasonable model. From the trend of the controller gains in the bottom plot in Fig. 9.24, you can see that although the model parameters look reasonably constant, the values of the individual controller gains are quite ‘excited’. LQR of the blackbox ∆t=1.5

output & setpoint

1

0.5

0

−0.5 1 2 q

Input

2 r

dlqr servo: σ = 1, σ =1000

0.5 0 −0.5 −1 4

RLS: λ=0.995,

θ

2

a 2 abbb2

0

301

a

1

10 0

1

K &K

2

−2 20

−10 −20

0

50

100

150

200

250

time (s)

Figure 9.24: An experimental application of LQR servo control of the black box with 103 q = r with online identification and state reconstruction. Plots: (a) output and setpoint; (b) input; (c) model parameters θ; (d) controller gains. The control started after 20 seconds.

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

444 Noise State

Measurement actual outputs, y

Plant

Input, u

Model

−g(x)

−ˆ y

+

Estimator

predicted states u = −Kˆ x

ˆ corrected states, x −K

State feedback

Software

Figure 9.25: A state-based estimation and state feedback control scheme.

9.5

E STIMATION OF STATE VARIABLES

Chapter 6 looked at various alternatives to identify the parameters of dynamic systems, given possibly noisy input/output measured data. This section looks at ways to estimate not the parameters, (they are assumed known by now), but the states themselves, x, by only measuring the inputs u, and the outputs y. This is a very powerful technique since if this estimation works, we can use state feedback control schemes without actually going to the trouble of measuring the states, but instead using only their estimates. In section 8.1.4 we built a simple state reconstructor which works well in the case of no measurement noise or model/plant mismatch. The problem is that in practical cases there will always be disturbing signals which will degrade our state estimates derived from such a straight forward algorithm. The question is: can we devise a better scheme that mitigates against this degradation? The answer is given in the form of an optimal state estimator. ˆ , which can be compared to the These estimators work by using a model to predict the output, y measured output, y. Based on this difference we can adjust, if necessary, the predicted states. The estimation of states in this way is known, for historical reasons, as a Kalman filter. A block scheme of how the estimator works is given in Fig. 9.25. Here we have a control scheme that while demanding state-feedback, actually uses only state estimates rather than the actual hard-to-obtain states. There are many texts describing the development (eg: [52, 96]) and applications (eg: [39, 191]) of the Kalman filter.

9.5.1

R ANDOM

PROCESSES

The only reason that our estimation scheme will not work perfectly, at least in theory, is due to the corrupting noise, and to a lesser degree, incorrect initial conditions. Now in practice, noise is the garbage we get superimposed when we sample a real process, but when we run pure simulations,

9.5. ESTIMATION OF STATE VARIABLES

445

we can approximate the noise using some sort of random number generator such as described previously in §6.2. These are correctly termed pseudo random number generators, and most of them are of the very simple linear congruent type, and are therefore deterministic and not strictly random. Whole books have been written on this subject and many of them quote John von Neumann saying in 1951 “Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin.” M ATLAB can generate two types of random numbers: namely uniform or rectangular, and normal or Gaussian. Using the uniform scheme, we can generate any other arbitrary distribution we may desire including Gaussian. However in most practical cases, it is the Gaussian distributed random number we are more interested in. To visualise the difference, try: 150 100

x = rand(1000,1); hist(x)

50

% uniformly distributed,

0 0

0.5

1

300 200

2

100

y = randn(1000,1); hist(y) % normally distributed,

0 −5

0

5

Sometimes it is interesting to compare the actual probability density function or PDF obtained using the hist command with the theoretical PDF. To directly compare these, we need to scale the output of the histogram such that the integral of the PDF is 1. Fig. 9.26 compares the actual (scaled) histogram from 104 samples of a random variable, N (µ = 12, σ = 2) with the theoretical probability density function. The PDF of a normal variate is, [192, p60],  2 ! 1 x−µ 1 exp − (9.73) f (x) = √ 2 σ 2πσ 2 0.25

0.2

frequency

mean = 12.0116 0.15 σ2 = 4.03369 0.1

0.05

0

2

4

6

8

10

12

14

16

18

20

x

Figure 9.26: The probability density function of a normally distributed random variable x ∈ N (µ = 12, σ = 2); theoretical (solid) and experimental (histogram).

Listing 9.18: Comparing the actual normally distributed random numbers with the theoretical probability density function.

3

8

xsig = 2; xmean = 12; % Variance, σ, & mean, x ¯ x = xsig*randn(10000,1)+xmean; % sample (realisation) of r.v. x. xv = linspace(min(x),max(x),100)'; % approx range of x pdf= @(xv) 1/sqrt(2*pi)/xsig*exp(-0.5*(((xv-xmean)/xsig).ˆ2));% expected PDF, Eqn. 9.73. area = trapz(xv,pdf(xv)); % Note integral of PDF should=1.0 [numx,xn] = hist(x,30); % Generate histogram [incidence #, x-value] dx = mean(diff(xn)); % assume evenly spaced

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

446

scale = 1/dx/sum(numx); % scale factor so total area = 1 freq = numx*scale; % normalise so total area =1 bar(xn,freq); hold on; plot(xv,pdf(xv));hold off % See Fig. 9.26

M ULTIVARIABLE RANDOM

NUMBERS

For a single time series xk of n elements, the variance is calculated from the familiar formula n

Var(x) = σx2 =

1X (xi − x)2 n i=1

(9.74)

where x is the mean of the vector of data. In the case of two time series of data (xk , yk ) the variances of x and y are calculated in the normal fashion using Eqn. 9.74, but the cross-variance or co-variance between x and y is calculated by n

Cov(x, y) =

1X (xi − x) (yi − y) n i=1

(9.75)

Note that while the variance must be positive or zero, the co-variance can be both positive and negative and that Cov(x, y) = Cov(y, x). For this two vector system, we could assemble all the variances and cross-variances in a matrix P known as the variance-covariance matrix   σx2 Cov(x, y) P= Cov(y, x) σy2 which is of course symmetric with positive diagonals for time series of real numbers. In normal usage, this name of this matrix is shortened to just the ‘covariance’ matrix. For an n variable system, the co-variance matrix P will be a positive definite symmetric n × n matrix. M ATLAB can calculate the standard deviation (σ) of a vector with the std command and it can calculate the co-variance matrix of a multiple series using the cov command. One characteristic about two independent random time series of data is that they should have no cross-correlation, or in other words, the co-variance between x and y, should be approximately zero. For this reason, most people assume that the co-variance matrices are simply positive diagonal matrices, and that the off-diagonal or cross-covariance terms are all zero. Suppose we create three random time series vectors in M ATLAB (x, y, z).

4

x=randn(1000,1) +3; y=randn(1000,1)*6+1; z=randn(1000,1)*2-2; a=[x y z]; cov(a)

% generate 3 independent vectors % standard deviation, σ = 6 % variance σ 2 = 22 = 4 % find co-variance

The co-variance matrix of the three time series is in my case, (yours will be slightly different), 

  1.02 −0.11 0.10 1 Cov(x, y, z) =  −0.11 37.74 0.30  ≈  0 0.10 0.30 4.22 0

 0 0 36 0  0 4

which shows the dominant diagonal terms which are approximately the true variances (1, 36, 4) and the small, almost zero, off-diagonal terms.

9.5. ESTIMATION OF STATE VARIABLES

C OVARIANCE

447

MATRICES AND ELLIPSES

In many cases, the random data does exhibit cross-correlation and we will see this clearly if the x, y data pairs fall in a patten described by a rotated ellipse. We can compute the 95% percent confidence limits of this ellipse using T

(x − µ) P−1 (x − µ) ≤

p(n − 1) Fα,p,(n−p) n−p

where p is the number of parameters (usually 2), and n is the number of data points (usually > 100). Fα,p,(n−p) is the F -distribution at a confidence of α, in this case 95%. To compute values from the F -distribution we could either use the functions given in Listing 9.19, Listing 9.19: Probability and inverse probability distributions for the F -distribution. pf = @(x,v1,v2) betainc(v1*x/(v1*x+v2),v1/2,v2/2); qf = @(P,v1,v2) fsolve(@(x) pf(x,v1,v2)-P,max(v1-1,1))

or use equivalent routines from S TIX B OX toolbox from Holtsberg mentioned on page 4 or alternatively use finv(P,v1,v2) from the Statistical toolbox. The easiest way to construct this ellipse is to first plot a circle parametrically, scale the x and y axes, rotate these axes, then finally translate the ellipse so that the center is at the mean of the data. The axes of the ellipse are given by the eigenvectors of P, and the lengths are given by the square root of the eigenvalues of P. Example of confidence limit plotting Here we will generate 1000 random (x, y) samples with deliberate correlation and we will compute the ellipse at a 95% confidence limit. Furthermore we can check this limit by using the inpolygon routine to calculate the actual proportion of data points that lie inside the ellipse. We expect around 95%. 1. First we generate some correlated random data, x ∈ N (−0.3, 1) and y ∈ N (2, 1.1) + x These data pairs are correlated because y is explicitly related to x. Listing 9.20: Generate some correlated random data.

3

N = 1e3; % # of samples in Monte-Carlo x = randn(N,1)-0.3; % N (¯ x = −0.3, σ = 1) y = 1.1*randn(N,1)+x + 2; % Deliberately introduce some cross-covariance plot(x,y,'.') % See Fig. 9.27(a).

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

448

We can plot the (x, y) points and we can see a that the ‘buckshot’ pattern falls in a rotated ellipse indicating correlation as expected as shown in Fig. 9.27(a). 2. The distribution of the data in Fig. 9.27(a) is more clearly shown in a density plot or a 3D histogram. Unfortunately M ATLAB does not have a 3D histogram routine, but we can use the normal 2D histogram in slices along the y-axis as demonstrated in the following code snippet. Listing 9.21: Plot a 3D histogram of the random data from Listing 9.20. 1

6

Nb = 30; % # of bins in a quick & dirty 3D histogram xmin = min(x); ymin = min(y); xmax = max(x); ymax = max(y); xv = linspace(xmin,xmax,Nb); yv = linspace(ymin,ymax,Nb); [X,Y] = meshgrid(xv,yv); Z= 0*X; for i=1:length(xv)-1 idx = find(x>xv(i)&x tol Kprev = K; M = Phi*P*Phi' + GQG; % Time update 17

% Square root form for the measmnt update S = chol(M)'; % Note transpose required from chol

9.5. ESTIMATION OF STATE VARIABLES

22

467

F = S'*C'; G = sqrtm(R+F'*F); Sp = S - (S*F/(G'))*((G+V)\F'); P = Sp*Sp'; % Will always be positive definite K = S*F/(G')/G; end % while return % end function potter.m

Modify the potter algorithm to make it more efficient. (For example the calculation of K is not strictly necessary within the loop.)

S EQUENTIAL PROCESSING The Kalman filter was developed and first implemented in the 1960s when computational power was considered expensive. For this reason, there has been much optimisation done on the algorithm. One of these improvements was to sequentially process the measurements one at a time rather than as a vector. [6, p142] gives two reasons for this advantage. First if the measurement noise covariance matrix R is diagonal, there is a reduction in processing time, and second, if for some reason there is not enough time between the samples to process the entire data, (owing to an interrupt for example), then there is only a partial loss of update information to the optimised states, rather than a full loss of improvement. In the sequential processing formulations of the Kalman filter equations, the Kalman gain matrix L is in fact a single column vector, and the measurement matrix C is a row vector.

D IAGONALISING

THE

R MATRIX

If the measurement noises are correlated, then R will not be diagonal. In this rare case, we can transform the system to where the measurements are uncorrelated, and work with these. The transformation uses a Cholesky factorisation. R is factored into R = SS ⊤

(9.126)

¯ = S −1 y y

(9.127)

Now the transformed measurements are

9.5.9

T HE E XTENDED K ALMAN F ILTER

The Extended Kalman Filter or EKF is used when the process or measurement models are nonlinear. Once we have a nonlinear process, all the guarantees that the Kalman filter will converge and that the filter will deliver the optimal state estimates no longer apply. Notwithstanding, it is still worth a try, since in most cases even sub-optimal estimates are better than none. Almost all the published accounts of using a Kalman filter in the chemical processing industries are actually an extended Kalman filter applied to a nonlinear model. The EKF is a logical extension from the normal linear Kalman filter. The estimation scheme is still divided up into a prediction step and an estimation step. However for the model and measurement prediction steps, the full nonlinear relations are used. Integrating the now nonlinear process differential equation may require a simple numerical integrator such as the Runge-Kutta algorithm. Thus the linear model update and measurement equations are now replaced by ˆ k|k−1 = f (ˆ x xk−1|k−1 , uk , d, θ, t) ˆ = g (ˆ y x)

(9.128) (9.129)

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

468

Given the nonlinear model and measurement relations, updating the covariance estimates is no longer a simple, or even tractable task. Here we assume that over our small prediction step of interest, the system is sufficiently linear that we can approximate the change in error statistics, which are only an approximate anyway, with a linear model. Now all the other equations of the Kalman filter algorithm are the same, except that the discrete state transition matrix Φ, is obtained from a linearised version of the nonlinear model evaluated at the best current estimate, ˆ k|k−1 following the techniques described in §3.5. The creation of the linear approximation to the x nonlinear model can be done at every time step, or less frequently, depending on the application and the availability of cheap computing. Linearising the measurement equation, Eqn 9.129 can be done in a similar manner.

E STIMATING

PARAMETERS ONLINE USING THE

K ALMAN FILTER

Chapter 6 discussed estimating model parameters online which could then be subsequently used in an adaptive control scheme. In addition, there are many text books explaining how to offline regress parameters from experimental data. While there is some debate about the nomenclature, parameters are generally considered to be constant, or at least change much more slowly than the states. In a heat exchanger for example, the heat exchange surface area is constant, thus a parameter, while the flow may vary depending on demand, thus a state. Some variables lie somewhere in between, they are essentially constant, but may vary over a long time period. The heat transfer coefficient may be one such variable. It will gradually get less as the exchanger fouls (owing to rust for example), but this occurs over time spans much larger than the expected state changes. In many engineering applications, however, there is some, (although possibly not complete) knowledge about the plant, and this partial knowledge should be exploited. This was not the case for the identification given in chapter 6, where only an arbitrary ARMA model was identified. The Kalman filter can be extended to estimate both states as given in §9.5 and also parameters. Of course once parameters are to be estimated, the extended system generally becomes nonlinear, (or if already nonlinear, now more nonlinear) so the Extended Kalman filter or EKF must be used. In this framework, the parameters to be estimated are treated just like states. I typically simply append the parameter vector (θ) to the bottom of the state vector x. Normally we would like to assume that the parameter is roughly constant, so it is reasonable to model the parameter variation as a random walk model, dθ =0+v dt

(9.130)

where v is a random vector of zero mean and known variance. A plot comparing normally distributed white noise with a random walk is shown in Fig. 9.35 and can be reproduced using: 1

v = randn(500,1); % white noise: mean zero, variance 1.0, Normal distbn. x = cumsum(v); % generate random walk plot([v x])

Eqn 9.130 is the simplest dynamic equation for a parameter. In some specialised examples, it may be worth while to have other simple dynamic models for the parameters, such as random ramps to model instrument drift for example, but in general this extension is not justified. Of course, with the extra parameters, the dimension of the estimation problem has increased, and the observability criteria must still hold.

9.5. ESTIMATION OF STATE VARIABLES

469

25 20 Random walk 15 10 5 0 −5 −10 −15

White noise 0

100

200 300 Sample #

400

500

Figure 9.35: An example of a random walk process: θi+1 = θi +ν . The solid line, ν, is white noise with a mean of zero, and a variance of 1. The dashed line is θ.

N ON -G AUSSIAN NOISE The assumption that the measurement and process noise sequences are Gaussian, un-correlated, and mean zero are usually not satisfied in practice. Coupled with the fact that the true system is probably nonlinear, it is surprising that an optimal linear estimator, such as the Kalman filter, or even the EKF works at all. Most textbooks on regression analysis approximate the residual between actual experimental data and a model as made up of two components; a systematic error and a random error. The Kalman filter is based on the assumption that the only error is due to a random component. This in general, is a very strong assumption, and not satisfied in practice. However coloured noise is treated theoretically in [52, Chpt 5]. One easy test to make is how the Kalman filter performs when the noise is correlated. We can generate correlated or coloured noise by passing uncorrelated or white noise through a low pass filter. Problem 9.6 Chapter 7 discussed auto-tuning by relay feedback. One of the key steps in constructing an automated tuning procedure is to identify the period and amplitude of the oscillation in the measurement. Often, however, this is difficult owing to the noise on the measurement, but we can use an Extended Kalman filter to estimate these quantities in the presence of noise. Further details and alternative schemes are discussed in (author?) [12]. A signal that is the sum of a sinusoid and a constant bias can be represented by the following equations; x2 x3 x1 x˙ 2 = −2π x3 x˙ 3 = 0

x˙ 1 = 2π

x˙ 4 = 0 y = x1 + x4 and the period and amplitude of the oscillation given by q P = x3 , ǫ = x21 + x22

The idea is to estimate the states of the process given the output y, and then to calculate the period and amplitude online.

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

470

1. Linearise the dynamic equations to the standard state space form. 2. Design a steady-state Kalman filter. 3. Simulate the estimation process. Plot the estimates of the period and amplitude for a noisy signal, and for a time varying signal.

9.5.10

C OMBINING

STATE ESTIMATION AND STATE FEEDBACK

One of the key aims of the Kalman filter is to provide estimates of unmeasured states that can then be used for feedback. When we combine state estimation with state feedback we have a Linear Quadratic Gaussian or LQG controller. Fig. 9.36 shows how we could implement such a combined estimator and controller in S IMULINK which essentially follows the scheme given in Fig. 8.7. The actual plant is in grey which is disturbed by green state and measurement noise. The estimator (Kalman) gain is in dark red, and the state feedback controller is in blue. measurement noise

plant noise 5

3

5 3

K*u

5

3

5

5

5

5 Add

Delta

1 z

5

x

5

K*u

3

3

Add1

C

Unit Delay

y

3

Phi 5

K*u

5

Ke 5

K*u

3

5 3

K*u

5

3 5

5

5 5 Add2

Delta est

5

1 z

xe

Unit Delay2

5

K*u

3

3 ye

3

Add3

C est

Phi est 5

K*u

5

Kc u

3

K*u

5

Figure 9.36: An example of a discrete LQG optimal estimator/controller implemented in S IMULINK. This is simply the estimator from Fig. 9.31 where the estimated states are fed back through a static gain. (See also Fig. 8.7.) The design of the discrete regulator, perhaps using dlqr, and the design of the estimator, perhaps using dlqe are completely independent. In this implementation we are using the 5 state aircraft model from [62] listed in Appendix D.2. In the listing below, we will assume the discrete time model sampled at Ts = 0.1s is Gd. The variance for both the state and measurement noise is set to 0.01.

9.5. ESTIMATION OF STATE VARIABLES

2

471

% Assume Gd is the discrete time model from Appendix D.2. n = size(Gd.a,1); m = size(Gd.b,2); p = size(Gd.c,1); % Plant dimensions, n, m, p Q = eye(n,n); R = eye(m); % Weighting for the LQR controller [Kc,S,E] = dlqr(Md.a, Md.b,Q,R); % Optimal controller gain, u = −Kc x

7

Qe = eye(n,n); Re = 100*eye(p); % Weighting for the LQG estimator [Ke,Se,Ee] = dlqe(Md.a,eye(n),Md.c, Qe,Re) % Kalman estimator gain, Ke

The controlled results are given in Fig. 9.37 which also shows the (typically invisible) true states in grey. 100 80

x

2

x & xe

60 40 20 x4 x5

0

x x 31

−20 −40 20

u1 u3

u

0

−20 u2

−40

0

1

2

3

4

5 time (s)

6

7

8

9

10

Figure 9.37: LQG regulatory control of an aircraft. The true, but hidden, states are given in grey. (See also Fig. 9.36.)

9.5.11

O PTIMAL

CONTROL USING ONLY MEASURED OUTPUTS

The optimal controllers of §9.4 had the major drawback that they required state feedback since the control law was a function of the states, namely Eqn. 9.50 or u = −Kx. We addressed this problem in §9.5 by first designing an observer to produce some estimated states which we could use in place of the true, but unknown, states in the controller. However it would be valuable to be able to design an optimal controller directly that uses output, rather than state feedback. Such an optimal output feedback controller in fact exists, but it is difficult to compute, and may only deliver a local optimum. The derivation of the optimum output controller can be found in [122, chapter 8], and improvements to the design algorithm are given in [166]. Once again we assume a linear plant x˙ = Ax + Bu, y = Cx

x(0) = x0

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

472

but this time we desire an output controller of the form u = −Ky which minimises the familiar quadratic performance index Z
1 ∞ T x Qx + uT Ru dt J (K) = 2 0

where A, Q ∈ ℜn×n , B ∈ ℜn×p , C ∈ ℜr×n , R ∈ ℜp×p , K ∈ ℜp×r . Also we assume, Q > 0 (positive semi-definite) and R > 0. The optimum gain Kis given by the simultaneous solution to the following design equations: 0 = ATc P + PAc + CT KT RKC + Q 0 = Ac S +

SATc

+X −1  K = R−1 BT PSCT CSCT

(9.131) (9.132) (9.133)

where the closed loop Ac and the covariance of the initial state are defined as Ac = A − BKC, and the performance index is given by

 def X = E x(0)xT (0)

J = trace (PX)

(9.134)

The coupled nature of the multiple Lyapunov equations mean that we will need to use a numerical technique to solve for the K that satisfies all three design equations. Our alternatives are: 1. To use a general-purpose unconstrained optimiser to search for the values of K that minimise Eqn. 9.134 directly where P is given by the solution to Eqn. 9.131. 2. To use an unconstrained optimiser and exploit the gradient information since the gradient of J with respect to K is   ∂J = 2 RKCSCT − BT PSCT ∂K

where we have computed the auxiliary matrices P, S by solving the two Lyapunov equations Eqns 9.131–9.132 perhaps using the M ATLAB function lyap. 3. To follow a simple fixed-position algorithm where we guess a value for K and then iterate around Eqns 9.131 to 9.133. Numerical experiments show that the direct optimisation is prone to infinite looping, and the fixed-point algorithm can converge to local optima.

9.6

S UMMARY

This chapter dealt with optimal control using classical methods. Optimal control is where we try to control a process in some “best” manner. The term best in the engineering field usually means most economic (either most profit, or least cost). While it is naive to expect our controller to perform optimally in the field, in practice they have been found to out-perform more conventional controllers designed via methods such as loop shaping or frequency response for example. I think

9.6. SUMMARY

473

of optimal controllers as simply another set of design rules to construct a workable, satisfactory controller, but would not expect them to be the “best” in any strict and absolute sense. A convenient objective function is the quadratic objective function that tries to minimise the weighted sum of state and manipulated deviations squared over time. The optimal controller design problem given nonlinear process dynamics and constraints is, in general, too hard to solve online, although offline open-loop optimal profiles are possible using numerical techniques. The optimisation problem given linear dynamics is tractable and the solution is termed the linear multivariable regulator (LMR or LQG). The LMR in its simple guise is a multivariable proportional controller. The fact that estimation is closely related to regulation was shown in chapter 8 and the optimal estimator is the Kalman filter.

474

CHAPTER 9. CLASSICAL OPTIMAL CONTROL

C HAPTER 10

P REDICTIVE

CONTROL

To be computationally practical, the optimal controllers of chapter 9 required that the system satisfy certain mathematical restrictions. To use variational calculus, the dynamic models must be differentiable; to use LQG, we require linear models with no saturation constraints. In contrast the controllers described in this chapter take into account constraints in the manipulated and control variables, something which is crucial for practical control applications.

10.1

M ODEL PREDICTIVE CONTROL

Model predictive control, or MPC, first started appearing in a variety of flavours in the late 1970s and has since then emerged as one of the most popular and potentially valuable control methodologies, particularly popular in the chemical processing industries. A good comprehensive survey of theoretical and practical applications is given in [54, 73, 162, 167]. The central theme of all these variants of predictive control is a controller that uses a process model to predict responses in a moving or receding horizon. Unlike the optimal control of section 9.3.2 or the linear cases following, this control strategy uses a finite optimisation horizon, (refer Fig. 10.1). At every sample time, the optimiser built into the controller selects the best finite number of future control moves over a horizon of 10 or so samples, of which the controller will subsequently implement just the first. At the next sample time, the whole optimisation procedure would be repeated over the new horizon, which has now been pushed back one step. This is also known as receding horizon control. Since an optimisation problem is to be solved each sampling time step, one hopes that the calculations are not too involved! This long-term predictive strategy coupled with the seeking of multiple optimal future control moves turns out to be rather a flexible control scheme where constraints in manipulated and state variables can be easily incorporated, and some degree of tuning achieved by lengthening or shortening the horizon. By predicting over the dead time and two or three time constants into the future, and by allowing the controller to approach the setpoint in a relaxed manner, this scheme avoids some of the pitfalls associated with the traditionally troublesome processes such as those with inverse response or with excessive dead-time. However relying predictions from models are not without their problems. I pity the Reserve Bank of New Zealand which has the unenviable task of predicting the future time-varying interest rates at quarterly intervals as shown in Fig. 10.2. The dark solid line shows the actual 90-day interest rates from March 1999 until March 2003 compared to the predictions made at the originating point. This data was obtained from [194, p18] where is it cautioned: “While we consider our 475

CHAPTER 10. PREDICTIVE CONTROL

476 Predicted outputs ⊕

⊕ ⊕

⊕

Past values b

b

⊕ b

b

⊕

⊕

⊕

⊕

⊕

⊕

⊕

Known future setpoints, r

Prediction horizon, Np b

Past

now

time, t Future

Ts Planned future manipulated inputs

current input

Control horizon, Nc current time Figure 10.1: The basic idea behind predictive control showing the control and prediction horizons. The controller tries to select future input values such that the output tends to the setpoint in some optimal manner.

approach generally useful, there are risks. For example, if the interest rate projections are taken as a resolute commitment to a particular course of policy action, then those that interpret it as such will be misled.”

Figure 10.2: Successive future 90-day interest predictions by the Reserve Bank at quarterly intervals.

The idea behind MPC is that we have a dynamic, possibly nonlinear, plant described as yˆi+1 = f (yi , ui )

(10.1)

where we wish the plant output y(t) to follow as closely as practical a possibly varying setpoint or reference r(t).

10.1. MODEL PREDICTIVE CONTROL

477

To establish the control move, we find a suitable u(t) solve that minimises the quantity Jt =

subject to some input constraints

Np X i=1

|

2

(rt+i − yˆt+i ) + {z

prediction horizon

}

Nc X

w∆u2t+i−1

(10.2)

i=1

|

{z

control horizon

ui ∈ U

} (10.3)

The set U in the constraint equation, Eqn. 10.3 is the set of allowable values for the input and typically takes into account upper and lower saturation limits in the actuator. For example in the case of a control valve, the allowable set would be from 0 (shut) to 100% (fully open). The optimisation of Eqn. 10.2 involves finding not just the one next control move, but a series of Nc future moves, one for each sample time over the future control horizon. In practice the control horizon is often about 5–15 samples long. The quantity optimised in Eqn. 10.2 is comprised of two parts. The first expression quantifies the expected error between the future model predictions yˆ and the future setpoints, r. Typically the future setpoint is a constant, but if changes in reference are known in advanced, they can be incorporated. The comparison between error and expected output is considered over the prediction horizon, Hp , which must be longer than the control horizon. The second term in Eqn. 10.2 quantifies the change in control moves. By making the weighting factor, w, large, one can discourage the optimal controller from making overly excited manipulated variable demands. At each sample time, a new constrained optimisation problem is solved producing an updated vector of future control. At the next sample instance, the horizons are shifted forward one sample time T , and the optimisation repeated. Since the horizons are continually moving, they are often termed receding horizons. The chief drawback with this scheme is the computational load of the optimisation every sample time. If the model, Eqn. 10.1, is linear, and we have no constraints, then we are able to analytically extract the optimum of Eqn. 10.2 without resorting to a time consuming numerical search. Conversely if we meet active constraints or have a nonlinear plant, then the computation of the optimum solution becomes a crucial step. Further details for nonlinear predictive control are given in the survey of [83]. Algorithm 10.1 Model Predictive Control (MPC) algorithm The basic general predictive control algorithm consists of a design phase and a implementation phase. The design phase is: 1. Identify a (discrete-time) model of the plant to be controlled. 2. Select a suitable sampling time and choose the lengths of the control and prediction horizons, Hc , Hp , where Hc ≤ Hp and the control move weightings, w. Typically Hp is chosen sufficiently long in order to capture the dominant dynamics of the plant step response. 3. For further flexibility, one can exponentially weight future control or output errors, or not include the output errors in the performance index for the first few samples. Making the prediction horizon longer than the control horizon effectively adds robustness to the algorithm, without excessively increasing the optimisation search space.

CHAPTER 10. PREDICTIVE CONTROL

478

4. Incorporate constraints in the manipulated variable if appropriate. In the absence of constraints, and as the prediction horizons tend to ∞, the MPC will lead to the standard linear quadratic regulator as described in §9.4. The implementation phase is carried out each sampling time. 1. Use an optimiser to search for the next future Nc optimum control moves ut starting from the current position. 2. Implement only the first optimum control move found from step 1. 3. Wait one sample period, then go back to step 1. Use the previous optimum ut vector, (shifted forward one place) for the initial estimate in the optimiser. The following sections demonstrate various predictive controllers. Section 10.1.1 applies a predictive controller to a linear plant, but with input constraints. Because this is a nonlinear optimisation problem, a generic nonlinear optimiser is used to solve for the control moves. As you will see if you run the example in Listing 10.1, this incurs a significant computational burden. If we ignore the constraints, then we can use a fast analytical solution as shown in section 10.1.2, but this loses one of the big advantages of MPC in the first place, namely the ability to handle constraints. The M ODEL P REDICTIVE C ONTROL T OOLBOX for M ATLAB described in [28] is another alternative to use for experimenting with MPC.

10.1.1

C ONSTRAINED

PREDICTIVE CONTROL

This section demonstrates a constrained predictive control example using the following linear non-minimum phase plant, 1.5(1 − 2.5s) Gp = 2 (10.4) 4s + 1.8s + 1 but where the manipulated variable is constrained to lie between the values −3.5 < u < 3. Sampling Eqn. 10.4 at T = 0.75 with a zeroth-order hold gives the discrete system Gp (z) =

−0.005z −1 − 0.4419z −2 1 − 1.596z −1 + 0.7136z −2

whose open loop response to a step change is illustrated in Fig. 10.3. To make the control problem more interesting, we will assume that Since we have a constraint in the manipulated variable, I will use the general nonlinear constrained optimiser fmincon from the O PTIMISATION T OOLBOX. The function to be minimised is fmodstpc in Listing 10.2 which returns the performance index. We can see from the discrete step response that a sample time of T = 0.75 is a reasonable choice. However the response drops before it climbs to the final steady-state, exhibiting an inverse response or non-minimum phase behaviour owing to the right hand plane zero at s = +2.5. With this sort of inverse behaviour, we must take care that the prediction horizon is sufficiently long to span the initial inverse response which again we can estimate approximately from the discrete step response. In this case I will set Hp = 20 and optimise for 10 inputs, Hc = 10. The control weightings are set to w = 1. The script file to run the simulation and to obtain results similar to Fig. 10.4 is given in Listing 10.1.

10.1. MODEL PREDICTIVE CONTROL

479

Step Response 2

Amplitude

1.5

1

0.5

0

Gp =

−0.5

−1

0

5

1.5(1−2.5s) 4s2 +1.8s+1 ,

10 15 Time (sec)

Ts = 0.75

20

25

Figure 10.3: An inverse plant model sampled at Ts = 0.75s.

Listing 10.1: Predictive control with input saturation constraints using a generic nonlinear optimiser ntot=150; dt=0.75; t=dt*[0:ntot-1]';

% time vector

2

r = [2,3,-2,-7,2]; % Interesting setpoint Yspt = ones(size(t(1:fix(length(t)/length(r))+1)))*r(:)'; Yspt = Yspt(1:length(t))'; % Cut off excess data points 7

12

tau = 2.0; zeta = 0.45; Kp=1.5;taup=2.5; % Create non-minimum phase plant Gss = ss(c2d(tf(Kp*[-taup 1],[tauˆ2 2*tau*zeta 1]),dt)); % discrete plant Hc = 10; % control horizon, Hc Hp = Hc+10; % prediction horizon, Hp > Hc Umin = randn(Hc+1,1); % start guess Ulb = -3.5*ones(size(Umin)); Uub = 3*ones(size(Ulb)); % bounds Uopt=zeros(size(t)); Y=zeros(size(t)); % initialise u(t) x = [0,0]'; % initialise starting point optns = optimset('Display','off','TolFun',1e-3,'TolX',1e-3); % optimiser options

17

for i=1:ntot-Hp-1; % for each sample time ... Umin = [Umin(2:end);Umin(end)]; % roll & repeat Umin = fmincon(@(u)fmodstpc(u,Gss,x,Yspt(i:i+Hp-1),Hp),Umin, ... [],[],[],[],Ulb,Uub,[],optns); 22

x = Gss.a*x + Gss.b*Umin(1,:)'; % do 1-step prediction Y(i,:) = Gss.c*x; Uopt(i,:) = Umin(1)'; % collect end % for i Yspt(1) = []; Yspt=[Yspt;1]; % shift due to fence-post error 27

32

subplot(2,1,1); % Now plot results plot(t,Y,t,Yspt,'r--') % plot results ylabel('output & setpoint') subplot(2,1,2); [ts,Us] = stairs(t,Uopt); plot(ts,Us,t,[ones(size(t))*mean([Ulb,Uub])],'k--') xlabel('time [s]'); ylabel('Input')

The objective function to be minimised called by the optimiser is given in Listing 10.2. Note that

CHAPTER 10. PREDICTIVE CONTROL

480

auxiliary variables, such as the model, and horizon lengths are passed as anonymous variables through to the function to be optimised. Listing 10.2: Objective function to be minimised for the predictive control algorithm with input saturation constraints 1

6

11

function Perf = fmodstpc(U,G,x0,Yspt,Hp) % Short-term-prediction error called via fmincon via TGPC.M if Hp > length(U), % normally the case U = [U;U(end)*ones(Hp-length(U),1)]; % pad extra end % if P c 2 ssdu = sum(diff(U).ˆ2); % Control weightings N i=1 ∆ut+i−1 [Y,t,X] = lsim(G,U,[],x0); % do linear model prediction Perf = sum((Y-Yspt).ˆ2) + ssdu; return %

% Performance index J

The controlled performance as shown in Fig. 10.4 is very good, as indeed it should be. MPC: T =0.75, H =20, H =10 s

p

c

output & setpoint

5 0 −5 −10 4

umax Input

2 0 −2 −4

umin 0

20

40

60 time [s]

80

100

120

Figure 10.4: Predictive control of a non-minimum phase system with constraints on u with a control horizon of Hc = 10 and a prediction horizon of Hp = 20. Refer also to Fig. 10.5 for a zoomed portion of this plot. Especially note that since we are making use of a vector of future setpoints, the control algorithm starts preparing for the setpoint change before the actual setpoint changes. This acausal behaviour, illustrated more clearly in Fig. 10.5, is allowed because we assume we know future setpoints, improves the response compared with a traditional feedback only PID controller. Naturally the control becomes looser as the manipulated variable hits the upper and lower bounds. You will soon realise, if you run this example, that the computational load for the optimal input trajectory is considerable. With such a performance load, one needs to investigate ways to increase the speed of the simulation. Reasonable starting estimates for the new set of future control moves are simply obtained from the previous optimisation problem. Good starting guesses for the nonlinear optimiser are essential to ensure that the controller manages to find a reasonable output in the time available. Even though our model, ignoring the manipulated variable constraints, is linear, we have used a general nonlinear optimiser to solve the control problem. This means that the same algorithm will work in principle for any plant model, whether linear

10.1. MODEL PREDICTIVE CONTROL

481

MPC: T =0.75, H =20, H =10 s

p

c

output & setpoint

5

0

−5 Hp

Hc

Hp

−10 4

Input

2 0 −2 −4

30

35

40 time [s]

45

50

Figure 10.5: A zoomed portion of Fig. 10.4 centered about the setpoint change around t = 46 showing clearly the acausal response possible if we take advantage of the vector of future setpoints.

or nonlinear, and any constraint formulation. Naturally there are many extensions to the basic algorithm that address this issue such as only optimising every n-th sample time, or lowering Nc . However substantial simplifications can be made if we restrict our attention to just linear plants and bounded output, input and input rate constraints. Under these restrictions, the optimal MPC formulation can be re-written as a standard quadratic program, or QP for which there are efficient solution algorithms. Further details of this are given in [26, 165], and a commercial M ATLAB implementation is described in [29].

VARYING

THE CONTROL AND PREDICTION HORIZONS

Investigating different controller design decisions is computationally expensive for this application, but we can relax the termination requirements for the optimiser with little noticeable change in results. From a computational complexity viewpoint, we want to reduce the search space, and hence reduce the horizons. However, reducing the horizons excessively, will result in a poor controlled response. Fig. 10.6 demonstrates what is likely to happen as the control and prediction horizons grow shorter. For a short control horizon, Nc = 2, and a reasonably short prediction horizon, Np = 8, but one that still covers inverse response, the controlled response is still quite reasonable as shown in Fig. 10.6(a). However decreasing the prediction horizon any further, even just slightly such as to Np = 7, gives problems as exhibited in Fig. 10.6(b). The inverse response horizon is ≈ 5. MPC

ON THE LABORATORY BLACKBOX

Fig. 10.7 shows the result of using the constrained MPC on the black box assuming a constant model of the blackbox as 0.98 M: G(s) = (3s + 1)(5s + 1) sampling at T = 0.75 seconds with a prediction horizon of Hp = 20, and control horizon of Hc = 10 samples. Clearly we should also estimate the controller model online perhaps using a recursive least-squares type algorithm.

CHAPTER 10. PREDICTIVE CONTROL

482 GPC: Ts=0.75, Hp= 8, Hc= 2

GPC: Ts=0.75, Hp= 7, Hc= 2 5 output & setpoint

output & setpoint

5 0

−5

0 −5

2

2 Input

−10 4

Input

−10 4

0 −2 −4

0 −2

0

20

40

60 time [s]

80

100

120

−4

0

20

40

60 time [s]

80

100

120

(b) Short control & prediction horizons,Hc = 2, Hp = 7.

(a) Short control & reasonable prediction horizons, Hc = 2, Hp = 8.

Figure 10.6: Varying the horizons of predictive control of a non-minimum phase system with constraints on u. Black−Box: MPC ∆t=0.75, H =20, H =10 p

c

output & setpoint

0.5

0

−0.5

−1 1

input

0.5 0 −0.5 −1

0

20

40

60

80

100 time (s)

120

140

160

180

200

Figure 10.7: Constrained MPC applied to the laboratory blackbox with sampling at T = 0.75 sec and prediction & control horizons Hp = 20, Hc = 10.

10.1.2

D YNAMIC

MATRIX CONTROL

If you actually ran the simulation of the general optimal predictive controller from §10.1.1, and waited for it to finish, you would have stumbled across the one big drawback — computational load. As previously anticipated, one way to quicken the search is to assume a linear system, ignore any constraints, and make use of the analytical optimisation solution. To do this, we must express our model as functions of input only, that is, a finite impulse response model, rather than the more parsimonious recursive infinite impulse response model. There are again many different flavours of this scheme, but the version here is based on the step response model, y(t) =

∞ X i=1

gi ∆u(t − i)

(10.5)

10.1. MODEL PREDICTIVE CONTROL

483

where the gi s are the step response coefficients, and ∆u(t) is the change in input, or ut − ut−1 , refer Fig. 10.8. Suppose we start from rest, and make a unit step change in ∆u at time t = 0. Using Eqn. 10.5, the first few output predictions are y1 = g1 ∆u0 + g2 ∆u−1 + · · · |{z} | {z } =1

(10.6)

=0

y2 = g1 ∆u1 + g2 ∆u0 + g3 ∆u−1 + · · · |{z} |{z} | {z } =0

=1

(10.7)

=0

y3 = g1 ∆u2 + g2 ∆u1 + g3 ∆u0 + · · ·

(10.8)

Since all the control moves changes are zero, except ∆u0 = 1, then it follows that y1 = g1 , y2 = g2 , y3 = g3 and so on. So the values of the output at each sample time yk in response to a unit step at k = 0 are the step response coefficients, gk . These coefficients form an infinite series in general, but for stable systems, converge to a limit as shown in Fig. 10.8. Output, y gNp g6

Impulse coeff. h6 = g 6 − g 5

b b

b

b

⊕b

b

Approximate steady state

b

g4 b b

b

b

0 g1

time, t b

b b

g2

g3

Ts Step response coefficients, gi Input

Past

now

Unit step

Future

Figure 10.8: Step response coefficients, gi , and impulse response coefficients, hi , for a stable system. We can write the response of the model in a compact matrix form as      g1 0 0 ··· 0 y1 ∆u0  g2  g 0 · · · 0 1  y2     ∆u1     g2 g1 ··· 0  ..  =  g3  ..    .  . .. .. .. .. .  ..  . . . . yN p ∆uNc −1 gNp gNp −1 gNp −2 · · · gNp −Nc +1 {z } |

    

dynamic matrix

yforced = G ∆u

(10.9)

The Np × Nc matrix G in Eqn. 10.9 is referred to as the dynamic matrix, hence the term dynamic matrix control or DMC. It is also closely related to a Hankel matrix which provides us with a convenient, but not particularly efficient, way to generate this matrix in M ATLAB. Eqn. 10.9 tells us how our system will respond to changes in future moves in the manipulated variable. Specifically Eqn. 10.9 will predict over the prediction horizon, Np , the effect of Nc control moves. As in

CHAPTER 10. PREDICTIVE CONTROL

484

the general optimisation case, the input is assumed constant, ∆u = 0, past the end of the control horizon. All we need to do for our optimal controller, is to find out what to set these future control moves to, in order that the future outputs are close to the desired setpoints. Actually Eqn. 10.9 only predicts how our system will behave given future control changes, (termed the forced response), but it ignores the ongoing effect of any previous moves, (termed the free response). We must add the effect of the free response, if any, to Eqn. 10.9 to obtain a valid model prediction. This simple addition of two separate responses is a consequence of the linear superposition principle applicable for linear systems. We can calculate the free response over the prediction horizon, yfree simply by integrating the model from the current state with the current constant input for Np steps. Therefore to ensure our future response is equal to the setpoint, r, at all sample times over the prediction horizon, we require r = yfree + yforced (10.10) Again, since we will make the requirement that the control horizon is shorter than the prediction horizon, our dynamic matrix, G, is non-square, so the solution of Eqn. 10.9 for the unknown control moves involves a pseudo inverse. Typically we further weight the control moves to prevent an over-excited input demands by a positive weight λ, so the optimal solution to the predictive control problem is  −1 ∆u = G⊤ G + λI G⊤ (r − yfree ) (10.11) This analytical solution to the optimal control problem is far preferred above the numerical search procedure required for the general nonlinear case. Note that the matrix inversion in Eqn. 10.11 is only done once, offline and thus does not constitute a severe online numerical problem. The effect of the control move weighting improves the numerical conditioning of the matrix inversion. Fig. 10.9 shows how this control philosophy works for a plant with a prediction horizon of 10 and a control horizon of 4. The output is intended to follow the setpoint, and in this case the setpoint change at k = 15 is known in advance and that information is available to the controller. (There is also a disturbance at k = 19 which is not known to the controller.) A small amount of noise is added to the output to continually stress the controller. The control law, Eqn. 10.11, generates 4 control moves in the future, but implements only the first. The grey trends in both the output and the input show at each sample time what the controller is thinking will happen in the future. Fig. 10.9(b) shows a zoomed portion of Fig. 10.9(a) just prior to the step change. The four future predictions are plotted, getting progressively lighter in colour further into the future.

A DYNAMIC

MATRIX CONTROL EXAMPLE

Fig. 10.10 shows the performance of a DMC controller for the non-minimum phase plant of Eqn. 10.4 from §10.1.1. I generated the step response coefficients elegantly from the transfer function model simply using the dstep command. For illustration purposes in this simulation I have varied the control move weighting, λ in Eqn. 10.11 as λ = 100 for t < 50, then λ = 1 for the interval 50 < t < 100, and then finally λ = 0. The controller uses a prediction horizon of Hp = 25, a control horizon of Hc = 8 at a sample time of T = 0.5. The manipulated variable constraints present in §10.1.1 are relaxed for this example. As the control move weighting is decreased, the output performance improves at the expense of vigorous manipulated variable movements. Note that a weighting λ = 0 is allowable, but will result in an over-excited controller. The m-file for this DMC example, but using a constant weight of λ = 1 is given in Listing 10.3.

10.1. MODEL PREDICTIVE CONTROL

485

Ts= 1, Hp=10, Hc= 4, λ=10.0 1

Y & spt

0.8 0.6 0.4 0.2 Disturbance here

0 −0.2 1

U

0.5

0 0

5

10

15

20 25 sample interval

30

35

40

(a) Dynamic matrix control showing the predictions for a step change and a disturbance. Ts= 1, Hp=10, Hc= 4, λ=10.0 0.025 0.02 0.015

yk & spt

0.01 0.005 0 −0.005 −0.01 −0.015 1 0.8

uk

0.6 0.4 0.2 0 −0.2

5

6

7

8 9 sample interval

10

11

12

(b) A zoomed portion of the DMC control showing the future predictions at each sample interval getting lighter in colour as the prediction gets longer.

Figure 10.9: Dynamic matrix predictive control showing at each sample time the future control moves and the resultant prediction of the plant’s output given those anticipated control moves. For example at each sample, the controller generates the next 4 control moves, but actually only implements the first.

CHAPTER 10. PREDICTIVE CONTROL

486

T =0.5, H =25, H = 8 s

p

c

1

Y & spt

0.5

0

−0.5

−1 1

U

0.5 0 −0.5 −1

λ = 100 0

λ = 20 50

λ = 0.1 100

150

sample time

Figure 10.10: DMC predictive control of a non-minimum phase system with Hc = 8, Hp = 25 showing the effect of varying the control move weighting λ. Listing 10.3: Dynamic Matric Control (DMC) control

4

ntot=250; dt=0.5; t=dt*[0:ntot-1]';% time vector r = [1,3,-1,-2.5,3.5,5.4,-4,-6,5,2.2]; % random square wave setpoint Yspt = ones(size(t(1:fix(length(t)/length(r))+1)))*r(:)'; Yspt = Yspt(1:length(t))'; % Cut off excess data points

9

Kp=1.5;taup=2.5; tau = 2.0; zeta = 0.45; % process model Gpc = tf(Kp*[-taup 1],[tauˆ2 2*tau*zeta 1]); % Kp (−τp s + 1)/(τ 2 s2 + 2ζτ s + 1) Gp = c2d(Gpc,dt); % discrete plant [Phi,Del,C,D] = ssdata(Gp); % state space for x0 Hc = 8; % Control horizon, Hc Hp = 25; % Prediction horizon, Hp > Hc

14

19

24

[g,ts] = step(Gp,dt*Hp); % step response coefficients g(1) = []; % only consider Hp points (ignore first zero) % Formulate DMC G matrix & inverse once; (offline) Hankel version G = flipud(hankel(flipud(g))); G(:,Hc+1:end) = []; weight=1; % control move weighting w ≥ 0 GG = G'*G; % G⊤ G
−1 ⊤ invG = (GG + weight*eye(size(GG)))\G'; % G⊤ G + λI G x = [0,0]'; y=C*x; u=0; p=0; % cheat here but Ok Uopt=zeros(size(t)); Y=zeros(size(t)); % initialise u(t) for i=1:ntot-Hp-1; du = invG*(Yspt(i:i+Hp-1)-p); % analytical optimisation u = du(1)+u ; % only take first change x = Phi*x + Del*u; y = C*x; % apply 1 control move

29

% Calculate future free response (assume all future du=0)

10.2. A MODEL PREDICTIVE CONTROL TOOLBOX

487

[p,px] = dlsim(Phi,Del,C,D,u*ones(Hp+1,1),x); p(1)=[]; % kill repeat of x0 34

39

Y(i,:) = y; Uopt(i,:) = u; end % for i Yspt(1) = []; Yspt=[Yspt;1]; subplot(2,1,1); plot(t,Y,t,Yspt,':') subplot(2,1,2); stairs(t,Uopt);

% plot results

The high fidelity of the controlled response of both predictive control algorithms, (Figures 10.4 and 10.10) is helped by the lack of model/plant mismatch in the previous simulations. In practice, faced with the inevitable mismatch, one can extend the basic DMC algorithm such as varying the weights and/or horizon lengths or by including an adaptive model and therefore performing the matrix inversion online. In some cases, particularly for unstable processes, or for cases where the sampling rate is either very fast or very slow compare with the step response of the system, the DMC algorithm will fail primarily owing to the poor conditioning of G. In these cases singular value decomposition may help.

DMC

OF THE BLACKBOX

A DMC controller is tried on the laboratory blackbox in Fig. 10.11. Unlike the GPC example on 482 however, here we will identify the model online using a recursive least-squares estimation algorithm. The model-based controller assumes the structure M:

G(q −1 ) =

b0 + b1 q −1 + b2 q −2 1 + a1 q −1 + a2 q −2 + a3 q −3

for the blackbox with a sample time of T = 1.5 seconds and a forgetting factor of λ = 0.995. The controller horizon is set to 8 samples, and the prediction horizon is set to 20 samples. The control move weighting is set to 0.2.

10.2

A M ODEL P REDICTIVE C ONTROL T OOLBOX

A Model Predictive Control toolbox developed by Currie, [58, 59] allows one to rapidly build and test various MPC configurations for a wide variety of linear, or linearisable nonlinear plants with both soft and hard constraints on the output, the input and input rate of change. The following sections describe this toolbox in more detail.

10.2.1

A

MODEL PREDICTIVE CONTROL

GUI

The easiest way to get started is to use the Model Predictive Control toolbox GUI as shown in Fig. 10.12. In this M ATLAB GUI, one can load and experiment with different plants (and models of those plants), adjust prediction horizons, operating constraints, and setpoints. What is interesting to observe in the GUI application are animations of the future output and input predictions given to the right of the solid line in the scrolling plot.

CHAPTER 10. PREDICTIVE CONTROL

488 0.8

DMC of the blackbox ∆t=1.50, Hp=25, Hc= 8, λ=0.20

0.6

output & setpoint

0.4 0.2 0 −0.2 −0.4 −0.6 −0.8 1 RLS Model ID: na= 3, nb= 2

Input

0.5 0 −0.5 −1

0

50

100

150 time (s)

200

250

300

Figure 10.11: Adaptive dynamic matrix control (DMC) of the blackbox with control horizon Hc = 8 and prediction horizon Hp = 25.

10.2.2

MPC

TOOLBOX IN

M ATLAB

The MPC toolbox is easy to use in script mode. This section shows how to build a multivariable MPC controller using the Wood-Berry column plant in Eqn. 3.24 from section 3.2.4. This plant has 2 inputs (reflux and steam) and two outputs (distillate and bottoms concentration) and we will use a sampling time of Ts = 2 seconds. For this application we will assume no model/plant mismatch.

Listing 10.4: Setting up an MPC controller 1

6

G = tf({12.8, -18.9; 6.6 -19.4}, ... { [16.7 1],[21 1]; [10.9 1],[14.4 1]}, ... 'ioDelayMatrix',[1,3; 7,3]) % The Wood-Berry column plant from Eqn. 3.24 Ts = 2; Gd = c2d(G,Ts); % Discretise the plant at Ts = 2 seconds Plant = jSS(Gd); % Convert the LTI model to a special form required for the MPC controller Model = Plant; % Assume (for the moment) no model-plant mismatch

Now we are ready to specify the controller tuning parameters which include the length of the control and prediction horizons, the values of the input and output saturation constraints, and the relative input and output weights in the objective function.

10.2. A MODEL PREDICTIVE CONTROL TOOLBOX

489

Figure 10.12: An MPC graphical user interface

Np = 10; % Prediction Horizon Nc = 5; % Control Horizon 3

con.u = [-1 -1 con.y = [-2 -2

1 0.1; ... % Input constraints: umin , umax , ∆umax 1 0.1]; % One row for each input 2; % Output constraints: ymin , ymax 2] ;

8

uwt = [1 1]'; % Individual weighting on the inputs ywt = [1 0.1]'; % We are less interested in controlling bottoms, y2 Kest = dlqe(Model); %Estimator Gain

Now we are ready to initialise the model (and plant). To that, we need to know how many internals states we have in our discrete model. That number depends on the discretisation method, and the sampling rate, so the easiest way is to inspect the properties of the plant and model.

3

>> Plant Plant = -- jMPC State Space Object -States: 10 Inputs: 2 Outputs: 2 Sampling Time: 2 sec

8

>> Plant.x0 = zeros(10,1); % Initial condition of the 10 states >> Model.x0 = zeros(10,1);

CHAPTER 10. PREDICTIVE CONTROL

490

Finally we are ready to design a setpoint trajectory, assemble the MPC controller, and simulate the results. In this case we are using the high-speed MEX version of the optimisation routines. T = 200; % Desired simulation length setp = 0.5*square(0.05*Ts*[0:T]',50)+0.7; % Design interesting setpoint setp(:,2) = 0.5*square(0.1*Ts*[0:T]',50)-0.7; setp(1:10,:) = 0; 5

MPC1 = jMPC(Model,Np,Nc,uwt,ywt,con,Kest); % Construct an MPC controller simopts = jSIM(MPC1,Plant,T,setp); % Set some simulation options

10

simresult = sim(MPC1,simopts,'Mex'); % Simulate & plot results plot(MPC1,simresult,'summary'); suptitle(sprintf('Horizons: N_p=%2i, N_c=%2i',Np,Nc))

The results for this multivariable MPC example are shown in Fig. 10.13. Note how the control of y2 is considerably looser than that for y1 . This is a consequence of the output weights which are ten times higher for y1 . Also note that the change in input ∆u never exceeds the required ±0.1. Horizons: N =10, N = 5 p

c

Outputs: yp(k)

States: xp(k) 6 4 Amplitude

Amplitude

1

0

2 0

−1 0

50

100

150

−2

200

0

50

0.1

0.4

0.05 Amplitude

Amplitude

150

200

150

200

Change in Input: ∆ u(k)

Input: u(k) 0.6

0.2 0 −0.2

100

0 −0.05

0

50

100 Sample

150

−0.1

200

0

50

100 Sample

Figure 10.13: Multivariable MPC of the Wood-Berry column.

10.2.3

U SING

THE

MPC

TOOLBOX IN

S IMULINK

It is also easy to implement MPC controllers in S IMULINK. Fig. 10.14 shows an MPC controller from the jMPC toolbox connected to an unstable, non-minimum phase plant G(s) =

s−1 (s − 2)(s + 1)

(10.12)

which is difficult to control and has been used in various benchmark control studies, [102]. To use the MPC controller block in S IMULINK, we first must design an MPC controller in the same way as we did in section 10.2.2. Once we have designed the controller, we can simply insert it into the discrete MPC controller block and connect it to our plant. However in this case instead of

10.2. A MODEL PREDICTIVE CONTROL TOOLBOX

u

mdist

491

(s−1)

double

Plant

delu

double

double

(s−2)(s+1) Scope

setp ym

Signal Generator

yp

xm Discrete MPC Controller

Figure 10.14: The MPC block in the S IMULINK environment. using the true plant inside the model-based MPC controller, we will inadvertently use the slightly different model ˆ = 0.95(s − 1) (10.13) G (s − 2.1)(s + 1) thereby introducing some model/plant mismatch. Model = jSS(zpk([1],[2.1 -1],0.95));% Mismatch: Compare Eqn. 10.12 with Eqn. 10.13. Ts = 0.1; Model = c2d(Model,Ts); % Discretize Model 4

Np = 40; Nc = 5; con.u = [-5 5 0.3]; con.y = [-2 2]; uwt = 1; ywt = 0.5; Kest = dlqe(Model);

% Prediction & control Horizons % Input & output constraints % input & output weightings

9

14

T = 800; setp = square(0.2*Ts*[0:T]',50); setp(1:10,:) = 0; Model.x0 = zeros(2,1); % Initial values

% Setpoint

MPC1 = jMPC(Model,Np,Nc,uwt,ywt,con,Kest);

% Build the MPC controller

The last line created an MPC controller object called MPC1 which we can place in the MPC controller block in the S IMULINK diagram in Fig. 10.14. Note that the true plant, Eqn. 10.12 is never actually used in the MPC controller. The results of the subsequent S IMULINK simulation in Fig. 10.15 show that we hit, but do not exceed, the input rate constraint during the transients.

10.2.4

MPC CONTROL OF AN UNSTABLE 3 DEGREE OF FREEDOM HELICOPTER

The 3-DOF helicopter manufactured by Quanser1 shown in Fig. 10.16 is a popular test rig for testing multivariable controllers, [57]. The helicopter can be modelled using 6 states, def

xT =



ǫ

p λ

ǫ˙ p˙

λ˙



which are the angles (measured in radians) of the elevation, ǫ, the pitch, p, and the travel, λ, from a nominal zero position, and their corresponding rates of change. The physical limitations of the elevation angle are about ±50◦ , while the pitch is limited to about ±100◦ . The travel angle 1 www.quanser.com

CHAPTER 10. PREDICTIVE CONTROL

492 2

y

1 0 −1 −2 4

u

2 0 −2

∆u

−4 0.5

Figure 10.15: MPC of a SISO non-minimum phase unstable plant with model/plant mismatch from Fig. 10.14.

0

−0.5

0

10

20

30

40

50

60

70

time

is unlimited due to the slip rings on the central shaft. These state constraints provide a good incentive to use MPC control. ˙ must be The angles are directly measured via a rotary encoder, but the time derivatives, (ǫ, ˙ p, ˙ λ) estimated either using a state estimator, or some sort of carefully designed differentiator. The two control inputs are the two independent voltages applied to the two motors driving the front and def

back rotors, uT = [Vf , Vb ].

Figure 10.16: A 3 degree of freedom helicopter manufactured by Quanser. The nonlinear equations of motion for the helicopter are as follows:   Tg K f la (Vf + Vb ) cos(p) − ǫ= ¨ Je Je   K f lh (Vf − Vb ) p¨ = Jp   K f la ¨ λ=− (Vf + Vb ) sin(p) Jt

(10.14) (10.15) (10.16)

with the model parameters either directly measurable from the equipment, or alternatively available from the Quanser documentation. Note that the sin(·) and cos(·) terms introduce both nonlinearities and cross-coupling to the model and that the double integrators for all three states mean that this system is ‘triply’ unstable.

10.2. A MODEL PREDICTIVE CONTROL TOOLBOX

493

Fig. 10.17 shows how we would control the 3DOF helicopter using an MPC controller. The plant block in Fig. 10.17(a) expanded in Fig. 10.17(b) contains the the dynamic equations given above, or we could directly control the actual helicopter. The former allows us to test the controller in a simulation mode, the latter allows us to actually control some external hardware provided we have suitable external connections.

(a) MPC control of a Helicopter in S IMULINK

(b) The internals of the nonlinear Helicopter plant above following Eqns 10.14–10.16.

Figure 10.17: MPC control of a helicopter.

In order to build a MPC controller of helicopter, the model equations, Eqns (10.14)–(10.16) are entered into M ATLAB as a normal ODE function file as shown in Listing 10.5. We will use this nonlinear model as the base for the MPC controller design. Listing 10.5: Nonlinear dynamics of a helicopter 1

function xdot = heli_nlQ(t,x,u,param) % Nonlinear helicopter dynamics following Eqns 10.14–10.16 [Je,La,Kf,Fg,Tg,Jp,Lh,Jt,m_f,m_b,m_w,Lw,g] = param{:}; %Assign Parameters

6

xdot(1,1) = x(4); % elevation velocity

CHAPTER 10. PREDICTIVE CONTROL

494 xdot(2,1) = x(5); % pitch velocity xdot(3,1) = x(6); % rotation (travel) velocity

11

xdot(4,1) = (Kf*La/Je)*(u(1)+u(2))*cos(x(2))-Tg/Je; % ǫ¨, Eqn. 10.14 xdot(5,1) = (Kf*Lh/Jp)*(u(1)-u(2)); % p¨, Eqn. 10.15 ¨ Eqn. 10.16 xdot(6,1) = -(Kf*La/Jt)*(u(1)+u(2))*sin(x(2)); % λ, return

We can either use this nonlinear model as the basis of the MPC design (as shown in the listing below), or we can deliberately use a different model, perhaps to test some model/plant mismatch.

2

[Kf,m_h,m_w,m_f,m_b,Lh,La,Lw, ... g,K_EC_T,K_EC_P,K_EC_E,Je,Jt,Jp,Fg,Tg] = setup_constants(); C = eye(3,6); % Output Matrix, C. We only measure the angles, not the rates of change Vop = g*(Lw*m_w - La*m_f - La*m_b)/(2*La*Kf); % Operating Point

7

param = {Je,La,Kf,Fg,Tg,Jp,Lh,Jt,m_f,m_b,m_w,Lw,g}; %parameter cell array Plant = jNL(@heli_nlQ,C,param); % See Listing 10.5 for the nonlinear ODEs u0 = [Vop;Vop];

Now we need to linearise the plant about some operating point in order to build the linear internal model that will be used in the MPC controller. 1

6

Model = linearize(Plant,u0,zeros(6,1)); % Automatically formulate a linearized model UD = Model.UserData; Ts = 0.04; % Discretize model at Ts & add 2 samples of deadtime set(Model,'iodelay',2*Ts); Model = c2d(Model,Ts); Model.UserData = UD; Model = jSS(Model); %Build jSS object

Now that we have a model, we need to specify the MPC tuning options such as control and prediction horizons, the input and output constraints, the various control weightings, and the design of the estimator. 1

6

Np = 45; Nc = 5; % Prediction, Np , & control, Nc , horizons con=[]; pitch = 30*pi/180; con.u = [-20 20 10; -20 20 10]; con.y = [-Inf Inf; -pitch pitch; -Inf Inf];

%Constraints

con.slack = [Inf 5e3 Inf];

% Heavy penalty for soft constraints

% Or could limit elevation to ±27.5◦ % Limit pitch to ±30◦ % Or could limit rotation to ±180◦ degrees

11

16

uwt = [1 1]'; %Input & output Weighting ywt = [10 5 15]'; %Estimator Gain Q = 2*eye(10); R = 0.1*eye(3); Q(4,4) = 1; Q(5,5) = 1; Q(6,6) = 0.1; Kest = dlqe(Model,Q,R);

10.2. A MODEL PREDICTIVE CONTROL TOOLBOX

495

Finally we are ready to construct the MPC controller. It might be useful to include descriptive variable names and units.

4

opts=jMPCset('InitialU',u0,'QPMaxIter',20,... 'InputNames',{'Motor 1 Input','Motor 2 Input'},... 'InputUnits',{'V','V'},... 'OutputNames',{'Elevation Angle','Pitch Angle','Rotation Angle'},... 'OutputUnits',{'rad','rad','rad'}); % Build MPC controller object & place in the S IMULINK block in Fig. 10.17(a). MPC1 = jMPC(Model,Np,Nc,uwt,ywt,con,Kest,opts);

The control object, MPC1 in the above listing, can now be inserted into the S IMULINK MPC control block in Fig. 10.17(a). We are now ready to test out MPC controller. The resultant MPC control given in Fig. 10.18 shows the results of an extremely large travel setpoint change of 400◦, over a full circle. Notice how the MPC controller does violate the (soft) state constraints at times, but does eventually manage to bring the helicopter under control. For comparison, a linear controller such as LQR/LQG would not be able to maintain stability for such a large setpoint change.

elevation [deg]

10 0 −10 −20

pitch [deg]

−30 40 20 0 −20

travel [deg]

0 −200 −400 −600

Motor voltages

20 10 0 −10 −20 0

10.2.5

5

10

F URTHER

15

20 time [s]

25

READINGS ON

30

35

40

Figure 10.18: MPC control of a helicopter for an extremely large travel setpoint change of over 1 full rotation.

MPC

Model predictive control is now sufficiently mature that it has appeared in numerous texts, [134, 174, 199], all of which give an overview of the underlying theory, and insight into the industrial practice. Industrial applications using commercial implementations such as Pavilion Technologies’ Pavilion8, Honeywell’s RMPCT or Perceptive Engineering have been surveyed in [162]. A commercial MPC toolbox for M ATLAB is available in [29] while freeware versions also exist from [58] and [4].

CHAPTER 10. PREDICTIVE CONTROL

496

10.3

O PTIMAL CONTROL USING LINEAR PROGRAMMING

Optimal control problems quickly demand excessive computation, which all but eliminates them from practical real-time applications. If however our model is a linear discrete-time state-space model, our performance objective is to minimise the sum of the absolute value, rather than the more normal least-squares, and we have linear constraints on the manipulated variables, then the general constrained optimisation problem becomes a linear program. Linear programs or LPs are much more attractive to solve than a general constrained nonlinear optimisation problem of comparable complexity because, in theory, they can be solved in a finite number of operations, and in practice they are far more successful at solving problems with more than 50 variables and 100 constraints on modest hardware. The theory of linear programming is well known, and if the problem is poorly conditioned, or not ‘well-posed’, then the software will quickly recognise this, ensuring that we do not waste time on impossible, or poorly thought out problems. There are many computer codes available to solve linear programs, including various routines in O PTI or linprog from the O PTIMIZATION T OOLBOX. An introduction to linear programming with emphasis on the numerical issues with M ATLAB is given in [204], and an experimental application is given in [148] from which this section was adapted. As in the predictive control development, we want a controlled response that minimises the error between setpoint, r, and the actual state, say something like Eqn. 9.11 Z ∞ f (x, r) J = 0

where we could either use the absolute value of the error, l1 norm, or perhaps minimise the maximum error, l∞ norm. In both cases these problems can be converted to a convex linear program. We will also consider only a finite time horizon, and since the model is discrete, we will use a simple sum in place of the integral. The optimisation problem is then to choose the set of future manipulated variables uk over the time horizon such that the performance index J =

N X n X

k=0 i=1

|ri − xi |

(10.17)

is minimised subject to the process model. In Eqn. 10.17, n is the number of states, and N is the number of samples in the time horizon, (typically around 10–20). If desired, Eqn. 10.17 could be further modified by adding state weighting if some states are more important than others.

10.3.1

D EVELOPMENT

OF THE

LP

PROBLEM

The hardest part of this optimal controller is formulating the LP problem. Once that is done, it can be exported to any number of standard numerical LP software packages to be solved. We will use a discrete state model for the predictive part of the controller. xk+1 = Φxk + ∆uk

(10.18)

Unwinding the recursion in Eqn. 10.18, we can calculate any subsequent state, xk given the manipulated variable history, and the stating condition, x0 , (refer Eqn. 2.88, xk = Φk x0 + Φk−1 ∆u0 + Φk−2 ∆2 u1 + · · · + ∆uk−1

(10.19)

If rk is the setpoint at sample time k, then the error is rk − xk = rk − Φk x0 − Φk−1 ∆u0 − · · · − ∆uk−1

(10.20)

10.3. OPTIMAL CONTROL USING LINEAR PROGRAMMING

497

We desire to minimise the sum of the absolute values of these errors, as given in Eqn. 10.17, but the absolute value sign is difficult to incorporate in a linear programming format. We can instead introduce two new non-negative variables, ǫ1i := ri − xi ǫ2i := 0



if

ri ≥ xi

(10.21)

ǫ1i := 0 ǫ2i := xi − ri



if

xi ≥ ri

(10.22)

otherwise

Note that we could write the above four relations in M ATLAB without the IF statement as something like:

2

e1 = abs(r-x)*round(sign(r-x)/2 + 0.5) e2 = abs(x-r)*round(sign(x-r)/2 + 0.5)

for scalar x and r. Using definitions Eqns 10.21–10.22, the absolute value is |ri − xi | = ǫ1i + ǫ2i

(10.23)

ri − xi = ǫ1i − ǫ2i

(10.24)

and the error is

Both relations are valid at all times because only one of either ǫ1 or ǫ2 is non-zero for any particular state at any particular time. Re-writing the objective function, Eqn. 10.17 in terms of the new variables using Eqn. 10.23 gives the linear relation without the troublesome absolute values. J =

N X n X

(ǫ1i + ǫ2i )k

(10.25)

k=0 i=1

The penalty paid is that we have doubled the number of decision variables, thereby doubling the optimisation space. Eqn. 10.25 is a linear objective function suitable for a linear program. The constraints arise from the model dynamics which must be obeyed. Substituting Eqn. 10.24 into Eqn. 10.20 gives N blocks of equality constraints, each with n equations. To make things clearer, the first few blocks are written explicitly out below. ǫ11 − ǫ21 +∆u0 = r1 − Φx0 | {z } r1 −x1 ǫ12 − ǫ22 +Φ2 ∆u0 + ∆u1 = r2 − Φ2 x0 | {z } r2 −x2 .. . ǫ − ǫ +Φk−1 ∆u0 + Φk−2 ∆u1 + · · · + ∆uk−1 = rk − Φk x0 | 1k {z 2k} rk −xk

CHAPTER 10. PREDICTIVE CONTROL

498 Naturally this is best written in a matrix form.



u0



   u  1      .   .   .     u    N −1     ∆ 0 · · · 0 I −I r1 − Φx0 ǫ11       2    r ∆ ··· 0 I −I  Φ∆   2 − Φ x0    ǫ21  = ..     .. .. .. .. .    . . . 0 .    ǫ12  r − ΦN x0 N    ΦN −1 ∆ ΦN −2 ∆ · · · ∆ I −I   ǫ22       ..   .      ǫ  1N    ǫ2N

    

(10.26)

The dimensions of the partitions in this large constraint matrix are given in Fig. 10.19 where N is the number of future sample horizons, n is the number of states, and m is the number of manipulated variables.

n

    

Nn   

m z }| { ∆ Φ∆ .. .

0 ∆ .. .

ΦN −1 ∆ ΦN −2 ∆ | {z

Nm

··· ··· .. .

0 0 .. .

···

∆

n z }| { I

−I

}|

I

−I

..

.

{z

2N n

I

n

−I }

Figure 10.19: LP constraint matrix dimensions The constraint matrix in Fig. 10.19 is a N n × (N m + 2N n) matrix, the decision variable vector has N m manipulated variables and 2N n auxiliary variables comprising of (N m + 2N n) × 1 vector, and the right-hand side is a N n × 1 vector. In general applications it is necessary for the control variable, u, to be unrestricted in sign, which is counter to the formulation required in most linear programming packages. To achieve this we, must introduce two further new variables, both non-negative, for each manipulated variable. u := s − t,

s, t ≥ 0

(10.27)

If the unrestricted u > 0, then t = 0, otherwise u is negative and s = 0. This substitution modifies

10.3. OPTIMAL CONTROL USING LINEAR PROGRAMMING

499

the A matrix in the constraint equation, Eqn. 10.26, to        

Φ

∆

0

··· 0

−∆

0

···

0

Φ∆

∆ .. .

−Φ∆

−∆

···

0

.. .

··· 0

.

0

N −1

N −2

∆Φ

..

.. .

. 0

N −1

∆ · · · ∆ −Φ

.. .

N −2

∆ −Φ

..



I −I I −I

..

∆ · · · −∆

. I −I

      

and the decision variables are now h

s0

s1

.. . t0

· · · sN −1

t1

· · · tN −1

.. .

· · · ǫ2N

ǫ11

i⊤

Using this ordering of decision variables, the objective function, Eqn. 10.25, is the sum of the ǫs. In linear programming, the objective function is written as J = c⊤ x, where c is termed the cost vector, and x is the decision variable vector to be optimised. In our case the cost vector is given as c=

"

0 0 |

··· 0 1 {z } |

2N m

1

1 1 {z

··· 1

2N n

}

#⊤

(10.28)

Note that the manipulated variables do not explicitly enter the objective function. Naturally the ones in Eqn. 10.28 could be replaced by different positive individual time and state weightings, if so desired. This completes the formal development of the LP as far as the unconstrained optimum controller is concerned. However since we are already solving a constrained optimisation problem, there is little overhead in adding extra constraints such as controller upper and lower bounds to the problem. In practice the manipulated space is limited, (a control valve can only open so far, an electric drive is limited in top speed), so we are interested in constraints of the form ulower ≤ u ≤ uupper

(10.29)

These constraints can be appended to the other constraint equations.

S IMPLIFICATIONS

WHEN USING THE OPTIMISATION TOOLBOX

When using linprog.m from the O PTIMIZATION T OOLBOX, certain simplifications are possible to the standard LP problem form. These simplifications reduce the problem size, and therefore also reduce the solution time. Since the algorithm does not make the non-negative constraint assumption, we do not need to introduce the variable transformation given by Eqn. 10.27, and suffer the associated growth of the constraint matrix. This is not the case in general, if we were to use other LP routines such as described in [204] or [78], then we would need to use non-negative variables. Most LP programs assume the constraints are given as Ax ≤ b, i.e. less-than inequalities rather than equalities as in Eqn. 10.26. While the conversion is trivial, it again increases the number of constraint equations. However the routine linprog conveniently allows the user to specify both equality and inequality constraints. The third simplification is to exploit how linprog handles constant upper and lower bounds such as needed in Eqn. 10.29. These are not added to the general constraint matrix, Ax ≤ b, but are supplied separately to the routine. All these modifications reduce the computation required to solve the problem.

CHAPTER 10. PREDICTIVE CONTROL

500 O PTIMAL

CONTROL USING LINEAR PROGRAMMING

Listing 10.6 demonstrates a finite time-horizon optimal controller with a randomly generated stable multiple-input/multiple output (MIMO) model. The optimisation problem is solved using the linear program linprog from the O PTIMISATION T OOLBOX. Listing 10.6: Optimal control using linear programming n = 2; m = 2; % # of states & manipulated variables [Ac,Bc,C,D]=rmodel(n,1,m); % Some arbitrary random model for testing 3

8

13

18

Gc = ss(Ac,Bc,C,D); dt = min(1. ./abs(eig(Ac)))/0.5; % sparse sample time reasonable guess G = c2d(Gc,dt,'zoh'); G.d = zeros(size(G.d)); % clear this if rank(ctrb(G)) 6= n error('System not controllable') end % if N = 20; % future optimisation horizon x0 = randn(n,1); % random start point r = [ones(floor(N/2),1)*randn(1,n); ... ones(ceil(N/2),1)*randn(1,n)]; % setpoint change % start construction of the LP matrix Phin = G.b; Aeq = G.b; % better for when m > 1 for i=2:N Phin = G.a*Phin; Aeq = [Aeq; Phin]; % rough method end

23

Z = zeros(size(G.b)); % padding for i=2:N Aeq = [Aeq, [Z;Aeq(1:(N-1)*n,(i-2)*m+1:(i-1)*m) ]]; end % for i 28

33

38

% weighting: Could use blkdiag here (but messy) W = eye(n); % change if want different state weighting A2 = zeros(N*n,2*N*n); % initialise for i=1:N A2((i-1)*n+1:i*n,2*(i-1)*n+1:2*i*n) = [W, -W]; end % for Aeq = [A2,Aeq]; % put +ve variables before general ± variables % Now right-hand side ..... U = zeros(N+1,m); % unforced, therefore zero input (drop start term) [yuf,t,xuf]=lsim(G,U,[],x0); % simulate unforced response xuf(1,:) = []; % drop start term since dlsim repeats it. b = r - xuf; % error b = b'; beq = b(:); % columnerise

43

% Now objective function [nr,nc] = size(Aeq); % current size c = zeros(1,nc); % row vector for the moment c(1:2*N*n) = 1; % unity weights for the moment ! 48

% Set up A*x < b (or -Ax > 0) b = zeros(2*N*n,1); % All #s are positive A = zeros(2*N*n,nc)

10.3. OPTIMAL CONTROL USING LINEAR PROGRAMMING

501

A(:,1:2*N*n) = -eye(2*N*n); 53

58

63

tic; % now ready to solve LP .................. ulow = -13; uhigh = 14; % upper/lower constraints xlower = [zeros(2*N*n,1); ulow*ones(N*m,1)]; xupper = [1e6*ones(2*N*n,1); uhigh*ones(N*m,1)]; xsol=linprog(c',A,b,Aeq,beq,xlower,xupper); % first 2Nn are +ve toc % extract solution, if found U = xsol(2*N*n+1:end); % must reshape if m= 6 1 U = reshape(U,m,length(U)/m)'; % MI system [y,t,xopt] = lsim(G,U,[],x0); % Now test system response %k = [0:length(xopt)-1]'; [ts,rs] = stairs(t,r); [ts,Us] = stairs(t,U);

68

73

78

83

subplot(2,1,2); plot(ts,Us,t,ones(size(t))*[ulow, uhigh],'--'); h = line(ts,Us); set(h,'LineWidth',2); %set(gca,'YLim',[-4,4.5]) xlabel('sample #'); ylabel('Manipulated') % Sub-sample output response nsub = 500; % # of sub-samples total (approx 500) ts2 = ts(:,1) + [1:length(ts)]'/1e10; ts2(1) = 0; % destroy monotonicity t2 = linspace(0,t(end),nsub)'; U2 = interp1(ts2,Us,t2); [y,t2,xoptsc] = lsim(Gc,U2,t2,x0); subplot(2,1,1); plot(ts(:,1),rs,':',t2,xoptsc); h = line(t,xopt); set(h,'Marker','.','MarkerSize',24,'LineStyle','none'); ylabel('state'); title('LP optimal control')

You should run the simulation a number of times with different models to see the behaviour of the optimal controller. One ideal 2-input/2 state case is given in Fig. 10.20. Here the states reach the setpoint in two sample times, and remain there. The manipulated variables remain inside the allowable upper (4) and lower (−3) bounds. It is clear that the trajectory is optimal, since the sum of the absolute error at the sample points is zero. One cannot do better than that. In another case, the manipulated bounds may be active and consequently the setpoint may not be reached in the minimum time, and our objective function will not reach zero. In Fig. 10.21, both steady-states are reachable, but owing to the upper and lower bounds on u, they cannot be met in just two sample times. In other words we have hit the manipulated variable constraints during the transients. Our system does not need to be square, Fig. 10.22 shows the results for a 3-state, 2 input problem. In this case, one state is left ‘floating’. Some small inter-sample ripple is also evident in this case. Note that since this is a predictive controller, and it knows about future setpoint changes, it anticipates the changes and starts moving the states before the setpoint change. Fig. 10.23 shows the case for a 2 × 3 system where owing to the active upper and lower bounds on u, the controller knows that it cannot reach the setpoint in one sample time, hence it chooses a trajectory that will minimise the absolute error and in doing so, starts the states moving before the setpoint.

CHAPTER 10. PREDICTIVE CONTROL

502

LP optimal control 3

state 1

2 1 0 −1 0

state 2

−0.2 −0.4 −0.6 −0.8

Manipulated

−1 20 10 0 −10 −20

0

1

2

3 sample #

4

5

6

Figure 10.20: LP optimal control with no active manipulated variable bounds. Note the “perfect” controlled performance. We hit the setpoint in one sample time, and stay there. One cannot do much better than that within the limitations of discrete control. The plot was generated by Listing 10.6.

LP optimal control 2

state

1 0 −1 −2 0

1

2

3

1

2

3

4

5

6

7

8

9

6

7

8

9

Manipulated

4

Figure 10.21: LP optimal control with active manipulated variable constraints. During transients, the manipulated variables hit the upper constraints, so compromising the controlled performance for a short period.

2 0 −2 −4 0

4 5 sample time

10.3. OPTIMAL CONTROL USING LINEAR PROGRAMMING

503

LP optimal control 2 1 state

0 −1 −2 −3 0

1

2

3

4

5

6

7

8

9

4 Manipulated

2 0 −2 −4 0

1

2

3

4 5 sample time

6

7

8

9

Figure 10.22: Non-square LP optimal control with 3 states and 2 inputs. Note the small inter-sample ripple.

LP optimal control 1 0.5 state

0 −0.5 −1 −1.5 −2 0

1

2

3

4

5

6

7

8

9

Manipulated

4 2 0 −2 −4 0

1

2

3

4 5 sample time

6

7

8

9

Figure 10.23: LP control showing behaviour

optimal acausal

504

CHAPTER 10. PREDICTIVE CONTROL

A PPENDIX A

L IST

OF SYMBOLS

S YMBOL

D ESCRIPTION

U NITS

t T, ∆t τ K ζ δ(·) f (t) f ∗ (T ) L(·) Z(·) s z q −1 ω f V(x) P, Q J num den

time sample time time constant Gain shape factor Dirac delta function function of time sampled function f (T ) Laplace transform z transform Laplace variable z transform variable backward shift operator radial velocity frequency scalar Lyapunov function positive definite matrices Jacobian matrix numerator denominator

s s s radians/s Hz -

M F ρ h A ∆P T H I cp ∆T Q, q J θ Λ

mass material flow density height cross sectional area pressure differential temperature Enthalpy current heat capacity temperature differential heat loss (gain) moment of inertia angle relative gain matrix

kg kg/s kg/m3 m m2 kPa K,◦ C kJ mA kJ/kg.K K kJ/s radians -

u ǫ τi τd Kc Ku ωu Pu

manipulated variable error integral time derivative time proportional gain ultimate gain ultimate frequency ultimate period

s s rad/s s

505

ωm γ, α d, θ P Φ W

phase margin actual PID parameters dead time period signal power weight

s samples, s s

M S θ ǫ t1−α/2 cov(·) C K P I λ f E{·}

model system vector of parameters error vector t-statistic co-variance past input/output data gain matrix co-variance matrix identity matrix forgetting factor factor expected value

-

∆t x u y d A B D C Φ ∆ Γ θ β z Co Ob K L

sample time s state vector n×1 input vector m×1 output vector r×1 disturbance vector p×1 system matrix n×n control matrix n×m disturbance matri x n×p measurement matrix r×n transition matrix n×n discrete control matrix n×m discrete disturbance matrix n × p time delay s # whole samples delay augmented state vector controllability matrix observability matrix controller gain observer gain -

J,j Q, R x ¯ σx λ, ϕ H

performance index weighting matrices mean of x standard deviation of x Lagrange multipliers Hamiltonian

kg

≈4

$ -

APPENDIX A. LIST OF SYMBOLS

506 φ

termination criteria

S UBSCRIPTS D ESCRIPTION N u,l ss

Nyquist upper & lower steady-state

S UPERSCRIPTS D ESCRIPTION x ˆ x ¯ x⋆

estimate (of x) mean (of x) desired value (of x)

A PPENDIX B

U SEFUL UTILITY M ATLAB

FUNCTIONS IN

The routine in Listing B.1 adds two or more polynomials of possibly differing lengths. The resultant polynomial order is the same as the maximum order of the input polynomials. Note that this routine pads with zeros to the left so that all the polynomials are of the same size. It assumed (but not checked) that the polynomials are row vectors in descending order. Listing B.1: Polynomial addition.

5

10

function R = polyadd(A,varargin) % Adds two or more row vectors of possibly differing lengths. % R(x) = A(x) + B(x) + C(x) + · · · . np = length(varargin); nmax = length(A); for i=1:np % find maximum order in the input argument list nmax = max(nmax,length(varargin{i})); end % if R = [zeros(1,nmax-length(A)),A]; for i=1:np varargin{i} = [zeros(1,nmax - length(varargin{i}),1), varargin{i}]; R = R+varargin{i}; end % if return

This routine is useful for checking the results of a Diophantine solution, A(x)R(x) + B(x)S(x) = T (x). Tc = polyadd(conv(A,R),conv(B,S)) The routine in Listing B.2 convolves or multiplies two or more polynomials. Unlike polynomial addition, we do not need to ensure that all the polynomials are of the same length. Listing B.2: Convolution (polynomial multiplication) of two or more polynomials. 2

7

function y = mconv(a,varargin) % Multiple convolution of polynomial vectors % R(x) = A(x)B(x)C(x) · · · . % Would be faster to do “divide & conquer”, and lots of smaller first. if nargin < 2 error('Need two or more arguments')

507

APPENDIX B. USEFUL UTILITY FUNCTIONS IN MATLAB

508 end % if

12

y = conv(a,varargin{1}); % do the first pair for i=2:nargin-1 % now do any remaining y = conv(y,varargin{i}); end return

Listing B.3 strips the leading zeros from a row polynomial, and optionally returns the number of zeros stripped which may, depending on the application, be the deadtime. Listing B.3: Strip leading zeros from a polynomial. 2

7

function [Br,i] = stripleadzeros(B) % Strip the leading zeros from a (vector) polynomial & return deadtime (# of leading zeros) % [Br,i] = stripleadzeros(B) for i=0:length(B)-1 if B(i+1) % if Bi 6= 0 Br = B(i+1:end); return % jump out of routine end end % for return

A PPENDIX C

T RANSFORM time function f (t) unit step

1

impulse

δ(t) t

unit ramp

t2 tn e−at te−at (1 − at)e−at sin(at) cos(at) − e−at a−b e−bt sin(−at) e

−bt

e−bt cos(−at) ae−bt − be−at 1 + ab ab(b − a)

PAIRS

Laplace transform F (s) 1 s 1 1 s2 2 s3 n! sn+1 1 s+a 1 (s + a)2 s (s + a)2 a s2 + a2 s s2 + a2 1 (s + a)(s + b) a (s + b)2 + a2 s+b (s + b)2 + a2 1 s(s + a)(s + b)

509

z-transform Z z z−1 Tz (z − 1)2 T 2 z(z + 1) 2(z − 1)3 z z − eaT T zeaT (z − eaT )2 z sin(aT ) z 2 − 2z cos(aT ) + 1

510

APPENDIX C. TRANSFORM PAIRS

A PPENDIX D

U SEFUL

TEST MODELS

The following are a collection of multivariable models in state-space format which can be used to test control and estimation algorithms.

D.1

A FORCED CIRCULATION EVAPORATOR

The linearised forced circulation evaporator developed in [147, chapter 2] and mentioned in §3.2.3 in continuous state-space form, x˙ = Ax + Bu, is  0 0.10455 0.37935  −0.1 0 A= 0 −2 −2 0 −1.034 · 10 −5.4738 · 10  −0.1 −0.37266 0 0 0 B =  −0.1 0 3.6914 · 10−2 −7.5272 · 10−3 

(D.1) 0.36676 0 3.6302 · 10−2

0.38605 0.1 3.2268 · 10−3

0 0.1 0

−3.636 · 10−2 0 3.5972 · 10−3

T200

iT

 0  0 1.7785 · 10−2 (D.2)

with state and input variables defined as in Table 3.3, x= u=



h

L2 F2

P2 P100

x2

T

F200

(D.3) .. .

F3

F1

x1

T1

(D.4)

In this case we assume that the concentration cannot be measured online, so the output equation is   1 0 0 y= x (D.5) 0 1 0

4

9

A=[0, 0.10455 , 0.37935 0, -0.1 , 0 0, -1.034e-2 , -5.4738e-2]; B=[-0.1, -0.37266 , 0 , 0.36676 , 0.38605 , -0.1, 0 , 0 , 0 , 0.1 , 0 , 3.6914e-2, -7.5272e-3, 3.6302e-2, 3.2268e-3, C= eye(2,3); D = zeros(2,8); G = ss(A,B,C,D); G.StateName = {'L2','P2','x2'}

511

0 , -3.636e-2, 0 0.1, 0 , 0 0 , 3.5972e-3, 1.7785e-2];

APPENDIX D. USEFUL TEST MODELS

512

G.InputName = {'F2','P100','F200','F3','F1','x1','T1','T200'} G.Notes = {'Newell & Lee Evaporator'};

D.2

A IRCRAFT MODEL

The following model is of an aircraft reported in [62, p31]. The state model is    0 0 0 1.132 0 −1   −0.12  0 −0.0538 −0.1712 0 0.0705     0 0 1 0  A= , B =  0  0  4.419  0 0.0485 0 −0.8556 −1.013  1.575 0 −0.2909 0 1.0532 −0.6859   1 0 0 0 0 C= 0 1 0 0 0  0 0 1 0 0

0 1 0 0 0

0 0 0 −1.665 −0.0732

     

(D.6)

(D.7)

where the aircraft states & control inputs are defined as:

state x1 x2 x3 x4 x5 u1 u2 u3

description altitude forward velocity pitch angle pitch rate, x˙3 vertical speed spoiler angle forward acceleration elevator angle

units m m/s degrees deg/s m/s deg/10 m/s2 deg

measured yes yes yes no no

Note that we only measure altitude, velocity and pitch angle, we do not measure the rate states. A reasonable initial condition is x0 =

3

8



10 100 −15 1

25

⊤

.

A = [0,0,1.132 0 -1; 0 -0.0538 -0.1712 0 0.0705; 0 0 0 1 0; ... 0. 0.0485, 0 -0.8556 -1.013; 0 -0.2909 0 1.0532 -0.6859]; B = [0 0 0; -0.12 1 0; 0 0 0; 4.419 0 -1.665; 1.575 0 -0.0732]; C = eye(3,5); D = zeros(3,3); G = ss(A,B,C,D); G.StateName={'altitude','forward vel','pitch angle','pitch rate','vertical speed'}; G.StateUnit={'m','m/s','degrees','deg/s','m/s'}; G.InputName={'spoiler angle','for acc''n','elevator angle'}; G.InputUnit={'deg/10','m/sˆ2','deg'}; G.notes ={'Aircraft model from Dorato et al'};

B IBLIOGRAPHY [1] J. Abate and P.P. Valko. ´ Multi-precision Laplace transform inversion. International Journal for Numerical Methods in Engineering, 60:979–993, 2004. 34, 36 [2] James L. Adams. Flying Buttresses, Entropy, and Orings: The World of an Engineer. Harvard University Press, 1991. 10 [3] Advantech Co. Ltd. PC-Multilab Card User’s Manual, 1992. 16 ˚ [4] Johan Akesson. MPCtools: A toolbox for simulation of MPC controllers in Matlab. Technical report, Lund Institute of Technology, Lund, Sweden, January 2006. 495 [5] Brian D. O. Anderson and John B. Moore. Optimal Control. Linear quadratic methods. Prentice– Hall, 1971. 429 [6] Brian D.O. Anderson and John B. Moore. Optimal Filtering. Information and System Sciences. Prentice-Hall, 1979. 465, 467 [7] Jim Anderson, Ton Backx, Joost Van Loon, and Myke King. Getting the most from Advanced Process Control. Chemical Engineering, pages 78–89, March 1994. 11, 12 [8] Anon. Experience with the X-15 Adaptive Flight Control System. Technical Report NASA TN D6208, NASA Flight Research Centre, Edwards, California, Washington, D.C., March 1971. 323 ˚ en. Realisation of expert system based feed[9] K-E Arz´ back control. PhD thesis, Department of Automatic control, Lund Institute of Technology, Lund, Sweden, 1987. 155 ˚ om. [10] K. J. Astr ¨ Computer Control of a Paper Machine – an Application of Linear Stochastic Control Theory. IBM Journal of Research and Development, 11(4):389–405, 1967. 328 [11] K. J. Astrom. Maximum likelihood and prediction error methods. Automatica, 16:551–574, 1980. 285 ˚ om. [12] K. J. Astr ¨ Ziegler–Nichols Auto-tuners. Technical Report LUTFD2/(TFRT–3167) /01–025/, Lund University, Lund, Sweden, 1982. 174, 178, 469

513

˚ om [13] K. J. Astr ¨ and B. Wittenmark. On Self Tuning Regulators. Automatica, 9:185–199, 1973. 328 ˚ om [14] Karl J. Astr ¨ and Tore H¨agglund. Advanced PID Control. ISA, Research Triangle Park, NC, USA, 2006. 138, 186 ˚ om. [15] Karl-Johan Astr ¨ Introduction to stochastic control theory. Academic press, 1970. 328 ˚ om [16] Karl-Johan Astr ¨ and Tore H¨agglund. Automatic tuning of PID controllers. Instrument Society of America, Research triangle park, NC, 1988. 138, 147, 160, 171, 172, 327 ˚ om [17] Karl-Johan Astr ¨ and Tore H¨agglund. Revisiting the Ziegler-Nichols step response method for PID control. Journal of Process Control, 14:635–650, 2004. 157, 158 ˚ om [18] Karl-Johan Astr ¨ and Bjorn ¨ Wittenmark. Adaptive control. Addision–Wesley, 1989. 13, 311, 312, 315 ˚ om [19] Karl-Johan Astr ¨ and Bjorn ¨ Wittenmark. Adaptive Control. Addision–Wesley, 2 edition, 1995. 247, 334, 355 ˚ om [20] Karl-Johan Astr ¨ and Bjorn ¨ Wittenmark. Computer-Controlled Systems: Theory and Design. Prentice–Hall, 3 edition, 1997. 65, 147, 363, 377 ˚ om [21] K.J. Astr ¨ and R.D. Bell. Drum-boiler dynamics. Automatica, 36(3):363–378, 2000. 186 [22] K.J. Astrom ¨ and B. Wittenmark. Adaptive Control. Addison–Wesley, 1989. 332 ˚ om [23] K.J. Astr ¨ and B. Wittenmark. Computer Controlled Systems: Theory and Design. Prentice–Hall, 2 edition, 1990. 316, 348, 351, 358 [24] Gregory L. Baker and J. P. Gollub. Chaotic dynamics: an introduction. Cambridge University Press, 1990. 126 [25] Yonathan Bard. Nonlinear parameter estimation. Academic Press, 1974. 279 [26] R.A. Bartlett, A. W¨achter, and L. T. Biegler. Active set vs. interior point strategies for model predictive control. In Proceedings of the American Control Conference, pages 4229–4233, Chicago, Illinois, USA, 2000. 481

514 [27] B. Bellingham and F.P. Lees. The detection of malfunction using a process control computer: A Kalman filtering technique for general control loops. Trans. IChemE, 55:253–265, 1977. 464 [28] Alberto Bemporad, Manfred Morari, and N. Lawrence Ricker. Model Predictive Control Toolbox. The MathWorks Inc., 2006. 478 [29] Alberto Bemporad, Manfred Morari, and N. Lawrence Ricker. Model predictive control toolbox 3. Technical report, The Mathworks, 2009. 481, 495

APPENDIX D. USEFUL TEST MODELS [42] F. Buchholt and M. Kummel. ¨ A Multivariable Selftuning Regulator to Control a Double Effect Evaporator. Automatica, 17(5):737–743, 1981. 97 [43] Richard L. Burden and J. Douglas Faires. Numerical Analysis. PWS Publishing, 5 edition, 1993. 124 [44] C. Sidney Burrus, James H. McClellan, Alan V. Oppenheim, Thomas W. Parks, Ronald W. Schafer, and Hans W. Schuessler. Computer-Based Exercises for Signal Processing using Matlab. Prentice–Hall, 1994. 19, 207

[30] B. Wayne Bequette. Nonlinear control of chemical processes: A review. Ind. Eng. Chem. Res., 30:1391– 1413, 1991. 389

[45] B. Carnahan and J.O. Wilkes. Numerical solution of differential equations - an overview. In R. Mah and W.O. Seider, editors, FOCAPD ’80, Henniker, New Hampshire, 1980. 90

[31] A. Besharati Rad, Wai Lun Lo, and K.M. Tsang. Self-tuning pid controller using newton-raphson search method. Industrial Electronics, IEEE Transactions on, 44(5):717–725, Oct 1997. 160, 260

˚ om [46] C.C. Hang and K. J. Astr ¨ and Q.G. Wang. Relay feedback auto-tuning of process controllers – A tutorial overview. J. Process Control, 12:143–162, 2002. 183

[32] Torsten Bohlin. A grey-box process identification tool: Theory and practice. Technical Report IRS3-REG-0103, Department of Signals Systems & Sensors, Royal Institute of technology, SE-100 44 Stockholm, Sweden, August 2001. 89

[47] B. Chachuat. Nonlinear and dynamic optimization: From theory to practice. Technical report, Automatic Control Laboratory, EPFL, Switzerland, 2007. Available from infoscience.epfl.ch/record/111939/files/. 409, 418

[33] C. Bohn and D.P. Atherton. An Analysis Package Comparing PID Anti-Windup Strategies. IEEE Control Systems, 15:34–40, 1995. 147 [34] G.E.P. Box and G.M. Jenkins. Time Series Analysis: Forecasting and Control. Holden–Day, 1970. 240, 247, 313, 314 [35] M. Braun. Differential Equations and their Applications. Springer–Verlag, 1975. 90, 124 [36] John W. Brewer. Kronecker products and matrix calculus in system theory. IEEE Transactions on Circuits and Systems, 25(9):772–781, September 1978. 84, 85 [37] E.H. Bristol. On a new measure of interaction for multivariable process control. IEEE transactions on Automatic Control, AC–11(133), 1966. 107

[48] Edward R. Champion. Numerical Methods for Engineering Applications. Marcel Dekker, Inc., 1993. 124 [49] Cheng-Liang Chen. A simple method of on-line identification and controller tuning. 35(12):2037– 2039, 1989. 164 [50] Cheng-Liang Chen. A closed loop reaction–curve method for controller tuning. Chemical Engineering Communications, 104:87–100, 1991. 164, 171 [51] Robert Chote. Why the Chancellor is always wrong. New Scientist, page 26, 31 October 1992. 95 [52] C.K. Chui and G. Chen. Kalman Filtering with RealTime Applications. Springer–Verlag, 1987. 444, 465, 469

[38] Jens Trample Broch. Principles of Experimental Frequency Analysis. Elsevier applied science, London & New York, 1990. 224

[53] C.K. Chui and G. Chen. Linear Systems and Optimal Control. Springer–Verlag, 1989. 409, 411, 420

[39] Robert Grover Brown and Patrick Y.C. Hwang. Introduction to Random Signals and Applied Kalman Filtering. John Wiley & Sons, 2 edition, 1992. 444

[54] David Clarke. Advances in Model-Based Predictive Control. Oxford University Press, 1994. 190, 324, 475

[40] Arthur E. Bryson and Yu-Chi Ho. Applied Optimal Control. Ginn & Co., Waltham, Mass., 1969. 409

[55] David W. Clarke. PID Algorithms and their Computer Implementation. Trans. Inst. Measurement and Control, 6(6):305–316, Oct-Dec 1984. 138

[41] James L. Buchanan and Peter R. Turner. Numerical Methods and Analysis. McGraw–Hill, 1992. 124

[56] J.D. Cryer. Time series analysis. PWS, 1986. 247

D.2. AIRCRAFT MODEL [57] Jonathan Currie, Arrian Prince-Pike, and David I. Wilson. A cost effective high-speed auto-coded embedded model predictive controller. International Journal of Intelligent Systems Technologies and Applications, 13(1/2):37–55, 2014. 491 [58] Jonathan Currie and David I. Wilson. A Model Predictive Control toolbox intended for rapid prototyping. In Tim Molteno, editor, 16th Electronics New Zealand Conference (ENZCon 2009), pages 7–12, Dunedin, New Zealand, 18–20 November 2009. 487, 495 [59] Jonathan Currie and David I. Wilson. Lightweight Model Predictive Control intended for embedded applications. In 9th International Symposium on Dynamics and Control of Process Systems (DYCOPS), pages 264–269, Leuven, Belgium, 5–7 July 2010. 487 [60] James B. Dabney and Thomas L. Harman. Mastering Simulink 4. Prentice–Hall, 2001. 404 [61] J.J. DiStefano, A.R. Stubberud, and I.J. Williams. Feedback and Control Systems. Schaum’s Outline Series, McGraw-Hill, 1990. 75 [62] Peter Dorato, Chaouki Abdallah, and Vito Cerone. Linear-quadratic Control: An Introduction. Prentice– Hall, 1995. 423, 470, 512 [63] Richard Dorf. Modern control systems. Addison– Wesley, 5 edition, 1989. 69 [64] M. Drouin, H. Abou-Kandil, and M. Mariton. Control of complex systems. Methods and technology. Plenum Press, 223 Spring St, New York NY 10013, 1 edition, 1991. 13, 409 [65] D.B. Ender. Control analysis and optimisation. Technical report, Instrument Soc. America, Techmation, Tempe, Arizona 85282, 1990. 155 [66] D. Grant Fisher and Dale E. Seborg. Multivariable Computer Control. North-Holland, 1976. 97 [67] Samuel C. Florman. The Existential Pleasures of Engineering. St. Martins Press, New York, 1976. 11 [68] T. R. Fortescue, L. S. Kershenbaum, and B. E. Ydstie. Implementation of self-tuning regulators with variable forgetting factors. Automatica, 17(6):831–835, 1981. 301 [69] Lloyd D. Fosdick, Elizabeth R. Jessup, and Carolyn J.C. Schauble. Elements of Matlab. High Performance Scientific Computing, University of Colorado, January 1995. Available from cs.colorado.edu/pub/HPSC. 4, 5 [70] A. S. Foss. Critique of chemical process control theory. American Inst. Chemical Engineers J, 19(2):209–214, 1973. 48

515 [71] G.F. Franklin and J.D. Powell. Digital control of dynamic systems, pages 131–183. Addison–Wesley, 1980. 45, 72, 376 [72] G.F. Franklin, J.D. Powell, and M.L. Workman. Digital Control of Dynamic Systems. Addison– Wesley, 3 edition, 1998. 279 [73] C.E. Garcia, D.M. Prett, and M. Morari. Model predictive control: Theory and practice — A survey. Automatica, 25(3):335–348, 1989. 475 [74] Paul Geladi and Bruce R. Kowalski. Partial LeastSquares Regression: A Tutorial. Analytica Chimica Acta, 185:1–17, 1986. 114 [75] S.F. Goldmann and R.W.H. Sargent. Applications of linear estimation theory to chemical processes: A feasibility study. Chemical Engineering Science, 26:1535–1553, 1971. 454 [76] Graham C. Goodwin, Stefan F. Graebe, and Mario E. Salgado. Control System Design. Prentice– Hall, 2001. 130 [77] Felix Gross, Dag Ravemark, Peter Terwiesch, and David Wilson. The Dynamics of Chocolate in Beer: The Kinetic Behaviour of theobroma cacao Paste in a CH3 CH2 OH–H2 O–CO2 Solution. Journal of Irreproducible Results, 37(4):2–4, 1992. 241 [78] Tore K. Gustafsson and Pertti M. M¨akil¨a. ˚ L1 Identification Toolbox for Matlab. Abo Akademi, Finland, August 1994. From ftp.abo.fi/pub/rt/l1idtools. 499 [79] I. Gustavsson. Survey of Applications of Identification in Chemical and Physical Processes. Automatica, 11:3–24, 1975. 240 [80] Juergen Hahn, Thomas Edison, and Thomas F. Edgar. A note on stability analysis using bode plots. Chemical Engineering Education, 35(3):208– 211, 2001. 76 [81] A. Helbig, W. Marquardt, and F. Allgower. Nonlinearity measures: definition, computation and applications. Journal of Process Control, pages 113– 123, 2000. 134 [82] M.A. Henson and D.E. Seborg. Input-output Linearization of General Nonlinear Processes. American Inst. Chemical Engineers J, 36(11):1753–1757, 1990. 127, 389 [83] Michael A. Henson. Nonlinear model predictive control: current status and future directions. Computers in Chemical Engineering, 23(2):187–202, 1998. 477 [84] Michael A. Henson and Dale E. Seborg. Nonlinear Process Control. Prentice Hall, Saddle River, New Jersey, 1997. 95

516 [85] David R. Hill. Experiments in Computational Matrix Algebra. The Random House, 1988. 58 [86] David M. Himmelblau. Process Analysis by Statistical Methods. John Wiley & Sons, 1970. 119, 262 [87] Jesse B. Hoagg and Dennis S. Bernstein. Nonminimum-phase zeros: Much to do about nothing. IEEE Control Systems Magazine, pages 45–57, June 2007. 54, 61, 187 [88] Roger A. Horn and Charles R. Johnson. Topics in Matrix Analysis. Cambridge University Press, 1991. 84 [89] Morten Hovd and Sigurd Skogestad. Pairing Criteria for Unstable Plants. In AIChE Annual Meeting, page Paper 149i, St. Louis, Nov 1993. 357

APPENDIX D. USEFUL TEST MODELS [101] Paul G. Kaminski, Arthur E. Bryson, and Stanley F. Schmidt. Discrete Square Root Filtering: A survey of current techniques. IEEE Trans. Automatic Control, AC-16:727–735, 1971. 465 [102] L.H. Keel and S.P. Bhattacharyya. Robust, fragile, or optimal? Automatic Control, IEEE Transactions on, 42(8):1098 –1105, Aug 1997. 490 [103] David Kincaid and Ward Cheney. Numerical Analysis. Mathematics of Scientific Computing. Brooks/Cole, 1991. 124 [104] Costas Kravaris and Jeffery C. Kantor. Geometric Methods for Nonlinear Process Control. 1 Background. Ind. Eng. Chem. Res., 29:2295–2310, 1990. 389

[90] P. J. Huber. Robust regression: Asymptotics, conjectures, and Monte Carlo. Ann. Math. Statist., 1(5):799–821, 1973. 285

[105] Costas Kravaris and Jeffery C. Kantor. Geometric Methods for Nonlinear Process Control. 2 Controller Synthesis. Ind. Eng. Chem. Res., 29:2310– 2323, 1990. 389

[91] Enso Ikonen and Kaddour Najim. Advanced Process Identification and Control. Marcel Dekker, 2002. 268

[106] Erwin Kreyszig. Advanced Engineering Mathematics. John Wiley & Sons, 7 edition, 1993. 224

[92] Vinay K. Ingle and John G. Proakis. Digital Signal Processing using Matlab V.4. PWS Publishing Company, 1997. 208

[107] B. Kristiansson and B. Lennartson. Robust and optimal tuning of PI and PID controllers. IEE Proc.Control Theory Applications, 149(1):17–25, January 2002. 195

[93] Rolf Isermann. Paramater adaptive control algorithms—A tutorial. Automatica, 18(5):513–528, 1982. 315

[108] R. Kulhavy´ and M.B. Zarrop. On a general concept of forgetting. Int. J. Control, 3(58):905–924, 1993. 317

[94] M.L. James, G. M. Smith, and J. C. Wolford. Applied Numerical Methods for Digital Computation with Fortran and CSMP. Harper & Row, 2 edition, 1977. 124

[109] Benjamin C. Kuo. Automatic Control Systems. Prentice–Hall, 6 edition, 1991. 43

[95] R.K. Jaspan and J. Coull. Trajectory Optimization Techniques in Chemical Reaction Engineering II: Comparison of Methods. American Inst. Chemical Engineers J, 18(4):867–869, July 1972. 413 [96] Andrew H. Jazwinski. Stochastic Processes and Filtering Theory. Academic Press, 111 Fifth Avenue, New York, 1970. 444, 465 [97] A. Johnson. LQG Applications in the Process Industries. Chemical Engineering Science, 48(16):2829– 2838, 1993. 409

[110] Benjamin C. Kuo. Automatic Control Systems. Prentice–Hall, 7 edition, 1995. 373, 374 [111] A. Kuznetsov. On the convergence of the GaverStehfest algorithm. SIAM J. Numerical Analysis, 51(6):2984–2998, May 2013. 35, 36 [112] I. D. Landau. Identification in closed loop: a powerful design tool (better design models, simpler controllers). Control Engineering Practice, 9:51–65, 2001. 314 [113] Yoan D. Landau. Adaptive Control: The model reference approach. Marcel Dekker, 1979. 304

[98] Michael A. Johnson and Mohammad H. Moradi. PID COntrol: New Identification and Design Methods. Springer–Verlag, London, UK, 2005. 138, 140

[114] Leon Lapidus and John H. Seinfeld. Numerical Solution of Ordinary Differential Equations. Academic Press, 1971. 90

[99] Arthur Jutan and E. S. Rodriguez II. Extension of a New method for On-Line Controller Tuning. The Canadian Journal of Chemical Engineering, 62:802– 807, December 1984. 164

[115] Alan J. Laub. Matrix Analysis for Scientists and Engineers. Society for Industrial and Applied Mathematics, 2004. 84

[100] Thomas Kailath. Linear Systems. Prentice–Hall, 1980. 409

[116] P.L. Lee and G.R. Sullivan. Generic Model Control (GMC). Computers in Chemical Engineering, 12(6):573–580, 1988. 379

D.2. AIRCRAFT MODEL [117] T.H. Lee, Q.G. Wang, and K.K. Tan. Knowledgebased process identification from relay feedback. Journal of Process Control, 5(6):387–397, 1995. 183, 184 [118] S. Lees and J.O. Hougen. Determination of pneumatic controller characteristics by frequency response. Ind. Eng. Chem., 48:1064, 1956. 250 [119] J. R. Leigh. Control Theory: A guided tour. IEE Control Series 45. Peter Peregrinus Ltd., 1992. 77

517 [133] Paul A. Lynn and Wolfgang Fuerst. Introductory Digital Singnal Processing with Computer Applications. John Wiley & Sons, 2 edition, 1994. 211, 224 [134] J. M. Maciejowski. Predictive Control with Constraints. Prentice–Hall, 2002. 495 [135] Sven Erik Mattsson. On Modelling and Differential/Algebraic Systems. Simulation, pages 24–32, January 1989. 134 [136] Cleve Moler and John Little. Matlab User’s Guide. The MathWorks Inc., December 1993. 1

[120] A. Leva, C. Cox, and A. Ruano. Hands-on pid autotuning: a guide to better utilisation. [137] B. De Moor, P. De Gersem, B. De Schutter, and In Marek Zaremba, editor, IFAC Professional W. Favoreel. DAISY: A database for identification Brief, chapter 3. IFAC, 2002. Available from of systems. Journal A, 38(3):4–5, 1997. 318 www.ifac-control.org/publications/pbriefs/PB_Final_LevaCoxRuano.pdf. [138] Manfred Morari. Three critiques of process control 243 revisited a decade later. In D.M. Prett, C.E. Grac´ıa, [121] Frank L. Lewis. Optimal Estimation. John Wiley & and B.L. Ramaker, editors, The Shell Process control Sons, 1986. 288, 451, 465 workshop, pages 309–321. Butterworths, 1987. 48, 367 [122] Frank L. Lewis and Vassilis L. Syrmos. Optimal [139] Manfred Morari. Process control theory: ReflecControl. John Wiley & Sons, 2 edition, 1995. 471 tions on the past and goals for the next decade. [123] R.J. Litchfield, K.S. Campbell, and A. Locke. The In D.M. Prett, C.E. Grac´ıa, and B.L. Ramaker, ediapplication of several Kalman filters to the contors, The second Shell Process control workshop, pages trol of a real chemical reactor. Trans. IChemE, 469–488. Butterworths, December 1988. 99 57(2):113–120, April 1979. 384, 385 [140] Manfred Morari. Some Control Problems in the Process Industries. In H.L. Trentelman and J.C. [124] Tao Liu, Qing-Guo Wang, and Hsiao-Ping Huang. Willems, editors, Essays on Control: Perspectives in A tutorial review on process identification from the Theory and its applications. Birkh¨auser, 1993. 147 step or relay feedback test. Journal of Process Control, 23(10):1597 – 1623, 2013. 172, 243 [141] Manfred Morari. Model Predictive Control: Multivariable Control Technique of Choice in the 1990s. [125] Lennart Ljung. System Identification: Theory for the In David Clarke, editor, Advances in Model-Based User. Prentice–Hall, 1987. 240, 247, 274, 286, 316 Predictive Control, pages 22–37. Oxford University Press, 1994. 48, 225 [126] Lennart Ljung. System Identification Toolbox, for use with Matlab. The MathWorks Inc., May 1991. 240, [142] Manfred Morari and Evanghelos Zafiriou. Robust 311, 315 Process Control. Prentice–Hall, 1989. 104, 128 [127] Lennart Ljung. System Identification: Theory for the User. Prentice–Hall, 2 edition, 1999. 282 [128] Lennart Ljung. Perspectives on system identification. Annual Reviews in Control, 34(1):1–12, 2010. 241 [129] Lennart Ljung and Torkel Glad. Modeling of Dynamic Systems. Prentice–Hall, 1994. 240 [130] Charles F. Van Loan. The ubiquitous kronecker product. Journal of Computational and Applied Mathematics, 123:85–100, 2000. 84 [131] W. L. Luyben. Process Modeling, Simulation, and Control for Chemical Engineers. McGraw-Hill, 2 edition, 1990. 90 [132] W.L. Luyben. Process modelling simulation and control for chemical engineers. McGraw-Hill, 1973. 102

[143] Jerzy Moscinski and Zbigniew Ogonowski. Advanced Control with Matlab and Simulink. Ellis Horwood, 1995. 250 [144] Frank Moss and Kurt Wiesenfeld. The benefits of Background Noise. Scientific American, pages 50– 53, August 1995. See also Amateur Scientist in the same issue. 199 [145] Ivan Nagy. Introduction to Chemical Process Instrumentation. Elsevier, 1992. 13 [146] Ioan Nascu and Robin De Keyser. A novel application of relay feedback for PID auto-tuning. In IFAC CSD03 Conference on Control systems Design, Bratislava, Slocak Republic, 2003. 183 [147] R.B. Newell and P.L. Lee. Applied Process Control – A Case Study. Prentice–Hall, 1989. 97, 98, 434, 460, 511

518 [148] R.E. Nieman and D.G. Fisher. Experimental Evaluation of Optimal Multivariable Servo-control in Conjunction with Conventional Regulatory Control. Chemical Engineering Communications, 1, 1973. 496 [149] R.E. Nieman, D.G. Fisher, and D.E. Seborg. A review of process identification and parameter estimation techniques. Int. J of control, 13(2):209–264, 1971. 240 [150] Katsuhiko Ogata. Discrete Time Control Systems. Prentice–Hall, 1987. 17, 25, 27, 29, 32, 39, 41, 53, 58, 66, 77, 79, 81, 86, 112, 235, 316, 345, 347, 358, 359, 361, 373, 376, 378, 410, 456 [151] Katsuhiko Ogata. Discrete-Time Control Systems. Prentice–Hall, 1987. 28 [152] Katsuhiko Ogata. Modern Control Engineering. Prentice–Hall, 2 edition, 1990. 75, 79, 81, 84, 112, 156, 173, 358, 359, 361, 400 [153] B. A. Ogunnaike, J. P. Lemaire, M. Morari, and W. H. Ray. Advanced multivariable control of a pilot-plant distillation column. 29(4):632–640, July 1983. 101

APPENDIX D. USEFUL TEST MODELS [163] R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2013. ISBN 3-90005107-0. 4, 122 [164] W. Fred Ramirez. Process Control and Identification. Academic Press, 1994. 409, 418, 423 [165] C. V. Rao, S. J. Wright, and J. B. Rawlings. Application of interior-point methods to model predictive control. Journal of Optimization Theory and Applications, 99(3):723–757, 1998. 481 [166] T. Rautert and E.W. Sachs. Computational design of optimal output feedback controllers. Technical Report Nr. 95-12, Universit¨at Trier, FB IV, Germany, June 1995. 471 [167] James B. Rawlings. Tutorial overview of Model Predictive Control. IEEE Control Systems Magazine, 20(3):38–52, June 2000. 475 [168] W. Harmon Ray and Julian Szekely. Process Optimization. John Wiley & Sons, 1973. 409, 413 [169] W.H. Ray. Advanced Process Control. McGraw–Hill, NY, 1 edition, 1981. 409

[154] Alan V. Oppenheim and Ronald W. Schafer. Digital Signal Processing. Prentice–Hall, 1975. 210

[170] John R. Rice. Numerical Methods, Software, and Analysis. McGraw–Hill, 1983. 124

[155] Overschee and De Moor. Subspace Identification. Kluwer Academic Publshers, 1996. 279

[171] C. Magnus Rimvall and Christopher P. Jobling. Computer Aided Control System Design. In W.S. Levine, editor, The Control Handbook, chapter 23, pages 429–442. CRC Press, 1995. 3

[156] Chris C. Paige. Properties of Numerical Algorithms Related to Computing Controllability. IEEE transactions on Automatic Control, AC–26(1):130– 138, 1981. 361 [157] Sudharkar Madhavrao Pandit and Shien-Ming Wu. Time Series and System Analysis with Applications. Wiley, 1983. 313 [158] Rajni V. Patel, Alan J. Laub, and Paul M. van Dooren. Numerical Linear Algebra Techniques for Systems and Control. IEEE Press, 1994. A selected reprint volume. 66, 361, 362 [159] Linda Petzold. Differential/Algebraic Equations are not ODE’s. SIAM J. Sci. Stat. Comput., 3(3):367– 384, 1982. 129 [160] C.L. Phillips and H.T. Nagle. Digital Control System Analysis and Design. Prentice–Hall, 2 edition, 1990. 204 [161] W.H. Press, B.P. Flannery, S.A Teukolsky, and W.T. Vetterling. Numerical Recipes: The Art of Scientific Computing. Cambridge University Press, 1986. 113, 224, 225, 232, 413 [162] S. Joe Qin and Thomas A. Badgwell. A survey of industrial model predictive control technology. Control Engineering Practice, 11(7):733–764, July 2003. 475, 495

[172] William H. Roadstrum and Dan H. Wolaver. Electrical Engineering for all Engineers. John Wiley & Sons, 2 edition, 1994. 15 [173] E.R. Robinson. Time Dependent Chemical Processes. Applied Science Publishers, 1975. 409 [174] J. A. Rossiter. Model-Based Predictive Control: A Practical Approach. CRC Press, 2003. 495 [175] Hadi Saadat. Computational Aids in Control Systems using Matlab. McGraw–Hill, 1993. 55 [176] S.P. Sanoff and P.E. Wellstead. Comments on: implementation of self-tuning regulators with variable forgetting factors. Automatica, 19(3):345 – 346, 1983. 301 [177] Robert Schoenfeld. The Chemist’s English. VCH Verlagsgesellschaft mbH, D-6940 Weinheim, Germany, 2 edition, 1986. 224 [178] J. Schoukens and R. Pintelon. Identification of Linear Systems. Pergamon, 1991. 113 [179] Tobias Schweickhardt and F. Allgower. Linear control of nonlinear systems based on nonlinearity measures. Journal of Process Control, 17:273–284, 2007. 134

D.2. AIRCRAFT MODEL

519

[180] Dale E. Seborg, Thomas F. Edgar, and Duncan A. Mellichamp. Process Dynamics and Control. Wiley, 2 edition, 2005. 157, 160

[195] O. Taiwo. Comparison of four methods of online identification and controller tuning. IEE Proceedings-D, 140(5):323–327, 1993. 164, 171

[181] D.E. Seborg, T.F. Edgar, and D.A. Mellichamp. Process Dynamics and Control. Wiley, 1989. 17, 32, 108, 144, 156, 158, 257

[196] Les Thede. Analog and Digital Filter Design using C. Prentice–Hall, 1996. 208

[182] D.E. Seborg, T.F Edgar, and S.L. Shah. Adaptive Control Strategies for Process Control: A survey. American Inst. Chemical Engineers J, 32(6):881–913, 1986. 324 [183] Warren D. Seider, J.D. Seader, and Daniel R. Lewin. Process Design Principles: Synthesis, Analysis and Evaluation. John Wiley & Sons, 1999. 108 [184] Bahram Shahian and Michael Hassul. Control System Design Using Matrixx . Prentice–Hall, 1992. 71 [185] Kermit Sigmon. Matlab primer, 3rd edition. Dept. of Mathematics, University of Florida, 1993. 4, 5 [186] Jonas Sjoberg, ¨ Qinghua Zhang, Lennart Ljung, Albert Benveniste, Bernard Delyon, Pierre-Yves Glorennec, H˚akan Hjalmarsson, and Anatoli Juditsky. Nonlinear black-box modeling in system identification: a unified overview. Automatica, 31(12):1691–1724, 1995. Trends in System Identification. 242, 285 [187] S. Skogestad. Dynamics and Control of Distillation Columns– A Critical Survey. In DYCORD+, pages 11–35. Int. Federation of Automatic Control, 1992. 99 [188] Sigurd Skogestad. Simple analytic rules for model reduction and PID controller tuning. Journal of Process Control, 13(4):291–309, 2003. 140, 159 [189] Jean-Jacques E. Slotine and Weiping Li. Applied Nonlinear Control. Prentice–Hall, 1991. 78, 79, 389 [190] T. Soderstr ¨ om ¨ and P. Stocia. System Identification. Prentice–Hall, 1989. 240, 316

[197] J. Villadsen and M.L. Michelsen. Solution of differential equation models by polynomial approximation. Prentice–Hall, 1978. 90 [198] Stanley M. Walas. Modeling with Differential Equations in Chemical Engineering. Butterworth– Heinemann, 1991. 90, 128, 413 [199] Liuping Wang. Model Predictive Control System Design and Implementation using Matlab. Springer, 2009. 495 [200] Liuping Wang and William R. Cluett. From Plant Data to Process Control. Taylor and Francis, 11 New Fetter Lane, London, EC4P 4EE, 2000. 262, 263 [201] Ya-Gang Wang, Zhi-Gang Shi, and Wen-Jian Cai. PID autotuner and its application in HVAC systems. In Proceedings of the American Control Conference, pages 2192–2196, Arlington, VA, 25–27 June 2001. 183 [202] P. E. WELLSTEAD and S. P. SANOFF. Extended self-tuning algorithm. International Journal of Control, 34(3):433–455, 1981. 301 [203] P.E. Wellstead and M.B. Zarrop. Self-tuning Sytems: Control and Signal Processing. John Wiley & Sons, 1991. 248, 301, 307, 308, 315, 316, 318, 320, 328, 465 [204] David I. Wilson. Introduction to Numerical Analysis with Matlab or what’s a NaN and why do I care? Auckland University of Technology, Auckland, New Zealand, July 2007. 465pp. 4, 124, 201, 208, 285, 401, 496, 499 [205] R.K. Wood and M.W. Berry. Terminal composition control of a binary distillation column. Chemical Engineering Science, 28(9):1707–1717, 1973. 98

[191] Harold W. Sorenson, editor. Kalman filtering: Theory and Application. Selected reprint series. IEEE Press, 1985. 444

[206] Minta Yuwana and Dale E. Seborg. A New Method of On-line Controller Tuning. American Inst. Chemical Engineers J, 28(3):434–440, May 1982. 164

[192] Henry Stark and John W. Woods. Probability, Random Processes, and Estimation Theory for Engineers. Prentice–Hall, 2 edition, 1994. 445

[207] J.G. Ziegler and N.B. Nichols. Optimum Settings for Automatic Controllers. Trans. ASME, 64:759– 768, 1942. 156

[193] George Stephanopoulos. Chemical Process Control: An introduction to Theory and Practice. Prentice– Hall, 1984. 108, 151 [194] Dominick Stephens. The Reserve Bank’s Forecasting and Policy System. Technical report, Economics Department, The Reserve Bank of New Zealand, Wellington, NZ, 2004. 475