Data-driven control system

Iterative feedback tuning

The iterative feedback tuning (IFT) method was introduced in 1994, starting from the observation that, in identification for control, each iteration is based on the (wrong) certainty equivalence principle.

IFT is a model-free technique for the direct iterative optimization of the parameters of a fixed-order controller; such parameters can be successively updated using information coming from standard (closed-loop) system operation.

Let ${\displaystyle y^d}$ be a desired output to the reference signal ${\displaystyle r}$; the error between the achieved and desired response is ${\displaystyle \tilde{y}(\rho )=y(\rho )-y^d}$. The control design objective can be formulated as the minimization of the objective function:

${\displaystyle J(\rho )=\frac{1}{2N}\sum _{t=1}^{N}E\left[\tilde{y}(t,\rho {)}^2\right].}$

Given the objective function to minimize, the quasi-Newton method can be applied, i.e. a gradient-based minimization using a gradient search of the type:

${\displaystyle {\rho }_{i+1}={\rho }_i-{\gamma }_i{R}_i^{-1}\frac{d\hat{J}}{d\rho }({\rho }_i).}$

The value ${\displaystyle {\gamma }_i}$ is the step size, ${\displaystyle R_i}$ is an appropriate positive definite matrix and ${\displaystyle \frac{d\hat{J}}{d\rho }}$ is an approximation of the gradient; the true value of the gradient is given by the following:

${\displaystyle \frac{dJ}{d\rho }(\rho )=\frac{1}{N}\sum _{t=1}^N\left[\tilde{y}(t,\rho )\frac{\delta y}{\delta \rho }(t,\rho )\right].}$

The value of ${\displaystyle \frac{\delta y}{\delta \rho }(t,\rho )}$ is obtained through the following three-step methodology:

1. Normal Experiment: Perform an experiment on the closed loop system with ${\displaystyle C(\rho )}$ as controller and ${\displaystyle r}$ as reference; collect N measurements of the output ${\displaystyle y(\rho )}$, denoted as ${\displaystyle y^{(1)}(\rho )}$.
2. Gradient Experiment: Perform an experiment on the closed loop system with ${\displaystyle C(\rho )}$ as controller and 0 as reference ${\displaystyle r}$; inject the signal ${\displaystyle r-y^{(1)}(\rho )}$ such that it is summed to the control variable output by ${\displaystyle C(\rho )}$, going as input into the plant. Collect the output, denoted as ${\displaystyle y^{(2)}(\rho )}$.
3. Take the following as gradient approximation: ${\displaystyle \frac{\delta \hat{y}}{\delta \rho }(\rho )=\frac{\delta C}{\delta \rho }(\rho )y^{(2)}(\rho )}$.

A crucial factor for the convergence speed of the algorithm is the choice of ${\displaystyle R_i}$; when ${\displaystyle \tilde{y}}$ is small, a good choice is the approximation given by the Gauss–Newton direction:

${\displaystyle R_i=\frac{1}{N}\sum _{t=1}^N\frac{\delta \hat{y}}{\delta \rho }({\rho }_i)\frac{\delta {\hat{y}}^T}{\delta \rho }({\rho }_i).}$

Noniterative correlation-based tuning

Noniterative correlation-based tuning (nCbT) is a noniterative method for data-driven tuning of a fixed-structure controller. It provides a one-shot method to directly synthesize a controller based on a single dataset.

Suppose that ${\displaystyle G}$ denotes an unknown LTI stable SISO plant, ${\displaystyle M}$ a user-defined reference model and ${\displaystyle F}$ a user-defined weighting function. An LTI fixed-order controller is indicated as ${\displaystyle K(\rho )={\beta }^T\rho }$, where ${\displaystyle \rho \in {\mathbb{R}}^n}$, and ${\displaystyle \beta }$ is a vector of LTI basis functions. Finally, ${\displaystyle K^{\ast }}$ is an ideal LTI controller of any structure, guaranteeing a closed-loop function ${\displaystyle M}$ when applied to ${\displaystyle G}$.

The goal is to minimize the following objective function:

${\displaystyle J(\rho )={\Vert F(\frac{K^{\ast }G-K(\rho )G}{(1+K^{\ast }G{)}^2})\Vert }_2^2.}$

${\displaystyle J(\rho )}$ is a convex approximation of the objective function obtained from a model reference problem, supposing that ${\displaystyle \frac{1}{(1+K(\rho )G)}\approx \frac{1}{(1+K^{\ast }G)}}$.

When ${\displaystyle G}$ is stable and minimum-phase, the approximated model reference problem is equivalent to the minimization of the norm of ${\displaystyle \epsilon (t)}$ in the scheme in figure.

The input signal ${\displaystyle r(t)}$ is supposed to be a persistently exciting input signal and ${\displaystyle v(t)}$ to be generated by a stable data-generation mechanism. The two signals are thus uncorrelated in an open-loop experiment; hence, the ideal error ${\displaystyle \epsilon (t,{\rho }^{\ast })}$ is uncorrelated with ${\displaystyle r(t)}$. The control objective thus consists in finding ${\displaystyle \rho }$ such that ${\displaystyle r(t)}$ and ${\displaystyle \epsilon (t,{\rho }^{\ast })}$ are uncorrelated.

The vector of instrumental variables ${\displaystyle \zeta (t)}$ is defined as:

${\displaystyle \zeta (t)=[r_W(t+{\ell }_1),r_W(t+{\ell }_1-1),\dots ,r_W(t),\dots ,r_W(t-{\ell }_1){]}^T}$

where ${\displaystyle {\ell }_1}$ is large enough and ${\displaystyle r_W(t)=Wr(t)}$, where ${\displaystyle W}$ is an appropriate filter.

The correlation function is:

${\displaystyle f_{N,{\ell }_1}(\rho )=\frac{1}{N}\sum _{t=1}^N\zeta (t)\epsilon (t,\rho )}$

and the optimization problem becomes:

${\displaystyle \hat{\rho }=\underset{\rho \in D_k}{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}{J}_{N,{\ell }_1}(\rho )=\underset{\rho \in D_k}{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}{f}_{N,{\ell }_1}^T{f}_{N,{\ell }_1}.}$

Denoting with ${\displaystyle {\varphi }_r(\omega )}$ the spectrum of ${\displaystyle r(t)}$, it can be demonstrated that, under some assumptions, if ${\displaystyle W}$ is selected as:

${\displaystyle W(e^{-j\omega })=\frac{F(e^{-j\omega })(1-M(e^{-j\omega }))}{{\varphi }_r(\omega )}}$

then, the following holds:

${\displaystyle \underset{N,{\ell }_1\to \mathrm{\infty },{\ell }_1/N\to \mathrm{\infty }}{lim}\hat{\rho }={\rho }^{\ast }.}$

Stability constraint

There is no guarantee that the controller ${\displaystyle K}$ that minimizes ${\displaystyle J_{N,{\ell }_1}}$ is stable. Instability may occur in the following cases:

• If ${\displaystyle G}$ is non-minimum phase, ${\displaystyle K^{\ast }}$ may lead to cancellations in the right-half complex plane.
• If ${\displaystyle K^{\ast }}$ (even if stabilizing) is not achievable, ${\displaystyle K(\rho )}$ may not be stabilizing.
• Due to measurement noise, even if ${\displaystyle K^{\ast }=K(\rho )}$ is stabilizing, data-estimated ${\displaystyle \hat{K}(\rho )}$ may not be so.

Consider a stabilizing controller ${\displaystyle K_s}$ and the closed loop transfer function ${\displaystyle M_s=\frac{K_{s}G}{1+K_{s}G}}$. Define:

${\displaystyle \mathrm{\Delta }(\rho ):=M_s-K(\rho )G(1-M_s)}$

${\displaystyle \delta (\rho ):={\Vert \mathrm{\Delta }(\rho )\Vert }_{\mathrm{\infty }}.}$

Theorem

The controller ${\displaystyle K(\rho )}$ stabilizes the plant ${\displaystyle G}$ if

1. ${\displaystyle \mathrm{\Delta }(\rho )}$ is stable
2. ${\displaystyle \mathrm{\exists }{\delta }_N\in (0,1)}$ s.t. ${\displaystyle \delta (\rho )\le {\delta }_N.}$

Condition 1. is enforced when:

• ${\displaystyle K(\rho )}$ is stable
• ${\displaystyle K(\rho )}$ contains an integrator (it is canceled).

The model reference design with stability constraint becomes:

${\displaystyle {\rho }_s=\underset{\rho \in D_k}{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}J(\rho )}$

${\displaystyle \text{s.t. }\delta (\rho )\le {\delta }_N.}$

A convex data-driven estimation of ${\displaystyle \delta (\rho )}$ can be obtained through the discrete Fourier transform.

Define the following:

${\displaystyle \begin{array}{cc}& {\hat{R}}_r(\tau )=\frac{1}{N}\sum _{t=1}^{N}r(t-\tau )r(t)\text{ for }\tau =-{\ell }_2,\dots ,{\ell }_2\\ & {\hat{R}}_{r\epsilon }(\tau )=\frac{1}{N}\sum _{t=1}^{N}r(t-\tau )\epsilon (t,\rho )\text{ for }\tau =-{\ell }_2,\dots ,{\ell }_2.\end{array}}$

For stable minimum phase plants, the following convex data-driven optimization problem is given:

${\displaystyle \begin{array}{cc}\hat{\rho }& =\underset{\rho \in D_k}{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}{J}_{N,{\ell }_1}(\rho )\\ & \text{s.t.}\\ & |\sum _{\tau =-{\ell }_2}^{{\ell }_2}{\hat{R}}_{r\epsilon }(\tau ,\rho )e^{-j\tau {\omega }_k}|\le {\delta }_N|\sum _{\tau =-{\ell }_2}^{{\ell }_2}{\hat{R}}_r(\tau ,\rho )e^{-j\tau {\omega }_k}|\\ {\omega }_k& =\frac{2\pi k}{2{\ell }_2+1},k=0,\dots ,{\ell }_2+1.\end{array}}$

Virtual reference feedback tuning

Virtual Reference Feedback Tuning (VRFT) is a noniterative method for data-driven tuning of a fixed-structure controller. It provides a one-shot method to directly synthesize a controller based on a single dataset.

VRFT was first proposed in and then extended to LPV systems. VRFT also builds on ideas given in as ${\displaystyle VR{D}^2}$.

The main idea is to define a desired closed loop model ${\displaystyle M}$ and to use its inverse dynamics to obtain a virtual reference ${\displaystyle r_v(t)}$ from the measured output signal ${\displaystyle y(t)}$.

The virtual signals are ${\displaystyle r_v(t)=M^{-1}y(t)}$ and ${\displaystyle e_v(t)=r_v(t)-y(t).}$

The optimal controller is obtained from noiseless data by solving the following optimization problem:

${\displaystyle {\hat{\rho }}_{\mathrm{\infty }}=\underset{\rho }{\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{m}\mathrm{i}\mathrm{n}}\underset{N\to \mathrm{\infty }}{lim}{J}_{vr}(\rho )}$

where the optimization function is given as follows:

${\displaystyle J_{vr}^N(\rho )=\frac{1}{N}\sum _{t=1}^N{\left(u(t)-K(\rho )e_v(t)\right)}^2.}$