Optimistic Inference Control (OIC)

Optimistic inference control (OIC) is a novel method which works as follows. Assume that after a fixed delay $T_c$, the desired goal is reached. That is, (some components of) the observations $\mathbf{x}$ are at the desired level $\mathbf{r}$. Given this optimistic assumption and the observations and control signals so far, infer what happens in between. Then choose the expectation of $q(\mathbf{u}(t_0))$ as before. An example situation is illustrated in Figure 2.

Figure 2: Optimistic inference control (see Section III-B). The inferred observations and control signals are plotted with confidence intervals. The current time is $t_0=0$ and after time $t_0+T_c=40$, the observation $\mathbf{x}(t)$ is assumed to be at the desired level $\mathbf{r}$.

                   
OIC in a nutshell:


Given observations 
$\dots,\mathbf{x}(t_0-2),\mathbf{x}(t_0-1)$ and


control signals 
$\dots,\mathbf{u}(t_0-2),\mathbf{u}(t_0-1)$


1:		Fix future 
$\mathbf{x}(t_0+T_c)=\mathbf{x}(t_0+T_c+1)=\dots=\mathbf{r}$


2:		Infer the distribution 
$q(\mathbf{u}(t),\mathbf{s}(t),\mathbf{x}(t))$ for all $t$


3:		Select the mean of 
$q(\mathbf{u}(t_0))$ as the control signal


4:		Observe 
$\mathbf{x}(t_0)$ and release 
$\mathbf{x}(t_0+T_c)$


5: Increase $t_0$ and loop from $1$

OIC propagates the same evidence forwards as the DC and additionally, the evidence from the desired future backwards. The inference is conceptually simple, but algorithmically difficult. The information from the future needs to flow through tens of nonlinear mappings $\mathbf{g}$ before it affects $\mathbf{u}(t_0)$. In case there are constraints for control signals or observations, they are forced after every inference iteration. If the horizon is set too short or the goal is otherwise overoptimistic, the method becomes unreliable. Even with a realistic goal, it is not in general guaranteed that the iteration will converge to the optimal control signal, as the iteration may get stuck in a local minimum. The inferred control signals can be validated by releasing the optimistic future and re-inferring. If the future changes a lot, the control is unreliable. Note that OIC does not require goal-oriented data, because different goals can be set by changing the desired future.