Skip to content

Instantly share code, notes, and snippets.

@ClementPinard
Created January 17, 2018 16:02
Show Gist options
  • Save ClementPinard/20b0f7c339b512850c80d60250c5d8d2 to your computer and use it in GitHub Desktop.
Save ClementPinard/20b0f7c339b512850c80d60250c5d8d2 to your computer and use it in GitHub Desktop.
Flow Augmentation

given $\theta_1$ and $\theta_2$ as affine transformations for $I_1$ and $I_2$ $$ \theta_1 = \left(\begin{array}{ccc}r_{0,1}&r_{1,1}&t_{0,1}\r_{2,1}&r_{3,1}&t_{1,1}\end{array} \right) = \left( R_1 | T_1 \right) $$ $$ \theta_2 = \left(\begin{array}{ccc}r_{0,2}&r_{1,2}&t_{0,2}\r_{2,2}&r_{3,2}&t_{1,2}\end{array} \right) = \left( R_2 | T_2 \right) $$

$$ \theta_2^{-1}= \left( R_2^{-1} | -R_2^{-1}T_2 \right) $$

Optical Flow is given by the function $F$ such as

$$ F(x,y) \in \mathbb R^2 ,~ I_1 \left(\begin{array}{c} x\y \end{array}\right) = I_2\left(\left(\begin{array}{c} x\y \end{array}\right) + F\right) $$

we want to go from $(x,y)$ in $I'_1$ to $(x_3,y_3)$ in $I'_2$. Main idea is to go from warped $I'_1$ to unwarped $I_1$ to unwarped $I_2$ (through $F$) to warped $I'_2$ .

NB : $(x,y)$ is homogeneous, obtained from $(u-u_0, v-v_0)$

To get corresponding $(x_1,y_1)$ in unwarped image, we apply $$ \left(\begin{array}{c} x_1\y_1 \end{array}\right) = R_1\left(\begin{array}{c}x\y\end{array}\right) + T_1 $$ $(x_2,y_2)$ in unwarped $I_2$, is obtained with $F$, evaluated on $(x_1, y_1)$ (named $F_1$) $$ \left(\begin{array}{c} x_2\y_2 \end{array}\right) = \left(\begin{array}{c} x_1\y_1 \end{array}\right) + F \left(\begin{array}{c} x_1\y_1 \end{array}\right) = R_1\left(\begin{array}{c}x\y\end{array}\right) + T_1 + F\left(R_1\left(\begin{array}{c}x\y\end{array}\right) + T_1 \right) = R_1\left(\begin{array}{c}x\y\end{array}\right) + T_1 + F_1 $$

$(x_3,y_3)$ is then obtained from $(x_2,y_2)$ with $$ \left(\begin{array}{c} x_2\y_2 \end{array}\right) = R_2\left(\begin{array}{c}x_3\y_3\end{array}\right) + T_2 $$

$$ \left(\begin{array}{c} x_3\y_3 \end{array}\right) = R_2^{-1}\left(\begin{array}{c}x_2\y_2\end{array}\right) - R_2^{-1}T_2 $$

$$ \left(\begin{array}{c} x_3\y_3 \end{array}\right) = R_2^{-1}\left(R_1\left(\begin{array}{c}x\y\end{array}\right) + T_1 + F_1\right) -R_2^{-1}T_2 $$

$$ \left(\begin{array}{c} x_3\y_3 \end{array}\right) = R_2^{-1}R_1\left(\begin{array}{c}x\y\end{array}\right) + R_2^{-1}(T_1-T_2) + R_2^{-1}F_1 $$

$$ F_{final} = \left(\begin{array}{c} x_3\y_3 \end{array}\right) - \left(\begin{array}{c} x\y \end{array}\right) = (R_2^{-1}R_1 - I_2)\left(\begin{array}{c}x\y\end{array}\right) + R_2^{-1}(T_1-T_2) + R_2^{-1}F_1 $$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment