4D Vision System

Action4D Online Action Recognition in the Crowd and Clutter


We propose a new method to track people in 4D, which can reliably detect and follow each person in real time. Then, we build a new deep neural network, the Action4DNet, to recognize the action of each tracked person. Such a model gives reliable and accurate results in the real-world settings. We also design an adaptive 3D convolution layer and a novel discriminative temporal feature learning objective to further improve the performance of our model. Our method is invariant to camera view angles, resistant to clutter and able to handle crowd. The experimental results show that the proposed method is fast, reliable and accurate. Our method paves the way to action recognition in the real-world applications and is ready to be deployed to enable smart homes, smart factories and smart stores.


People classification network and the tracking association algorithm.
Our proposed attention Action4DNet.
We recognize actions using volumes (color represents the height of each voxel).
  • Projection of human circle on each camera view
  • Examples

    Example images from a new environment with three network cameras (best viewed in color).



    Quanzeng You and Hao Jiang. Action4D: Online Action Recognition in the Crowd and Clutter.