This presentation summarizes the latest scientific advances that occurred at the VITALab (Sherbrooke, Canada) and underline unsolved questions that we ought to work on in the upcoming years. In the last two years, the VITALab (like a large proportion of the digital imaging community) has been carried by the deep learning wave. Work has been done in various areas of video analytics as well as medical imaging. This includes traffic analytics, motion detection, background estimation, brain tumor segmentation, and MRI cardiac analysis. In parallel, we organized 5 international challenges each time providing the scientific community with large annotated datasets. The combination of these challenges and the latest deep learning models make us believe that we might be at the eve of solving issues that have been the topic of thousands of papers. More specifically, I will present solve and unsolved issues in areas of vehicle recognition and localization, video annotation, brain tumor segmentation, and cardiac segmentation