Brief: Here, we investigated spatio-temporal data analysis methods in motor-based early ASD (autism spectrum disorder) detection application. The challenge here is processing the stream multi-modal (Motion, fMRI, EEG, and DTI) information with the goal of detecting early signs of ASD in the human subject. We propose two solutions, with the ability to achieve 80% accuracy. Additionally, we have extracted a set of entropy-based features that can help explain the decision-making process of the network and improve confidence in the results.