Pubudu: Deep learning based screening and intervention of dyslexia, dysgraphia and dyscalculia

Abstract

Dyslexia, Dysgraphia and Dyscalculia are significant learning disabilities that affect around 10% of children in the world. Despite the advancement of technology literacy in the community, limited attention has been given for screening and intervention of these disabilities using mobile applications in Sri Lanka. In this research, one of the first deep learning and machine learning based mobile applications, named “Pubudu” was developed for screening and intervention of dyslexia, dysgraphia and dyscalculia supporting local languages. In “Pubudu” we have followed up clinical screening and diagnostic procedures recommended by health professionals for screening and intervention. The screening of dyslexia, letter dysgraphia and numeric dysgraphia was carried out using deep neural network and the screening for dyscalculia was carried out using machine learning techniques. Intervention techniques are implemented using gamified environments. System testing was carried out using 50 differently abled children and 50 typical children. With the initial dataset 88%, 58%, 99% screening accuracies are achieved in neural networks for letter dysgraphia, dyslexia and numeric dysgraphia screening while dysgraphia, whereas 90% accuracy was achieved for dyscalculia. Handwritten letters and numbers were fed as inputs to CNN model in letter dysgraphia and numeric dysgraphia while embedded audio clips of letter pronunciation were fed in to voice recognition CNN model in dyslexia. “Pubudu” shows significant potential for screening and intervention of dyslexia, dysgraphia and dyscalculia in local languages motivating children and interactively making them able and would be an enabling app for most of the underprivileged children in Sri Lanka.