ABSTRACT: Evaluation of Transfer Learning Models on Detection of COVID-19 Using Multi-Modal Data

The outbreak of the 2019 novel Coronavirus (COVID-19) has affected millions of people worldwide. The structural manifestations of COVID-19 can be visualized on radiographic images of the chest, including computerized tomography (CT) and X-ray. Advancements in machine learning have led to the development of transfer learning models as tools for faster and more accurate diagnoses of COVID-19 when compared to the reverse transcription-polymerase chain reaction (RT-PCR) test. Transfer learning utilizes pre-trained convolutional neural network (CNN) models to classify images from an unseen dataset. We analyzed the performance of six different transfer learning architectures on their ability to classify 2482 single slice chest CT images from 60 COVID-19 positive patients and 60 COVID-19 negative patients. The best performing model was InceptionV3, with 98.71 ± 1.43% validation accuracy, 97.64 ± 3.06% specificity, 99.76 ± 0.36% sensitivity, 97.73 ± 2.78% precision, an F1 score of 98.74 ± 1.37% and an AUC of 0.9997 ± 0.0003. These preliminary results suggest that applying an InceptionV3 transfer learning-based framework on chest CT scans is a very effective method for the classification of COVID-19 patients. However, the diagnostic capacity of machine learning models would greatly improve with the introduction of additional classification groups, such as disease severity or other lung diseases. Therefore, we hope to adapt a machine learning architecture to receive multiple inputs and apply it to multi-modal data types, such as imaging, demographic and clinical. In addition to chest images, other inputs would include sex, age, comorbidities, or clinical lab results that can potentially improve COVID-19 prediction. An explicit comparison between classification on imaging data, clinical data, and a combination of the modalities will provide great insight into the clinical relevance of machine learning methods.