Data Augmentation and Multimodal Learning for Predicting Drug Response in Patient-derived Xenografts from Gene Expressions and Histology Images

Patient-derived xenografts (PDXs) are an appealing platform for preclinical drug studies. A primary challenge in modeling drug response prediction (DRP) with PDXs and neural networks (NNs) is the limited number of drug response samples. The authors investigate multimodal neural network (MM-Net) and data augmentation for DRP in PDXs. The MM-Net learns to predict response using drug descriptors, gene expressions (GE), and histology whole-slide images (WSIs). The authors explore whether combining WSIs with GE improves predictions as compared with models that use GE alone. The authors propose the following data augmentation methods which allow the authors to train multimodal and unimodal NNs without changing architectures with a single larger dataset:

• Combine single-drug and drug-pair treatments by homogenizing drug representations, and
• Augment drug-pairs which doubles the sample size of all drug-pair samples.

The authors compared unimodal NNs which use GE to assess the contribution of data augmentation. The NN that uses the original and the augmented drug-pair treatments as well as single-drug treatments outperforms NNs that ignore either the augmented drug-pairs or the single-drug treatments. In assessing the multimodal learning based on the Matthews correlation coefficient (MCC) metric, MM-Net outperforms all the baselines. The results show that data augmentation and integration of histology images with GE can improve prediction performance of drug response in PDXs.