Purpose: To assess the use of deep learning for high-performance image classification of color-coded corneal maps obtained using a Scheimpflug camera. Methods:We used a domain-specific convolutional neural network (CNN) to implement deep learning.CNNperformancewas assessed using standard metrics and detailed error analyses, including network activation maps. Results: The CNN classified fourmap-selectable display images with average accuracies of 0.983 and 0.958 for the training and test sets, respectively. Network activation maps revealed that the model was heavily influenced by clinically relevant spatial regions. Conclusions: Deep learning using color-coded Scheimpflug images achieved high diagnostic performance with regard to discriminating keratoconus, subclinical keratoconus, and normal corneal images at levels that may be useful in clinical practice when screening refractive surgery candidates. Translational Relevance: Deep learning can assist human graders in keratoconus detection in Scheimpflug camera color-coded corneal tomography maps.