This paper proposes a novel grading method of apples, in an automated grading device that uses convolutional neural networks to extract the size, color, texture, and roundness of an apple. The developed machine learning method uses the ability of learning representative features by means of a convolutional neural network (CNN), to determine suitable features of apples for the grading process. This information is fed into a one-to-one classifier that uses a support vector machine (SVM), instead of the softmax output layer of the CNN. In this manner, Yantai apples with similar shapes and low discrimination are graded using four different approaches. The fusion model using both CNN and SVM classifiers is much more accurate than the simple k-nearest neighbor (KNN), SVM, and CNN model when used separately for grading, and the learning ability and the generalization ability of the model is correspondingly increased by the combined method. Grading tests are carried out using the automated grading device that is developed in the present work. It is verified that the actual effect of apple grading using the combined CNN-SVM model is fast and accurate, which greatly reduces the manpower and labor costs of manual grading, and has important commercial prospects.