Recently, people have been paying more and more attention to mental health, such as depression, autism, and other common mental diseases. In order to achieve a mental disease diagnosis, intelligent methods have been actively studied. However, the existing models suffer the accuracy degradation caused by the clarity and occlusion of human faces in practical applications. This paper, thus, proposes a multi-scale feature fusion network that obtains feature information at three scales by locating the sentiment region in the image, and integrates global feature information and local feature information. In addition, a focal cross-entropy loss function is designed to improve the network's focus on difficult samples during training, enhance the training effect, and increase the model recognition accuracy. Experimental results on the challenging RAF_DB dataset show that the proposed model exhibits better facial expression recognition accuracy than existing techniques.