Electroencephalographic (EEG)-based emotion recognition has received increasing attention in the field of human-computer interaction (HCI) recently, there however remains a number of challenges in building a generalized emotion recognition model, one of which includes the difficulty of an EEG-based emotion classifier trained on a specific task to handle other tasks. Little attention has been paid to this issue. The current study is to determine the feasibility of coping with this challenge using feature selection. 12 healthy volunteers were emotionally elicited when conducting picture induced and video induced tasks. Firstly, support vector machine (SVM) classifier was examined under within-task conditions (trained and tested on the same task) and cross-task conditions (trained on one task and tested on another task) for picture induced and video induced tasks. The within-task classification performed fairly well (classification accuracy: 51.6% for picture task and 94.4% for video task). Cross-task classification, however, deteriorated to low levels (around 44%). Trained and tested with the most robust feature subset selected by SVM-recursive feature elimination (RFE), the performance of cross-task classifier was significantly improved to above 68%. These results suggest that cross-task emotion recognition is feasible with proper methods and bring EEG-based emotion recognition models closer to being able to discriminate emotion states for any tasks.