This paper studies the problem of Cover Source Mismatch (CSM) in steganalysis, i.e. the impact of a testing set which does not originate from the same source than the training set. In this study, the trained steganalyzer uses state of the art deep-learning architecture prone to better generalization than feature-based steganalysis. Different sources such as the sensor model, the ISO sensitivity, the processing pipeline and the content, are investigated. Our conclusions are that, on one hand, deep learning steganalysis is still very sensitive to the CSM, on the other hand, the holistic strategy leverages the good generalization properties of deep learning to reduce the CSM with a relatively small number of training samples.