In recent years, deep learning makes great breakthroughs in a variety of applications. Its motivation is to learn feature from raw data. However, the performance of deep learning method applied to security-sensitive applications is poor. In adversarial environment, there are adversaries who attempt to evade valid classification by manipulating training and test samples. Thus, test data does not follow the same distribution of training data. Robustness of stacked autoencoder must be concerned under adversarial environment. In this paper, we investigate the attacking principle and try to improve the robustness of stacked autoencoder. A simple but effective objective function that not only minimizes empirical risks but also improves the robustness of stacked autoencoder against evasion attack is proposed. Our proposed method is based on weight evenness which presents better classification performance under evasion attack. We verify this method on the MNIST dataset. The robustness of our model is compared with conventional stacked autoencoder. The experimental results show that the proposed method is effective against evasion attack and enhances the security of stacked autoencoder under adversarial environment.