Cellular network is facing a severe traffic overload problem caused by the phenomenal growth of mobile data. Offloading part of the mobile data traffic from the cellular network to alternative networks is a promising solution. In this paper, we study the mobile data offloading problem under the architecture of mobile cloud computing, where mobile data can be delivered by WiFi network and device-to-device communication. In order to minimize the overall cost for the data delivery task, it is crucial to reduce cellular network usage while satisfying delay requirements. In our proposed model, we formulate the data offloading task as a finite horizon Markov decision process. We first propose a hybrid offloading algorithm for mobile data with different delay requirements. Moreover, we establish sufficient conditions for the existence of threshold policy. Then, we propose a monotone offloading algorithm based on threshold policy in order to reduce the computational complexity. The simulation results show that the proposed offloading approach can achieve minimal communication cost compared with the other three offloading schemes.