Given historical sequential sets of elements such as purchasing items from time to time, could be formalized as sequential sets, namely temporal sets. In practice, most of the existing research focuses on time series and temporal events. Different from the previous research, this paper aims at developing prediction methods for temporal sets. If we formalize the time series as numerical values with timestamps and the temporal events as nominal items with timestamps, then temporal sets could be seen as a sequence of sets with timestamps, where each set consists of an irregular number of items. It is very challenging to model and predict such temporal sets due to the difficulty of set representation, dynamic temporal dependence of historical sets, and fusion of user preference. To address these issues, we propose a novel Deep Heterogeneous Network for Temporal Sets Prediction (DHNTSP) in this paper. We first provide a set representation method based on Heterogeneous Information Network (HIN) embedding, where HIN is used to model the multiple-perspective relationships among sets, items, users and categories, and matrix factorization is used to vectorize the set nodes of HIN. Then, an attention-based recurrent module is designed to learn the temporal dependence of next-period set with historical sets. Next, we integrate the current temporal dynamics of set sequence with user preference to get the representation of next-period set, and then predict the set by an end-to-end model. Finally, experiments are conducted on real-world datasets, and results demonstrate that DHNTSP outperforms the state-of-the-art methods.