Building energy consumption prediction needs both time series features and categorical features, but traditional models can only deal with one of the features. Aiming at this problem, a new neural network integrating one-dimensional convolutional kernel and word embedding was proposed in this paper. The one-dimensional convolutional kernel can extract the continuous time series features, and the word embedding model can embed the discrete categorical features, based on which a building energy consumption prediction model is established that can simultaneously process both time series features and categorical features. By comparing with the gradient boosting decision regression tree and the long short time memory network, it is proved that the proposed model has good performance in efficiency and accuracy. In terms of hyperparameter adjustment, the automatic hyperparameter optimization algorithm based on Bayesian optimization was adopted, and the algorithm can find the optimal hyperparameter in the search space. Compared with manual optimization, the automatic hyperparameter optimization algorithm can improve the perfor-mance of the model in a relatively short time. Finally, simulation studies were conducted and the results demonstrate that the proposed model is better in performance than the ensemble learning model and the long short-time memory network.