Building energy consumption modeling and forecasting is essential to address buildings energy efficiency problems and take up current challenges of human comfort, urbanization growth and the consequent energy consumption increase. In a context of integrated smart infrastructures, data-driven techniques rely on data analysis and machine learning to provide flexible methods for building energy prediction. The present paper offers a review of studies developing data-driven models for building scale applications. The prevalent methods are introduced with a focus on the input data characteristics and data pre-processing methods, the building typologies considered, the targeted energy end-uses and forecasting horizons, and accuracy assessment. A special attention is also given to different machine learning approaches. Based on the results of this review, the latest technical improvements and research efforts are synthesized. The key role of occupants’ behavior integration in data-driven modeling is discussed. Limitations and research gaps are highlighted. Future research opportunities are also identified.