Plant phenomics is a new avenue for linking plant genomics and environmental studies, thereby improving plant breeding and management. Remote sensing techniques have improved high-throughput plant phenotyping. However, the accuracy, efficiency, and applicability of three-dimensional (3D) phenotyping are still challenging, especially in field environments. Light detection and ranging (lidar) provides a powerful new tool for 3D phenotyping with the rapid development of facilities and algorithms. Numerous efforts have been devoted to studying static and dynamic changes of structural and functional phenotypes using lidar in agriculture. These progresses also improve 3D plant modeling across different spatial-temporal scales and disciplines, providing easier and less expensive association with genes and analysis of environmental practices and affords new insights into breeding and management. Beyond agriculture phenotyping, lidar shows great potential in forestry, horticultural, and grass phenotyping. Although lidar has resulted in remarkable improvements in plant phenotyping and modeling, the synthetization of lidar-based phenotyping for breeding and management has not been fully explored. We identify three main challenges in lidar-based phenotyping development: 1) developing low cost, high spatial-temporal, and hyperspectral lidar facilities, 2) moving into multi-dimensional phenotyping with an endeavor to generate new algorithms and models, and 3) embracing open source and big data.