Chromatin conformation capture (3C)(1) and high-throughput 3C (Hi-C)(2) assays allow the study of three-dimensional (3D) genome structures in cell populations or tissues, based on average proximities of folded DNA. However, differences between cells can be observed only by single-cell measurements that avoid ensemble averaging(3-5). To study 3D chromatin organization and dynamics before and after fertilization in flowering plants, we analysed the 3D genomes of rice eggs, sperm cells, unicellular zygotes and shoot mesophyll cells. We show that chromatin architectures of rice eggs and sperm cells are comparable to those of mesophyll cells and are reorganized after fertilization. The rice single-cell 3D genomes display specific features of chromosome compartments and telomere/centromere configuration compared to those in mammalian single cells. Active and silent chromatin domains combine to form multiple foci in the nuclear space. Notably, the 3D genomes of the eggs and unicellular zygotes contain a compact silent centre (CSC) that is absent in sperm cells. CSC appears to be reorganized after fertilization, and may be involved in the regulation of zygotic genome activation (ZGA). Our results reveal specific 3D genome features of plant gametes and the unicellular zygote, and provide a spatial chromatin basis for ZGA and epigenetic regulation in plants.