Plant height is one of the most important agronomical traits in rice (Oryza sativa L.). Introducing the semidwarf rice in the 1960s significantly enhanced the rice yield potential in Asia. Implementing near-isogenic lines (NILs) is the most powerful tool for the identification and fine mapping of quantitative trait loci (QTLs). In this study, 176 NILs were produced from the crossing and back-crossing of two rice cultivars. Specifically, the indica rice cultivar Jiafuzhan served as a recipient, and the restorer japonica cultivar Hui1586 served as a donor. Using the 176 NILs, we identified a novel major QTL for reduced plant height in the NIL36 line. The qph12 QTL was mapped to a 31 kb genomic region between the indel markers Indel12-29 and Indel12-31. The rice genome annotation indicated the presence of three candidate genes in this genomic region. Through gene prediction and cDNA sequencing, we confirmed that LOC_Os12g40890 (qPH12) is the target gene in the NIL36 line. Further analysis showed that the qph12 QTL is caused by a 1 bp deletion in the first exon that resulted in premature termination of the qPH12. Knockout experiments showed that the qph12 QTL is responsible for the reduced plant height phenotype of the NIL36 line. Although the qph12 gene from the NIL36 line showed a shorter panicle length, fewer spikelets per panicle and a lower plant grain yield, the plant also exhibited a lower plant height. Taken together, our results revealed that the qph12 have good specific application prospects in future rice breeding.