A borehole radar (BHR) prototype system was developed for the exploration of complicated oil and gas reservoirs. To verify the performance of the system, single-hole reflection imaging experiments were carried out in an abandoned limestone mine. In the physical experiments, the cliff wall and a metal plate were selected as the imaging targets to evaluate the detection performance of the prototype system. The average filter method was used to remove the background noise, then the frequency–wave number (F-K) imaging algorithm was adopted for radar imaging. The unknown fractures surrounding the borehole produced complex reflections that were not beneficial to effectively extract the target echo when the down-hole sensor was shifted along the borehole. However, by fixing the down-hole sensor and shifting the target, the detection range of the radar system extended up to about 10 m in the limestone formation. A 2-D finite-difference time-domain (FDTD) modeling method was also implemented to simulate the experimental procedure, and demonstrated that the prototype system can provide enough accuracy to predict the echo signal characteristics and reproduce the radar response in the formation. The combination of field experiment, theoretical analysis, and numerical simulation not only objectively validated the fundamental performance of the radar prototype, but also generated some new concepts for further improvement on the radar system design.