In recent years, surface‐wave dispersion has been used to image lithospheric structure jointly with receiver function or Rayleigh‐wave ellipticity. Because surface‐wave dispersion is the total propagation effect of the travel path, the joint inversion relies on dense seismic arrays or high seismicity to obtain local velocity structure. However, both receiver function and Rayleigh‐wave ellipticity are single‐station measurements with localized sensitivities and could be combined for joint inversion naturally. In this article, we explored the feasibility of the joint inversion of Rayleigh‐wave ellipticity and receiver function. We performed sensitivity tests with forward modeling and found that the receiver function is sensitive to sharp velocity interfaces but shows weak sensitivity to long‐wavelength structure, almost complementary to Rayleigh‐wave ellipticity. Therefore, joint inversion with two single‐station measurements provides tighter constraints on the velocity structure beneath the seismic station. A joint inversion algorithm based on the fast simulated‐annealing method is developed to invert Rayleigh‐wave ellipticity and receiver function for the lithospheric structure. Application of the algorithm to the Indian craton and the Williston basin in the United States demonstrates its effectiveness in reducing the nonuniqueness of the inversion. However, the joint inversion may fail to resolve the average crustal velocity, suggesting the need to combine surface‐wave dispersion (or other type of observations), receiver function, and Rayleigh‐wave ellipticity to more accurately resolve the velocity structure.