Rayleigh wave dispersion curve inversion is a non-linear iterative optimization process with multi-parameter and multi-extrema. It is difficult to carry out inversion and reconstruction of stratigraphic parameters quickly and accurately with a single linear or non-linear inversion for the data processing of Rayleigh waves with complex seismic geological conditions. We proposed a new method that combines artificial bee colony algorithm (ABC) and damped least squares algorithm (DLS) to invert Rayleigh wave dispersion curve. First, food sources are initialized in a large scale of the model based on the prior geological information. Then, after three kinds of bee operators (employed bees, onlooker bees and scout bees) transform each other and perform search optimization with several iterations, the targets are converged near the optimal solution to obtain an initial S-wave velocity model. Finally, the final S-wave velocity model is obtained by local optimization of DLS inversion with fast convergence and strong stability. The correctness of the method has been verified by one high-velocity interlayer model, and it was further applied to a real Rayleigh wave dataset. The results show that our method not only absorbs the advantages of ABC global search optimization and strong adaptability, but also makes full use of the advantages of DLS inversion, such as high accuracy and fast convergence speed. The inversion strategy can effectively suppress the inversion falling into local extrema, get rid of the dependence on an initial model, enhance the inversion stability, further improve the convergence speed and inversion accuracy, while has good anti-noise ability.