Random noise and drifts often degrade the quality of the seismic data and harm further processing. The support vector machine (SVM) is a universal machine learning method based on the statistical learning theory, which has been widely used in classification and regression. On the basis of the least squares support vector regression (LS-SVR), we propose the least squares support vector regression with implicit bias term (ILS-SVR) to reduce the random noise and remove the drifts in seismic data. The proposed method stacks the weight vector and the bias term into a new vector, and incorporates the bias term into the objective function of the optimization problem in the LS-SVR model. As a result, for the translation invariant kernel we obtain a simpler solution that can be computed more efficiently than the LS-SVR. More importantly, the ILS-SVR with a Ricker wavelet kernel can not only effectively suppress the random noise, but also remove drifts in seismic data. Behind the proposed method, the implicit bias term and the special choice of the kernel (such as the Ricker wavelet kernel) are two key ingredients to remove drifts. We test the proposed method on both synthetic and real seismic data, and compare it with the ILS-SVR using the popular radial basis function (RBF) kernel, the f-x prediction filtering, and the wavelet transform-based method. Our experimental results illustrate that the proposed algorithm achieves better performance over the other three algorithms.