The seismic horizon is a critical input for the structure and stratigraphy modeling of reservoirs. It is extremely hard to automatically obtain an accurate horizon interpretation for seismic data in which the lateral continuity of reflections is interrupted by faults and unconformities. The process of seismic horizon interpretation can be viewed as segmenting the seismic traces into different parts and each part is a unique object. Thus, we have considered the horizon interpretation as an object detection problem. We use the encoder-decoder convolutional neural network (CNN) to detect the “objects” contained in the seismic traces. The boundary of the objects is regarded as the horizons. The training data are the seismic traces located on a user-defined coarse grid. We give a unique training label to the time window of seismic traces bounded by two manually picked horizons. To efficiently learn the waveform pattern that is bounded by two adjacent horizons, we use variable sizes for the convolution filters, which is different than current CNN-based image segmentation methods. Two field data examples demonstrate that our method is capable of producing accurate horizons across the fault surface and near the unconformity which is beyond the current capability of horizon picking method.