This article presents numerical computations of two-dimensional transient responses of canyons and alluvium deposits with various configurations subject to vertically and obliquely incident SH waves. The time domain boundary element method (BEM) is used for the analysis. Elementwise double integrals over space and time domains are performed analytically for the first time. The substructure formulation is used for the analysis of alluvium deposits. The validity of the present solution is examined for a semi-circular canyon and an alluvium whose frequency domain solutions are available from the past publications. In the following analyses, the transient responses are computed for specific Ricker wavelets under different angles of incidences for various site configurations. The results show that the proposed technique is reliable and therefore can be used to study the seismic response at irregular sites.