In modeling waves radiated from a borehole seismic source in layered isotropic or anisotropic media, the commonly used numerical methods (e.g., finite difference and finite element) encounter difficulties because of the large scale difference between the borehole diameter and the formation extent. To get around this problem, we apply the indirect boundary element method to establish a general algorithm for modeling source radiation from open and cased boreholes in layered transversely isotropic (TI) media. The essence of the algorithm is to use discrete secondary sources (unknowns) on both sides of the borehole wall (formation/cement interface) to represent the influence of the interface on wave scattering, so that wave propagation inside and outside the borehole can be carried out by Green's functions. The discrete distribution of the secondary sources is determined by matching boundary conditions on the borehole wall. Comparison with the discrete wavenumber method validates the implementation. Applications to fluid-filled open and cased boreholes in three-layer media demonstrate the creation of guided modes in low velocity layers. Presence of anisotropy complicates the guided modes as a result of dispersion and P- and SV-waves coupling in homogeneous TI media. Presence of casing and cement enhances the visibility of the guided modes.