Phase transformation of amorphous calcium carbonate (ACC) into vaterite and its subsequent stability was investigated at a constant pH (~8.2), ionic strength, and temperature that simulated the biological environment. Solutions containing the same concentrations of CaCl2, Na2CO3, and tris(hydroxymethyl)aminomethane buffer and various concentrations of PO4 (0–62.5 μM) were prepared, and precipitates in the solutions were sampled at a constant interval to observe the morphology and type of calcium carbonate polymorphs that appeared. The change in the Ca-ion concentration over time, which served as a guide for phase transformation of ACC into crystalline phases, was measured in relation to the PO4 concentration. The starting time of phase transformation was at the minimum point when the concentration was ~2–3 μM. Vaterite spherulites consisting of needle-like crystals (0.5–2 μm in length) formed only in this PO4 range and survived the experimental procedure (~2.5 h). In contrast, the starting time of phase transformation increased exponentially with the PO4 concentration when it was higher than 5 μM. The vaterite spherulites and calcite crystals co-precipitated, and both polymorphs grew over time. The PO4 was shown to be an accelerator for phase transformation from ACC into vaterite at low concentrations (Ca/PO4 molar ratio <3000) and an inhibitor for transformation at high concentrations. We investigated the kinetics of vaterite formation in the presence of PO4 and derived an advanced concept for cluster-based phase transformation. This investigation showed that the appearance and stability of calcium carbonate polymorphs is easily controlled by adjusting the PO4 concentration.