Geochemical investigations on samples of pumice and lava extracted from the mortars and concretes of the Forum of Caesar and the Forum and Markets of Trajan are presented and integrated with previous analyses of lava samples from Rome and Pompeii, and of pumices from the Colosseum and the Dioclethian’s Bath, with the aim to identify the lithological provenance of these materials. Discrimination diagrams based on ratios of selected trace elements (i.e., Zr/Y, Nb/Y, Th/Ta, Nb/Zr) allow us to recognize the geochemical signature of the different volcanic regions of central Italy. These diagrams indicate that in the mortars of the Forum of Caesar and Forum of Trajan, the Roman builders integrated the scanty pumices from deposits of the Monti Sabatini Volcanic District near Rome with others coming from the Vesuvius and Phlegrean Fields. Some of the pumices employed in the vaulted ceilings of these monuments display a peculiar geochemistry that has a unique correspondence with a pyroclastic deposit recovered in a borehole drilled on the southern flanks of Vesuvius. Similarly, lava flows from this borehole display a different trace-element signature with respect to that of other coeval lava flows of the northern flanks of Vesuvius, and it has a closer correspondence to that of the archaeological lava samples. These observations, integrated with historical and archaeological data, indicate that the Roman builders had a profound empirical knowledge of the physical properties of the volcanic rocks from different regions, and suggest that a systematic cultivation of lightweight volcanic material occurred in the area of Pompeii for exportation to Rome and, likely, to other regions of the Mediterranean.