The crystal structure of halloysite, despite being one of the most commonly occurring clay minerals on the Earth’s surface, remains elusive. This paper reports on a multi-methodological study to shed more light on the atomic arrangement of halloysite from Olkhon Island, Lake Baikal, Russia, which has a prismatic morphology. It has been investigated using X-ray diffraction (XRD), scanning electron microscopy (SEM), selected-area electron diffraction (SAED) and, in particular, high-resolution transmission electron microscopy (HRTEM), to reveal its atomic structure and formation process. XRD analysis indicated a basal spacing of ca. 7.2 Å and two characteristic peaks with d = 4.28 and 4.03 Å on the tail of the 02,11 band. The halloysite grain cross sections displayed various prismatic forms, ranging from a rectangle to a regular 18-sectored polygon in SEM and TEM. The SAED pattern from a sector of the polygonal prisms with the incident beam parallel to the prism axes showed a regular one-layer oblique reciprocal lattice, similar to that of kaolinite along Yi-directions. HRTEM imaging performed with the new computer-assisted minimal-dose system and an incident beam perpendicular to the prism axis showed stacking of most dioctahedral 1:1 layers with their pseudo-mirror plane perpendicular to the prism axis and an almost random, or rather, alternating lateral stagger to the right or left from the preceding layer, which corresponds to interlayer displacements of τ+ and τ− in Zvyagin symbols, or layer displacements of t1 and t2 used to describe the stacking in kaolinite. This stacking feature explains the SAED pattern from the side of the prismatic grains and the two characteristic peaks on the tail of the 02,11 band in the XRD pattern. Based on these results, it is proposed that tubular halloysite initially forms as a hydrated one with the pseudo-mirror plane of the kaolinite layers perpendicular to the tube axis, then dehydrates with, possibly, partially hydrogen-bonded interlayers, and finally transforms to a prismatic one consisting of sectored flat layers with the complete hydrogen-bonded interlayers. During this transformation, stacking with comparable ratio and frequent alternation of τ+ and τ− is formed, to minimize morphological change of the tubes.