The use of three-dimensional (3-D), photo-textured representations of topography from laser scanning and photogrammetry is becoming increasingly common across the geosciences. This rapid adoption is driven by recent innovations in acquisition hardware, software automation, and sensor platforms, including unmanned aerial vehicles. In addition, fusion of surface geometry with imaging sensors, such as multispectral, hyperspectral, thermal, and ground-based radar, and geophysical methods creates complex and visual data sets that provide a fundamental spatial framework to address open geoscience research questions.
Despite the current ease of acquiring and processing 3-D photo-textured models, the accessibility of tools for analyzing and presenting data remains problematic, characterized by steep learning curves and custom solutions for individual geoscience applications. Interpretation and measurement is essential for quantitative analysis of 3-D data sets, and qualitative methods are valuable for presentation purposes, for planning, and in education. This contribution presents LIME, a lightweight and high-performance 3-D software for interpreting and co-visualizing 3-D models and related image data. The software allows measurement and interpretation via digitizing in the 3-D scene. In addition, it features novel data integration and visualization of 3-D topography with image sources such as logs and interpretation panels, supplementary wavelength imagery, geophysical data sets, and georeferenced maps and images. High-quality visual output can be generated for dissemination to aid researchers with communication of their results. The motivation and an overview of the software are described, illustrated by example usage scenarios from outcrop geology, multi-sensor data fusion, and geophysical-geospatial data integration.
The use of digital spatial data is becoming commonplace in many areas of the geosciences (for example in geology, geomorphology, cryospheric science, volcanology, natural hazards, hydrology, energy, infrastructure, and mining). Laser scanning (lidar), global navigation satellite systems (GNSS), digital photogrammetry (also referred to as structure from motion [SfM]), and imaging remote sensing have all evolved heavily over the last two decades, and are distinguished by unprecedented resolution, precision, and ease of use. Computing hardware has developed to facilitate field-based acquisition (Weng et al., 2012; Kehl et al., 2017), and more analysis tools are available in geographical information systems (GIS). The evolving state of the art can be followed through the scientific literature, from early adoption to the status quo. This is exemplified by rapid adoption across the many arms of the geoscience subdisciplines (see, e.g., McCaffrey et al. , Pringle et al. , Buckley et al. [2008a], Kääb , Pavlis et al. , Jaboyedoff et al. , Hodgetts , Bemis et al. , Eitel et al. , Eltner et al. , Kehl et al. , and Squelch  as a small subset of papers providing snapshots of developments through time).
The reinvention of photogrammetry—a technique that has advanced in line with photographic innovations—has recently intensified the adoption of three-dimensional (3-D) spatial data by geoscientists. This has been driven by the ubiquity of digital cameras and major increases in automation arising from scientific outputs from the computer vision discipline (e.g., interest operators, feature point matching, and dense point cloud extraction; Granshaw and Fraser, 2015), which have been implemented in easy-to-use software packages. In addition, new dynamic sensor platforms such as unmanned aerial vehicles (UAVs) and mobile mapping systems allow 3-D acquisition in a wide range of configurations, and extended-spectral-range imaging sensors (multispectral and hyperspectral, thermal, radar; Eitel et al., 2016) provide new possibilities for complementing purely geometric approaches (Buckley et al., 2013). Finally, societal policy shifts have moved toward making geospatial data freely available to the public through public- or private-sector initiatives (Krishnan et al., 2011; U.S. Geological Survey, 2017; Norwegian Mapping Authority, 2017).
These developments have facilitated workflows for obtaining 3-D digital representations of surface topography across the range of scales (i.e., hand sample to regional elevation models). Consequently, geoscientists are empowered to obtain their own spatial data, which once would have been unachievable without expensive equipment and high expertise levels. The result is the current rapid adoption of digital spatial data across the diverse arms of the geosciences (Buckley et al., 2017).
Although data acquisition and processing have become more automated—embodied by the empowerment of field geoscientists who, armed with digital cameras and easy-to-use SfM software, are now able to capture and process image sets to make 3-D models, even during a field campaign—the exploitation of this data for many geoscience applications continues to lag behind. Applications commonly demand custom analysis solutions; however, interpretation, visualization, and dissemination needs are shared by many different geoscience subdisciplines. The limited availability of accessible and fit-for-purpose software tools has been a continued barrier to adoption of 3-D spatial data for new users (e.g., Hodgetts, 2013). At the same time, quantitative and qualitative analysis is critical to maximizing the full value of the 3-D spatial data sets now achievable. Thus, there is still a need for cross-disciplinary software tools that allow geoscientists to achieve simple measurement, interpretation, and visualization tasks, as well as share their results with collaborators. To address this need, this paper presents LIME as a software resource for the geoscience community and beyond. LIME focuses on the high-performance rendering of meshes, with or without texture, and provides simple tools for measurement and interpretation. In addition, new co-visualization methods allow placement and integration of 3-D models and two-dimensional (2-D) image data. We describe the motivation for creating LIME, the architecture, and main features, and illustrate our contribution with relevant usage scenarios in geoscience, primarily geology, applications.
BACKGROUND AND MOTIVATION
LIME has its origins in the need for geologists to have simple tools for performing measurements and interpretation on 3-D terrestrial lidar models in research and in education. In the mid-2000s, procedures for acquiring and, especially, processing terrestrial lidar data were relatively cumbersome compared to today’s possibilities. Large point clouds and photo-textured meshes frequently pushed standard computing hardware to the limits. In addition, software tools designed for geoscience purposes were not widely available, requiring users to utilize, e.g., complex lidar post-processing or GIS software to perform relatively modest measurement or interpretation operations (Hodgetts, 2013). This resulted in a significant barrier for researchers and students to take advantage of the benefits of including 3-D data sets in their studies. Indeed, early work with lidar data required geoscience students with little background in geomatics or geospatial data handling to process data sets to point clouds or photo-textured models, a stage both inefficient and not central to their project outcomes, though not for their general development and understanding of spatial data sets and their potential limitations.
The need for a high-performance, simple-to-use software for geoscience users to quickly start working with visual 3-D data sets was the initial motivation for creating LIME. Although data acquisition and processing have become easier, and computing power has been vastly improved with 64-bit processors and more powerful graphics processing units (GPUs), issues in these areas continue to exist (e.g., Hodgetts, 2013). Initially, LIME provided measurement and digitization tools to aid quantitative interpretation, as well as level-of-detail rendering of large digital topographic 3-D models, where model resolution is adaptively loaded according to the user’s current position and zoom when navigating the 3-D scene. Over time, the software has additionally become increasingly focused on novel co-visualization of disparate geoscience data types (e.g., Buckley et al., 2013) and visual presentation (dissemination) of 3-D environments. This has resulted in LIME as a software framework for compiling spatial and non-spatial data sets, to aid the interpretation process by integrating tools in a coherent implementation useful to geoscientists.
Another important motivation for LIME has been to facilitate the dissemination of 3-D model data sets and resulting measurements, interpretations, and visualizations, either by sharing data sets with collaborators or as outputs for academic presentation. As a lightweight executable, the software can be shared easily with partners and in its most basic form functions as an uncluttered 3-D viewer. The advanced graphical data fusion framework can be used to generate high-quality figures for use in journal articles or posters, images for use in slideshows, and input image sequences for generating movies. In addition, orthorectified images can be created based on a user-defined viewport, which can be exported to give new synthetic views of the 3-D environment, e.g., where interpretations or additional images are overlain on loaded data objects.
LIME is not a unique initiative for solving the lack of available analysis and visualization software within the geosciences. Indeed, parallel efforts have been presented by a number of groups. In natural hazards research, Jaboyedoff et al. (2007) developed Coltop3D for structural characterization of lidar point clouds. For exposed outcrops in geology and petroleum geology, software such as Virtual Reality Geological Studio (Hodgetts, 2013) allows extensive 3-D interpretation and modeling of sedimentary and structural features in point clouds and photo-textured models. Hardebol and Bertotti (2013) presented DigiFract for characterizing fracture data in images of geological outcrops. For point cloud analysis, CloudCompare (Girardeau-Montaut, 2017) is currently the software of choice for many working with photogrammetric and laser scanning data sets. This software is extended with plugins, such as qFacets (Dewez et al., 2016) for structural orientation analysis needed, e.g., in geology and natural hazards applications.
Within this constellation of relevant software packages, LIME currently focuses on the importation of triangular surface meshes rather than point clouds. This was an early design principle motivated by the general need of geologists (though not exclusively) to see both geological overviews and close-up details at the pixel scale of the overlaid photographic texture, where the discontinuous nature of point clouds can be problematic for viewing purposes (Buckley et al., 2008a; Hodgetts, 2013). Point clouds require a simpler data structure and can therefore be efficiently rendered, whereas triangular meshes can be more complex to create and require more graphical elements to be displayed. However, the continuous surface representation obtained using triangular meshes is highly beneficial if the rendering of the larger data sets is handled in an appropriate manner. To this end, LIME allows display of level-of-detail photo-textured models, such as the octree-based processing described by Buckley et al. (2008b), allowing scalable level-of-detail rendering of large-area terrain data.
Combining measurement and interpretation tools with high-performance rendering and co-visualization of different data sets makes LIME a simple tool for analyzing and presenting digital geoscience data sets.
IMPLEMENTATION AND FRAMEWORK
LIME is written in the C++ software language and makes use of the cross-platform libraries OpenSceneGraph (OSG) for 3-D rendering (e.g., Wang and Qian, 2010) and wxWidgets (wxWidgets, 2017) for the graphical user interface (GUI). Both are long-running and stable open-source projects widely employed in scientific and industry projects. OSG presents a high-level interface to OpenGL, which is itself an industry-standard library for high-performance 2-D and 3-D graphics applications. While OpenGL provides advanced real-time rendering, it has few methods for loading and organizing 3-D objects in a common spatial environment. OSG encapsulates OpenGL and combines it with methods to create and interact with scene graphs—a concept used in 3-D graphics to allow organizing, positioning, transforming, and applying individual or grouped rendering settings to objects in a 3-D scene (e.g., Wang and Qian, 2010). In addition, OSG includes data loaders for many common file formats (e.g., .obj, .ply, .stl, .3ds) and makes use of the OpenGL Shading Language (GLSL) to define flexible and advanced GPU-based rendering using shaders (Rost, 2006). Currently, LIME is compiled for Microsoft Windows 64-bit platforms, though because of the software’s reliance on cross-platform libraries, this may be extended to other operating systems in the future.
The software features a GUI based around a 3-D viewer linked to a project database window, listing the available loaded data. The user can interact with the loaded data using standard operations (e.g., show/hide, remove, rename, etc.), create new elements, as well as change the visual properties of the elements contained within the 3-D scene (Fig. 1). Menus and toolbars give options to change properties of the global 3-D scene, e.g., the use of vertical exaggeration to aid perception of subtle geometric features such as changes in dip over wide areas. The user can use standard along-axis views, as well as define customized viewports to position the virtual camera relative to the 3-D scene. Viewports can be used to store location of static 2-D images rendered as publication figures or presentations, or for sharing with collaborators. Perspective and orthographic projections are available, the latter being useful for viewing the 3-D scene without perspective effects, such as for correlating layers in non-continuous model sections.
LIME uses an Extensible Markup Language (XML) format for storing project data. This hierarchical text format is readily editable and facilitates data exchange. All loaded project data are stored, though links to 3-D models and other larger files are used to save duplicating or reformatting original data sets, which may be gigabytes in size. A search wizard for updating links is provided in case the referred elements are moved to other disk locations. In addition to the project data, the LIME XML project describes all visual properties of loaded elements, applied viewports, and object visibility status, allowing the elements of a scene to be compiled and reloaded at a later time (e.g., when saving a particular 3-D scene created for a journal figure, or opening a project for a project meeting). Creation of a project file is optional, to allow 3-D models to be quickly loaded in a single operation.
Global projected coordinate systems, such as Universal Transverse Mercator (UTM), are supported directly in LIME, without associated problems caused by GPU hardware handling only single-precision 3-D transformations, and without the common requirement to offset x-y coordinates to a local system (e.g., Mach and Petschek, 2007). This allows large areas to be viewed, especially for large terrain databases requiring out-of-core memory management techniques to be employed (Lindstrom and Pascucci, 2002). At present, data are assumed to be loaded in the required project coordinate system, and with Cartesian coordinates.
The main features of LIME are centered on the import and handling of model data, visualization of surface and texture properties, measurement and interpretation tools, and integration of ancillary data sets. The key features and possibilities are outlined in the following section.
Data Import and Large-Data Handling
LIME can handle a number of industry-standard 3-D model formats commonly found in graphics software (examples are .obj, .ply, or .stl). In addition, OSG native binary formats allow for high-performance loading and rendering with graphics-card optimization such as texture compression; models can be exported to binary after import into LIME. Using the native graphics library support for out-of-core data loading, hierarchical terrain databases can be loaded and viewed without being restricted to physical hardware memory. LIME handles digital elevation models (DEMs), typically organized as quadtrees owing to the raster (2-D gridded) input format, and 3-D photo-textured models from lidar or photogrammetry, organized as octrees due to the additional high resolution in the vertical (z) direction (described in Buckley et al., 2008b). As well as 3-D mesh formats, the software can import ASCII grids such as those derived in GIS or reservoir modeling software. Support for a limited range of SfM and lidar software project formats is also present, allowing the 3-D footprints of registered image sets to be displayed relative to the loaded 3-D scene.
Visualization of Surface and Texture Properties
One of the main features of LIME is the possibility to perform advanced manipulation of the visual properties of individual or multiple objects in the 3-D scene. All visual properties of the model are dynamically editable, and rely on composition of shaders (Rost, 2006), which are compiled and run on the GPU to achieve flexible and high-performance rendering. The software uses a layer-based approach to visualization, where the terrain surface is the base layer, which can be manipulated in different ways to change the visual presentation of the 3-D mesh. Image textures are shown as additional layers on top of the terrain surface, and the transparency of these layers may be individually altered to build up complex and visually appealing combinations of topographic and image data. The object terrain can be viewed with preset parameters, including elevation range, dip (slope; Fig. 2A), dip direction (aspect), strike, surface normal (Fig. 2B), and a combined dip–dip direction color wheel as introduced by Jaboyedoff et al. (2007) (Fig. 2C). These parameters are all calculated in real time using the GPU and OpenGL shaders (Rost, 2006), and are based on the surface normals of the loaded 3-D geometry. These surface normals are either included in the loaded 3-D data file or calculated based on 3-D vertices during file loading.
Multiple texture layers are supported, which may be conventional photo-textures linked to 3-D models (e.g., from photogrammetric, terrestrial lidar, airborne, or satellite terrain and imagery), additional image sources (e.g., multispectral or hyperspectral images; Buckley et al., 2013), attributes of the underlying terrain, or other arbitrary non-photorealistic or non-spatial image sources (described below). The number of texture layers that can be added is determined by the GPU hardware, typically a minimum of 16 layers, although performance may decrease with a higher number of textures as more memory is allocated. Contours and grids are calculated in real time and can be overlain as new layers (Fig. 2). Textures coming from image sources (as opposed to procedurally generated) can be dynamically modified for brightness and contrast, filtering of high-frequency detail, and setting of a specific color to be transparent (e.g., a background or legend class). All textures can be given a general layer transparency, and in addition, horizontal or vertical layer transparency can be altered, with or without a gradient transition, to allow visual comparison between layers and preparation of graphical material for export.
Artificial lighting can be applied to the individual 3-D models in the scene, giving the flexibility to choose which objects are lit. This is advantageous when meshes or surfaces without texture are combined with photo-textured models in the same view, as artificially lighting naturally lit photo-textures is often undesirable. Even finer-grained control is possible for 3-D models where the image texture extent does not equal the entire area of the available triangles. In this case, lighting may be applied to the underlying terrain surface only (Fig. 3).
Interpretation and Measurement
Provided interpretation tools within LIME allow digitization of line features and orientation planes based on intersection of cursor positions with the visible 3-D scene. Lines can be closed to form polygon outlines, and visual properties can be set to change the color, thickness, stipple pattern, etc. A line can be modified, allowing vertices to be changed or removed and new vertices inserted. Three-point planes can be plotted to measure and represent the orientation of surfaces, with visual display properties available as for 3-D meshes. Objects can be added into hierarchies of folders to group common feature types. LIME exports ASCII and XML project data to facilitate interchange with GIS, reservoir modeling, or other 3-D viewing software.
In addition to lines and planes, LIME introduces panel objects representing projection planes that may have one or more associated image textures. Panels are rectangular planar objects whose placement is defined using a three-point plane, by intersecting two points in the 3-D scene to create vertical panels, or numerically. Once placed, a panel acts as a blank image canvas for receiving projected 3-D scene objects, such as lines, planes, or 3-D models. Based on a user-defined pixel resolution, the projected result is an orthorectified image—where the image scale matches the 3-D scene—which can be exported and manipulated, e.g., in image-editing software. Furthermore, panels can be projected back into the 3-D scene to provide new overlay textures on 3-D models. Using this approach, we give two example usage scenarios: creating interpretation panels and annotating 3-D models.
The first example scenario can be used to create “filled” interpretations, rather than using the digitized lines alone, which results in more intuitive representations of interpreted features. For example, in geology, instead of representing a layer, fault zone, or architectural element with a digitized polygon around the feature, using a filled polygon is closer to the approach used in conventional interpretation panels (Ford and Pyles, 2014) and provides a more aesthetic result for presentation purposes. In LIME, lines are digitized using the editing tool and then projected to a panel object (Fig. 4A) to create a new texture image with transparent background (.png or .tif format). This image is exported and a simple flood fill operation applied to all polygons using the corresponding fill color (Fig. 4B). At this stage, further symbols or text can also be added in a drawing package. The saved image is then imported back into LIME as a new texture layer on the same panel object (Fig. 4C); as long as the image size and/or aspect ratio does not change, then the modified image retains the spatial properties of the LIME 3-D scene. Finally, the modified interpretation panel can be projected from panel space onto selected models in the 3-D scene, resulting in new texture layers being added to these models (Fig. 4D).
The second usage scenario relies on using panels for the creation of orthorectified images of the 3-D scene. Because orthorectified images can be used for direct measurement, and creation through the panel process preserves all visual properties of the captured scene, they are a useful output independent of further analysis or modification. However, edits to the exported orthorectified imagery can be again applied in image processing software, similar to in the first scenario, to create visual products for interpretation or communication purposes. In image processing software or a drawing package, it is possible to add new layers, which can be used as the basis for sketching on or annotating the orthorectified image. Saving only the modified image layer, containing the annotations on an otherwise transparent background, provides the input for adding a new panel texture layer in LIME. The panel texture can then be projected onto 3-D scene objects, as above (Fig. 5).
Placement, Co-Visualization, and Multi-Texture Overlay of Other Geoscience Data
In addition to using panels for aiding the interpretation process and presentation of analysis, LIME facilitates the introduction of additional spatial and non-spatial data into the 3-D scene. Surfaces or horizons created in reservoir modeling or GIS software (Enge et al., 2007) can be imported and viewed simultaneously with the 3-D photo-textured models (Fig. 1). This can be useful in outcrop analogue studies in geology, e.g., for quality controlling the veracity of interpolated or extrapolated 3-D data derived from input sparse interpretations.
Georeferenced imagery—such as topographic or geological maps, classifications, and orthoimages from UAV, aerial, or satellite imagery—can be loaded and draped onto models, surfaces, and panels in the 3-D scene (Fig. 6).
Panel objects can be used to position images in the 3-D scene, either using one of the placement options described above or by choosing a single 3-D point to anchor the image bottom left-hand corner. The user defines the dimensions of the image in 3-D space. Examples of usage are placing field or sample photos, interpretation panels, cross sections, or background slides into relevant positions in the 3-D environment. Such data may not have accurate, or indeed any, georeferencing, but can confer benefits when co-visualized with digital spatial data sets. As well as being able to be viewed in the 3-D scene, any panel image may be overlaid on available terrain models using the projection approach described. A further example is placing a sedimentological log image relative to a digital outcrop model (Fig. 7).
Panels can also be used to integrate spatial and geophysical data. In this case, geophysical sections of the subsurface, acquired using techniques such as ground-penetrating radar, can be placed as 2-D images into the 3-D scene relative to 3-D models or elevation data. Where geophysical acquisition is georeferenced, such as using GNSS data, this can be used to correctly position the resulting sections. The exposed topography can thus be co-visualized with the subsurface information. Furthermore, geophysical sections can be projected onto the 3-D models as additional texture layers, as for all panels (Fig. 8). Panels in LIME provide a flexible and powerful approach to linking accurate and high-resolution digital spatial data with ancillary field, laboratory, GIS, and multidisciplinary data sets.
Multi-sensor image data, acquired from multispectral, hyperspectral, or thermal cameras, are increasingly being integrated into geoscience studies, requiring new tools for co-visualization and data fusion. In LIME, the multiple-texture-layer approach allows registered additional sensor imagery to be overlaid on loaded 3-D data sets, as well as simultaneous display of models textured using multiple image sources. Buckley et al. (2013) described the integration of close-range hyperspectral imagery with 3-D models using the multi-layer display framework in LIME, and Figure 9 illustrates a further example where thermal image results from a UAV acquisition are co-visualized with a photogrammetric 3-D model generated from conventional camera images. A current limitation of LIME is that the co-registration of such additional image types must be performed externally, prior to overlay within the software.
Export of Presentation Material
High-resolution renderings of the 3-D scene can be captured using LIME, in lossy or lossless formats (.jpg, .tif). The user can choose the dimensions of the output image to create high-quality figures for presentations and journal articles or poster printing. The software can be set to output a sequence of .jpg images while the user navigates in the 3-D scene, which can then be compiled into movie files outside of LIME. Custom fly-throughs of the 3-D environment are a planned future feature. Finally, stereo rendering is supported, currently for anaglyph or quad-buffered outputs.
This paper presents LIME as an available software resource for the geoscientific community, motivated by the ongoing need to ease barriers to adoption of 3-D spatial data sets in all branches of the discipline. We described the background of the software, and outlined the current scope of LIME as a lightweight viewer focusing on visual inspection and interpretation, integration, and co-visualization of disparate spatial and non-spatial data sources, and high-quality presentation of results. The software is applicable for quantitative and qualitative analysis, as a planning tool for fieldwork, for educational purposes, as well as for dissemination. Although written primarily for geoscientists, LIME is applicable to other relevant fields, such as archaeology, cultural heritage, and environmental studies, and for supporting infrastructure projects.
As the convergence between geoscience, computer science, geomatics, and related disciplines becomes more apparent, increasingly complex techniques are available for geoscientists to make use of in their specific research areas. More than ever, there is the need for accessible and inclusive software tools for geoscientists operating at this interface between disciplines. Open issues remain, particularly related to data standards and interoperability, which requires the academic community to work with the public and private sectors to address.
LIME is a work in progress; it has developed in line with the needs and demands of many different research projects, and it continues to evolve. Planned future additions relate to adding to the different geoscience data types supported and increasing the range of interpretation options available. The software can be downloaded at http://virtualoutcrop.com/lime.
Parts of LIME have been developed to address research requirements in projects funded by the Research Council of Norway (RCN) through the Petromaks and Petromaks 2 programs. The following grants are acknowledged: 153264 (VOG [Virtual Outcrop Geology]; with Statoil ASA), 163316 (Carbonate Reservoir Geomodels [IRIS (International Research Institute of Stavanger)]), 176132 (Paleokarst Reservoirs [Uni Research CIPR]), 193059 (EUSA; with FORCE Sedimentology and Stratigraphy Group), 234152 (Trias North [University of Oslo]; with Deutsche Erdoel AG, Edison, Lundin, Statoil, and Tullow), 234111 (VOM2MPS [Uni Research CIPR]; with FORCE Sedimentology and Stratigraphy Group), as well as SkatteFUNN (RCN) project 266740. In addition, the SAFARI project consortium (http://safaridb.com) is thanked for its continued support. The OSG and wxWidgets communities are acknowledged for ongoing commitment to providing mature and powerful software libraries. All authors thank colleagues past and present for studies culminating in the presented figures: Kristine Smaadal and Aleksandra Sima (Figs. 1 and 4); Colm Pierce (Fig. 2A); Eivind Bastesen, Roy Gabrielsen and Haakon Fossen (Fig. 3); Christian Haug Eide (Fig. 7); Ivar Grunnaleite and Gunnar Sælen (Fig. 8); and Magda Chmielewska (Fig. 9). Isabelle Lecomte contributed to discussions on geospatial-geophysical data fusion. Bowei Tong and Joris Vanbiervliet are acknowledged for internal discussions during article revision. The lead author thanks Uni Research for providing a base funding grant to refine some of the presented features. Finally, authors Buckley and Dewez are grateful to Institut Carnot BRGM for the RADIOGEOM mobility grant supporting the writing of this paper. Corbin Kling and one anonymous reviewer helped improve the final manuscript.