From Data Discovery to Data Visualization and Data Dissemination: Handling Large LiDAR datasets for a Statewide Customer in the Public Cloud
Significant challenges exist when managing data discovery and dissemination for Lidar data sources, ranging from data storage to data discovery to feature extraction through to data visualization. In this talk, we present the complete Lidar data pipeline as managed in our spatial database framework hosted in the public cloud for a statewide customer. This involves four distinct sections: (a) populating the database, (b) building a spatial hierarchy that supports lookup of the available data sources, (c) generating a visualization of Lidar data (or its subset) in a WebGL enabled web-application viewer, and (d) data analytics. The application has been designed to provide different levels of authenticated access based on user credentials, e.g., state employees see more data than the general public. Each of the footprints are displayed as wireframes and contain metadata as attributes. They can be previewed in the web-application viewer or downloaded directly. The users can also query for data footprints that meet specified conditions. Once a selection is made, users are presented with a list of attributes for all polygons selected. Restrictions can be applied on the download of multiple tiles or images at one time, based on overall size of download to user or the area size. Procedures can also be setup to convert all available data into multiple file formats and/or typical derivative products in anticipation of user needs. The data analytics tools supported in the web-application viewer include elevation profiling, area, height and length measurements. Users can change the point cloud display based on the elevation, intensity, RGB or classification. Vector layers (point, polyline or polygon shapefiles/geodatabases) can be overlaid on the Lidar data to allow users to compare the alignment of the vector data relative to the Lidar datasets, manipulate the point cloud in 3D, or annotate individual points or features. Standard deliverable products such as DEM or Intensity Rasters can be shown in the web-application viewer. The end user is able to download the data of interest or stream a raster version of the data (served as a WMS/WMTS) for consumption in any OGC-compliant software. Since the entire pipeline is hosted and managed on the public cloud, it is infinitely scalable, responsive to increased user loads or processes in real-time and has a 99.9% SLA and guaranteed uptime.