Spatial Locality as the Governing Design Constraint for Petabyte-Scale Point Cloud Platforms
Keywords:
Distributed Storage Systems, LiDAR Data Infrastructure, Point Cloud Data Management, Spatial Indexing, Spatial PartitioningAbstract
Point cloud platforms built for autonomous vehicles, robotics, and digital twin systems tend to inherit their architecture from general-purpose big data infrastructure, with spatial partitioning bolted on as an implementation detail rather than treated as a governing constraint. This article takes the opposite position: spatial locality — the principle that data representing nearby physical regions should sit close together in storage and retrieval paths — belongs at the center of point cloud platform design, from ingestion through retrieval, not layered on as an afterthought. The argument traces how point cloud infrastructure draws on two separate lineages, spatial indexing structures and distributed storage systems, without being fully served by either on its own. It examines spatial indexing as the foundational design problem, distributed storage architecture adapted for spatial partitioning, metadata as the coordination layer governing usability at scale, and retrieval and visualization as the point where these design choices pay off or fail. The discussion extends to autonomous transportation, robotics, and digital twins, framed as consumers of a well-designed spatial platform rather than the source of its design requirements. Platforms treating spatial locality as secondary accumulate storage capacity while retrieval performance degrades as datasets grow — the reverse of what these workloads need.
Downloads
References
Bentley, J. L. (1975). Multidimensional binary search trees used for associative searching. Communications of the ACM, 18(9), 509–517. https://doi.org/10.1145/361002.361007
Meagher, D. (1982). Geometric modeling using octree encoding. Computer Graphics and Image Processing, 19(2), 129–147.
Rusu, R. B., & Cousins, S. (2011). 3D is here: Point Cloud Library (PCL). Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA), 1–4.
Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3354–3361.
Ghemawat, S., Gobioff, H., & Leung, S.-T. (2003). The Google file system. Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP), 29–43. https://doi.org/10.1145/945445.945450
Dean, J., & Ghemawat, S. (2004). MapReduce: Simplified data processing on large clusters. Proceedings of the 6th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 137–150.
Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauley, M., Franklin, M. J., Shenker, S., & Stoica, I. (2012). Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. Proceedings of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 15–28.
Graziosi, D., Nakagami, O., Kuma, S., Zaghetto, A., Suzuki, T., & Tabatabai, A. (2020). An overview of ongoing point cloud compression standardization activities: Video-based (V-PCC) and geometry-based (G-PCC). APSIPA Transactions on Signal and Information Processing, 9, e13. https://doi.org/10.1017/ATSIP.2020.12
Chang, F., Dean, J., Ghemawat, S., Hsieh, W. C., Wallach, D. A., Burrows, M., Chandra, T., Fikes, A., & Gruber, R. E. (2008). Bigtable: A distributed storage system for structured data. ACM Transactions on Computer Systems, 26(2), Article 4. https://doi.org/10.1145/1365815.1365816
Corbett, J. C., Dean, J., Epstein, M., Fikes, A., Frost, C., Furman, J. J., Ghemawat, S., Gubarev, A., Heiser, C., Hochschild, P., Hsieh, W., Kanthak, S., Kogan, E., Li, H., Lloyd, A., Melnik, S., Mwaura, D., Nagle, D., Quinlan, S., Rao, R., Rolig, L., Saito, Y., Szymaniak, M., Taylor, C., Wang, R., & Woodford, D. (2012). Spanner: Google's globally-distributed database. Proceedings of the 10th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 251–264.
Zaharia, M., Xin, R. S., Wendell, P., Das, T., Armbrust, M., Dave, A., Meng, X., Rosen, J., Venkataraman, S., Franklin, M. J., Ghodsi, A., Gonzalez, J., Shenker, S., & Stoica, I. (2016). Apache Spark: A unified engine for big data processing. Communications of the ACM, 59(11), 56–65. https://doi.org/10.1145/2934664
Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., Caine, B., Vasudevan, V., Han, W., Ngiam, J., Zhao, H., Timofeev, A., Ettinger, S., Krivokon, M., Gao, A., Joshi, A., Zhang, Y., Shlens, J., Chen, Z., & Anguelov, D. (2020). Scalability in perception for autonomous driving: Waymo Open Dataset. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2446–2454.
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020). nuScenes: A multimodal dataset for autonomous driving. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11621–11631.
Wilson, B., Qi, W., Agarwal, T., Lambert, J., Singh, J., Khandelwal, S., Pan, B., Kumar, R., Hartnett, A., Pontes, J. K., Ramanan, D., Carr, P., & Hays, J. (2023). Argoverse 2: Next generation datasets for self-driving perception and forecasting. Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track.
Lokugam Hewage, C. N., Laefer, D. F., Vo, A.-V., Le-Khac, N.-A., & Bertolotto, M. (2022). Scalability and performance of LiDAR point cloud data management systems: A state-of-the-art review. Remote Sensing, 14(20), 5277. https://doi.org/10.3390/rs14205277
Teuscher, B., & Werner, M. (2025). Point cloud data management for analytics in a lakehouse. AGILE GIScience Series, 6, Article 47.
Teijeiro, D., Amor, M., Doallo, R., & Deibe, D. (2023). Interactive visualization of large point clouds using an autotuning multiresolution out-of-core strategy. The Computer Journal, 66(7), 1802–1816. https://doi.org/10.1093/comjnl/bxac179
Nguyen, M. H., Yoon, S., Ju, S., Park, S., & Heo, J. (2022). B-EagleV: Visualization of big point cloud datasets in civil engineering using a distributed computing solution. Journal of Computing in Civil Engineering, 36(3), Article 04022005. https://doi.org/10.1061/(ASCE)CP.1943-5487.0001021
Pajić, V., Govedarica, M., & Amović, M. (2018). Model of point cloud data management system in big data paradigm. ISPRS International Journal of Geo-Information, 7(7), 265. https://doi.org/10.3390/ijgi7070265
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
