IGSNRR OpenIR
GeoSpark SQL: An Effective Framework Enabling Spatial Queries on Spark
Huang, Zhou1,2,3; Chen, Yiran1,2; Wan, Lin4; Peng, Xia5,6
2017-09-01
Source PublicationISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION
ISSN2220-9964
Volume6Issue:9Pages:20
Corresponding AuthorPeng, Xia(ivy_px@163.com)
AbstractIn the era of big data, Internet-based geospatial information services such as various LBS apps are deployed everywhere, followed by an increasing number of queries against the massive spatial data. As a result, the traditional relational spatial database (e.g., PostgreSQL with PostGIS and Oracle Spatial) cannot adapt well to the needs of large-scale spatial query processing. Spark is an emerging outstanding distributed computing framework in the Hadoop ecosystem. This paper aims to address the increasingly large-scale spatial query-processing requirement in the era of big data, and proposes an effective framework GeoSpark SQL, which enables spatial queries on Spark. On the one hand, GeoSpark SQL provides a convenient SQL interface; on the other hand, GeoSpark SQL achieves both efficient storage management and high-performance parallel computing through integrating Hive and Spark. In this study, the following key issues are discussed and addressed: (1) storage management methods under the GeoSpark SQL framework, (2) the spatial operator implementation approach in the Spark environment, and (3) spatial query optimization methods under Spark. Experimental evaluation is also performed and the results show that GeoSpark SQL is able to achieve real-time query processing. It should be noted that Spark is not a panacea. It is observed that the traditional spatial database PostGIS/PostgreSQL performs better than GeoSpark SQL in some query scenarios, especially for the spatial queries with high selectivity, such as the point query and the window query. In general, GeoSpark SQL performs better when dealing with compute-intensive spatial queries such as the kNN query and the spatial join query.
Keywordbig data GeoSpark SQL Spark spatial query processing spatial database
DOI10.3390/ijgi6090285
Indexed BySCI
Language英语
Funding ProjectNational Key Research and Development Program of China[2017YFB0503602] ; National Natural Science Foundation of China[41401449] ; National Natural Science Foundation of China[41501162] ; National Natural Science Foundation of China[41771425] ; Scientific Research Key Program of Beijing Municipal Commission of Education[KM201611417004] ; Beijing Philosophy and Social Science Foundation ; Talent Optimization Program of Beijing Union University ; State Key Laboratory of Resources and Environmental Information System
Funding OrganizationNational Key Research and Development Program of China ; National Natural Science Foundation of China ; Scientific Research Key Program of Beijing Municipal Commission of Education ; Beijing Philosophy and Social Science Foundation ; Talent Optimization Program of Beijing Union University ; State Key Laboratory of Resources and Environmental Information System
WOS Research AreaPhysical Geography ; Remote Sensing
WOS SubjectGeography, Physical ; Remote Sensing
WOS IDWOS:000416386100026
PublisherMDPI AG
Citation statistics
Document Type期刊论文
Identifierhttp://ir.igsnrr.ac.cn/handle/311030/56704
Collection中国科学院地理科学与资源研究所
Corresponding AuthorPeng, Xia
Affiliation1.Peking Univ, Inst Remote Sensing, Beijing 100871, Peoples R China
2.Peking Univ, GIS, Beijing 100871, Peoples R China
3.Beijing Adv Innovat Ctr Future Internet Technol, Beijing 100124, Peoples R China
4.China Univ Geosci, Fac Informat Engn, Wuhan 430074, Peoples R China
5.Beijing Union Univ, Inst Tourism, Collaborat Innovat Ctr eTourism, Beijing 100101, Peoples R China
6.Chinese Acad Sci, Inst Geog Sci & Nat Resources Res, State Key Lab Resources & Environm Informat Syst, Beijing 100101, Peoples R China
Recommended Citation
GB/T 7714
Huang, Zhou,Chen, Yiran,Wan, Lin,et al. GeoSpark SQL: An Effective Framework Enabling Spatial Queries on Spark[J]. ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION,2017,6(9):20.
APA Huang, Zhou,Chen, Yiran,Wan, Lin,&Peng, Xia.(2017).GeoSpark SQL: An Effective Framework Enabling Spatial Queries on Spark.ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION,6(9),20.
MLA Huang, Zhou,et al."GeoSpark SQL: An Effective Framework Enabling Spatial Queries on Spark".ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION 6.9(2017):20.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Huang, Zhou]'s Articles
[Chen, Yiran]'s Articles
[Wan, Lin]'s Articles
Baidu academic
Similar articles in Baidu academic
[Huang, Zhou]'s Articles
[Chen, Yiran]'s Articles
[Wan, Lin]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Huang, Zhou]'s Articles
[Chen, Yiran]'s Articles
[Wan, Lin]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.