Crowdsourcing Water Quality with the Simile App

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B4-2020, 2020 XXIV ISPRS Congress (2020 edition) CROWDSOURCING WATER QUALITY WITH THE SIMILE APP D. Carrion1,*, E. Pessina1, C. A. Biraghi1, G. Bratic1 1 Politecnico di Milano - Lecco Campus, Department of Civil and Environmental Engineering, Via Gaetano Previati, 1/c, 23900 Lecco, Italy - (daniela.carrion, edoardo.pessina, carloandrea.biraghi, gorica.bratic)@polimi.it Commission IV, WG IV/4 KEY WORDS: Citizen Science, Water quality, Mobile application, SIMILE, Free and Open Source ABSTRACT: This paper aims at presenting the application for lake water quality monitoring which has been developed in the framework of SIMILE (Informative System for the Integrated Monitoring of Insubric Lakes and their Ecosystems) Interreg Italy-Switzerland project. The objective of SIMILE project is to facilitate the monitoring of the Maggiore, Como, and Lugano lakes through the integration of different techniques: in situ monitoring with buoys, remote sensing and citizen science. A mobile application has been designed in agreement with the project partners, who are also actors working for lake quality monitoring, such as CNR (Italian National Research Council) and ARPA (Agency for prevention and environmental protection). The developed application allows to collect data over the area of interest, such as pictures and parameters which can be acquired by visual inspection as well as with appropriate tools, depending on the user typology. The application has then been implemented with open source software to foster its use also for other projects with similar goals. In the paper, the design choices, the architecture and the implementation details are described. 1. INTRODUCTION 1.1 Lake quality monitoring and SIMILE project Water scarcity is considered as one of the largest global risks in almost any geographical context (Liu et al., 2017). As a consequence, the importance of ensuring and protecting the quality of surface sources of freshwater is pressing and everincreasing. In particular, lakes are a fundamental resource for the territory, not only in terms of water consumption for agricultural and domestic usages, but also considering the benefits the touristic and leisure activities derive from them. It’s precisely in this context that project SIMILE (Informative System for the Integrated Monitoring of Insubric Lakes and their Ecosystems, Brovelli et al., 2019) finds its collocation. Financed by the Interreg Italy-Switzerland program, this initiative involves technical/scientific partners (Politecnico di Milano – Lecco Campus, Fondazione Politecnico, Water Research Institute - National Research Council and SUPSI University of Applied Sciences and Arts of Southern Switzerland) as well as institutional partners (Lombardy Region and Ticino Canton) in order to address the problem of monitoring and handling the quality of the three lnsubric lakes: Maggiore, Como and Lugano. These water bodies are a crossborder resource of extreme relevance and their quality needs to be protected from the anthropic pressure and the effects of climate change (Solimini et al., 2006). The main goal of the project is the creation of a business intelligence platform to support decision and policy making through the combination of hight-quality data from different, innovative sources. Satellite images, high frequency buoys sensors measures and data collected by citizens are integrated to create an open system capable of offering an effective and easily maintainable solution for facing the challenges related to the management of the lakes’ waters. This paper focuses on the Citizen Science side of the project and specifically on the design, development and deployment of the cross-platform mobile application which allows citizens to produce Volunteered Geographic Information (VGI, Brovelli et al., 2018; Fritz et al., 2017, Olteanu-Raimond et al., 2017). In fact, citizen science can be a a very relevant resource to contribute to the water monitoring (Fienen et al., 2012; Fraternali et al., 2012; Lowry et al., 2013), and it can also help increasing the consciousness of people with respect to the protection of the environment. 1.2 Mobile applications for water quality monitoring The current state of the art in this field has been explored in order to find any existing solution that could satisfy SIMILE project requirements regarding citizen science. Such investigation has been conducted taking into account both applications specifically related to lake and water quality assessment (Jovanovic et al., 2019) and products generically built for data collection. Figure 1. A Secchi disk is a simple tool (20-30 cm diameter disk) to measure water transparency: the depth to be measured corresponds to when the disk is not visible anymore. The first considered mobile application is EyeOnWater with its supporting website. It has been developed by Royal Netherlands Institute for Sea Research, Vrije University of Amsterdam, ________________________________ * Corresponding author This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B4-2020-245-2020 | © Authors 2020. CC BY 4.0 License. 245 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B4-2020, 2020 XXIV ISPRS Congress (2020 edition) MARIS and Veeder within Citclops project (EU H2020, Citclops, 2014). The application has been designed to assess water colour (Busch et al., 2016, Wernand et al., 2012). In this case, the user must follow specific instructions in order to take images of the water with the right illumination in order to allow to effectively compare the water colour with a digital colour comparator. In the application the user can answer to questions and provide, optionally additional parameters which require proper tools (e.g. a Secchi disk, see Figure 1). A similar approach is considered by the HydroColor application (Leeuw et al., 2018). Also in this case, pictures must be acquired by the user with appropriate illumination characteristics in order to allow the automatic calculation of reflectance. The computation is done in the smartphone thanks to the implementation of models which have been properly tuned for the area of interest for the application. It is likely that the same models could be applied to other lakes, however it should be verified. Another interesting example of citizen science application for water quality monitoring is SecchiApp, which has been developed by the University of Plymouth (Plymouth Marine Laboratory, 2020), AeroCODE and Pixalytics. The application allows to store the Secchi disk depth and additional information, such as pictures and water temperature. Lake Observer mobile application has been developed by members of the Global Lake Ecological Observatory Network (GLEON, Lake Observer, 2020) and allows to collect data through a form. Many parameters are considered, including temperature, Secchi depth, weather and aquatic vegetation. Three of the applications presented so far are free, while HydroColor has a cost of 3.5 €, all four are not open source. A more detailed review of these and other mobile applications for water quality monitoring can be found in Jovanovic et al. (2019). In addition to applications specifically devoted to water quality monitoring, also applications designed for generic infield data collection have been explored. One very interesting example is QField, which is developed by OPENGIS.ch (QField, 2020). QField is designed to be compatible with QGIS and allows to collect features, to display them on a map customizing legend and symbols and to attach pictures. Epicollect5 is an interesting project developed and maintained by Oxford Big Data Institute (Epicollect5, 2020). It allows to collect data and pictures. Both QField and Epicollect5 are free and open source. The results of this review, however, highlighted no usable preexisting system for SIMILE project, since the considered applications are too tied to specific geographic locations or too broad presenting no real connection with the lake ecosystem. Moreover, the majority of the explored solutions are not free and open source. Presenting the various stages of the development process, from the conceptualization to the actual coding, this paper aims at giving some general guidelines on how to approach the technological side of a citizen science project taking in consideration both the user-friendliness and the technical precision of the final product. 2. THE MOBILE APPLICATION 2.1 The mobile application design The first and, arguably, most important step of the process is the precise definition of the system requirements. Conducting this action carefully, with the involvement of all project partners, provides with a solid base for the following steps and greatly cuts the costs of the development. The results of this planning process can be summarized in the following goals. Figure 2. The main map screen of the mockup distributed to the users. 1. 2. 3. A user can localize himself on a map. It is essential for the user, to be able to see his position on the map, in order to check if the observation that he is about to submit is in the expected location. A user can perform an observation to assess the quality of an area of a lake. This is the main goal of the mobile application: to submit data that can be used to monitor the lake water quality. The basic observation to be provided as mandatory, is a picture. A user can add details to the main mandatory observation (a picture), such as weather conditions, the presence of algae, foams, oil stains, litters, odours, drains, strange behaviours of fauna as well as additional pictures, to better describe the content to be communicated, including its surroundings. In fact, additional details have been included to provide further pieces of information that can provide added value for the use of the observation by the water quality monitoring actors. These details include weather conditions, because it can help interpret for example the colours of water in the pictures with respect to the illumination and with respect to the colour of the sky; but also elements which can help single out the cause of what has been observed, for example the presence of a drain. This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B4-2020-245-2020 | © Authors 2020. CC BY 4.0 License. 246 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B4-2020, 2020 XXIV ISPRS Congress (2020 edition) 4. 5. 6. A user can visualize on the map all his and other users’ entries. It is important, to foster the use of the lake and the attention to its waters, to show the presence of multiple observations on the map. A user can be notified about initiatives and events organized by local actors. This can be a useful way to let the actors who are monitoring the lake quality to communicate with the citizens who are cooperating through citizen science and to let them to feel involved in the lake life and preservation. The system must be free and open source. This is a design decision which has been taken in the framework of SIMILE project, in order to promote the FOSS (Free and Open Source Software) and the sharing of knowledge. Having established the need of a brand-new system, specifically tailored for the project, the design process of the application can start. This phase is crucial, since the high-fidelity mock-ups that are produced give to the partners a good idea of the look and feel of the final product and guide the developers in the building process. Some examples of the mock-ups which have been shared with the project partners can be seen in figures 2 and 3. working for monitoring the lake quality, such as CNR and ARPA. 2.2 The mobile application architecture With the completion of the mock-ups, the project can transition to the actual development phase. The hight-level three-tier architecture of the system is shown in Figure 4. No processing is expected client side, we can consider this a thin client case (Sinclair et al., 1999). The client is represented by the mobile application created using two free and open source JavaScript frameworks, AngularJS and Ionic, which ease the development of cross-platform applications based on HTML5 from a single code base. Iconic is an open source Software Development Kit for mobile applications which is base on AngularJS and Apache Cordova. The Iconic SDK provides the great advantage of allowing the production of an application which can run on different operating systems. In this way not only the development phase is simplified, but also the maintenance, reducing in general the costs. Figure 4. Hight-level architecture of the system. The application is powered on server-side by a RESTFul API (Masse, 2011) also written in JavaScript, using Node.js. The API is publicly accessible, and it offers carefully built endpoints to retrieve, save and modify data regarding users, observations, measurements and events. The client communicates with the server through a series of REST (Representational State Transfer) Calls (Fielding and Taylor, 2000). The RESTFul API that powers the system leverages on the free and open source framework Express.js, which allows to structure the flow in middlewares, small functions executed in chain to provide the right response to any request (Middleware.org, 2008). This approach produces cleaner, more flexible and more easily testable code. The result is a Web API that exposes a series of publicly accessible endpoints that specify where resources lie and can be accessed. In this way a request-response message system is created with a client that accesses the endpoints through URIs posting a HTTP request and a server that responds to this request typically using the JSON format. Figure 3. The page of the mockup dedicated to insert a new observation. SIMILE data storage is based on MongoDB, a NoSQL and object-oriented database (Connolly et al., 2005), which stores data in documents instead of using tables like any traditional relational database. This choice guaranties to the system the needed flexibility both in data structure and in horizontal scalability, without sacrificing the useful capabilities of SQL databases (MongoDB, 2020). This phase has been crucial to define the parameters to include in the application, how to present the activities to the user, which type of interaction consider with respect to the actors The source code of the software can be found on GitHub (https://github.com/interreg-simile/mobile-application). The application is provided both in Italian and in English. This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B4-2020-245-2020 | © Authors 2020. CC BY 4.0 License. 247 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B4-2020, 2020 XXIV ISPRS Congress (2020 edition) 2.3 The mobile application implementation and user interface The user interface is on purpose very simple: the first access is through the map, which shows the user location, with few buttons. The map that has been used as a background is OpenStreetMap (OSM). In Figure 5, the map screen is shown with labels for every element which are described in the following list. 1. 2. 3. 4. 5. 6. 7. 8. 9. Menu button, which allows to switch among Map, News, Project description and Settings (see Figure 6). Title of the current view. Bell notifying the availability of news. Map refresh. Visible layers: it is possible to switch on and off the Observation and the Events layers. Observation marker. Events marker. GPS position: this button allows the user to centre the map and the position marker on his location if the positioning settings of the mobile phone are active. If the positioning is not active, the shown symbol is different. New observation: this button allows the user to insert his observations. Figure 5. Map screen with labels for every element. Figure 6. App menu (accessible with button 1 of the map screen of Figure 4). The main purpose of the application is the acquisition by the user of water quality parameters. To add an observation the “+” button of the main screen (label 9 on Figure 5) must be chosen. In order to be sure that data concern a lake of interest for the project, a check is made on the user coordinates. If the position is outside the area of interest, a message is displayed, communicating that the collected data will be stored but not showed on the map. If the reason why the position is outside the area of interest is due to the internet connection or to the accuracy of the positioning, the user can place manually his observation on the map and then proceed with the data acquisition. Figure 7. New observation dialog window. It appears right after that the mandatory picture is taken by the user. This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B4-2020-245-2020 | © Authors 2020. CC BY 4.0 License. 248 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B4-2020, 2020 XXIV ISPRS Congress (2020 edition) Comparing Figures 2 and 3 with Figures 4 and 5, respectively, it is possible to observe that the mock-ups, which have been produced during the design phase, are very similar to the how the actual application looks like after the implementation. When acquiring data, the first request is to take a picture, which is mandatory. Then, the user is redirected to a dialog window (Figure 7) which allows to add attributes to the acquired picture. The parameters that it is possible to acquire are grouped into categories: Weather, Details, Measures and Other Information. In addition, it is possible to take extra pictures. ▪ “Weather” is automatically taken from the OpenWeatherMap Web service (OpenWeatherMap, 2020). The user can optionally refine the details (temperature, sky and wind). These pieces of information can be useful for interpreting the acquired data, for example to consider the colour of water with respect to the presence of clouds. ▪ “Details” have been agreed with the project partners and include Algae, Foams, Oil stains, Litters, Odours, Drains and Fauna. For each one of these parameters, it is possible to insert specific characteristics and attributes. As an example, in Figure 8, the attributes which can be filled for Algae are shown. ▪ “Measures” which can be taken are: Transparency, Temperature, pH, Oxygen, Bacteria. ▪ “Other information” is an open field (500 characters maximum) that the user can fill if he needs to add other specifications. In agreement with the project partners, it has been decided to leave only one mandatory acquisition feature, which is a picture. At the same time, it has been agreed to leave wider the options for data collection in order to allow the contribution from different typologies of users: for example the occasional user could have no tools to help him in the measurement, so he can contribute answering questions which need visual inspection only. Another category of users could be regular users of the lake, such as fishermen or rowers, who could be specifically trained and equipped with some tools, such as a Secchi disk or a thermometer, in order to make regular monitoring. Other users could be more expert ones, who could take advantage of the application to collect measures, and they could be equipped with professional limnology quality instruments. Every application section and, in some cases, every parameter, is provided with the “help” functionality, which, where possible, provides very specific indications in order to explain which details to pay attention to and how to store the pieces of information. When the observation is complete and all pieces of information have been added, it is possible to submit it. Then, the observation will be visible on the map (if it has been acquired within the area of interest, otherwise the observation will be stored but not displayed) and it will be sent to the database. The actors working for lake monitoring will be able to access the observations through the API. From the main map it is possible to access a “News” section as well (Figure 9), which must be filled by the project partners or the lake monitoring actors to keep the user involved in the lake preservation. It can show events which regard lake life and monitoring as well as alerts, to inform the users, for example, regarding known phenomena, such as natural algae blooms. Figure 9. “News” section of the mobile application. Figure 8. Additional specifics that can be provided for Algae observation. This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B4-2020-245-2020 | © Authors 2020. CC BY 4.0 License. 249 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B4-2020, 2020 XXIV ISPRS Congress (2020 edition) After the implementation, a test phase has been performed, sharing the application with volunteer users, including project partners. The test, which included a questionnaire and exercises (Biraghi et al., 2020) allowed to find and fix bugs and to refine some details of the user interface. The next phase of the activity will require to present the application to schools and administrations and associations in order to promote its use. 3. CONCLUSIONS Figure 10. Example of “Event” section of the mobile application, from which it is possible to see the event address and, clicking on it, it is possible to obtain the directions to reach it in a navigation application. In this paper, a mobile application for lake water quality monitoring has been presented. The application has been developed to contribute to the environment protection through citizen science in the framework of SIMILE Interreg ItalySwitzerland project. The contribution of all partners, who include actors working actively for lake quality monitoring has been essential in order to define the proper parameters to be observable. The application allows to collect data, including pictures, and parameters which can be visually evaluated, but it provides also with the possibility to collect data with the use of proper tools (e.g. Secchi disk or thermometer). The application has already been tested by a group of users, including the project partners. After a bug fixing phase, the application has proved to fulfil the project requests. In the framework of SIMILE project, it is foreseen to advertise the mobile application through public events, involving schools, public administrations, regular lake users (e.g. fishermen) and leisure associations (e.g. rowers) in order to teach how to use it and to attract interest for the lake water preservation. The choice of producing the application with open source software, will allow for the possibility to exploit it also in other projects with similar needs. ACKNOWLEDGEMENTS Clicking on an “Event” it is possible to show its location and also to access the directions to reach it (see Figure 10): the application will automatically redirect the user to a navigation application, such as Google Maps. Finally, in the “Settings” section of the mobile application it is possible to choose the language, Italian or English, to see the application version and to contact the developer through the email. An important matter in the application development was to allow the user to be able to take observations while offline. This can be necessary for the case of interest, not only because close to mountain areas the mobile connection can be not always reliable, but also for the specific needs of the cross-border project SIMILE. In fact, the lakes under consideration are both in Italy and Switzerland, where the EU agreements for no cost data roaming are not valid. So, in order to allow citizens coming from both nations to collect data for all lakes of interest, the offline acquisition is permitted. For this reason, it has been necessary to store locally the background OSM map. In order to have a good compromise between data storage and number of details useful to help the user locate himself on the map, the level of detail for the map visualization has been set to 12. This corresponds approximately to a map scale of 1:150000 on screen, for offline use only (of course when online the level of detail can be much higher), allowing the user to locate himself with respect to the lake boundaries and the main points of interest. The data acquired during the offline time are sent to the SIMILE database as soon as the mobile phone is back online. The work outlined in this paper is part of project SIMILE (Informative System for the Integrated Monitoring of Insubric Lakes and their Ecosystems) and as such it is financed by the European Commission in the frame of Interreg ItalySwitzerland 2014-2021 program. REFERENCES Brovelli, M. A., Cannata, M., and Rogora, M., 2019. SIMILE, a geospatial enabler of the monitoring of sustainable development goal 6 (ensure availability and sustainability of water for all), Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLII4/W20, 3–10, https://doi.org/10.5194/isprs-archives-XLII-4W20-3-2019. Brovelli, M., Wu, H., Minghini, M., Molinari, M., Kilsedar, C., Zheng, X., . . . Chen, J., 2018. Open source software and open educational material on land. XLII-4, pp. 61-68. Delft, The Netherlands: Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. doi:10.5194/isprs-archives-XLII-4-61-2018 Biraghi, C.A, Pessina, E., Carrion, D., Brovelli, M.A., 2020. VGI visualisation to support participatory lake monitoring: the case study of SIMILE project, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. Busch, J. A., Bardaji, R., Ceccaroni, L., Friedrichs, A., Jaume, P., Simon, C., …, Zielinski, O., 2016: Citizen Bio-Optical Observations from Coast- and Ocean and Their Compatibility with Ocean Colour. Remote Sensing, 879-898. doi:10.3390/rs8110879 This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B4-2020-245-2020 | © Authors 2020. CC BY 4.0 License. 250 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLIII-B4-2020, 2020 XXIV ISPRS Congress (2020 edition) Citclops, 2014. Measuring water color. Retrieved April 25, 2020, from http://www.citclops.eu/water-colour/measuringwater-colour Connolly, T. M. and Begg, C. E., 2005. Database systems: a practical approach to design, implementation, and management, London, UK, Pearson Education. Epicollect5, 2020. Epicollect5, Free and easy-to-use mobile data-gathering platform. Retrieved April 25, 2020, from https://five.epicollect.net/ Fielding, R. T. and Taylor, R. N., 2000. Architectural styles and the design of network-based software architectures, University of California, Irvine Doctoral dissertation, vol. 7. Fienen, M. N., & Lowry, C. S., 2012. Social. Water—A crowdsourcing tool for environmental data acquisition. Computers & Geosciences, 49, 164-169. Fraternali, P., Castelletti, A., Soncini-Sessa, R., Ruiz, C. V., & Rizzoli, A. E., 2012. Putting humans in the loop: Social computing for Water Resources Management. Environmental Modelling & Software, 37, 68-77. Fritz, S., See, L. and Brovelli, M., 2017. Motivating and Sustaining Participation in VGI. In G. Foody, L. See, S. Fritz, P. Mooney, A.-M. Olteanu-Raimond, C. C. Fonte, & V. Antoniou (Eds.), Mapping and Citizen Sensor (pp. 93-117). London: Ubiquity Press. doi:10.5334/bbf.e. License: CC-BY 4.0 Jovanovic, S., Carrion, D., and Brovelli, M. A., 2019. CITIZEN SCIENCE FOR WATER QUALITY MONITORING APPLYING FOSS, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLII-4/W14, 119–126, https://doi.org/10.5194/isprs-archives-XLII-4-W14-119-2019. Novoa, S., Wernand, M., & Van der Woerd, H., 2015. A generic algorithm to derive the intrinsic color of natural waters from digital images. Limnology and Oceanography Methods, (13), 697-711. doi:10.1002/lom3.10059 Olteanu-Raimond, A.-M., Laakso, M., Antoniou, V., Fonte, C. C., Fonseca, A., Grus, M., . . . Skopeliti, A., 2017. VGI in National Mapping Agencies: Experiences and Recommendations. In G. Foody, L. See, S. Fritz, P. Mooney, A.-M. Olteanu-Raimond, C. C. Fonte, & V. Antoniou, Mapping and the Citizen Sensor (pp. 299-326). London: Ubiquity Press. OpenWeatherMap, 2020. One Call API. Retrieved 25 April, 2020 from https://openweathermap.org Plymouth Marine Laboratory, 2020. MONOCLE. Retrieved 25 April, 2020, from http://monocle-h2020.eu/About QField, 2020. Qfield. Retrieved 25 April, 2020 from https://qfield.org/ Sinclair, J. T. and Merkow, M., 1999. Thin Clients clearly explained, Morgan Kaufmann Publishers Inc., Burlington, USA Solimini, A. G., Cardoso, A. C., and Heiskanen, A.-S., 2006. Indicators and methods for the ecological status assessment under the Water Framework Directive. Luxembpurg: Office for Official Publications of the European Communities. Wernand, M. R., Ceccaroni, L., Piera, J., & Zielinski, O., 2012. Crowdsourcing technologies for the monitoring of the colour, transparency and fluorescence of the sea. Proceedings of Ocean Optics XXI, 8-12. Lake Observer, 2020. Lake Observer, A mobile app for recording lake and water observations, Retrieved 25 April, 2020, from https://www.lakeobserver.org/ Leeuw, T., and Boss, E., 2018. The HydroColor App: Above Water Measurements of Remote Sensing Reflectance and Turbidity Using a Smartphone Camera. Sensors, 18(1), 15. doi:10.3390/s18010256 Liu, J., Yang, H., Gosling, S. N., Kummu, M., Flörke, M., Pfister, S., . . . Oki, T., 2017. Water scarcity assessments in the past, present, and future. Earth's Future, 5(6), 545-559. Lowry, C. S., & Fienen, M. N., 2013. CrowdHydrology: crowdsourcing hydrologic data and engaging citizen scientists. GroundWater, 51(1), 151-156. Masse, M., 2011. REST API Design Rulebook: Designing Consistent RESTful Web Service Interfaces, O’Reilly Media, Inc., Sebastopol, USA. Middleware.org, 2008. What is Middleware? Retrieved 01 May, 2020 from https://web.archive.org/web/20120629211518/http://www.midd leware.org/whatis.html MongoDB, 2020. “Relational Vs Non Relational Database”. Retrieved 01 May, 2020 from https://www.mongodb.com/scale/relationalvs-non-relationaldatabase. This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLIII-B4-2020-245-2020 | © Authors 2020. CC BY 4.0 License. 251