VR for Cultural Heritage

VR for Cultural Heritage A VR-WEB-BIM for the Future Maintenance of Milan’s Cathedral Francesco Fassi1(&), Alessandro Mandelli1, Simone Teruggi1, Fabrizio Rechichi1, Fausta Fiorillo2, and Cristiana Achille1 1 ABC, Politecnico di Milano, Via Ponzio, 31, 20133 Milan, Italy {francesco.fassi,alessandro.mandelli, fabrizio.rechichi,cristiana.achille}@polimi.it, [email protected] 2 DICIV, Università degli studi di Salerno, Via Giovanni Paolo II, 132, 84084 Fisciano, SA, Italy [email protected] Abstract. The work presented here is the final step of a multidisciplinary research project conducted on the Milan Cathedral for eight years (2008–2015). Three main topics, consequentially related, will be here addressed: (i) the survey of the structure, meant to update the old drawings; (ii) the construction of an accurate and detailed 3D model to be used to produce measurements at a 1:20– 1:50 representation scale; (iii) the development of a Building Information System (BIM) to collect all the data relating to the restoration projects, as well as all information relating to past, current and future maintenance activities of the cathedral. The result of this research project is a complex and accurate digital 3D model of the main spire of the cathedral and of other parts of the building. This model can be visualized, navigated and used by the Veneranda Fabbrica technicians as an info-data catalogue, thanks to a common web browser connected with the remote BIM System Server and the modelling software where ad hoc I/O plugins are implemented. The last step of this long project was to take advantage of the nascent potential of immersive visualization techniques and to transpose the BIM system in a VR environment, thus obtaining two main results. The first was a high-appeal visualization system that allows a virtual visit of the Main Spire of the cathedral, the building’s highest part that has been closed to visitors since the beginning of the XX century. The second was the possibility to use this technology to virtually explore the cathedral from a technical point of view: by using an immersive visualization technology, operators can improve their understanding of the structure and obtain real-time information about the state of conservation, including current and past maintenance activities, in a sort of “augmented reality system in a virtual environment”. Keywords: Survey
3D reality-based modelling Virtual reality
BIM
Architecture
Immersive visualization
© Springer International Publishing Switzerland 2016 L.T. De Paolis and A. Mongelli (Eds.): AVR 2016, Part II, LNCS 9769, pp. 139–157, 2016. DOI: 10.1007/978-3-319-40651-0_12 140 F. Fassi et al. 1 Introduction “Virtual Reality is a computer technology that gives the illusion, to those who use it, of being immersed in a virtual environment that does not really exist. It is a computer simulation of a real situation where the human subject may interact with the virtual environment, sometimes by means of non-conventional interfaces like glasses and helmets on which the scene is represented and the sounds reproduced. A data glove is equipped with sensors to simulate tactile stimuli and to translate the movements into instructions for the software. The purpose of virtual reality is to simulate a real environment in order to give the impression of being truly immersed in that environment” [1]. Initially, in the wide world of Cultural Heritage, the aim of Virtual Reality was to implement (i) high-impact applications and (ii) functional purposes. The first corresponded to beautifully performed virtual reconstruction without ‘metric’ accuracy; the second to reconstructions of sites with the relocation of environment and objects, used, for example, when they are closed to the public and/or no longer exist. Today the trend is to ‘mix’ all aspects by creating scientific virtual reconstructions (metric) that are, at the same time, able to convey an impressive effect. A virtual model of Cultural Heritage can have different values (scientific, educational, historical, …) depending on the information that it provides, and it can also be used to assess the evolution of the environment by comparing 3D reconstructions made at different times in the object’s life. It thus becomes easy to compare present and past, and to assess and quantify the changes caused by time. The reality-based model of the Main Spire of Milan Cathedral was commissioned by the Veneranda Fabbrica del Duomo1 (from here onwards abbreviated as VDF); made with great care and designed to support the ordinary and extraordinary maintenance operations, it is used not only to allow ‘the virtual tour’ (typical case of cultural heritage not accessible to the wide public) but also as a support for the display of information (technical, historical, …) georeferenced on the single piece (single block of marble). In this sense, it is not just a virtual visit, but also a real exploration of the 3D model. In other words a virtual visit to a restoration yard. In the virtual scenario of the geometric model of the Main Spire the work in progress on its pieces, it is linked in a sort of Augmented Reality inside the Virtual Reality. The substantial difference between Virtual Reality (VR) and Augmented Reality (AR) is that, in the first case, the reconstruction of an entirely fictional world and consisting of virtual objects is observed on a screen; the second case is not tied to a video display, but to any surface/object that can become a support to visualize the additional information of the scene; what we observe is an integration between real images and virtual objects. In the case of the Duomo, VR and AR models are blended. The user is immersed in the virtual model and can access computer-generated information contained in the Database. The virtual world is ‘increased’, and further enriched 1 “Veneranda Fabbrica del Duomo di Milano” (VFD) is the historic organisation responsible for preservation and restoration of the Cathedral. Gian Galeazzo Visconti established VFD in 1387, for design and construction of the Cathedral, and it has been operating for over six hundred years to ensure the preservation and architectural and artistic restoration of the Duomo, safeguarding and providing services for the Cathedral’s religious activity [2]. VR for Cultural Heritage 141 by additional information (graphic and textual) georeferenced and synchronized on the model. It is possible to virtually visit the Spire and simultaneously know how the operations are carried out by the VFD staff (consolidation, cleaning, replacement, tessellation, etc.). The careful organization of the data will allow the public to visit the monument and understand its complex organization; in this sense, therefore, the virtual tour will help visitors to understand the complexity of the preservation of Cultural Heritage. In a futuristic scenario, the combination of VR and AR will allow the technical staff to simulate the start and the arrangement of the restoration yards. These virtual operations made on reality-based models managed in a BIM process will help to improve the prediction of costs and the duration of operations, as well as to simulate future scenarios, etc. The complex system of information (structured DB) visible on the model will allow technicians to optimize all operations, helping to identify the needs of the yard. The use of the model by VFD will contribute to the dissemination of innovative systems for the use of data and will simultaneously allow VFD to self-evaluate needs-time-cost-answers. VR and AR will become a management tool of complex Cultural Heritage yards both for researchers and users in general. It is necessary to overcome the initial effect of ‘wonder’ that this system inspires, and learn to use it effectively as a new work tool, and thus create the necessary professionals and specializations. VR and AR models will never replace the in situ visit, but they will certainly increase the understanding of the Cultural Heritage by the visitor as well as support studies and scientific analyses. This paper starts with a brief description of the previous research activities regarding the survey and modelling phase of the project that led to the creation of the accurate 3D model of the Main Spire of the Milan Cathedral. Then it moves on to briefly describe the WEBBIM system, created ad hoc and already in use by the Veneranda Fabbrica del Duomo to support its continuous maintenance activities. Finally, it will describe the process necessary to modify and adapt this 3D model to a VR immersive environment. The paper does not aim at presenting new technical developments in the VR technologies. Rather, as the final chapter of a long and multidisciplinary research work, it aims at illustrating the investigation that the team carried out over the years whilst exploring the possible uses of high detailed BIM 3D model inside VR and AR, as support for professional activities in the field of Cultural Heritage conservation. 1.1 Virtual Reality for the Cultural Heritage (State of the Art) The integration of the research in the area of computer graphics, computer vision and 3D survey systems found as a common action field the virtual reconstructions of Cultural Heritage. A 2004 paper by Jiri Zara [3] presented an overview of techniques for the creation and visualization of cultural heritage objects and site on the web. The discussion included dealing with acquiring techniques, modelling, optimization, and final presentation of digital reconstructions using Virtual Reality paradigms that are still relevant today. The study was illustrated by practical experience from the implementation of the EU project Virtual Heart of Central Europe [4]. The aim of the project 142 F. Fassi et al. were: (i) the digital reconstruction of selected historical buildings of four cities from different countries – Bratislava (SK), Graz (AT), Maribor (SI), and Prague (CZ); (ii) the creation of a digital storytelling; (iii) and the publication of the results on the web. The pioneering experiment was the Virtual Old Prague project [5], a web application based on progressive downloading city parts in correspondence to current user’s position and consists of several components: 3D scene; HTML document, and 2D navigation map synchronized with the user’s movement in simulated scene. Although today the technology is evolved and the interface used for the navigation in the virtual space is old-fashion and not very user-friendly, topics discussed and conclusions are still current: “The web definitely has a potential to bring cultural heritage on the screen of any interested Internet users”. The article “Visualization and Virtual Reality for Cultural Heritage Diagnostic” [6] promoted the use of Virtual Reality in Cultural Heritage applications for diagnostic purpose, introducing the term of Virtual Cultural Heritage. The work developed within into the SIDART (Integrated System for Cultural Heritage Diagnostic) project and its objective was the development of hardware and software package to acquire, process, integrate and visualize information produced by different survey instruments (laser scanner, multi-spectral camera, calibrated metric photogrammetric camera, thermography). Using a VR device (Table Projector Baron with a workstation equipped with nVidia Quadro 4500 graphic card), the system allowed passive and active stereoscopic visualization. In order to correctly visualize the model, the user had to wear a special eyewear (HMD). The most innovative idea of the study was the possibility to not only visualize but also work interacting with the survey data in an immersive 3D environment: “This lets the operator perceive the third dimension and the virtual investigation of the object becomes more realistic. This lets us take into consideration in a more simple, natural and correct way and also reduce the possibility to make wrong evaluation due to the false prospective of the classic visualization.” The current virtual technologies (VR and AR), devices and sensors are able to immerse the user in a virtual environment, offering the possibility of interacting with additional contextual heritage data (reality-based 3D model, pictures, technical documents, historical information, tourist guides, etc.) [7]. The VR applied to Cultural Heritage has already become a potential tool for ‘tourist’ users to navigate and interact in a virtual scene in total symbiosis with the environment. The system has an attractive impact for dissemination purposes because it adds extra information and allows a direct and intuitive access of digital objects [8]. VR offers more than a replica of the real context; it allows virtual hypothetical reconstructions of the past aspect that can be developed for different epochs and can be related or overlaid with the archaeological surviving evidences [9]. VR creates a direct connection between tangible data (museum collection, archaeological remains etc.) and ancient invisible space, reproducing the scene, the environment and the atmosphere (i.e. ancient illumination condition and sound) of past context in an immersive ad attractive mode [10]. The potentiality of VR applications to promote knowledge Cultural Heritage is proven by the development of a significant number of projects in this research area [11]. An increasingly large number of museums possess a virtual gallery where the VR for Cultural Heritage 143 users can explored online digital reproductions enriched with technical and/or historical documentations [12] and organized in a virtual rooms of the collections [13]. The “museum of the future” [14] developed by European and ArchiVision contains the 3D models of the masterpieces from the Dutch Rijksmuseum. Using a development kit for the Oculus Rift virtual reality glasses, the user can visualize and walk around the paintings at much closer range than would be possible in reality. Furthermore, in order to make digital contents available to the Europeana collection portal, the 3D-ICONS project was founded [15]. The aim is the definition of a standard workflow for the creation of a 3D replica from the selection of methods and tools, data acquisition, post-processing, online publication of content, and metadata capture. Virtual Museum Transnational Network (V-MusT) [16], instead, was created to provide the heritage sector with tools and support to develop virtual museums with communicative and educational purposes using VR technologies and devices. “Apa Game” [17] is a case-study selected by V-MUST project to address the issues of re-use of digital assets in online and mobile applications. The first case study was the project “Apa discovering Bologna” made by CINECA for the museum of Bologna Genus Bononiae, which in ITABC CNR participated for the reconstruction of the Etruscan and Roman age landscape of the city. The result is a 3D cartoon on the history of Bologna that combines high-performance graphics and rigorous virtual reconstruction (based on laser scanner and photogrammetry integrated system) [18]. Since its release in 2011, Apa was considered an innovation in the panorama of educational museum movies, it is the first 3D stereoscopic movie with historical references applied to an entire city with four different geo-referenced scenarios and seven historical periods: Etruscan, Roman, Medieval, Renaissance, XVII and XVIII century and the present day. “Ati discovering Veio” [19] is the successive short 3D animation of the CINECA; the 3D reconstructions are the result of joint work of archaeologists and experts in modelling and computer vision [20]. The protagonist Ati (Apa’s cusin), guides the tourists in the rooms of Villa Giulia museum (Rome) dedicated to the masterpieces of Etruscan sculpture, with the famous Apollo of Veii, and in the sanctuary of the “Portonaccio”, leading them to the roof of the temple. The preview of the animation film was made during the inauguration of the exhibition “Apa, Etruscan lands in Rome” [21] at the National Etruscan Museum of Villa Giulia. The initiative was the result of a partnership between the Roman museum and Genus Bononiae, and a scientific and technological project coordinated by CINECA. At the same time, the exhibition “The journey beyond life. The Etruscans and the afterlife among masterpieces and virtual reality” was opened in Palazzo Pepoli-Museum of the History of Bologn [22]. The starting point of these initiatives is always a complete historical and scientific study combined with the application of the most advanced reality-based reconstructions, thus combining science, art and spectacular effects. One of the attractions of the exhibition “Apa Etruscan lands in Rome” was the holographic life-size installation of “Situla della Certosa” [23] (precious bronze vase symbol of Northern Etruria). Virtual animations include the actual geometry of the vase rich in decorative details, and an artificially restored version, that brings it back to the time of its production. In the same way, the “Sarcofago degli Sposi” [24], symbolic monument of the Etruscan civilization (permanently exhibited at Villa Giulia) was the 144 F. Fassi et al. subject of a virtual reconstruction in the museum of Bologna. The digital survey of the sarcophagus was built starting from the fourth team of researchers acquisition campaigns (3DOM-Fondazione Bruno Kessler [25], CNR-ISTI, CNR-ITABC, Leica, and University of Bologna), coordinated by CINECA. The installation was a performance combining holography, projections 3D video mapping, sound and advanced visualization techniques in an immersive audiovisual environment. In conclusion, it is possible to summarize the principal purpose of a VR application in function of the final consumer: (i) the ‘technical user’ can use metric data and functional information for preservation and maintenance purpose; (ii) the ‘common user’ can take advantages for knowledge and dissemination purpose. In any case, the 3D reconstructions of the present and of the past require a reality-based acquisition and modelling, connected with spatial and geographical data. 2 The Case-Study: The Milan Cathedral The case study presented here is the Milan Cathedral: to be precise, the research activities that accompanied the restoration works of the monument over the last 8 years. The original purpose of the work, commissioned by VFD, was to update the old drawings and produce new plans, sections and elevations of the Main Spire of Milan Cathedral [26]. The occasion was represented by the extraordinary maintenance activities that were about to be implemented on that part of the monument. Over the years, however, the research project evolved into something more stimulating and challenging. The classical approach was immediately abandoned in favour of testing new solutions: in particular, survey methods able to directly reproduce the 3D geometry of the structure, as well as different modelling strategies. The final, ambitious aim became building a complete high resolution and high accuracy three-dimensional virtual model of the entire structure. The idea was to adopt a method that would generate plans, sections, elevations and, in general, all measurements at every point of the building. The resulting virtual model would respect, in fact, the requirements of metrical accuracy, as well as the richness of details, and would easily allow various representation scales, including 1:50 for structural parts and 1:5 for the statues and decorations. It was clear that a mere extraction of 2D drawings from a very complex 3D model was important and mandatory (and immediately useful for to Veneranda Fabbrica del Duomo), but reductive in relation to the inherent high potential of the high-resolution 3D model [27]. An important point was how to take advantage of a detailed three-dimensional model and make it usable for all the operators and workers involved in the Cathedral’s maintenance. A careful study was conducted in order to better understand the modus operandi of VFD. Its main activities are the supervision of the monument, in order to cyclically identify the damaged parts, and the consequent maintenance operations, which consists of the restoration of the cracked or chipped marble blocks or the complete substitution in the most extreme cases. The restoration of the monument is an endless, continuous work of repairing, cleaning and replacement that has to be documented to help future activities. The modern answer to this necessity is the management of all the works inside a BIM system, as it combines the digital three-dimensional or multidimensional representation of an object with an information database (spatial VR for Cultural Heritage 145 position, technical features, properties of the materials, realization phases, maintenance operation, etc.) [28]. Thanks to this working method, it is possible to create a proper geographic and informative system of the heritage, including the management of its life cycle, from the phase of the project, to the phases of the use and maintenance. Therefore, the main effort was spent in the last years to build a dedicated BIM system with this specific goal: supporting to the operations of the restoration yard. Since 2009, the research work involved different areas of the Milan Cathedral following or, to be more precise, anticipating the restoration yards. As of today (2016) the system includes the Main Spire, the Dome Cladding, the two Altars of the Transept, the 18th Spire, the Lantern and the Big Dome. 2.1 The Survey: Measuring and Modelling Gothic Architecture The Milan Cathedral is one of most important example of Gothic architecture in Italy. It is an architectural style born and developed in the early and late medieval period; it evolved from Romanesque architecture and ended with the advent of Renaissance architecture. This style had great influence in ecclesiastical architecture. Churches, abbeys and monasteries were built using a unique combination of existing technologies such as ogival or pointed arches, ribbed vaults, buttresses, window traceries, clustered columns, towers, pinnacles and spires. When applied to religious buildings this style emphasizes the verticality and the light. Pointed arches gave greater flexibility to architectural form allowing a vertical visual result; they are used in every location (doorways, windows, arcades and galleries), both for structural and decorative purposes. Externally, towers and spires, both large and small, are characteristics of Gothic churches. Gothic architecture and especially its characteristic decorations exhibit quite elaborated geometric shape configurations. This is achieved by combining only a few basic geometric patterns, namely circles and straight lines. The results are obtained combining them with one another thanks to a limited set of operations, such as intersection, offsetting and extrusions. It was not possible to preventively decide which were the correct and more efficient methods and tools in order to study such a complex architecture. Therefore, the survey phase involved different approaches and instrumentations and evolved over the time. Although a first topographic survey was not sufficient to completely describe the building, these measurements were essential to georeference all data together. The use of a laser scanner (2009) seemed to be the obvious choice to complete an exhaustive survey of the Cathedral [27]. Over the years (2010–2015) the laser approach was substituted by the photogrammetric method. This technique permitted to overcome problems linked to the penetration of the laser beam in the structure that prevents accurate and reliable measurement of structures. Moreover, photogrammetry is more flexible, as it allows to better negotiate with the presence of narrow spaces, and can be easily used on moving scaffoldings. As a result, a high number of photos were acquired and were useful not only to build the 3D model but also to complete the description of the Cathedral with the addition of information about the state of health, degraded parts and colour of the marble surfaces. These data also allowed to elaborate high definition orthophoto, useful to subdivide the models according the disposition of the constructive 146 F. Fassi et al. elements. This represented the starting point to build up a multi-scale model aimed at the precise description of every single marble block of the yards involved in the maintenance activities. 2.2 The Modelling The goal of the modelling part was to develop a method to generate 3D models of complex objects starting from dense point clouds generated from range or image based techniques. The modelling phase nowadays still represents the bottleneck of the entire workflow, especially when it comes to reality-based modelling objects rich of details and freeform shapes [29]. The model was created using Rhinoceros, a widespread 3D software working with NURBS curves and surfaces. It is capable to manage the complexity of the information obtained from the survey phase and it shows flexibility both in the modelling phase as well as in importing/exporting file formats. It can be integrated with external plug-ins able to manage dense point clouds (Pointools2) or to apply aided parametric design (Grasshopper). Moreover it is possible create some ad hoc personalized processes using SDK tools free for developers. The protagonist of all the maintenance activities is the marble block: for this reason, every single marble block is singularly modelled. The whole model results in a big LEGO structure composed by simple (structural) and complex (artistic) blocks that are close and topologically connected elements. If all objects are closed and valid (there are no auto-intersecting surfaces), it is possible to calculate the volume, the external surface and the mass of each element. These values, properly organized, provide useful information that can be employed in the daily activities of the yard: for economic, logistics and structural analysis purposes. An interesting aspect of this system is that it provides not only information on each object, but also on their position and their proximity, thus resembling a geographic information system, and therefore allowing the user to operate spatial queries. During the modelling phase, three types of objects were identified as requiring different modelling strategies. Line-based items compose the first family: this is the case of structural elements, that are normally quite simple and with a geometry easily mouldable with sweeping, revolution and extrusion operations. The second family is that of freeform objects, which include all round statues, ornaments and decorations with no straight lines at all [29]. Every component of this category is different from one another, so they were acquired one by one moving from point clouds obtained by image-based techniques to meshes, to mesh and finally to NURBS, editable in Rhinoceros. The transformation is realized through re-topology operations and reverse engineering software. It is a mandatory step in order to lighten the model inside the modelling software and to merge different family models. This allows to virtualize the last type of objects born from the combination of the two previous ones: they are “mixed” elements in which appear at the same time geometric and free form shapes. Usually, these correspond to high-decorated structural elements. 2 Today no longer available. VR for Cultural Heritage 147 Concerning the modelling strategies, two different approaches were tested: direct and parametric. Today, the most used modelling technique to create reality-based complex 3D objects is the direct approach. This working method is based on the extraction of features from surveyed data and on the extrusion of these profiles along tracks. The parametric approach is used to model similar object avoiding the time consuming work to model separately one by one blocks that are different to the extent but identical in shape. This approach can reduce the time requested for modelling because it allows to adapt a first parametric model to all similar objects, thus allowing the creation of an abacus of ad hoc parametric models that include elements belonging to a specific architectonical family. 2.3 The WebBIMDuomo A dedicated system called WebBIMDuomo was developed in order to manage all the restoration activities of the yards through a BIM system. WebBIMDuomo is an ad hoc system created for Cultural Heritage applications, as the BIM commercial software are not yet capable to handle completely the uniqueness and the complexity of the objects belonging to this field to be described by three-dimensional models. It promotes a shared knowledge of an architectural or artistic object among different participants. It allows to collect and share information and data useful for the study of the past and to design the interventions for the future. The system is divided into three parts: the I/O plug-in inside Rhinoceros, the database and a web-based visualization system. The first part belongs to the realm of the technical officers, who have the possibility to load parts or the whole model, through specific queries, as well as to modify the 3D model itself, update or add information to the imported objects in any file formats. The second module is the database, which collects all the 3D models, all the information and files related to the objects, both in the NURBS original format and the JSON one, which is used for the web visualization. The database, remotely installed on the cloud, allows the users to share and to synchronize of all reviews in real time [30]. The web module has been developed for non-technical users and to be used during the works in the yard. In fact, through any mobile device endowed with an internet connection it is possible to add information, photos and notes taken during inspections directly on the model, to plan replacement of parts of the cathedral and to “create all type of maintenance operations”. For each planned activity, the system is able to create the corresponding records in the database, to register the input data and share it in real time, allowing the updating of the virtual model in function of the real physical modification of the structure. WebBIMDuomo was developed taking into account the specific needs expressed by the workers of VFD: the system was designed to be able to manage all the ordinary and extraordinary activities or interventions that daily occurs in the yards of the Cathedral [31]. Usually, the main interventions regard the substitution, in whole or in part, of the degraded marble blocks, anchorages and reinforcement structures. When an intervention is created inside the system, automatically, a “father-son” relationship is established between original and modified object; in this way, it is possible to derive in real-time the history of modifications for each single element. 148 F. Fassi et al. All the relations are included in the system: one-to-one, one-to-many, many-to-many, many to one. This feature allows to cover all possible interventions, from cleaning and tessellation to partial or entire substitutions. The BIM system automatically prepares all the necessary records in the DB to insert images and information about the maintenance operation and about the new objects (3D model, data and info) that will up uploaded in the system only. The changes are modelled by the technical office on the basis of the data and instructions stored in the system during the yard activity. The process ends when the new block is housed in its position in the yard and the 3D model is uploaded into the system with the related information. In this way, the visualized model is always up-do-date, as it also includes the latest modifications. A precise color-coding was adopted in order to simplify the understanding of the blocks involved in the restoration activities and to have an immediate perception of the progress on a specific date: the opening procedure of an intervention is highlighted in blue, the survived part and the tessellation in red and green (Fig. 1). Fig. 1. (a) Main Spire 4.825 blocks; (b) North Altar 3.358 blocks; (c) South Altar 2.883 blocks; (d) (e) Dome Cladding + Big Dome 4.194 blocks; (f) 18th Spire 379 blocks (Color figure online) 3 The VR System As anticipated, the main aim of this branch of the research was to investigate the possibility to develop a specific VR application in order to use the BIM system in an immersive environment. Using this type of technology not only for general public dissemination but also for technical purposes would represent an important VR for Cultural Heritage 149 Fig. 2. Synthetic scheme of WebBIMDuomo system performance achievement. The most obvious and immediate application, in fact, is using the model for museum, virtual navigation, game, virtual reconstruction, documentation or teaching purposes. The following step, however, that is creating a VR immersive system for a professional use in the cultural heritage field still represents a significant challenge. The direct fruition of the virtual model through a fully immersive VR tour could open the path for an innovative development in the field of architectonic maintenance, improving our understanding of the spaces and allowing to immediately georeference the information. The idea was to focus on the highest part of the Milan Cathedral because it has been closed since the beginning of XX century. It is also hardly accessible, the only access now being represented by yard elevators. For this reason, a system that would allow a virtual visit of this area could be attractive both for a general public and for technician that are not allowed to enter in the yard. The chosen development tools are Unity 3D, well-known software designed for videogame development, and Oculus DK2, the Head Mounted Display (HMD), cheap and fast, and endowed with an open development kit. The main idea was to use the data that already exist (3D model and info) and adapt them in order to build an immersive system. The main issue was the huge amount of data due to the high details of the 3D reality-based model. In order to solve this issue, it was necessary to implement a multi-step simplification process: 1. A fix mesh decimation that allows a number of polygons forming the initial 2. A logical segmentation of the model structure of the cathedral and following used in the technical BIM system. 3.1 fix carefully considered reduction of the NURBS model. following the natural distribution of the the subdivision in Area-Zone–Sector-Type The Mesh Simplification The first process of simplification has been carried out directly inside Rhinoceros 3D. The expected result was a system of simplified objects topologically well connected 150 F. Fassi et al. that retained a good graphic quality without stressing too much on hardware performances. In order to start the reduction process, it was very important to define the starting level of detail, the “zero”, which is the base level from where is possible to develop the various cuts of size and quality. In Rhino there are many different ways to create a mesh surface, for example through the export command or through the panel “Mesh/Mesh from NURBS object”. Both commands allow to choose some parameter values that influence the outcome of the transformation by modifying the level of simplification and the structure of the mesh. A series of experiments empirically led to the definition of the optimal level of simplification of the model. The tests have been carried out on the model of an ornament of the Belvedere balcony. The aim was to evaluate the behaviour of the different parameters in connection with the typology of the object, taking care in particular of the degree of complexity (linear simple object or artistic freeform surface) and of the dimensions. The ornament is a good test object because shows both linear simple surfaces and complex free-form shapes. The two main objectives were: on the one hand, to decimate the mesh as much as possible without losing accuracy and detail, and in particular paying attention to preserve the continuity between all the parts that compose the models; on the other hand, to find common parameters for every type of object in order to simplify the two processes of automatization of the decimation, and exportation of the models. The conducted tests suggest that not only the number of the polygons affects the handling of a model’s heaviness, but that also the orderliness of the mesh is very important. In fact, a structured mesh is more manageable by the software compared to a disordered one, and ordering the structure is essential during the decimation process when the simplification algorithm is being applied to a large amount of data. It became clear that some parameters had more influence in decimating the polygons than others. The “maximum angle parameter” sets the maximum allowable angle between the input surface normal and the neighbouring mesh vertices. It defines a substantial change in the computed mesh by establishing the curvature degree of simplification. Within the range between 0 and 90°, after checking which portion of the test sample is further simplified and which is not, the value 45° resulted optimal. The second relevant field is the “aspect ratio of the surface”, which varies between 1 and 100. Smaller values result in slower meshing and a higher polygon count with more equilateral and nicely shaped polygons. This value directly influences the degree of regularity that the structure of the mesh will retain during the computation of the new simplified model. Since it affects the proportions of the polygons, it is important that this number remains close to 1 as much as possible: this means that the mesh triangles approach the shape of an equilateral triangle after decimation process. The parameter was set to 5, the best compromise between quality, regularity of mesh structure. Directly tied to the aspect ratio there are the maximum and minimum edge lengths that directly control the dimensions of the polygons forming the surface. The choice was to keep the same values for the two parameters, so that the resulting polygon would remain as regular as possible. The selection of the best value for these fields was carried out by taking into account the scale of representation of the processed model; however, the optimal theoretical value (0,008–0,01 m) resulted in a mesh surface still difficult to navigate. The chosen minimum appreciable distance was reduced drastically VR for Cultural Heritage 151 at 0.08 m, resulting in a lower definition of the initial surface (in particular for decorated objects) but that nevertheless preserved enough quality and resolution for more simple and linear objects. The test comparisons on this elaborated 3D model with the original detailed model demonstrate that this value guarantees a representation scale of about 1:50 typical of large-scale architecture. To keep the reduction of details within an acceptable interval, the “minimum initial grid quad” was set on value 10. This parameter represents the initial resolution of mesh grid that is a quad mesh that Rhino creates on each NURBS surface in the first stage of meshing. In practice, Rhino will use at least this number of polygons to describe each surface. This parameter controls the minimum number of polygons of a surface and consequently its final definition. In order to conduct the simplification tests, a zero level mesh was used. This was the mesh automatically created by Rhino using default parameters. It is a not regular mesh and present 110949 polygons. The surface, after the application of the simplification algorithm resulted as composed of 34741 polygons. The reduction has been substantial (66 %) and the processed mesh surface keeps the desired acceptable level of detail. On the ornament of the Belvedere section the algorithm for simplification proved to be quite successful so the work proceeded by testing its effectiveness on a larger part of the 3D model, precisely on the central cylindrical staircase section of the Main Spire. 3.2 Other Simplification Methods Other simplification methods were tested starting from how videogames techniques manage environments and objects: that is, their Level of Approach to Details (LOD) and their logical sequential segmentation of the scene. These two methods have been applied to the Milan Cathedral as a case study (Fig. 3). Fig. 3. Central cylindrical staircase decimated using first experiment parameters (on the left) and the same parts using new parameters (on the right) LOD Levels. The LOD approach, applied to the VR tour development, simulates the effect of human vision for which the eye catches more details on near objects than on distant ones. For this reason, objects can be loaded in the scene with a different level of detail: farther objects are shown at a lower definition while near objects are formed by more dense and accurate surfaces. The different quality between near and far objects goes unnoticed, because of the effect of distance and the fact that the scene moves fast. Unity can simulate automatically the effect thanks to the “LOD Group” component. 152 F. Fassi et al. It must be properly tuned depending on the scale of the different pieces that form the model. A different number of meshes with different levels of details can be assigned. This depends on the desired range of zoom in which they become active, that is a percentage corresponding to the ratio between bounding box and screen heights. In our case, to produce a substantial improvement in performance, the component needs at least three versions of the same mesh. The first (high definition - LOD 0) is the decimated mesh automatically exported from Rhinoceros following the previously described consideration. The second and the third retain a lower amount of detail (LOD 1 and LOD2) that reciprocally correspond to a reduction of 70 %. The result was two very light meshes with a great amount of defects and deformations due to their strong simplification. They have been taken into account only for objects really far from the observer where only the silhouette and approximate shape of the block are important. A LOD approach works very well in this kind of situation due to the outdoor environment, the dimensions and the high degree of verticality of these structures. The application of human sight rules surely further lightened the impact of the model on the system, but the complexity of the Main Spire proved still too heavy to be computed as a whole. This approach needed to build an ad hoc scene subdivision and to design a correct setup. Scene Set Up. The logical solution was to subdivide the environment into levels as for videogames, so that only the blocks related to a specific section would be loaded at the same time at higher definitions. The whole complex of Milan Cathedral can be seen as formed by different constituting areas, and the same concept can apply to the Main Spire. The Spire has been divided into its four constituting areas as in the technical BIM environment. Each one corresponds to a specific scene: upper cladding of the dome, first balcony, central cylindrical staircase and Belvedere. Each has been assigned to a specific scene inside Unity creating four virtual container for objects and each scene is initially separated and independent from the others. There are many advantages in choosing this segmentation. First, in each section there are few pieces with a lot of details, significantly reducing the time computation and generally making the system faster; secondly in this way it is possible to load just the interesting area allows for easier and faster consultation. To create the whole version of the tour, all the scenes have to be connected so that the user can automatically tour through the whole Main Spire. “Triggers” handle the connection between different areas. The position of the triggers follows the sequential order of the areas of the main spire. From the upper cladding of the dome the user passes to the first balcony, than to the central cylindrical staircase and only after going through all the sections reaches the Belvedere and vice versa from the top of the Spire, it is possible to descend until the cladding level (Fig. 4). Layer Structure. Since the application is thought also for the facility management and the restoration works of the Milan Cathedral by the Veneranda Fabbrica, it is important to maintain the name and the layer structure of the blocks coming from the NURBS model inside Rhinoceros and used for the technical application of the BIM system [30]. It is required to connect the application to the central database, that is the core of the BIM system where are stored data, info, photos and the original high-resolution models of the marble blocks. Maintaining the correct naming of the blocks is important for yet VR for Cultural Heritage 153 Fig. 4. Background seen from the upper cladding of the dome another reason: in the original model, the marble blocks are named according to the elapsed intervention works. There are three different name types, an integer number (i.e. 1), a number followed by an underscore (i.e. 1_1) and the letter “n” followed by a three digits number (i.e. n532); changes in this convention would result in the wrong identification of the marble blocks and of the corresponding maintenance activity. Keeping this information during the export operation is not an automatic and easy task, due to different behaviour of Unity and Rhino in managing objects and their names. The problem lies in the fact that whereas Rhinoceros automatically accepts objects with the same name but on different layers, Unity 3d does not. Therefore exporting the model (from Rhino to Unity) using a spatial “area based” query, as logically, resulted in an automatic random variation of the names assigned to the blocks. The solution to avoid this unacceptable situation was found by taking advantage of the LEGO structure of the model and by importing the pieces from the subclass “typology”. In this way the software does not meet blocks with the same name inside the same asset and keeps the correct structure of the blocks. This implies that the export has to be done piece by piece and in the future needs to be automated in order to easier and faster export a huge number of 3D blocks. Collisions. A general common task in this type of applications is the definition of the “physics rules” inside the virtual environment. The tour, to be effective, must be realistic, which means that the interactions between objects must follow real world physics rules. The main scope is to create useful interactions between the user and the model in order to (i) navigate the complex space of the spire, (ii) display information about the marble blocks, statues and decorations. Inside the software, the correct use of “Colliders” represents the basis for both interactions. Different types of colliders (static and convex) are available. The choice obviously fell on the static type of collider. This is a key point because on this assumption the Unity engine regulates the physics of the object and makes useful optimizations. It is possible to assign automatically a single static collider to each object, so that they are selectable in order to get the specific data related to the marble piece and its conservation process. For static object that 154 F. Fassi et al. sometimes need to be moved or scaled inside the application it is possible to add to the desired block a “RigidBody” component with the “IsKinematic” property set to active, this would be necessary for example for the implementation of interaction through a “Leap Motion” device (Fig. 5). Fig. 5. Trigger sequence Model User Interactions. Interactions between the user and the model are thought for the facility management of the cathedral. Before the models are displayed through the Oculus device, a special function takes care of colouring objects involved in maintenance activities: green blocks are the remaining parts of large blocks that survived the interventions, while red ones are the new additions, following the same encoding used in the BIM system (Fig. 6). Fig. 6. Result of the process and general view through Oculus device (on screen display) (Color figure online) The user interaction with the model is very simple: it is enough to look at the desired object and click on it for its selection. To retrieve the right object, it is necessary to shot the RayCast directly from an object inside the model space, starting from the crosshair cursor. The crosshair cursor is a custom version adapted to be used inside the stereoscopic rendering used by the oculus device, managing the problems of double VR for Cultural Heritage 155 vision, offset from the centre of the screen, facing the camera changing its position and orientation in space according to head movements. When an object is selected, the marble block is highlighted in yellow. It is necessary to take into account of all multipart meshes as well as different models of the same block imported for LOD level purposes, changing the colour for all of them. After the selection, the system displays the connected BIM data on screen. Tabs have been designed ad hoc, covering all corresponding fields of the WebBIMDuomo. Some tabs are also designed to display the image gallery: one for the images of the object, one dedicated to the performed restoration works and one for the post-intervention, in the same way as is the WebBIMDuomo technical release. All information is gathered in real time from the database of WebBIMDuomo, through an internet or local connection to the database. The connection to the database was not always smooth because Unity is still locked to a very old version of Mono 2.0 and it has an incomplete support of system.data.dll. This leads to the inability to use the .NET ODBC class, which is not fully implemented, thus preventing the connection to the PostgreSQL database, used for WebBIMDuomo. To solve this problem, it was necessary to use an old version of Npgsql, compiled for the old .NET corresponding to Mono 2.0, and to mix the Npgsql class with system.data general class. Npgsql must be used only to the connection, while for the commands and readers it is required to use IDbCommand and IDataReader. Otherwise, the queries are executed, but the connection closes unexpectedly when the system tries to access the data. At first, the object is identified using its layer structure and its name; then a set of queries are executed onto the database and all data table are filled with the results. 4 Conclusion The research presented here aimed at applying new VR immersive techniques to the field of architecture and facility management. The main goal was to evaluate how VR could be used in the sector of Cultural Heritage as a support for conservation and maintenance activities. The work represented the follow-up of the previous research activities of survey and modelling of the Main Spire of the Milan Cathedral. The research work led to the creation of a complete high-resolution 3D model, currently used as a support of the maintenance activities of the Milan Cathedral. Furthermore, in order to promote a wider use of the 3D virtual model to a larger group of users also including non-experts of 3D, an ad hoc BIM system has been developed. This system is able to connect the modelling software with an external navigation system that encourages a simple and shared use of the 3D model by different stakeholders of the conservation process. It works like a catalyst that collects and connects data and information. The presented work succeeded in adding a further step in terms of virtualization by creating a VR immersive system that not only allows the classical virtual visit of the object, but also provides the technical information stored in the BIM system. The research was split into different steps ranging from the optimization and simplification of the original real-based models for VR environment, to the creation of the VR system inside Unity software, and finally to the connection of the VR application to the technical BIM database. The mix of high accuracy 3D survey, 156 F. Fassi et al. reality-based modelling, information system and virtual navigation allows a multidisciplinary professional use of the final virtual model. Differently from the industrial realm, the immersive virtualization of a BIM system in the Cultural Heritage field is nowadays not as widespread as the current technology could, in theory, allow. As of today, for the Milan’s Cathedral it is perhaps still a futuristic vision. The research demonstrated how high resolution models with a given maximal representation scale, that have been created for technical purposes, can nevertheless be easily adapted to the use in virtual environments without an excessive loss of detail; it also shows how the connection with external BIM data can create an augmented system inside a virtual environment. Moreover, this type of systems is surely optimal to better understand the geometry of a very complex and extensive architecture, and to immediately georeference the information. In perspective, the tested system may represent a fundamental tool to address the main difficulties faced by the VFD: the enormous dimension of the cathedral, and the consequent huge amount of “objects” and information that must be managed during the endless works of maintenance. The next phase of this line of investigation will have to focus on improving the graphic quality and performances, and to add more interactive functions. The future developments of our work will head in the following directions: (i) the development of the automatic exporting of the optimized model and its LOD levels for VR visualization, (ii) the possibility, as in the BIM system, to dynamically load the object from the cloud without restrictions or pre-constituted sub-divisions, and give the possibility to the user to choose from time to time what to display, (iii) the integration of external devices as “Leap motion” to improve the immersive experience and interaction with the model, (iv) the possibility to actively interact with the VR BIM system not only in reading- but also in writing-mode. References 1. Encyclopaedia Treccani. http://www.treccani.it 2. Veneranda Fabbrica del Duomo di Milano. http://www.duomomilano.it/en/ 3. Zara, J.: Virtual reality and cultural heritage on the web. In: 7th International Conference on Computer Graphics and Artificial Intelligence, pp. 101–112 (2004) 4. Virtual Heart of Central Europe project site. http://www.sccg.sk/*projects/virtual-heart 5. Virtual Old Prague project site. http://www.cgg.cvut.cz/vsp 6. Fassi, F., Colizzi, L., De Pascalis, F.: 3D visualization and virtual reality for cultural heritage diagnostic. Conserv. Sci. Cult. Herit. 7, 31–42 (2007) 7. Calori, L., Camporesi, C., Pescarin, S., Guidazzoli, A.: Open heritage: an integrated approach to web 3D publication of virtual landscapes. 3D-ARCH 2005: Virtual Reconstruction and Visualization of Complex Architectures, pp. 1–5 (2005) 8. Forte, M., Pescarin, S., Pujol Tost, L.: VR applications, new devices and museums: visitor’s feedback and learning: a preliminary report. In: 7th International Symposium on Virtual Reality, Archaeology and Cultural Heritage VAST (2006) 9. Forte, M., Danelon, N.: Regium@Lepidi 2200 Project. Archeomatica 6(1), 42–48 (2015) 10. Navvab, M., Bisegna, F., Gugliermetti, F.: Experiencing the tangible past through virtual reconstruction: cultural heritage of buildings and their environmental boundaries. Archeomatica 4(3), 36–41 (2013) VR for Cultural Heritage 157 11. Pescarin, S., Pagano, A., Wallergard, M., Hupperetz, W., Ray, C.: Evaluating virtual museums: archeovirtual case study. In: 40th Conference on Computer Applications and Quantitative Methods in Archaeology (2012) 12. Smithsonia X3D. http://3d.si.edu/ 13. The virtual museum of Iraq. http://www.virtualmuseumiraq.cnr.it/prehome.htm 14. Europeana. http://blog.europeana.eu/2013/12/virtual-reality-and-the-museum-of-the-future 15. DICONS home page. http://3dicons-project.eu/ 16. Virtual Museum Transnational Network. http://www.v-must.net/ 17. Apa Game. https://hpc-forge.cineca.it/files/visit_Dissemination/public/ApaGame/ApaGame 2013/ 18. Guidazzoli, A., Baglivo, A., De Luca, D., Imboden, S., Liguori, M.C., Rivalta, A.: Crossmedia integration of 3D contents for cultural communication. In: 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (2014) 19. ATI alla scoperta di Veio. http://www.glietruschielaldila.it/ati-alla-scoperta-di-veio 20. Russo, A., De Lucia, M.A., Cosentino, R., Guidazzoli, A., Imboden, S., De Luca, D., Liguori, M.C., Verri, L., Bellavia, G.: Apa l’Etrusco sbarca a Roma e passa il testimone ad Ati. Un cortometraggio tridimensionale per raccontare il santuario di Portonaccio a Veio. Archeomatica 6(2), 42–47 (2015) 21. CINECA. http://www.cineca.it/it/comunicatistampa/apa-letrusco-sbarca-roma 22. Mostra. http://www.glietruschielaldila.it/la-mostra 23. La situla della certosa. http://www.glietruschielaldila.it/la-situla-della-certosa 24. Il sarcofago degli sposi. http://www.glietruschielaldila.it/il-sarcofago-degli-sposi 25. FBK news. http://www.fbk.eu/news/sarcophagus-spouses-reconstructed-3d-fbk-researchers 26. Fassi, F., Achille, C., Fregonese, L., Monti, C.: Multiple data source for survey and modelling of very complex architecture. In: The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. XXXVIII, Part 5, pp. 234–239 (2010) 27. Fassi, F., Achille, C., Fregonese, L.: Surveying and modelling the main spire of milan cathedral using multiple data sources. Photogram. Rec. 26, 462–487 (2011) 28. Achille, C., Fassi, F., Fregonese, L.: 4 year history: from 2D to BIM for CH. In: 18th International Conference on Virtual Systems and Multimedia, pp. 377–382 (2012) 29. Mandelli, A.: Surveying, modelling and management of cultural heritage. Definition of good practices through applications on monuments and artworks. Ph.D. thesis (2016) 30. Fassi, F., Achille, C., Mandelli, A., Rechichi, F., Parri, S.: A new idea of BIM system for visualization, web sharing and using huge complex 3D models for facility management. In: The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. XL-5/W4, pp. 359–366 (2015) 31. Fassi, F., Parri, S.: Complex architectures in 3D: from survey to web. Int. J. Herit. Digit. Era 1(3), 379–398 (2012)