Keywords

1 Introduction

Virtual museums are becoming increasingly popular as they allow “to complement, enhance, or augment the museum through personalization, interactivity, user experience and richness of content” [1]. Creating a virtual museum commonly requires a lot of effort in terms of design, content management, and visual rendering, effort unavoidable whenever a given quality of the experience is to be achieved. However, it is possible to think at digital collections which, even if not carefully designed as proper virtual museums, provide an immersive experience richer and, expectantly, more engaging than digital galleries as commonly found in standard web pages.

The aim of the project described in this paper is to verify the effectiveness of automatic strategies to create a virtual museum of paintings, whose digital images are retrieved directly from repositories freely available in the network. The virtual museum is based on a virtual environment, initially empty, that can be populated with various digital elements based on specific user queries. Since the number of artworks can be variable, the structure of the museum is modular, i.e. it changes shape based on the number of works selected by the user. The goal of this project is to provide a tool enabling the exploration of a wide variety of artworks taken from public heritage repositories such as Europeana and Google Art Project, or from other generic repositories of multimedia information like Google Images.

Images and the related information are extracted using specific API making use of languages such as Javascript, C++, JSON and XML. The same approach is then applied in order to create an augmented reality application transforming any environment, purposely instrumented with markers, in a virtual museum.

2 Previous Work

Since the advent of VRML, 3D virtual museums have shown their potential as a tool for disseminating culture and knowledge using digital technologies and taking advantage of the advanced features provided by VR such as immersion and presence.

One of the first VR projects applied to cultural heritage was the Nu.M.E, or “New Electronic Museum of the city of Bologna” [2]. Launched in the first half of the 1990s, it can be considered one of the reference points for subsequent works in the field of virtual technologies applied to cultural heritage.

The MUVI [3] (Virtual Museum of Daily Life of the twentieth century) stems from a series of reflections on the memory of people and the events that preceded us above all in relation to the “danger” of losing all the collective heritage that brings together the knowledge related to everyday life and the identity of the various communities. The project, launched in 1999 at the VisIT Lab of CINECA, focuses on the transmission and conservation of past knowledge that can be favored by appropriate technological solutions.

A different approach was proposed in the Museum of Pure Forms [4]: in addition to presenting a VR system including graphic software and sterescopic visualization, it also includes a haptic interface to provide touch feedback. This enables a novel way of presenting historical/artistic artifacts to visitors, allowing them to touch artifacts, albeit in a virtual environment.

The ARCO project [5] aims to develop a series of technologies to “help museums create, manipulate, manage and present digitized cultural objects in virtual exhibitions accessible both inside and outside museums”. In the project there is a database storing information regarding digital representations, which can be managed by a system called ACMA (acronym of “ARCO Content Management Application”). ACMA associates each 3D model with a series of metadata outlining four main aspects of the museum’s objects: the curatorial aspect, the technical aspect, the exploration of resources and administration.

An investigation on how to go beyond 3D web-based virtual museums and embed virtual technologies in the offer of real physical museums is instead proposed in [6].

An approach similar to the one proposed in this paper was pursued in DynaMus [7]. In this case, also, the virtual content is dynamically downloaded from open data repositories, albeit the generated museum is made available on standard web browsers without any immersive/augmented option.

3 Architecture of the VR Museum

The Virtual Museum architecture (Fig. 1) is organized as follows:

Fig. 1.
figure 1

Data flow of the Virtual Museum generation procedure

Full size image

The initial phase is characterized by the selection of the search parameters, i.e. what the user wants to view in the virtual gallery. In this phase, HTML and JavaScript are used for the management of the parameters.

In the second phase the requested information based on the selected search criteria are retrieved. A C++ library is used to this purpose.

The third and last phase consists in the creation of the virtual museum, using the XVR [8] software.

3.1 Search Criteria

The selection of the search parameters in order to perform queries to the desired databases is made with a simple HTML page (Fig. 2). From this page, users can select three different repositories: Google Images, Wikimedia Commons and Europeana.

Fig. 2.
figure 2

HTML starting page

Full size image

Depending on the repository chosen, the user can perform two types of searches: enter the name of an artist or, in the case of Wikimedia Commons, a collection of works of art. Afterwards the user can choose the number of paintings to be visualize inside the virtual museum (in order to keep the download time within reasonable limits it was decided to set a maximum of 50 artworks, although no investigation has been performed about the optimal value of this parameter).

In addition to the searching on Google Images and Europeana, currently about 255 collections are indexed through an XML file created starting from Wikimedia Commons collected information regarding the works on the Google Cultural Institute. Each component of the web page is controlled by JS code in order to allow only valid queries. Once the user has outlined the parameters to perform the search, the XVR graphics engine has to be started.

This is obtained through Javascrip inside the same HTML home page: a new HTML page is created embedding the XVR engine as a plug-in and inserting customized parameters thanks to the “UserParam” attribute which contains the following data:

  • the first element is an ID corresponding to the chosen repository (Google Images, Wikimedia Commons, Europeana);

  • the second element is a string corresponding to the name of the artist or the name of the art gallery;

  • the third element is the amount of artworks to be inserted in the virtual museum.

These elements are concatenated into a single text string that is passed to the XVR plugin.

3.2 Resource Retrieval

Once the user XVR application has started, a preliminary phase begins consisting in the retrieval of information that takes place by means of a DLL library written in C++ compiled as a XVR external module, in charge of communicating with the server API of the selected repository.

In the case of Google Images, information retrieval is not a very laborious process, as the only information to be recovered is that relating to the URLs of the images to be downloaded. The solution adopted is to make a simple GET Request to Google’s servers to return an HTML page containing high-resolution images links. The HTML page is parsed in order to fill an array containing all the needed links, which are first examined and checked prior to the actual download of the image.

Checks ensure that:

  • the resource is actually reachable and returns a response code of 0;

  • the image to be downloaded is not be empty;

  • the images size is at least 10 K in order to avoid low-resolution resources.

A very different process is used to retrieve resources from Wikimedia Commons. In fact, it is possible to interface with an API in order to request, in addition to links to high-resolution images, also metadata containing information about the works. In addition, instead of receiving an HTML page in response to the GET REQUEST, a JSON object is received that is easily “parsed” through additional library methods.

Initially, a pattern is always used, which must be constructed according to the request that one wants to make. There are two phases for information retrieval:

  • the first phase consists in the getting a list of the items present in a given collection;

  • the second phase consists in the retrieval of the connection to high resolution images and related metadata.

In Europeana the process is somehow similar to that of Wikimedia Commons. Initially a list of items is obtained accessing the Europeana search engine and, subsequently, the application queries each item in order to retrieve related metadata and multimedia resources. Europeana provides several tools to retrieve information programmatically; dedicated APIs are available to perform searches in databases or to query entities and other open data. The tool used for file retrieval is the Search API.

4 Dynamic Generation of Virtual Museum

After retrieving all the information and resources related to the artworks, the next step is the creation of the virtual museum: a dynamic structure with three-dimensional models and multimedia resources placed in the appropriate spaces.

One of the peculiarities of the gallery is modularity: it does not have a fixed size, but varies according to the number of works requested by the user. 3DS Max software was used for the creation of three-dimensional models.

The virtual museum is divided into blocks: its general shape, corresponding to a box, is initially divided into four parts, with models for the top right, top left, bottom right and bottom left parts. Subsequently, “corridor” blocks are added to create a larger museum, with a shape depending on the side they are placed.

The structure of each block (Fig. 3) is basic: it is composed of a floor of about 7 m2 and three walls, a door and four quads, arranged two by two on two of the walls of the room. The procedural construction starts from the upper left room and then proceeds clockwise (Fig. 4).

Fig. 3.
figure 3

3D model of the museum room module

Full size image
Fig. 4.
figure 4

A possible arrangement of the room modules (top) and the resulting Virtual Museum including paintings mapped as textures onto pre-defined quads

Full size image

The interactive visit (Fig. 5, top) enables viewing the metadata related to each work (if available) and a simplified navigation of the rooms inside the museum. Metadata (author name, artwork name, period, etc.) are automatically displayed as overlaid text when the user approaches a painting. Information is only displayed when the user is at close range (Fig. 5, bottom).

Fig. 5.
figure 5

Close-up view of a procedurally generated Picasso exhibition (top) Visualization of artist/artwork information automatically retrieved from metadata (bottom)

Full size image

An assisted navigation mode facilitates the exploration of the virtual museum and improves the user experience in visiting the environment. The system consists in labeling the rooms of the museum and providing a list of the artworks present in the different rooms, so that the visitor can move from one room to another to see the desired paintings without having to complete the whole itinerary.

5 Architecture of the AR Museum

5.1 First Itinerary: Panels

The mobile augmented reality version of the procedural museum has been implemented as an Android app, using the Unity3D and Vuforia tools.

The application is based on four Android activities implementing the following functionalities:

  • the home menu

  • the interface to retrieve data from Wikimedia Commons (Fig. 6a, b)

    Fig. 6.
    figure 6

    Selection of the exhibition from Wikimedia (a), selection of parameters (b), free search on Google Images (c), picture of the real environment hosting the test AR exhibition

    Full size image

  • the interface to retrieve data from Google Images (Fig. 6c)

  • a Unity-based activity managing the real-time AR application.

It can be seen that, with respect to the VR museum, in this case only two repositories are supported. The procedure of data retrieval and resource downloading is similar in many respects to the one used in the VR museum. Once the data has been collected from the various activities, the Unity3D graphical engine is started.

In this case, Unity3D is used in combination with Vuforia, a library allowing to easily marker-based augmented reality apps. Vuforia makes available to Unity an AR Camera and imports the defined set of markers as asset files. The Unity 3D scene is composed of quads on which markers are placed following a predefined order. Prior to the execution of the real-time application, each marker is associated to a certain downloaded resource. Once a marker is recognized, it will be substituted by the painting automatically associated to the marker ID.

Once the user points the device towards a marker placed in the physical scene (Fig. 4d), the associated painting is applied to the surface of the quad hosting it, modifying its size according to the proportions of the image. The user can then see the artwork appearing on the designated surface (Fig. 7).

Fig. 7.
figure 7

Selection of the exhibition from Wikimedia (a), selection of parameters (b), free search on Google Images (c), picture of the real environment hosting the test AR exhibition

Full size image

6 Performance Test

Several runs of the application of the VR Museum have been performed, creating each time different virtual museums based on queries on different authors, composed of 30 paintings. The fastest repository (Fig. 8) was found to be Google Images, with an average of 27.06 s on average, Europeana being the slowest.

Fig. 8.
figure 8

Virtual Museum generation time by used repository

Full size image

As for the average download time by single image, Wikimedia Commons was the fastest with 0.80 s per image on average (Fig. 9).

Fig. 9.
figure 9

Averaged image download time by used repository

Full size image

7 Pilot User Study

To collect data for usability tests, questionnaires were drawn up subdivided into various sections ranging from the purely aesthetic aspect of the programs to the performative one. Questionnaires were distributed on a sample of twenty-five subjects. Three questions in particular sere aimed at evaluating, on a 7-point Likert scale, usability, quality of the cultural experience and the desire of reusing the application, with good results in these three respects in the case of the VR museum (Fig. 10).

Fig. 10.
figure 10

Usability evaluation of the VR Museum experience

Full size image

As far as the AR museum is concerned, results turned out to be extremely positive. All users find the augmented reality system suitable for a cultural experience; 92% of users confirmed that they want to reuse the application; 96% said they wanted to recommend this application to their friends; finally, 96% of users expressed the desire to see this type of technology adopted by museums (Fig. 11).

Fig. 11.
figure 11

Usability evaluation of the AR Museum experience

Full size image

8 Conclusions

The obtained results show a good potential of the methodology adopted and of the pilot implementation sketched out and used in this study.

The VR museum constitutes a good alternative of showing images of paintings in a sort a virtual 3D slideshow. However better results in terms of engagement could be obtained by presenting the environment in a fully immersive setup such as an HMD or a CAVE, while in this first pilot tests users were using a desktop environment.

Particularly promising seems the use of an AR paradigm as it has aroused great curiosity and proved to be extremely engaging, with the users amazed to transform the environment they actually are inside in an on-demand digital gallery.

A possible improvement will be the implementation of a vocal interface to perform queries or some actions within the museum, in order to increase immersion. An interesting feature in the augmented reality application would be the use of a markerless implementation, in order to avoid a preliminary instrumentation of the physical space hosting the experience.