Home networking as a distributed file system view

Home Networking as a Distributed File System View Serge Defrance, Rémy Gendrot, Jean Le Roux, Gilles Straub and Thierry Tapie Technicolor R&D France 1, Avenue de Belle Fontaine 35576 Cesson-Sévigné, France [email protected] other devices handled by third parties. Unfortunately, these devices have different capabilities and interfaces. Consequently, many users are discouraged by the complex challenge of accessing, moving, organizing, and formatting their large digital content libraries. Indeed, many protocols and technologies can be used to solve the home network content access and sharing issues: the popular computer’s network storage protocols (NFS, SMB, FTP, HTTP), the established UPnP/DLNA residential networking protocol and the IETF Web DAV (Web based Distributed Authoring and Versioning) http extension protocol. Anyway, if these different protocols are effective in their own domains, none of them covers all the home network requirements. ABSTRACT Devices forming a Home Network have different capabilities and interfaces, discouraging users to organize their large digital content libraries. To help users, we propose to organize the Home Network according to a gateway-centric architecture, where the content access unification is realized at the file system level and where no additional software installation on devices is required. Solutions for realizing this unification individually exist for the various devices making up the Home Network (UPnP/DLNA devices, personal computers, cloud storage systems, etc). Unifying the content access at the file system level offers a powerful lever for many legacy applications, as far as these applications can access all shared data in the Home Network. Users can thus continue to use their PC’s file manager or favorite media player to browse or display shared content. An indexing application, running on the gateway, possibly managed by the ISP and accessible from any device via a simple web interface, enables more powerful content retrieval and user experience. Such application may be enriched to offer additional services like content format adaptation, duplication detection or automatic backup. Lastly we describe how this gateway-centric architecture can be leveraged by cloud applications such as distributed storage systems. In this paper, we propose an innovative solution based on a gateway-centric architecture where this central equipment is in charge of translating the native content sharing protocol of any connected device. Documentation, Design, Experimentation, The structure of the paper is as follows: after a review of the related work from which we drew our thinking in general, we detail the motivations of our approach in section 3. Then we introduce our solution and describe how we can operate with major content access and sharing protocols, in Section 4. In Section 5 we describe how the distributed file system view can be leveraged by various applications. Section 6 outlines potential impact for contributing storage applications, while Section 7 concludes this paper. Keywords 2. RELATED WORK Categories and Subject Descriptors C.2.2 [Network Protocols]: Applications, C.2.3 [Network Operations]: Network management. General Terms Home Network, Virtual File System, Content Sharing, Indexation, Backup, Storage, Distributed, Cloud. Many studies point out the diversity and the complexity of the home network eco-system. Different approaches are proposed to move towards a better organization of home networks. [1] and [2] discuss the management of the home network for the sake of security and smooth running, with the idea of a central equipment collecting traffic information. UPnP/DLNA (see [3]) is a well-established standard , currently supported by an increasing number of connected CE devices. It defines protocols for interoperable home networking requiring neither a central device nor any configuration operation from the user. [4] also points out the difficulty for an unskilled user to operate a home network, further arguing that the zero-configuration UPnP, due to neglected security aspects, is not the ideal candidate as networking protocol. Intel raises the problem of a unified access to the content of the home network. In [5], content access unification is realized at the file system level : content stored on multiple devices appear to the user 1. INTRODUCTION Today, users enjoy creating and collecting a rapidly going up amount of digital content like digital photos, music and videos. New Internet storage services increase this trend. Users expect to share their content with their family, friends or even wider communities making their content available not only over their own devices but also over Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. HomeNet’11, August 15, 2011, Toronto, Ontario, Canada. Copyright 2011 ACM 978-1-4503-0798-7/11/08 ...$10.00. 67 For these reasons, we choose to explore an alternative solution, not imposing any additional software installation on the devices of the home network. Consequently we propose to use their native communication protocols and, to solve the interoperability problem, we propose to connect each device to a central equipment, to which we devote the role of protocols translator. Software updates are now necessary only on this central equipment. This equipment should be connected and powered on most of the time. It may be under the control of a service provider so as to be upgraded when appropriate, as is typical for a home gateway. Additionally we propose to implement this translation at the file system level. This choice, driven by content management storage concerns, ensures to keep compatibility with all applications manipulating stored content. as a familiar directory. However, it does not address heterogeneous protocols as the prototype focusses on SMB protocol (no access outside the home network is considered). [6] presents a system which aim is to share content between users. To this purpose, the system proposes an unification to content access based on device capability reports to a central server, and the use of ad-hoc (depending on the device and/or its environment) communication protocols for data exchange. The unification is yet realized at the application level, which means a constraint for adding new applications and services. Unification at the file system level is again discussed in [7] with the sake for « efficiently utilize freely available disk space on Desktop machines ». The system however requires the implementation of an agent on each Desktop machine. Storage, which is the main topic in [7], is also a real concern, especially regarding safety and security of personnal data. Online storage answers this concern. Solutions based on a client/server architecture are proposed like Dropbox [8] or Amazon S3 [9]. Online storage can also be provided as a direct access under VFS to online account, like for instance Picasa with GdataFS [10]. Alternative architectures are peer to peer systems [11] which reduce operation cost by dint of a loss in performance, or hybrid systems, corresponding to a medium term between pure peer to peer and client / server architectures. In Wuala [12] for example, data is stored on a central server and partially on peers. Peer assisted solution, as described in [13], aims to take benefit of storage capabilities at the edge of the cloud (i.e. in home networks) in order to offload the central server, cumulating performances of a client/server system and the reduced cost of a peer to peer system. 4. DISTRIBUTED FILE SYSTEM VIEW Our solution consists in a gateway centric architecture where we realize the content access and the protocol translation at the file system level (Figure 1). To this purpose, we take benefit of the Virtual File System (VFS) layer from Linux operating system. The VFS was developed in order to ease the addition of new file systems into the Linux kernel. It is frequently used in Unix like operating systems to facilitate the integration and the use of several file system types [14] [15]. 3. MOTIVATIONS The most accomplished system in terms of access unification is based on the unified UPnP/DLNA protocol . However UPnP/DLNA still faces some limitations. First, as new technologies or opportunities appear, the standard needs to evolve, establishing different releases and creating backward compatibility issues. For instance, UPnP/DLNA initialy solved the problem of sharing AV content in the home and is currently being extended in order to also support commercial content delivery in the home. Figure 1: Centric Gateway Architecture Taking benefit of VFS capabilities, we propose, in addition, to represent the shared content of every connected device as a dedicated folder in the tree structure of the gateway. Each dedicated folder is the mounting point of the corresponding device in its native network protocol. (Figure 2), for each of which VFS offers an adaptation module. Second, the approach requires a device to either natively support the protocol or to provide ways to be upgraded (at least to allow further sofware installation), which is generally very problematic in many Consumer Electronics (CE) devices as TV sets, mobile phones, etc. In the UPnP/DLNA case for example most TV sets now claim being compliant with UPnP/DLNA, but they generally cannot be upgraded. In order to benefit from the next release of the standard (as commercial content delivery support), users will have to change their TV set. However, the implementation of the module, its configuration and the creation of the mounting points require computer skills that not everybody has. The challenge is to make possible that any user overcomes this obstacle. To this purpose, we have developped a piece of software, called “Network Organizer”, which realizes an automation of the Home Network structuration and, in particular, of the VFS adaption modules installation. This 68 piece of software runs on the gateway. It may be provided by the ISP. We experimented in our prototype the integration of Picasa and Youtube by adding GdataFS to the VFS [10]. We also added a raw data storage service by integrating a Dropbox software to the VFS [8]. Such software installations need to be done only once and could be facilitated via the Network Organizer. Indeed, during the software installation user credentials (login/password) need to be provided, which could be done with the assistance of the Network Organizer (dynamic provisioning, etc). We have realized this assistance in our test bed for the Dropbox module installation. 4.3 SMB/NFS devices A last important part of home network devices consists in the personal computers themselves. As file system level network protocol, they natively support SMB (for Microsoft Windows systems) and NFS (for Unix like systems). Figure 2: Virtual File System View 4.1 UPnP/DLNA devices In such devices, the Network Organizer creates dedicated folders, basically one devoted to shared content exported by the device and one devoted to import content shared by other devices. At the same time, the shared content of each device appears in the gateway as a part of its own folders tree. UPnP/DLNA protocol can be interfaced to the VFS at two levels. A UPnP/DLNA Media Server running on the gateway on top of the VFS is in position to offer to UPnP/DLNA clients any visible user’s space connected to the VFS. Therefore the simple installation of a UPnP Media Server allows to expose the files being available at the Gateway level towards any UPnP/DLNA Player and/or Controller. We experimented the PS3 Media Server [16] and the MiniDLNA Server [17]. The network protocols (SMB/NFS) expect the devices going through a process when entering (connecting) or leaving (disconnecting) the system. This raises the issue of sorting out these complex manipulations without any additional software installation on every users’ devices. The Network Organizer is in charge of managing the registration and the connection or disconnection of users’ devices. Let’s describe these different procedures more in detail. In the opposite direction, a UPnP FUSE (File system in UserSpacE) module like DJmount [18] will map under the VFS all the available UPnP/DLNA Media Servers on the network and their attached offered content. Local applications running on the gateway can therefore see this content (present on remote UPnP/DLNA Media Servers) as if they were locally available from the local file system interface. It can also be exported towards remote client using other network file system specific export functions (NFS, SMB) as being described later. 4.3.1 Registration The new coming user shall first contact the Network Organizer via its web browser. His first operation is to register himself in the Home Network. The registration procedure is done only once, when the device joins the home system for the first time. Thanks to the initial http connection, the Network Organizer collects a certain amount of information on the device and the attached user account like operating system, IP address, login, password, etc (Figure 3). The collected information will be used by the Network Organizer first to build export/import folders on its side devoted to the new comer, and secondly, to generate a script (bash/shell) dedicated to the operating system of the user’s device. The script is proposed to the user’s device for upload and execution via the web browser. When executed, the script builds on the user’s device appropriate folders (import and export). At this point in time, everything has been prepared so that the new user/device can join (connect) the home system. UPnP/DLNA natively includes a discovery protocol so that there is no specific action to carry out when UPnP/DLNA devices enter or leave the home network. The UPnP/DLNA installation on the VFS is performed by the Network Organizer. 4.2 Cloud storage systems Nowadays, an increasing number of web servers offer storage services to Internet users, supporting a wide range of applications from raw data storage (Dropbox, Carbonite, etc) to photo (Picasa, Flickr, Facebook, etc) or video sharing (YouTube, Dailymotion, etc). Most of these servers provide FUSE compliant software that can be installed at the VFS level so as to ease cloud storage integrations. When they do not provide the code themselves, they generally provide an open interface so that third parties can develop applications, easing the integration of those cloud storage services with other applications. Oftenly the Open Source community develops a FUSE compliant software. 69 User information 5.1 Navigation in home network content User information HTTP HTTP HTTP server Windows Client Linux Client Network Organizer Gateway Mount Share The first common benefit of the distributed file system view comes from the file system interface itself: any legacy application is accessing data via a local file system interface, as if it would manipulate local content. Typically, PC users can continue to use their file manager application (i.e. Windows Explorer or Linux File Browser) for browsing their files (Figure 4), or their popular media players (e.g. Windows or VLC Media Player) to play their content. The advantage is that users have now access to all home network distributed content thanks to the specific folders created in their PC to import content shared by all connected devices; it could be content shared by another PC, a local NAS or content stored in cloud areas which accounts are mounted in the gateway file system. Mount Export VFS NFS SMB Figure 3: Registration Procedure 4.3.2 Connection DLNA devices (typically TV sets or game consoles) can access shared content via the DLNA Media Server (as described in Section 3.1) once it has scanned properly configured folders (indeed any content shared by any connected device appears as a folder of the gateway local file system, including the cloud accounts). The connection procedure may be done just after the registration and has to be done each time the user/device joins the home network, for instance at each power on of the user’s device. The connection procedure makes effective the mounts prepared by the registration procedure. So the content of exported folder of the user’s device is now available into the tree folders of the gateway, as it is part of the local gateway file system. This content is exported from the gateway to the rest of the network. The content is now visible by every connected device. And, in the other direction, the content shared by the other devices of the home network appears as a mounted folder in the tree structures of the local file system of the device. Note that, for the sake of security, this content may appear as read only, and only selected devices appear. Security aspects may be further investigated. However we consider this issue being out of scope of this paper. 5.2 Indexing of home network content For the moment, thanks to the VFS abstraction layer, users do not need to care about the physical location of the accessed content on the distributed network; they just need to remember the name and path. The system is confortable for a user if the number of files stored is not so high. But when a huge number of files is available in the home network, it becomes impossible for a human-being to remember all the content names and paths. Some DLNA Media Players are proposing support in this direction (e.g. artist classification, album identification); the main issue is that these services concern exclusively DLNA compliant devices. 4.3.3 Disconnection The disconnection procedure shall take place when the user/device leaves the network. It consists in unmounting the user’s exported folders in the gateway. The user’s device will not be part of the network anymore. The other devices still connected to the home network will not be able to access to the shared content of the disconnected device, which now appears as “disconnected” to them. The disconnection procedure can be initiated by the device itself (as part of the shutdown/standy procedure). But this cannot be the only way, as devices may be disconnected from the network by simple cable removal. Consequently the Network Organizer has to monitor devices presence, for instance by regularly pinging operations. Gateway UPnP/DLNA DLNA Media Server DLNA TV VFS Gdatafs Dropbox Player NFS SMB Local FS SMB Windows Computer Player Local FS On Line Storage NAS NFS Linux Computer 5. BENEFITS FOR THE APPLICATIONS This section illustrates how the distributed file system view can be leveraged by various sets of applications. We here describe such applications that we have deployed on our test bed. Figure 4: Navigation in Home Content Alternatively, we have implemented a web based Digital Asset Management (DAM) application on the gateway, requiring only the support of a web browser on an end device. The term DAM is used here to refer to all the different tasks and applications used to ingest, to index, to 70 Thus, when a user identified a content using the DAM tools, and expects to render it on his TV, the DAM resolves the content path (from the distributed file system view path into the UPnP Media Server path) and uses the local UPnP Control point to invoke the UPnP Media renderer of the TV with the relevant content path. This example demonstrates how the gateway, ensuring communications with devices using their respective native protocols and translating locally these protocols, leverages significantly users’ home network experience. catalogue and retrieve digital assets (content) such as documents, digital photos, music and videos based on metadata embedded in the digital assets or at least file names (Figure 5). Whole home network content can be indexed by the DAM application and the results (i.e. the name of the content, the location of the content, and eventually a preview or a jacket) stored in a database on the gateway. Users can browse this database from their PCs, their Smartphones, or any device equipped with a web browser, the DAM being in charge of building the web pages adapted to the device screen size and resolution. The support of query services by the DAM, using key word search is also straight forward and enables more powerful content retrieval end user experience. Lastly, to play content, users will transparently run the local media players or take advantage of the ones supported by the DAM itself. 5.4 Home network content backup Despite the fact that the majority of users place a high value on their own content, many users do not backup their files. In our test bed, we offer to each user a secured storage space located on the gateway, secured because regularly and transparently (for the user) backed up in the cloud (Dropbox). The proposed gateway centric architecture implementing the content access and the protocol translation at the file system level gives interesting opportunities to ISPs to propose backup applications based on well established and widespread file system and data backup tools & process. In addition clever choices could be done to ensure complementary high bandwidth local backup onto the local NAS and high secure storage services onto the cloud. Hierarchical storage management strategies could be designed to automatically move content between these in home and outside storage spaces quite easily. Furthermore, the DAM may be enriched (possibly under the control of the ISP) to offer additional services (not implemented in our test bed) like content format adaptation (i.e. transcoding) and content duplicate detection. Depending on the gateway resource availability, transcoding and duplicate detection may be launched in background to avoid gateway overloading. 5.3 Advanced home network remote control New users’ home network experience is possible with the proposed distributed home file system view. A tablet, or a Smartphone, can advantageously replace the outdated TV remote control as well as the keyboard or the mouse of any PC. Users will just take advantage of the web browser, the virtual keyboard and the touch screen offered by these devices to query the DAM database via key words and auto completion support. As an alternative to install a dedicated UPnP Control Point application on the tablet, we propose to run the UPnP Control Point software on the Gateway, and to make it available to the user via the DAM interface. This also allows preserving the user interface experience of the system even when rendering content on external TVs. 6. TOWARDS NEW HOME STORAGE ARCHITECTURE There has been a lot of work over the last years to propose distributed storage architectures where peer to peer applications contribute storage space and network bandwidth so as to build a global and scalable distributed storage system that can be used for various purposes (backup, sharing, access, etc). However all the previous work considered a contributing application as single agent (generally a piece of software running on a PC) contributing a fix quantity of storage with a certain (and generally low) availability. Installing such an application on top of the home network distributed file system view creates a certain number of new opportunities. First it allows contributing resources from other devices of the same home network, via the file system view without requiring any software installation on each contributing device. This is going therefore to ease the inclusion of new storage areas in the system as their integration gets automated. Second, considering the huge deployment of low cost and high capacity storage into connected devices, we can argue that storage is increasingly and massively available at the edge of the network. Indeed several studies show that end user hard drives are on average half empty and the authors in [19] reveal, after a five-year study, that the ratio of free space over the total file system capacity remained constant over the years. We can reasonably extrapolate this fact to the storage available at every connected device and Gateway HTTP Control HTTP Server Tablet DLNA ControlPoint DAM Indexation DLNA Media Server UPnP Control VFS Gdatafs Dropbox NFS SMB DLNA TV On Line Storage NAS Linux Computer Windows Computer Figure 5: Indexation/Retrieval of Home Content 71 consider that aggregating this free space, represents a real and scalable Eldorado of available space that can be contributed to such systems providing it can be made easily available. The home network distributed file system view can in fact be an enabler of distributed storage systems by boosting the availability of resources. Third a contributing application running on top of such a file system view will behave very differently from what has been considered so far: a contributing application running on a single device is either connected or disconnected with a significant churn. When it is connected, it contributes a fix amount of resources. When it is disconnected, its resources become unavailable. A contributing application running on top of the home network distributed file system view, will contribute a variable amount of resources depending whether local devices are available or not via the VFS. Moreover since gateways provide generally much higher availability than other devices, the contributing application will enhance the overall stability. Indeed, such a distributed storage system will be built of very stable agents contributing a variable size of resources. This new model is therefore introducing some changes that can be leveraged by distributed systems in various ways that can be further explored. [2] W.K. Edwards, N. Feamste, R.E. Grinter, Y. Deng, X. Zhou K.L. Calvert, "Instrumenting Home Network" , New Dehli, 2010. [3] DLNA, http://www.dlna.org/home. [4] H. Kinkelin, S.K. Ghai, G. Carle A. Müller, "A Secure Service Infrastructure for Interconnecting Future Home Networks based on DPWS and XACML" , New Dehli, 2010. [5] M. Payne E. Hady, A Consummer's Eye View of Whole Home Storage, http://www.intelconsumerelectronics.com/ConsumerElectronics-3.0/Whole-Home-Storage.aspx. [6] A. Saaranen P. Belimpasakis, "Sharing with people: a system for user-centric content sharing" Multimedia Systems, vol. 16, no. 6, 2010. [7] K.N. Honwadkar S.V. Patil, "Unified Virtual Storage: virtualisation of distributed storage in a network" International Journal of Computer Applications, no. 1, 2010. [8] Dropbox, http://www.dropbox.com. [9] Amazon Simple Storage Service (S3), http://aws.amazon.com/s3/. 7. CONCLUSION [10] GDataFS, Fuse filesystem for PicassaWeb, http://gdatafs.sourceforge.net/. To solve the problem of interoperability in heterogeneous home networks, we have proposed a Gateway centric architecture where the content access unification is realized at the file system level, allowing each device to access the entire home network using its native protocol. The unification is automated thanks to a software module called Network Organizer running on the gateway. Our solution does not require any software installation on end-devices, relying on the gateway as the unique device for software installation or upgrade. Our architecture therefore enables flexible migration scenarios, smooth integration of new features, protocols or technologies as they appear. Regarding devices where the manufacturer policy prevents direct access to the file system of the device (typically iPhone), access unification has to be envisioned at the application level. Further studies are required on this topic, as well as on security aspects which have not been addressed in this paper. [11] Z. Anwar, W. Yurcik, L. Brumbaugh, R. Campbell R. Hasan, "A Survey of Peer-To-Peer Storage Techniques for Distributed File System" , Washington, 2005. [12] Wuala by LaCie, secure online storage, http://www.wuala.com/. [13] M. Dell'Amico, P. Michiardi L. Toka, "Online data backup: a Peer-Assisted Approach" , Delft, 2010. [14] A. Tanenbaum, "Operating Systems: Design and Implementation", Prentice Hall, 1987. [15] S. Kleiman, "Vnodes: An Architecture for Multiple File System Types in Sun UNIX" In Proceedings of the Summer USENIX Conference, June 1986. [16] PS3 Media Server, http://www.ps3mediaserver.org/forum. 8. ACKNOWLEDGMENTS This study was partially funded by the FIGARO (Future Internet Gateway-based Architecture of Residential networks; FP7-ICT-258378) collaborative project. [17] MiniDLNA, http://sourceforge.net/projects/minidlna/. [18] DJmount, mount AV device as filesystem, http://sourceforge.net/projects/djmount/. 9. REFERENCES [19] W.J. Bolosky, J.R. Douceur, J.R. Lorch N. Agrawal, "A five-year study of file-system metadata" ACM Transaction on Storage, vol. 3, 2007. [1] N. Feamster, "Outsourcing Home Network security" , New Dehli, 2010. 72