ECOHOOD:Constructing Configured DBMS based on Frameworks

ECOHOOD: Constructing Configured DBMSs based on Frameworks Rubens Melo Fábio Porto Fernanda Lima Álvaro Barbosa Computer Science Department (Departamento de Informática) Pontifícia Universidade Católica do Rio de Janeiro – PUC/RJ Rua Marquês de São Vicente, 255 - Rio de Janeiro - RJ 22453-900 – Brasil E-mails: {rubens, porto, ferlima, alvaro} @ inf.puc-rio.br Telephone: 55-21-5299385 Fax: 55-21-2592232 Abstract New types of applications, such as: multimedia, workflow, web based and datawarehousing require different functionalities for a Database Management System (DBMS). The extension of general purpose DBMSs attending to these requirements introduces several performance, administration and development difficulties. On the other hand, producing specific DBMSs tailored to applications requirements imposes development burdens. This work presents an alternative approach, that is, the configuration of DBMSs. A configured DBMS is a set of selected inter-related services required for the data management in specific application domain. The services used in a configuration involve not only traditional, but also non-traditional DBMS functionalities, such as video and audio playback synchronism needed in multimedia application. In the ECOHOOD (Environment of COnfigurable Heterogeneous Object-Oriented DBMS) Project, we propose a novel approach for constructing DBMSs based on software engineering concepts of Frameworks which defines an Object Oriented architecture supporting reusability. Keywords: DBMS construction, DBMS extensibility, frameworks. 1. Introduction The adoption of Database Management Systems for the storage of data by traditional applications is a reality in the software industry, and gained extra power with the emergence of relational databases. Today, the appearance of new types of applications, such as: CAD/CAM, CASE software, multimedia, GIS, all requiring management of huge volume of complex data poses new challenges to the database research community. Relational databases were initially conceived to support applications with simple data structures and business oriented transaction types of access. The new areas of applications require special data treatment. These new requirements can be exemplified as: long transactions, complex query processing, storage of large objects such as video and audio, and the addition of new services to DBMS functionality, such as synchronism for continuous media. The answer to these new data requisites has been given by systems classified by Catell [Catt94] as Extended DBMS and DBMS Generators. As examples of the former, Postgres [SR86] and Starbust [S+86] attempt to satisfy the requisites of new applications by extending the DBMS with new data types which provide the required structure and behavior. Exemplifying the latter, EXODUS [C+90] is a code generator that adapt specific DBMS functionality and GENESIS [B+90] is a DBMS constructors based on modeled composition. According to Silberschatz and Zdonik [SZ97], extensibility and componentization are prominent research topics to be addressed. They state that it is necessary «to look for ways to open the architecture of DBMSs in such a way that new services can be incorporated and that database functionality can be configured in more flexible ways according to application needs». We present in this paper a more general approach for the construction of configured DBMSs. Our objective is to provide an architecture where a Database Implementor (DBI) can, at a reasonable cost, adapt any aspect of a DBMS software previously identified as «flexible». The adaptation will be possible for all the basic DBMS services, such as: query processor, transaction manager, rule manager and storage manager, besides allowing the introduction of new special services. The proposal is based on the software engineering concepts of Frameworks. Frameworks introduce an Object-Oriented architecture with predefined abstract classes to be specialized and ready-to-use components to be plugged in the architecture. This technology provides the required flexibility to allow the configuration of DBMSs to support new types of application with proper behavior. ECOHOOD (Environment of COnfigurable Heterogeneous Object-Oriented DBMS) is an on-going project of the PUC-Rio Computer Science Department and is based on the experience acquired in two of our previous projects: HEROS (Heterogeneous Object System) [USM94] and EITIS (Environment of Integrated Tools for Interactive Systems) [Melo87]. The research reported in this paper is a contribution to the ECOHOOD project. The remainder of this paper is organized as follows. In Section 2, we summarize the researches related to this work. Section 3 defines the main concepts involved in our project and describes the proposed architecture. In Section 4, we address the main DBMS services and exemplify one of the services framework used in the configuration of a DBMSs. Finally, in Section 5, some conclusions and suggestions for future works are presented. 2. Related Works Two main works may be considered as the principal researches related to ours. First, the extensible DBMS and the DBMS generators that have been proposed as means of constructing specific DBMS since the mid 80’s. Second, the recent research on Frameworks more usual in the Software Engineering literature. The following sub-sections survey related works in both areas. 2.1 Extensible DBMSs and DBMS Generators Extended database systems are complete DBMSs with part of its functionality extended to support new data types. The overall architecture is static. Differently, database system generators produce customizable DMBSs. There are two variations of database system generators, classified, according to their architecture, on Toolkits and Constructors [Catt94]. In the toolkit approach, «plug-and-play» customizable components are provided for the configuration of DBMSs. In the constructor approach, the system provides a database architecture from which a DBMS can be generated by inter-connecting existing modules. New DBMS modules may also be constructed. The extensibility in extended database systems is provided mainly by the creation of new data types, access methods, and by the extension of the query language. POSTGRES [SR86] is an example of an extended relational DBMS. It uses an extended data model with object facilities, such as: object identifiers, composite objects management, versioning treatment, and inheritance. Its query language, POSTQUEL is an extended relational query language that supports complex objects, and user-defined functions. EXODUS [C+90] and GENESIS [B+90] were the first projects that proposed a more general approach for DBMS construction through DBMS generators. Subsequently, other projects like Open OODB [WBT92], P2 [BT95] and Shore [Shor94] continued the research in this area. P2 and Shore are successors of GENESIS and EXODUS, respectively. The ECOHOOD project provides an architecture for configuration. The architecture is entirely flexible as to configure basic DBMS and non DBMS services providing support for new data type management. As a general DBMS framework, our proposal may configure an extended or a DBMS generator being, in this way, a more general approach. 2.2 Frameworks Frameworks are a promising software engineering technology for producing a reusable, «semi-complete» architecture that can be specialized to produce custom applications [FS97]. The reuse supported by a framework is at a larger granularity than classes, contributing to organize large systems development. Roberts and Johnson [RJ97] state the important concept of frameworks as a reusable design represented by a set of abstract classes and the way their instances interact. The idea is to define a several single components with predefined cooperation. Some of these components, designed to be replaceable, are called hot-spots [Pree94]. As cited in [Pree96], white-box frameworks consist of several incomplete classes that contain methods without meaningful implementations. In order to modify the behavior of white-box frameworks, it is necessary to apply inheritance to override methods in subclasses of framework classes. Black-box frameworks, on the other hand, offer ready components for adaptations. Modifications are achieved by composition, not by programming. It is important to highlight that frameworks are neither pure white or black. The tendency is that mature frameworks will evolve more into black-box through intense reuse. The following section describes our approach to applying Software Engineering techniques to solve the Database research challenge of configuring DBMSs. 3. The ECOHOOD Project The ECOHOOD project proposes the design and use of frameworks as a means of constructing configured DBMSs. The DBMS services, traditional or not, will be implemented by frameworks. Each framework is constituted of a static view, expressed by its classes, and a dynamic view, composed by messages intra and inter frameworks. The exchange of messages between frameworks occurs due to the necessity of communication between DBMS services. In order to deal with the complexity involved in constructing configured DBMSs, we propose an architecture consisting of: • the DBMS Framework that represents the architecture to be configured; • the Framework Instance that represents the configured DBMS; • the DBMS «Meta-framework» Schema that synthesize the framework description information; and • the Framework Instance Schema describing the composition of each instanced DBMS. The last two schemas constitute a repository of metadata describing the framework and the process of configuring of DBMSs. Different users will access this repository. The Database Implementor (DBI) is the user responsible for extending the existing framework modules or creating new ones whenever necessary. The Database Configurer (DBC) is the user who selects the appropriate classes from the framework to produce a configured DBMSs. Considering the amount of classes and messages he/she must deal with, his/her job will find great assistance in high-level information describing the framework abstractions through an appropriate GUI type of application. The Database Administrator (DBA) is the well-known user who keeps his regular responsibility. The following sub-sections describe the individual components of this architecture. 3.1 The DBMS «Meta-Framework» Schema The first component of the architecture is the «Meta-Framework» schema . Its responsibility is to describe the important framework concepts to be used by the DBI in the DBMS construction. It is a model for the framework. It represents the architecture of the framework by identifying its classes, messages and hot-spots. At this level, one can specify a single DBMS framework or link various frameworks producing another DBMS framework. As an example of the latter, a query processor framework can communicate with the storage manager framework composing a DBMS framework. Another important role of the «Meta-Framework» is to aid in the composition process by informing the DBI of composition rules, based on the framework architecture. These composition rules are represented as message calls showing the dependencies among classes. Figure 1 shows the UML class diagram [BP98], corresponding to the classes of this schema. Figure 1 - The "Meta-Framework" Schema The «Meta-Framework» schema presents the following concepts: • the class DBMS Framework expresses all the existent frameworks in an environment 1; • the class Composed DBMS framework represents an architecture for DBMS configuration. Various different architectures might exist promoting a more specific type of configuration. Each composed DBMS framework is made of 1 A set of frameworks available for user configuration is called an environment. DBMS service frameworks from which the DBC will construct a configured DBMS; • The single DBMS-Framework depicts a complete configurable architecture including all the DBMS services needed; • the class DBMS Services Framework describes the services to be utilized in a DBMS framework. New DBMS service frameworks can be created and joined in a DBMS configuration; • the Generalization class represents the superclasses. Most of the superclasses provide an opportunity for flexibility. • the sender/receiver association defines the glue between classes, defining the framework architecture; • The Component class represents the black-box, ready to use, components to be plugged into a framework hot-spot; • A hot-spot is a flexibility point. This is the aspect to be analyzed by the DBI in order to express adequate behavior and/or structure. 3.2 The DBMS Framework The DBMS Framework represents the configuration environment. This is the core level of the architecture where code and design are ready to be utilized. The use of the framework may happen either by filling the flexibility points with the desired behavior, or by plugging-in ready to use components. The development of the DBMS framework considers the following aspects: • adequacy of the framework - a DBMS framework may be developed aiming the refinement of DBMS services to attend different requirements, such as: specific application domains, distributed, heterogeneous or parallel DBMS. • Selection flexibility of the DBMS services frameworks - the DBI is free to choose which services the DBMS will require. The only mandatory DBMS service frameworks are the Query processor and the Storage Manager frameworks. These DBMS services form the basic structure of a DBMS. • flexibility of DBMS services - the framework developer identifies the hot spots within each DBMS service. The DBI may choose between transaction models, buffer manager swap policy, access methods, query optimization strategies and different rule coupling execution mode, etc. The level of flexibility is entirely user adaptable. It may be as large as the whole service and as small as a simple method override. A large grained flexibility could be a multimedia DBMS requiring a special new DBMS service, like the synchronism for the playback of continuous media, and an small grained flexibility example in a relational DBMS, could be the relational join method that can be chosen between the merge-join, nested loop or hash join algorithms. It is important to observe that framework is a semi-complete code made of objectoriented classes. Once the DBI has filled the hot-spots, what is obtained is a entirely functional DBMS. The next level is an instance of DBMS Framework, that is, a configured DBMS. 3.3 The DBMS Framework instance The product of a framework configuration is a functional configured DBMS. The configured DBMS provides the functionality selected from the framework, and possibly refined, by the DBI according to the requirements of the specific client application. This is the only level perceived by the Database Administrator (DBA) and application users. The configurations available at this level are those offered by the selected components, as blackbox modules, and are not different than the ones provided by commercial DBMSs. 3.4 The DBMS Framework instance schema This schema is responsible for the description of the generated configured DBMSs. A company may construct different DBMSs to attend requirements of specific departmental applications. The instance schema will store detailed information about each configured DBMS helping the DBA in the environment administration. It also provides useful information associating framework structures with configured DBMSs which use them. This information helps the framework maintenance, indicating DBMSs that might need to be reconstructed. 4. Example of DBMS Services and their Hot Spots A configured DBMS implements a selected set of interconnected specialized DBMS services. We provide a basic set of DBMS service [EN94, SK97] frameworks as a starting point for our environment. In Figure 2, we present the inter-connection between DBMS services framework according to [BP98]. Figure 2 – Inter-connected DBMS service frameworks The RuleManager framework implements active DBMS [WC96] behavior with flexibility points that permit configuring it with different execution coupling mode and special query extension treatment. The CatalogManager framework is designed to offer flexibility for defining new schema type elements. The QueryProcessor framework contains flexibility points that allows the definition of optimization strategies [YM98], complex and simple data type queries and also structured and semi-structured data access. The TransactionManager framework offers flexibility for the specification of adequate transaction models [GR93,ELMA92], different serialization enforcement protocols and special recovery policies. The StorageManager framework classes support different object type storage and special access methods and configurable units of data transfer between primary and secondary storage. Finally, the BufferManager permits a flexible configuration of paging flush strategy and paging size. In the following sub-sections we briefly describe the Rule Manager and Catalog frameworks. We also present the flexibility points (hot-spots) and discuss the configuration process involved. 4.1 Rule Manager Active DBMSs must provide an architecture for supporting the definition, storage and execution of rule objects. Our framework implements a meta-catalog for storing rules. It also includes a RuleManager class responsible for managing the storage, compiling and manipulating rule objects and a rule executor responsible for managing the execution of rules. In Figure 3, we show the Rule Manager Framework Figure 3 – The RuleManager Framework The RuleManager class is a flexibility point. Its is composed of a Query Re-writer object. The flexibility point retracts a possible implementation of rule execution where the user query is re-written including the associated rule text. The Query Re-writer class will provide the adequate implementation for the operation extendQuery. The RuleExecutor class has an abstract operation checkEvent. Depending on the coupling mode of rule execution, the DBC defines a proper specialization class that implements the adequate mode: immediate, deferred or decoupled. The abstract operation is another flexible point for the Rule Manager DBMS service. 4.2 Catalog The catalog is a DBMS service responsible for storing and providing access to schema information. Its framework is presented in Figure 4. The Catalog abstract class is a flexible point being optionally composed by different types of schemas. As an example of the catalog configuration, the DBC can compose a MetaSchema class and a RuleSchema class. The SchemaType class is also a flexible point that generalizes the possible types of schemas providing a generic interface for the catalog services. The Catalog abstract class is specialized with a concrete Catalog class with schema components chosen in a configuration. Figure 4 – The Catalog Framework 5. Conclusions Creating DBMSs to cope with the new functionality required by new types of applications is a challenge for the Database area. We believe that, instead of overloading a DBMS with several services that might not be used, the DBMS can be configured just with the absolutely necessary services to attend specific domain requirements. In this paper, we propose a new approach that offers great flexibility and produces a more adequate software. The configured DBMS is composed of software modules implementing the services needed to support new application data. We adopted Frameworks to produce a flexible architecture. DBMS services are implemented by an hierarchy of configured classes that together produce a software architecture. The implementation of software behavior is done by specifying new classes or composing existing modules in predefined flexible points. The main contributions of this work can be cited as: the proposed architecture that isolates the difficulties of developing a complex software as a DBMS; the use of schemas in a framework environment that usually just leave unorganized classes to be chosen and the modeling of database services with the identification of hot-spots as points of flexibility. In particular, we presented two examples of DBMS services framework, the Rule Manager framework and the Catalog framework. We also highlight the identified hot spots. As near future works, we intend to extend the idea of DBMS frameworks to configure new DBMSs with new services and functionalities. We also plan to develop several components to evolve our white-box framework into a black-box framework and construct a GUI tool to aid the Database Configurer (DBC) in the task of choosing components of the framework. References [BP98] [BT95] [B+90] [Catt94] [C+90] [Elma92] [EN94] [FS97] [GR93] [Melo87] [Pree95] [Pree96] [RJ97] [SKS97] [SR86] [SZ97] [Shor94] Blaha, M., and Premerlani, W. Object-Oriented Modeling and Design for Database Applications, Prentice-Hall, 1998. Batory, D.S., and Thomas, J. «P2: A Lightweight DBMS Generator», Department of Computer Sciences, University of Texas at Austin, Technical Report TR-9526, http://www.cs.utexas.edu, August 1995. Batory, D.S., Barnett, J.R., Garza, J.F., Smith, K.P., Tsukuda, K., Twichell, B.C., and Wise, T.E. «GENESIS: An Extensible Database Management System», in Maier, D., and Zdonik, S. (editors), Readings on Object-Oriented Database Systems, Morgan-Kaufmann, 1990. Cattell, R.G.G. «Object Data Management, Object-Oriented and Extended Relational Database Systems», Addison-Wesley, 1994. Carey, M.J., DeWitt, D.J., Graefe, G., Haight, D.M., Richardson, J.E., Schuh, D.T., Shekita, E.J., and Vandenberg, S.L. «The EXODUS Extensible DBMS Project: an Overview», in Maier, D., and Zdonik, S. (editors), Readings on Object-Oriented Database Systems, Morgan-Kaufmann, 1990. Elmagarmid, A.K., editor, Database Transaction Models For Advanced Applications, Morgan-Kaufmann, 1992. Elmasri, R., and Navathe, S.B. Fundamentals of Database Systems, 2nd Edition, Addison-Wesley, 1994. Fayad, M., and Schmidt, D. «Object-Oriented Application Frameworks», Communications of the ACM, vol. 40, n. 10, October 1997, pp.32-38. Gray, J., and Reuter, A. Transaction Processing: Concepts and Techniques, Morgan-Kaufmann, 1993. Melo, R.N. «EITIS: Environment of Integrated Tools for Interactive Systems», In Proceedings of the 1st Software Engineering Symposium , Petropolis, Rio de Janeiro, Brazil, 1987. Pree, W. Design Patterns for Object-Oriented Software Development, AddisonWesley/ACM Press, 1995. Pree, W. Software Patterns, SIGS Books & Multimedia, 1996. Roberts, D. and Johnson, R. Evolving Frameworks: A Pattern Language for Developing Frameworks, Addison-Wesley Reading, Massachusetts, 1997. Silberschatz, A., Korth, H.F., and Sudarshan,S. Database System Concepts, 3rd Edition, McGraw-Hill, 1997r. Stonebraker, M., and Rowe, L. «The Design of POSTGRES», Proceedings of the SIGMOD Conference, May 1986. Silberschatz, A., and Zdonik, S. «Database Systems – Breaking Out of the Box», SIGMOD Record, vol. 26, n. 3, September 1997, pp. 36-50. The Shore Project Group, «An Overview of Shore», Computer Sciences Department, http://www.cs.winsc.edu/shore/doc/overview, University of Wisconsin, 1994-97. [S+86] [USM94] [WC96] [YM98] Schwarz, P. et al, «Extensibility in the Starbust Database Systems», Proceedings of the International Workshop on Object-Oriented Database Systems, September 1986. Uchoa, E.M., Silva, S.D., and Melo, R.N. «HEROS Data Model – Object Oriented Heterogeneous Database System», In Proceedings of the 9th Brazilian Database Symposium, São Carlos, São Paulo, Brazil, 1994. Widom, J., and Ceri, S., editors, Active Database Systems - Triggers and Rules For Advanced Database Processing, Morgan-Kaufmann, 1996. Yu, C.T., and Meng, W. Principles of Database Query Processing for Advanced Applications, Morgan-Kaufmann, 1998.