Aggregation in disjunctive deductive databases

Aggregation in Disjunctive Deductive Databases Dietmar Seipel University of Wurzburg, Dept. of Computer Science Am Hubland, D { 97074 Wurzburg, Germany [email protected] Tel. +49 931 888 5026, Fax. +49 931 888 4600 Abstract 1 Introduction Practical applications of deductive databases often require the use of extended features like disjunctive information , aggregation operators or default negation . But it has been unclear how one could deal with aggregation in the presense of recursion and disjunction . Usually, there is the requirement that deductive databases must be strati ed w.r.t. aggregation , i.e. a predicate q that is de ned using aggregation based on some other predicate p may not be used for deriving this other predicate p. Disjunctive deductive databases extend the classical Datalog language by disjunction, default negation, and function symbols.1 There are practical examples that require non{ strati ed aggregation in disjunctive deductive databases. We will present a clear declarative set semantics for disjunctive deductive databases P with aggregation. The semantics is based on a special program transformation , which replaces an aggregation by a suitable construct using default negation and the function symbol for lists. The transformed database Pagg is free of aggregation, and it can be evaluated based on the well{known concept of stable models . Many practical applications of relational or deductive databases require the use of aggregation operators . For instance a study of data mining and decision support applications ([15, 9]) shows that aggregation is used heavily { often even user{de ned aggregates are required { and moreover aggregation is frequently used in forms that cannot be supported well or eciently by the current Sql{standard ([15]). In Sql3, aggregates and default negation can only be used in strati ed queries, and in relational or deductive databases so far there is no support for uncertain, disjunctive information. Typical and well{known forms of aggregation are the following: temporal aggregation in spatio{temporal databases ([8, 14]), aggregation in online{analytic processing (OLAP, [9]), computing a bill{of{materials, nding the shortest path in a graph, market analysis for transitive ownership and control of corporations for nding trusts ([1, 15]). In the DisLog{project ([11, 12]) a system for reasoning with disjunctive information in deductive databases has been developed. DisLog has been used for diagnostic reasoning , reasoning in spatial databases , or for solving con guration problems. So far, aggregation had not been considered in disjunctive deductive databases. Keywords disjunctive logic programming, aggregation, strati cation, non{monotonic reasoning, stable models 1 In this paper we will use the term disjunctive deductive database as an equivalent to disjunctive logic program. 1.1 An Example with Non{Strati ed Aggregation The following example of packing a knapsack requires disjunctive deductive databases with non{strati ed aggregation. There are similar con guration examples in other domains. Example 1.1 (Packing a Knapsack) The problem is to pack a knapsack with items from a relation item(Item; Weight ); such that the following conditions are satis ed: (i) the total weight of the knapsack does not exceed a limit given by weight limit (LimitW ); (ii) if the total weight `Total Weight ' of the knapsack exceeds a limit given by energy limit (LimitE ); then the knapsack must contain an `Energy Drink '; (iii) if `Total Weight ' does not exceed this limit, then the knapsack may not contain an `Energy Drink '; (iv) the knapsack must contain either an `Umbrella' or a `Trenchcoat', or both. The problem can be solved using the following disjunctive deductive database Pkn with aggregation. An atom knapsack(Item; Weight ) indicates that the knapsack contains the item `Item' with the corresponding weight `Weight '. The total weight of the knapsack is computed by the aggregation rule r0 . In Pkn disjunction is used in rule r4 for formulating the choice (iv) on the fact level. Furthermore, the integrity constraints given in (i) and (iii) are formulated as disjunctive rules r1 and r3 with zero head atoms. r0 = knapsack weight(< Weight >sum ) knapsack(Item; Weight ) r1 = weight limit(LimitW ) ^ knapsack weight(Total Weight ) ^ Total Weight > LimitW r2 = knapsack('Energy Drink'; Weight ) item('Energy Drink'; Weight ) ^ energy limit (LimitE ) ^ knapsack weight(Total Weight ) ^ Total Weight > LimitE r3 = knapsack('Energy Drink'; Weight ) ^ energy limit (LimitE ) ^ knapsack weight(Total Weight ) ^ Total Weight  LimitE r4 = knapsack('Umbrella'; Weight 1 ) _ knapsack('Trenchcoat'; Weight 2 ) item('Umbrella'; Weight 1 ) ^ item('Trenchcoat'; Weight 2 ) 1.2 The Transformation Approach In our new approach, a disjunctive deductive database P with aggregation will be evaluated by transforming it into another disjunctive deductive database Pagg without aggregation, which then can be evaluated using the well{ known semantics of stable models. Even though the database P can be disjunctive, aggregation in P will be treated in a definite manner based on the individual stable models of Pagg . This means that the aggregation condition will hold within each of the stable models (which can be considered as ordinary relational databases, possibly with function symbols). The aggregation operator agg can be dened by arbitrary (disjunctive) rules for a corresponding binary predicate agg , which are added to Pagg : a call `agg (Vs; V )' must compute the value V of the aggregation operator agg on the set corresponding to the list Vs. Thus, in our approach arbitrary user{de ned aggregation operators are possible. The transformation adds a binary function symbol for lists to P . If P did not contain any function symbols, then Pagg only contains this restricted use of the list function symbol, which is used for collecting the values to be aggregated. If the original database P was strati ed w.r.t. negation and aggregation, then Pagg is stratied w.r.t. negation, too. Then the semantics of perfect models can be used.2 2 Perfect models coincide with stable models on databases that are strati ed w.r.t. negation, but they are conceptually simpler and they can be computed more eciently. The rest of the paper is organized as follows: In Section 2 we present the syntax of disjunctive deductive databases with aggregation, and their predicate dependency graph. In Section 3 we recapitulate the well{known stable model semantics of disjunctive deductive databases (without aggregation). In Section 4 the declarative semantics of disjunctive deductive databases with aggregation is de ned based on a special program transformation. In Section 5 we give some concluding remarks. 2 Disjunctive Deductive Databases with Aggregation Given a rst order language , a disjunctive deductive database (cf. [5]) consists of logical inference rules of the form r = A1 : : : A k (1) L P _ _ B1 : : : Bm not C1 : : : not Cn ; 1; k , Bi , i 1; m , and ^ ^ ^ ^ ^ where Ai , i Ci , i 1; n , are atoms in the language , k; m; n IN0 , and not is the negation{ 2 2 h L h i 2 h i i 2 by{default operator.3 A rule is called a fact if m = n = 0. The set of all ground instances of the rules and facts in is denoted by gnd ( ). A rule (or database) is called positive{disjunctive if it does not contain default negation (i.e. n = 0). It is called normal if k = 1, and de nite if k = 1 and n = 0. A rule r of the form (1) is denoted for short as r= not ; where = A1 : : : Ak , = B1 : : : Bm , and = C1 : : : Cn .4 is called the head , is called the positive body , and not is called the negative body of r. P P ^ ^
_ ^ _ _ _
Aggregation A disjunctive rule r with aggregation is given by a rule of the form (1) above, such that the head atoms Ai = ai (Xi1; : : : ; Xin ) may have aggregation arguments Xij of the form Xij = < V >agg ; i 3 By IN+ we denote the set f 1; 2; 3; : : : g of positive natural numbers, whereas IN0 denotes the set 0; 1; 2; : : : of all natural numbers. n; m denotes the interval n; n + 1; : : : ; m of natural numbers. 4 Note that is a disjunction, and, according to De Morgan's law, not is taken to be a conjunction. f g h f g
i where `agg' can be one of the common aggregation operators `sum', `max', `min', `count', `avg' (average), `set' (cf. [8]), and `V ' is a variable symbol that also occurs in some positive body atom Bj . A disjunctive deductive database with aggregation is given by a set of disjunctive rules with aggregation. In our approach even user{de ned aggregation operators are possible, which can be de ned by arbitrary (disjunctive) rules. Note that we allow for aggregation arguments only in rule heads. This is not really a restriction, since an aggregation in the body of a rule can be simulated by introducing a new rule { with this aggregation in the head { de ning a new predicate, which is then used in the body of the original rule. For example, the following rule joins two relations p and a, groups the result by X , and then computes the sum of the di erent V {values r = q(X; < V >sum ) p(X; Y; V ) a(Y ): The following Sql{statement corresponds to the rule r: INSERT INTO q SELECT p:attr(1); sum(p:attr(3)) FROM p; a WHERE p:attr(2) = a:attr(1) GROUP BY p:attr(1) P ^ When we compare the expressive power of disjunctive rules with aggregation and of Sql{ statements with aggregation, then the question arises, whether we can also represent a HAVING{clause in the form of a logical inference rule. Roughly speaking, this can be done by rst de ning a new predicate without using the selection condition of the HAVING{clause and by then applying the selection condition to the resulting predicate. The Predicate Dependency Graph The predicate dependency graph GP = NP ; EP of a disjunctive deductive database with aggregation is given by a set NP = r2P Nr of nodes and a set EP = r2P Er of labeled edges, where each rule r results in a set of nodes and edges. For a rule r of the h i P [ [ 2 P form (1) above, let the predicate symbols in the atoms Ai = ai (X ), Bj = bj (Y ), Cl = cl (X ) be ai , bj , cl , respectively. Then Nr = f ai j i 2 h 1; k i g[ f bj j j 2 h 1; m i g [ f cl j l 2 h 1; n i g; + a ) j i; j 2 h 1; k i; i 6= j g[ Er = f (ai ?! j + f (ai ?! bj ) j i 2 h 1; k i; j 2 h 1; m i; Ai contains no aggregation g [ agg bj ) j i 2 h 1; k i; j 2 h 1; m i; f (ai ?! Ai contains an aggregation < V >agg g [ ? c ) j i 2 h 1; k i; j 2 h 1; n i g: f (ai ?! j A cycle for a predicate symbol p in GP is given by a sequence i j l l p )) Cp = ((pi ?! i+1 1ih i of labeled edges, such that p = p1 = ph+1. (i) P is called strati ed w.r.t. negation (or just strati ed ), if there exists no cycle Cp in GP , l p ) is such that one of the edges (pi ?! i+1 labeled by `li = ?'. (ii) P is called strati ed w.r.t. aggregation , if there exists no cycle Cp in GP , such that l p ) is labeled by one of the edges (pi ?! i+1 an aggregation operator `li = agg'. i i For example, both of the following disjunctive deductive databases are strati ed w.r.t. negation: (i) The disjunctive deductive database P = f r g consisting of the rule r = q(X; < V >sum ) p(X; Y; V ) ^ a(Y ) ^ not b(X; V ) is also strati ed w.r.t. aggregation, and its predicate dependency graph is given by: q (ii) The knapsack packing example Pkn of the introduction is non{strati ed w.r.t. aggregation, since there exits the following cycle through aggregation: sum knapsack ); (knapsack weight ?! + (knapsack ?! knapsack weight): 3 Stable Models for Disjunctive Deductive Databases The Herbrand base HBP of a disjunctive deductive database P contains all ground atoms over the language of P . An Herbrand interpretation of P is given by a subset I  HBP . It de nes a mapping I : HBP ! f t; f g that assigns a truth value \t" (true ) or \f" (false ) to each atom in HBP : I (A) = t , A 2 I: 5 I is an Herbrand model of a ground rule r = ^ not
if I ( ) ^ :I ( ) = t ) I ( ) = t: I is an Herbrand model of P if I is an Herbrand model of all ground instances r 2 gnd (P ) of the elements of P . I is a minimal model of P if I is an Herbrand model of P and there is no other Herbrand model J of P such that J ( I . The set of all minimal models of P is denoted by MM(P ). For positive{disjunctive deductive databases P , the semantics is given by MM(P ), and a formula  is considered to be a consequence of P under the minimal model semantics if  holds in all minimal models of P . The declarative semantics of disjunctive deductive databases in general is de ned based on a more general class of Herbrand models, the stable models , which are de ned in terms of the minimal models of the Gelfond{Lifschitz transformation (GL{transformation, [2, 6]) of P . The GL{transformation of P w.r.t. an Herbrand interpretation M is obtained from the ground instance gnd (P ) of P by replacing in every rule the negative body by its truth value:6 For Ai HBP , i 1; k , and a connective  we de ne I ( A1  : : : Ak ) = I (A1 ) : : : I (Ak ): For k = 0, the empty disjunction (i.e.  = ) evaluates to f, whereas the empty conjunction (i.e.  = ) evaluates to t. 6 It holds M (not ) = :M ( ): If this truth value is \t", then \t" can be deleted from the body. If it is \f", then the whole rule can be deleted from M . 5 ; 2 2 h i 2 f _ ^ g sum p sum ? a _ ? R ^ b
P (i) For r = rM = not
2 gnd (P ) we de ne ^ :M ( ): ^ (ii) The GL{transformation of M = rM r gnd ( ) : P P f j 2 P P w.r.t. M is M: get the following GL{transformation win P P M = win g with the truth values t and f as additional atoms,7 where rules with f in the body can be deleted from M . P M ). Obviously it holds M MM( win All of the atoms in I and the disjunction move (a; b) move (a; c) are consequences of win under the stable model semantics. 2 2 P (ii) The set of all stable models of is denoted by TAB( ). P P Stable models are also minimal models, i.e. TAB( ) MM( ); but the converse in general is not true. A formula  is considered to be a consequence of a disjunctive deductive database under the stable model semantics if  holds in all stable models of . S P  P 4 The Declarative Semantics for Aggregation P S move (a; b) _ move (a; c); move (b; d); move (c; d) g: P Let M be a Herbrand interpretation of a disjunctive deductive database . if M move (x; y) j _ De nition 3.1 (Stable Models, [2, 6]) P win (x) x 2 fa; b; c; dg; y 2 fa; dg g [ f M is a ground positive{disjunctive database (i) M is called stable model of MM( M ). f P P P Example 3.2 We will present a clear declarative semantics for disjunctive deductive databases with aggregation. The semantics is based on a special program transformation, which replaces all aggregations of by suitable constructs using default negation . The transformed database agg is free of aggregation, and it can be evaluated based on the concept of stable models. In a preprocessing step all rules r = not are normalized by replacing the complete rule body not by a single atom Br = body r (X1 ; : : : ; Xh), which has a new predicate symbol body r and has as arguments all variable symbols X1 ; : : : ; Xh that occur in not . Thus, we can replace r by the following two rules r1 and r2 : P P P ^
^
^ For the disjunctive deductive database win consisting of the following rules and facts P win (X ) move (X; Y ) ^ not win (Y ); move (a; b) _ move (a; c); move (b; d); move (c; d); we get a set with two stable models, which coincide on the subset I of atoms: TAB(Pwin ) = f f move (a; b) g [ I; f move (a; c) g [ I g; I = f move (b; d); move (c; d); win (b); win (c) g: S For M = f move (a; b) g [ I we get that M (not win (y)) = t i y 2 fa; dg, and thus we 7 Note that these truth values must evaluate to themselves under all Herbrand interpretations I of P M .
r1 = B r ^ not
; r2 = Br : Only the rule r2 can contain aggregation. It will be treated like described below, where we assume that the bodies of the rules with aggregation consist of exactly one atom. Transformation of De nite Rules Consider the following prototypical disjunctive rule r with aggregation, which computes the aggregation of V {values of a predicate `p' after grouping by X {values. If a V {value occurs with several (di erent) Y {values, then it contributes several times to the aggregation. r = q(X; < V >agg ) p(X; Y; V ) Note that `X ', `Y ' and `V ' could also be vectors of argument positions, and correspondingly `agg' could be a vector of aggregation functions for the components of `V '. The rule r is replaced by the following normal rules, which are free of aggregation:  ragg = pagg (X; [(Y; V )jYVs]; [V jVs]) pagg (X; YVs; Vs) ^ p(X; Y; V ) ^ not member((Y; V ); YVs)   eagg = pagg (X; [(Y; V )]; [V ]) p(X; Y; V ) ragg = pagg (X; V ) pagg (X; YVs; Vs) ^ not pext agg (X; YVs) ^ agg (Vs; V ) eagg = pext agg (X; YVs) p(X; Y; V ) ^ not member((Y; V ); YVs) 0 r = q(X; V ) pagg (X; V ) The transformed database is given by r ; Pagg = ( P n f r g ) [ Pagg where r = f r ; e ; ragg ; eagg ; r0 g: Pagg agg agg Instead of aggregation, Pagg contains rules with Prolog{style function symbols `[ ]' for lists. We also need the usual de nition of the predicate `member'. The predicate `pagg ' serves for constructing lists `YVs', which consist of pairs `(Y; V )' such that `p(X; Y; V )' holds, and for constructing the corresponding lists `Vs' of the values `V '. The call `pext agg (X; YVs)' in ragg checks whether the list `YVs' can be extended by other pairs `(Y; V )' that are not yet contained in it. If it cannot be extended, then the call `agg (Vs; V )' applies the aggregation function `agg' to the list `Vs' and produces the nal result `V '.8 Obviously, in our approach the aggregation function `agg' can be de ned by arbitrary (dis- junctive) rules for agg . We only have to require that the same result pagg (X; V ) is derived by the call agg (Vs; V ) in ragg from pagg (X; YVs; Vs) for all permutations YVs . In an implementation of the list approach the ineciency problem arising from permutations can be avoided by simply using ordered lists or ordered sets , depending on whether we are preferring a bag semantics or a set semantics. In the declarative description of our set semantics, however, we have not addressed this problem. r The predicate dependency graph GPagg is given in Figure 1, where labels \+" are omitsum p) due to r in ted. Instead of the edge (q ?! GP , the graph GPagg contains a path with a negative edge: + p): ? pext ); (pext ?! + p ); (p ?! (q ?! agg agg agg agg I.e., the aggregation has been transformed into a negation. If the original database P was strati ed w.r.t. negation and aggregation , then Pagg is strati ed w.r.t. negation , too. 8 If `agg ' is based on an associative binary operator, which holds for `sum', `max', `min', `count', then the aggregation could of course be evaluated incrementally without rst constructing the whole list `Vs'. with k1 pairwise di erent, fresh variable symbols Vi0 . Note that the atom with the predicate symbol p is necessary in the body of r0 , Transformation of Disjunctive Rules Disjunctive rules r with more than one head atom, some of which contain aggregation, can also be treated by this replacement technique: r = q1 (X1 ; < V1 >agg 1 ) _ : : : _ qk1 (Xk1 ; < Vk1 >agg k1 ) _ A1 _ : : : _ Ak2 p(X1 ; : : : ; Xk1 ; Y; V1 ; : : : ; Vk1 ) All the de nite rules ri = qi (Xi ; < Vi >agg i ) p(X1 ; : : : ; Xk1 ; Y; V1 ; : : : ; Vk1 ) are transformed as described above into rules  , eagg , raggi , eaggi , which de ne predicates ragg i i pagg i (Xi ; Vi ). Then the new rule is r0 = q1 (X1 ; V10 ) _ : : : _ qk1 (Xk1 ; Vk01 ) _ A1 _ : : : _ Ak2 pagg 1 (X1 ; V10 ) ^ : : : ^ pagg k1 (Xk1 ; Vk01 ) ^ p(X1 ; : : : ; Xk1 ; Y; V1 ; : : : ; Vk1 ); q ? pagg ? * ......... ................ ............ ....... ................ .......... .......... pagg R ext pagg R ? p q agg ? U member r Figure 1: The Predicate Dependency Graph of Pagg since it can share arguments with the atoms in A1 _ : : : _ Ak . 2 The Stable Model Semantics with Aggregation The declarative semantics of a disjunctive deductive database P with aggregation is de ned based on its transformed database Pagg using the concept of stable models: STABagg (P ) = f I \ HBP j I 2 STAB(Pagg ) g: For each stable model of Pagg only the regular atoms A 2 I \ HBP are considered; all arti cial atoms with predicate symbols that were introduced during the program transformation (i.e. pagg , pagg , pext agg , member) are thrown away. The Knapsack Example The aggregation rule r = q(X; < V >agg ) p(X; Y; V ) of Pkn is de nite. It is r0 = knapsack weight(< Weight >sum ) knapsack(Item; Weight ): The predicate on which we have to aggregate is `p = knapsack', the resulting predicate is `q = knapsack weight', the argument vector `X ' is empty, `Y = Item', and `V = Weight '. Pkn is non{strati ed w.r.t. aggregation, since `knapsack weight' is used for de ning `knapsack'. Thus, (Pkn )agg is non{strati ed w.r.t. negation, and it must be evaluated by stable model semantics (rather than perfect model semantics, which would be simpler). Given the basic facts in I = f weight limit(10); energy limit(6); item('Umbrella'; 2); item('Trenchcoat'; 1); item('Energy Drink'; 2); item('Mineral Water'; 3); item('Bread'; 2) g; and the rule r5 = knapsack(Item; Weight ) _ leave at home(Item; Weight ) item(Item; Weight ) ^ Item 6= 'Umbrella' ^ Item 6= 'Trenchcoat' ^ Item 6= 'Energy Drink'; in principle all items t into the knapsack. There exist 10 allowed packings, i.e. stable models of (Pkn )agg , out of 32 possible combinations of the 5 items.9 5 Conclusions Currently a number of prototypical deductive database systems are available, e.g. Ldl++, CORAL, Aditi, Glue{NAIL! ([13, 8]), which basically allow for strati ed aggregation. E.g. Ldl++ supports aggregates with choice in XY{ strati ed programs. Our approach extends deductive database technology by allowing for disjunction and aggregation; even user{de ned aggregation operators are possible. It is not necessary that the aggregation operator agg is associative or that it allows for an incremental evaluation, which is necessary for the approach of [15] for deductive databases with aggregation. We are currently implementing the new approach in DisLog using an ordered sets representation for the values that have to be aggregated. In the special case of Datalog with associative, strati ed aggregation, our semantics coincides with the traditional semantics that is used by current deductive database systems. References [1] S. Ceri, G. Gottlob, L. Tanca: Logic Programming and Databases, Springer, 1990. [2] M. Gelfond, V. Lifschitz: The Stable Model Semantics for Logic Programming, Proc. Fifth Intl. Conference and Symposium on Logic Programming 1988 (ICSLP'88), MIT Press, 1988, pp. 1070{1080. [3] F. Giannotti, D. Pedreschi, C. Zaniolo: Semantics and Expressive Power of Non{ Deterministic Constructs in Deductive Databases, to appear in JCCS. 9 All allowed knapsacks contain exactly one of the items 'Umbrella' and 'Trenchcoat'. There are 4 packings where the knapsack does not contain 'Mineral Water' (in this case it cannot contain an 'Energy Drink' and 'Bread' is optional) and 6 packings where it contains 'Mineral Water' (in this case an 'Energy Drink' is contained in three packings). [4] J.W. Lloyd: Foundations of Logic Programming, second edition, Springer, 1987. [5] J. Lobo, J. Minker, A. Rajasekar: Foundations of Disjunctive Logic Programming, MIT Press, 1992. [6] T.C. Przymusinski: Stable Semantics for Disjunctive Programs, New Generation Computing, vol. 9, 1991, pp. 401{424. [7] W.G. Roth, R. Ramakrishnan: MIMSY: A System for Analyzing Time Series in the Stock Market Domain, Proc. Workshop on Applications of Deductive Systems at the Intl. Symposium on Logic Programming 1993 (ILPS'93). [8] R. Ramakrishnan, D. Srivastava, S. Sudarshan: CORAL { Control, Relations and Logic, Proc. Intl. Conference on Very Large Databases 1992 (VLDB'92). [9] R. Ramakrishnan: Database Management Systems, McGraw{Hill, 1998. [10] D. Seipel, J. Minker, C. Ruiz: Model Generation and State Generation for Disjunctive Logic Programs, Journal of Logic Programming, vol. 32(1), 1997, pp. 48{69. [11] D. Seipel, H. Thone: DisLog { A System for Reasoning in Disjunctive Deductive Databases, Proc. Intl. Workshop on the Deductive Approach to Information Systems and Databases 1994 (DAISD'94), pp. 325{343. [12] D. Seipel: DisLog { A Disjunctive Deductive Database Prototype, Proc. Twelfth Workshop on Logic Programming (WLP'97), 1997, pp. 136{143. DisLog is available on the WWW at "http://www-info1.informatik.uniwuerzburg.de/databases/DisLog". [13] C. Zaniolo, N. Arni, K. Ong: Negation and Aggregates in Recursive Rules: the Ldl++ Approach, Proc. Intl. Conf. on Deductive and Object{Oriented Database systems 1993 (DOOD'93), Springer, 1993, pp. 204{221. [14] C. Zaniolo, S. Ceri, C. Faloutsos, R.T. Snodgrass, V.S. Subrahmanian, R. Zicari: Advanced Database Systems, Morgan Kaufmann, 1997. [15] C. Zaniolo, H. Wang: Logic{Based User{ De ned Aggregates for the Next Generation of Database Systems, in: The Logic Programming Paradigm: Current Trends and Future Directions, K.R. Apt, V.W. Marek, M. Truszczynski, D.S. Warren (eds.), Springer, 1999.