Syntactic Theories in Practice

BRICS Basic Research in Computer Science BRICS RS-02-4 Danvy & Nielsen: Syntactic Theories in Practice Syntactic Theories in Practice Olivier Danvy Lasse R. Nielsen BRICS Report Series ISSN 0909-0878 RS-02-4 January 2002 Copyright c 2002, Olivier Danvy & Lasse R. Nielsen. BRICS, Department of Computer Science University of Aarhus. All rights reserved. Reproduction of all or part of this work is permitted for educational or research use on condition that this copyright notice is included in any copy. See back inner page for a list of recent BRICS Report Series publications. Copies may be obtained by contacting: BRICS Department of Computer Science University of Aarhus Ny Munkegade, building 540 DK–8000 Aarhus C Denmark Telephone: +45 8942 3360 Telefax: +45 8942 3255 Internet: [email protected] BRICS publications are in general accessible through the World Wide Web and anonymous FTP through these URLs: http://www.brics.dk ftp://ftp.brics.dk This document in subdirectory RS/02/4/ Syntactic Theories in Practice ∗ Olivier Danvy and Lasse R. Nielsen BRICS † Department of Computer Science University of Aarhus ‡ January 31, 2002 Abstract The evaluation function of a syntactic theory is canonically defined as the transitive closure of (1) decomposing a program into an evaluation context and a redex, (2) contracting this redex, and (3) plugging the contractum in the context. Directly implementing this evaluation function therefore yields an interpreter with a worst-case overhead, for each step, that is linear in the size of the input program. We present sufficient conditions over a syntactic theory to circumvent this overhead, by replacing the composition of (3) plugging and (1) decomposing by a single “refocusing” function mapping a contractum and a context into a new context and a new redex, if there are any. We also show how to construct such a refocusing function, we prove its correctness, and we analyze its complexity. We illustrate refocusing with two examples: a programming-language interpreter and a transformation into continuation-passing style. As a byproduct, the time complexity of this program transformation is mechanically changed from quadratic in the worst case to linear. Keywords: syntactic theories, interpreters, refocusing. ∗A preliminary version of this work was presented at RULE 2001 [4, 10]. Research in Computer Science (www.brics.dk), funded by the Danish National Research Foundation. ‡ Ny Munkegade, Building 540, DK-8000 Aarhus C, Denmark E-mail: {danvy,lrn}@brics.dk † Basic 1 Contents 1 Introduction 1.1 Related work . . . . . . . . . . . . . . 1.2 Interpreters for syntactic theories . . . 1.3 A canonical example: the call-by-value 1.4 This work . . . . . . . . . . . . . . . . . . . . . . . . . . . . λ-calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 3 3 4 2 The call-by-value λ-calculus 2.1 An interpreter . . . . . . . . . . . . 2.1.1 The original specification . 2.1.2 The refocused specification 2.2 A CPS transformer . . . . . . . . . 2.2.1 The original specification . 2.2.2 The refocused specification 2.3 Refocusing efficiently . . . . . . . . 2.3.1 The interpreter . . . . . . . 2.3.2 The CPS transformer . . . 2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 5 5 6 6 6 7 8 9 9 9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Refocusing in a syntactic theory 3.1 The language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Syntactic theories . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Evaluation contexts . . . . . . . . . . . . . . . . . . . . . 3.2.3 Decompositions and potential redexes . . . . . . . . . . . 3.2.4 Unique decomposition . . . . . . . . . . . . . . . . . . . . 3.2.5 Reduction rules . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 No redundant constructs . . . . . . . . . . . . . . . . . . . 3.3.2 Distinct value constructors and potential-redex constructors 3.3.3 Left-to-right evaluation of sub-terms . . . . . . . . . . . . 3.4 Consequences of the requirements . . . . . . . . . . . . . . . . . . 3.5 Constructing a refocus function . . . . . . . . . . . . . . . . . . . 3.6 Correctness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 10 10 10 10 11 11 12 12 12 13 13 13 15 17 19 19 4 Conclusion and issues 20 A Arithmetic expressions with precedence A.1 A syntactic theory . . . . . . . . . . . . . . A.2 Evaluation contexts with right-composition A.3 Implementation . . . . . . . . . . . . . . . . A.4 Refocusing . . . . . . . . . . . . . . . . . . . A.5 Summary . . . . . . . . . . . . . . . . . . . 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 20 22 23 27 33 1 Introduction A syntactic theory provides a uniform, concise, and elegant framework for specifying the semantics of a programming language and reasoning about programs [5, 6, 7]. We consider the issue of implementing the evaluation function of a syntactic theory in the form of an interpreter. Our emphasis, however, is not on automating this process, as in Xiao and Ariola’s SL project [17, 18].1 Instead, we identify that the direct implementation of an evaluation function entails an overhead and we show how to circumvent it. 1.1 Related work Formal semantics has always offered a tempting format for implementing specifications or even for designing programming languages—witness denotational semantics and functional programming [13], Moggi’s computational monads and monad-based functional programming [15], operational semantics and logic programming [9], etc. Formal semantics can also provide cost models for computations [8]. The present article describes how to bypass the cost of consecutive plug-and-decompose operations when implementing an interpreter for a syntactic theory, an issue that is usually dealt with on a case-by-case basis to construct abstract machines [6, Chapter 7]. 1.2 Interpreters for syntactic theories A syntactic theory is a small-step operational semantics where evaluation is defined as the transitive closure of single reductions, each performed by (1) decomposing a program into a context and a potential redex, (2) contracting the redex, if possible,2 and (3) plugging the contractum in the context. Most syntactic theories are developed for deterministic programming languages and satisfy a unique-decomposition property. The interpreter for a syntactic theory implements its evaluation function. It naturally consists of a decompose-contract-plug loop. Decomposition is usually implemented with a depth-first search in the abstract syntax tree. The decompose step therefore introduces a significant overhead, proportional to the program size. Likewise, plugging takes time linear in the size of the context and it always takes at most as long as the following decomposition, if there is one. 1.3 A canonical example: the call-by-value λ-calculus Here is a syntactic theory of the call-by-value λ-calculus: e ::= x | λx.e | e e v ::= x | λx.e e∈Λ v ∈ Value ⊆ Λ x ∈ Var 1 http://www.cs.uoregon.edu/~ariola/SL/ 2 Some potential redexes are not actual ones—they are stuck terms [11]. 3 E ::= [ ] | E[[ ] e] | E[v [ ]] r ::= v v E ∈ EvCont r ∈ PotRedex ⊆ Λ Among potential redexes, only β-redexes are actual redexes. Therefore, the only reduction rule is the following one: E[(λx.e) v] → E[e[v/x]] Plugging the hole of an evaluation context with an expression is defined by induction on the evaluation context, and thus it operates in time proportional to the depth of the context: ([ ])[e] (E[[ ] e′ ])[e] (E[v [ ]])[e] = e = E[e e′ ] = E[v e] Likewise, decomposition operates in time proportional, in the worst case, to the size of the term to decompose. Let us illustrate this overhead with Church numerals: the Church numeral for the number n is λs.λz. s(s(s(. . . (s z) . . .))) and is noted ⌈n⌉. | {z } n Example 1 We consider the term ⌈n⌉ (λx.x) v where v is any value. This term reduces in two steps to (λx.x)((λx.x)((λx.x)(. . . ((λx.x) v) . . .))). From then on, {z } | n each decomposition into a context and a redex (where the redex is always (λx.x)v) takes time proportional to the number of remaining applications. The total time taken by decomposition during evaluation is thus at least proportional to n2 .  1.4 This work We state conditions under which consecutive plug-and-decompose operations can be replaced by a more efficient refocus operation. These conditions pertain to the order in which a redex occurs, in the depth-first traversal of the decompose operation. We show that these conditions hold if the syntactic theory is given in the “standard” way, i.e., by a context-free grammar of values and evaluation contexts, and if it satisfies a unique-decomposition property. We also show how to mechanically construct such a refocus function. The rest of this article is structured as follows. We first illustrate refocusing with two concrete examples: a syntactic theory for the call-by-value λ-calculus and a transformation of call-by-value λ-terms into continuation-passing style due to Sabry and Felleisen [12]. We then list sufficient conditions to refocus a syntactic theory. The proof that these conditions are sufficient is constructive. It indicates how to mechanically rephrase the syntactic theory to circumvent the overhead. These conditions are general enough to make refocusing practically useful—for example, they are fulfilled by all the examples documented in the SL project [17, 18]. 4 2 The call-by-value λ-calculus Let us go back to the call-by-value λ-calculus, initially considered in Section 1.3. First, we uniformly restate the grammars of the language and of the syntactic theory in the format used in the rest of this article. These grammars can be expressed directly as ML data types and the corresponding functions as ML functions. e ::= var(x) | lam(x, e) | app(e, e) v ::= var(x) | lam(x, e) E ::= [ ] | E[app([ ], e)] | E[app(v, [ ])] r ::= app(v, v) e v x E r ∈ ∈ ∈ ∈ ∈ Exp Val Var EvCont PotRedex Var is the syntactic category of variables. We make decomposition and plugging explicit with two functions: decompose : Exp → Val + (EvCont × PotRedex) plug : Exp × EvCont → Exp We define the refocusing function refocus as the composition of decompose and plug. Its type is thus as follows. refocus : Exp × EvCont → Val + (EvCont × PotRedex) We now consider an interpreter and then a CPS transformer for the call-byvalue λ-calculus (Sections 2.1 and 2.2). Each uses decompose and plug, which we fuse into refocus (Section 2.3). 2.1 2.1.1 An interpreter The original specification The following interpreter implements the evaluation function of the syntactic theory. It takes one expression as argument and repeatedly decomposes, contracts, and plugs until either a value or a stuck term is reached, if any. eval : Exp → Val + (EvCont × PotRedex) eval (e) = eval ′ (decompose(e)) eval ′ : Val + (EvCont × PotRedex) eval ′ (v) eval ′ (E, app(lam(x, e), v)) eval ′ (E, app(x, v)) → = = = Val + (EvCont × PotRedex) v eval (plug(e[v/x], E)) (E, app(x, v)) The evaluation of an expression e is eval (e). (NB: The last rule of eval ′ is used for stuck terms.) The interpreter contains two calls to eval : one in the second rule of eval ′ and one in the initial call. In the second rule of eval ′ , the argument of eval is the 5 result of plug. In some sense, so is it too in the initial call, since e = plug(e, [ ]). In that sense, decompose (in the definition of eval ) is always called with the result of plug. Let us re-express the interpreter with a refocusing function that combines the effects of decompose and plug. 2.1.2 The refocused specification We change the interpreter to use refocus instead of decompose and plug. eval : Exp → Val + (EvCont × PotRedex) eval (e) = eval ′ (refocus(e, [ ])) eval ′ : Val + (EvCont × PotRedex) eval ′ (v) eval ′ (E, app(lam(x, e), v)) eval ′ (E, app(x, v)) → = = = Val + (EvCont × PotRedex) v eval ′ (refocus(e[v/x], E)) (E, app(x, v)) The evaluation of an expression e is eval (e). We are now free to use any implementation of refocus that is extensionally equivalent to decompose ◦ plug. 2.2 A CPS transformer In their work on reasoning about programs in continuation-passing style (CPS), Sabry and Felleisen designed a new CPS transformation [12, Definition 5]. This CPS transformation integrates a notion of generalized reduction and thus yields very compact CPS programs [3]. It is also unusual in the sense that it builds on the notion of a syntactic theory, rather than on operational semantics [2, 11] or denotational semantics [16]. Therefore, it is defined as the transitive closure of decomposing, performing an elementary CPS transformation, and plugging, which makes it a candidate for refocusing. 2.2.1 The original specification Definition 1 (Sabry and Felleisen, 1993) The following CPS transformation uses three mutually recursive functions: Ck (and the auxiliary function Ck′ ) to transform expressions, Φ to transform values, and Kk to transform evaluation contexts. The functions Ck , Ck′ , and Kk are parameterized over a variable k that represents the current continuation. Ck : Exp → Exp Ck (e) = Ck′ (decompose(e)) Ck′ : Val + (EvCont × PotRedex) Ck′ (v) Ck (E, app(x, v)) Ck (E, app(lam(x, e), v)) 6 → = = = Exp app(k, Φ(v)) app(app(x, Kk (E)), Φ(v)) app(lam(x, Ck (plug (e, E))), Φ(v)) Φ : Val → Exp Φ(x) = x Φ(lam(x, e)) = lam(k, lam(x, Ck (e))) where k is fresh Kk : EvCont Kk ([ ]) Kk (E[app(x, [ ])]) Kk (E[app(lam(x, e), [ ])]) Kk (E[app([ ], e)]) → = = = = Exp k app(x, Kk (E)) lam(x, Ck (plug(e, E))) lam(ui , Ck (plug(app(ui , e), E))) where ui is fresh The CPS counterpart of an expression e is lam(k, Ck (e)), where k is fresh.  The CPS transformer contains five calls to Ck . In three cases, the argument of Ck is the result of plug, and in two cases, the argument is e. As in Section 2.1.1, and since e = plug(e, [ ]), we can say that decompose (in the definition of Ck ) is always called with the result of plug. Let us re-express the CPS transformer with a refocusing function that combines the effects of decompose and plug. 2.2.2 The refocused specification We change the CPS transformer to use refocus instead of decompose and plug. Ck : Exp → Exp Ck (e) = Ck′ (refocus(e, [ ])) Ck′ : Val + (EvCont × PotRedex) Ck′ (v) Ck′ (E, app(x, v)) ′ Ck (E, app(lam(x, e), v)) → = = = Exp app(k, Φ(v)) app(app(x, Kk (E)), Φ(v)) app(lam(x, Ck′ (refocus(e, E))), Φ(v)) Φ : Val → Exp Φ(x) = x Φ(lam(x, M )) = lam(k, lam(x, Ck (e))) where k is fresh Kk : EvCont Kk ([ ]) Kk (E[app(x, [ ])]) Kk (E[app(lam(x, e), [ ])]) Kk (E[app([ ], e)]) → = = = = Exp k app(x, Kk (E)) lam(x, Ck′ (refocus(e, E))) lam(ui , Ck′ (refocus(app(ui , e), E))) where ui is fresh The CPS transformation of an expression e is lam(k, Ck (e)), where k is fresh. 7 We are now free to use any implementation of refocus that is extensionally equivalent to decompose ◦ plug. 2.3 Refocusing efficiently Let us fuse plug and decompose into one function. First, here is their definition. plug(e, [ ]) = e plug(e, E[app([ ], e2 )]) = plug(app(e, e2 ), E) plug(e, E[app(v1 , [ ])]) = plug(app(v1 , e), E) decompose(e) = decompose ′ (e, [ ]) decompose ′ : Exp × EvCont → Val + (EvCont × PotRedex) decompose ′ (v, E) = decompose ′aux (E, v) decompose ′ (app(e1 , e2 ), E) = decompose ′ (e1 , E[app([ ], e2 )]) decompose ′aux : EvCont × Val decompose ′aux ([ ], v) decompose ′aux (E[app([ ], e2 )], v) decompose ′aux (E[app(v1 , [ ])], v) → = = = Val + (EvCont × PotRedex) v decompose ′ (e2 , E[app(v, [ ])]) (E, app(v1 , v)) The definition of plug is a transliteration of the one in Section 1.3. As for decompose, it implements a depth-first search for the first application of one value to another, and uses two auxiliary functions: one, decompose ′ , recursively descends into an expression and accumulates the corresponding context, and the other, decompose ′aux , dispatches on the accumulated context. Let us now construct a more efficient version of refocus = decompose ◦ plug. Our key observation is a consequence of the unique-decomposition property of a syntactic theory of the λ-calculus and of the fact that the grammar of evaluation contexts is context-free: decompose(plug(e, E)) = decompose ′ (e, E) In words: E uniquely determines the partial decomposition of the result of plug(e, E) into the expression e and the evaluation context E. (In general, this decomposition is partial because e is not necessarily a potential redex.) A similar consequence holds for values: decompose(plug(v, E)) = decompose ′aux (E, v) A simple fold-unfold calculation3 lets us calculate the following version of refocus. 3 For example, refocus (app(e1 , e2 ), E) = = = = = decompose (plug (app(e1 , e2 ), E)) decompose ′ (app(e1 , e2 ), E) decompose ′ (e1 , E[app([ ], e2 )]) decompose (plug (e1 , E[app([ ], e2 )])) refocus (e1 , E[app([ ], e2 )]) 8 refocus : Exp × EvCont → Val + (EvCont × PotRedex) refocus(v, E) = refocus aux (E, v) refocus(app(e1 , e2 ), E) = refocus(e1 , E[app([ ], e2 )]) refocus aux : EvCont × Val refocus aux ([ ], v) refocus aux (E[app([ ], e2 )], v) refocus aux (E[app(v1 , [ ])], v) → = = = Val + (EvCont × PotRedex) v refocus(e2 , E[app(v, [ ])]) (E, app(v1 , v)) The resulting function refocus is extensionally equivalent to decompose ◦ plug because fold-unfold transformations are correct [1]. It is also more efficient because it avoids building an intermediate expression for the sole purpose of decomposing it straight away.4 2.3.1 The interpreter The refocused interpreter produces the same result as the original one, but it operates more efficiently. For example (cf. Example 1 in Section 1.3), when evaluating a term such as ⌈n⌉ (λx.x) v, the time taken to refocus from a contractum and a context to a new context and a redex is independent of the number of remaining applications. 2.3.2 The CPS transformer The refocused CPS transformer produces the same compact terms as the original one, but it operates in linear time. More precisely, it operates in one pass over its input.5 2.4 Summary We have described how to refocus a syntactic theory for the call-by-value λ-calculus, by deforesting the intermediate expressions produced in each decomposecontract-plug cycle. As a byproduct, we have shown how refocusing makes it possible to derive a linear-time program transformation out of a quadratic-time one. 4 Replacing the composition of two functions that produce and consume an intermediate data structure is an instance of deforestation [14]. 5 Actually, the transformation could have been optimized to read as follows: Kk (E[app([ ], e)]) = lam(ui , Ck (refocus (e, E[app(ui , [ ])]))) This way, each ui would not be inspected as an expression. 9 3 Refocusing in a syntactic theory In this section we show how to construct a refocus function for a syntactic theory, as an alternative to plugging and decomposing. We prove that this function computes the correct result and we give an exact measure of its computational complexity. Finally we show that refocusing is always at least as efficient as plugging and then decomposing. 3.1 The language A programming language is defined by its syntax and it semantics. Here, its semantics is specified by a syntactic theory. Without loss of generality we assume that its abstract syntax is specified by a context-free grammar (or BNF) with only productions of the form e ::= c(e1 , . . . , en ) where c ranges over a set of constructor names and e, e1 , . . . , en are nonterminals corresponding to the syntactic categories of the language. Each constructor may occur only once in a grammar. 3.2 Syntactic theories A syntactic theory is a specification of a small-steps operational semantics, i.e., it specifies a reduction relation between programs. Syntactic theories traditionally come with context-free grammars for values and evaluation contexts, along with reduction rules of the form E[r] → e, where the term r is called a redex (short for reducible expression). Let us review each of these concepts in turn. 3.2.1 Values The grammar of values extends the BNF of the language with new non-terminals, v’s, one for each non-terminal e. The set of values ranged over by a v is included in the set of terms ranged over by the matching e. The productions for values are all of the form v ::= c(x1 , . . . , xn ) where e ::= c(e1 , . . . , en ) is a production of the language grammar and each xi is either ei or vi . If a syntactic category of the language contains only values, then it is irrelevant whether we use vi or ei in place of xi . From here on we will consistently use a ei for such a syntactic category. 3.2.2 Evaluation contexts Evaluation contexts are also specified by context-free grammars, and their meaning is given by the associated plug function. For example, the evaluation contexts of the call-by-value λ-calculus and the associated plug function are shown in Section 2. 10 All the evaluation contexts of a syntactic theory can be written as the composition of elementary evaluation contexts. Composition is a binary function on evaluation contexts, ◦, that together with the plug function satisfies (E1 ◦ E2 )[e] = E1 [E2 [e]], and is thus associative. Elementary contexts can themselves not be written as compositions of other non-empty contexts.6 For the call-by-value λ-calculus, the elementary contexts are app([ ], e) and app(v, [ ]). The construction of evaluation contexts corresponds to composing an elementary context to the right of another context, e.g., E[app([ ], e)] = E ◦ app([ ], e). In many articles on syntactic theories, though, compound contexts are constructed by composing an elementary context on the left rather than on the right. For the λ-calculus, the grammar of evaluation contexts then reads as follows. E ::= [ ] | app(E, e) | app(v, E) This notation induces the same elementary contexts. 3.2.3 Decompositions and potential redexes If a term, e, is the result of plugging term e ′ into an evaluation context E, i.e., E[e ′ ] = e, then we say that E and e form a decomposition (into an evaluation context and a term) of e ′ , or alternatively that e ′ can be decomposed into E and e. A term can be decomposed in many different ways. If e = E[e ′ ] then the decomposition is trivial if either E is the empty context or e ′ is a value. Values are normal forms: they cannot have a non-value sub-term in a position where it can be evaluated, so all decompositions must be trivial.7 Non-value terms that can only be decomposed trivially are called potential redexes. For the λ-calculus, the potential redexes are exactly the terms of the form app(v, v). If e = E[e ′ ] then the decomposition is complete if e ′ is a potential redex. The reduction rules of a syntactic theory contain only complete decompositions. If a term has a decomposition, E[e], that is not complete, then e can be further decomposed in a non-trivial way as E ′ [e ′ ]. Then (E ◦ E ′ )[e ′ ] is again a non-trivial decomposition of the original term. The context of a complete decomposition is maximal in the sense that further decompositions would be trivial. 3.2.4 Unique decomposition Unique decomposition is a property of syntactic theories that guarantees the existence and uniqueness of complete decompositions for arbitrary terms. Definition 2 (Unique decomposition) A syntactic theory is said to satisfy the unique-decomposition property if any non-value term e can be uniquely decomposed as e = E[r] where r is a potential redex. 6 We note that evaluation contexts, together with composition and the empty context, form a monoid. 7 This does not preclude, e.g., weak head normal forms or lazy pairs. 11 Unique decomposition is so fundamental to syntactic theories for deterministic languages that it is almost always the first property to be established. Its proof is often technically simple, but because of its many small cases, it tends to be tedious and error-prone. This state of affairs motivated Xiao, Sabry, and Ariola to develop an automated support for proving unique-decomposition properties [18].8 3.2.5 Reduction rules The reduction rules of a syntactic theory are of the form E[r] → e where r is a potential redex and e depends only on E and on the sub-terms of r. Often e is written as E[e ′ ] for some e ′ depending on r. The potential redexes that occur in a reduction rule are the actual redexes of the syntactic theory. The remaining potential redexes stick, to use Plotkin’s word [11]. Stuck terms encompass type-incorrect terms (e.g., the application of a literal) and undefined terms (e.g., the application of an identifier). The only reduction rule of the call-by-value λ-calculus is E[app(lam(x, e), x)] → e [v/x], and thus app(lam(x, e), x) is a redex. The remaining potential redexes (of the form app(x, v)) are stuck terms. 3.3 Requirements We state three requirements that are sufficient for constructing a refocus function. We consider syntactic theories whose structure is as described in Section 3.2 and that satisfy unique decomposition. 3.3.1 No redundant constructs Syntactic theories specify a reduction relation. Any part of a specification that does not affect the specified relation is redundant and can be ignored. Specifically: • If the language has a grammar production e ::= c(e1 , . . . , en ) and the syntactic category ranged over by ei contains only values, then an elementary evaluation context of the form c(x1 , . . . , xi−1 , [ ], xi+1 , . . . , xn ) is redundant. Since only values can be plugged in the hole, and values can only be trivially decomposed, then no evaluation context constructed by composing this elementary context can ever be part of a complete decomposition. • If a syntactic category, ranged over by e, contains no values, i.e., the corresponding values ranged over by v is the empty set, then any occurrence of v in a rule makes this rule redundant. (Syntactic categories with no values are not useless; they can occur meaningfully in, e.g., languages with control effects.) 8 Xiao, Sabry, and Ariola also point out that proving unique decomposition reduces to proving equivalence and unambiguity of context-free grammars—two undecidable properties. In SL, they reduce the problem to regular tree languages, for which equivalence and unambiguity are decidable. Similarly, the grammars we consider here are regular tree grammars. 12 • If the syntactic category ranged over by e is empty, then that entire syntactic category and all other productions containing e are redundant and can safely be ignored. The above properties are all decidable. Deciding whether the set of terms ranged over by a non-terminal is empty is decidable for context-free grammars. The specific structure of the productions of values guarantees that values must be a subset of terms, and that it is decidable whether e and v generate the same set of terms, i.e., whether e only ranges over values. 3.3.2 Distinct value constructors and potential-redex constructors By definition, potential redexes are distinct from values, and values are defined by a context-free grammar of the form shown in Section 3.2.1. If an instance of a syntactic construct can become a value by evaluating its sub-terms, then another instance must not also be able to become a redex, and vice-versa. The only way a syntactic construct can become both a value and a potential redex is if the value is given by a production of the form v ::= c(. . . , vi , . . .) and there is no elementary context of the form c(. . . , [ ], . . .) with a hole in the i’th position. Then c(. . . , ei , . . .) is a potential redex when ei is not a value. There are several ways to achieve the property that no syntactic construct can become both a value and a potential redex. We use the equivalent requirement that the potential redexes can be specified by a context-free grammar with productions of the same type as for values. In other words, the productions are of the form r ::= c(x1 , . . . , xn ) where xi is either vi or ei . 3.3.3 Left-to-right evaluation of sub-terms Without loss of generality, we assume that sub-terms of a syntactic construct are ordered so that evaluating them proceeds from left to right. In other words, the unique decomposition into evaluation context and potential redex will always have the hole of the context occurring in the left-most non-evaluated sub-term, in the ordering of sub-terms chosen for the abstract syntax. 3.4 Consequences of the requirements The requirements of Section 3.3 guarantee that a syntactic theory contains elementary evaluation contexts, values, and potential redexes of a specific form. This form is described in Figure 1. An example where the m in Figure 1 is strictly smaller than n is a conditional expression such as if(e, e, e). The only elementary evaluation context is if([ ], e, e), since the other two sub-terms should not be evaluated until the conditional expression has itself been reduced. Another example is the call-by-name λ-calculus where only the function part of an application is evaluated. Lemma 1 A syntactic theory satisfying the requirements stated in Section 3.3 contains productions corresponding to Figure 1. 13 For a syntactic construct with n sub-terms, e ::= c(e1 , . . . , en ), there is a number 0 ≤ m ≤ n such that the elementary evaluation contexts for c are exactly: c([ ], e2 , . . . , en ) c(v1 , [ ], e3 , . . . , en ) .. . c(v1 , . . . , vm−1 , [ ], em+1 , . . . , en ) If terms of the form c(v1 , . . . , vm , em+1 , . . . , en ) exist (i.e., the syntactic category ranged over by em contains values) then they are either all values or all potential redexes. In other words, either v ::= c(v1 , . . . , vm , em+1 , . . . , en ) or r ::= c(v1 , . . . , vm , em+1 , . . . , en ) is a production, but not both. We then say that c needs to evaluate m sub-terms. If c(v1 , . . . , vm , em+1 , . . . , en ) is a value we say that c becomes a value. If it is a potential redex we say that c becomes a potential redex. Figure 1: Elementary contexts, values, and potential redexes Proof (sketch): Take the production e ::= c(e1 , . . . , en ). If there is a value production, v ::= c(e1 , . . . , en ) or potential-redex production r ::= c(e1 , . . . , en ), then m = 0 and c becomes a value or a potential redex. If there is no such production then since e is not redundant, there must exist a term, c(e1 , . . . , en ), that has a non-trivial decomposition and thus there must exist an elementary evaluation context c(e1 , . . . , ei−1 , [ ], ei+1 , . . . , en ). We assume that evaluation proceeds from left to right, so the hole must be in the left-most term, i.e., the elementary context is c([ ], e2 , . . . , en ). Assume that there exists an elementary evaluation context of the form c(v1 , . . . , vi−1 , [ ], ei+1 , . . . , en ). Three cases occur: 1. If ei only ranges over non-values then m = i and c becomes neither a value nor a potential redex. 2. If ei ranges over values and there exists a production v ::= c(v1 , . . . , vi , ei+1 , . . . , en ) or r ::= c(v1 , . . . , vi , ei+1 , . . . , en ) then m = i and c becomes either a value or a potential redex. 14 3. If ei ranges over values, but terms of the form c(v1 , . . . , vi , ei+1 , . . . , en ) are not values or potential redexes, then they must have non-trivial decompositions, so there is a corresponding elementary evaluation context. Since we assume left-to-right evaluation, the context must be c(v1 , . . . , vi , [ ], ei+2 , . . . , en ). In other words, either (1) c(v1 , . . . , vi , ei+1 , . . . , en ) does not exist, or it is a value or a potential redex, and then m = i, or (2) there exists an elementary evaluation context c(v1 , . . . , vi , [ ], ei+2 , . . . , en ), and then m > i. An induction argument shows that m is the smallest i such that (1) holds. There exists such a smallest i: a term of the form c(v1 , . . . , vn ) either does not exist or it has only trivial decompositions, i.e., is either a value or a potential redex, and thus m ≤ n as required.  We have shown that a syntactic theory satisfying the requirements contain the productions of Figure 1. We do not show that there are no further productions. However, we show in the next section that one can compute the unique decomposition of any term using only the productions of Figure 1, and thus any further productions would either be redundant or violate the uniquedecomposition property. 3.5 Constructing a refocus function We construct a function, refocus, that is extensionally equivalent to the composition of the plug function and the decompose function. It uses a stack of elementary contexts to represent evaluation contexts. Such a stack directly corresponds to the representation of contexts of Section 2, and it allows an efficient implementation of composing an elementary evaluation context and plugging a term in a context. We use [ ] for the empty context/stack and E ◦ c(v1 , . . . , vi−1 , [ ], ei+1 , . . . , en ) for composing/pushing an elementary context. The refocusing function is defined with two mutually recursive functions. The first, refocus, takes a stack of elementary evaluation contexts and a term, and is defined by cases of the term. The other function, refocus aux , takes a stack of elementary contexts and a value as arguments, and is defined by cases on the top-most elementary evaluation context on the stack. For each e ::= c(e1 , . . . , en ) in the language grammar, there exists one corresponding rule in refocus. If c needs to evaluate m sub-terms then there are m rules in refocus aux corresponding to the elementary contexts c([ ], e2 , . . . , en ) through c(v1 , . . . , vm−1 , [ ], em+1 , . . . , en ). If refocus and refocus aux are implemented in a typed setting, there is one refocus and one refocus aux for each syntactic category of the language. The rules of refocus mentioned above go in the functions corresponding to the syntactic category ranged over by e. The rules of refocus aux go in the functions corresponding to the syntactic category of the hole, which is the same as the syntactic category of the value argument. 1. If c needs to evaluate zero sub-terms, then we know that c(e1 , . . . , en ) is a value or a potential redex. 15 (a) If c becomes a value, then refocus(c(e1 , . . . , en ), E) = refocus aux (E, c(e1 , . . . , en )) (b) If instead c becomes a potential redex, then refocus(c(e1 , . . . , en ), E) = (E, c(e1 , . . . , en )) since we have found the decomposition. 2. If c needs to evaluate more than zero sub-terms then the first expression must be reduced first, and we simply refocus on it: refocus(c(e1 , . . . , en ), E) = refocus(e1 , E ◦ c([ ], e2 , . . . , en )) Likewise, the refocus aux function is defined by cases on the evaluation context on top of the stack. If c(v1 , . . . , vi−1 , [ ], ei+1 , . . . , en ) is the top-most elementary context on the stack, the corresponding case is one of the following. 1. If c needs to evaluate exactly i sub-terms, then c(v1 , . . . , vi , ei+1 , . . . , en ), if such a term exists, is either a value or a potential redex. (a) If c becomes a value then the case is refocus aux (E ◦ c(v1 , . . . , vi−1 , [ ], ei+1 , . . . , en ), vi ) = refocus aux (E, c(v1 , . . . , vi−1 , vi , ei+1 , . . . , en )) (b) If c becomes a potential redex then we have found a decomposition into evaluation context and potential redex, and the case is refocus aux (E ◦ c(v1 , . . . , vi−1 , [ ], ei+1 , . . . , en ), vi ) = (E, c(v1 , . . . , vi−1 , vi , ei+1 , . . . , en )) (c) If ei ranges over only non-values then there is no rule. In a typed setting there is no refocus aux corresponding to that syntactic category. 2. If c needs to evaluate more than i sub-terms, then we continue searching for a potential redex in the next sub-term, and the rule is: refocus aux (E ◦ c(v1 , . . . , vi−1 , [ ], ei+1 , . . . , en ), vi ) = refocus(ei+1 , E ◦ c(v1 , . . . , vi , [ ], en )) 3. Finally there is one rule for the empty context. refocus aux ([ ], v) = v This base case accounts for the situation where the entire program is a value, so the decomposition has to return a value rather than a a context and a potential redex. 16 In Appendix A, we illustrate the construction of a refocus function for a syntactic theory of arithmetic expressions with precedence. In the following section we show that the refocus function generated by these rules computes a correct decomposition into evaluation context and potential redex, and that it does so at least as efficiently as the combination of plug and decompose. 3.6 Correctness By construction, refocus can only return either a value or a pair of evaluation context and potential redex, since the remaining cases all perform a tail-recursive call. Both refocus and refocus aux are passed a decomposition of a term, i.e., an evaluation context and another term, and if they make a recursive call, it is on another decomposition of the same term. Therefore, if refocus(e, E) returns a decomposition, then it is a complete decomposition of E[e]. It thus suffices to prove that refocus terminates on any argument. Let us show that if E[e] = E ′ [r], where r is a potential redex, then refocus(e, E) performs T (E ′ ) + Taux (r) − T (E) tail-recursive calls, where T and Taux are defined as follows: T : EvCont → N T ([ ]) = 0 Pk T (E ◦ c(v1 , . . . , vk−1 , [ ], ek+1 , . . . , een )) = 1 + T (E) + i=1 (1 + Taux (vi )) Taux : Val + PotRedex → N P m Taux (c(v1 , . . . , vm , em+1 , . . . , en )) = 1 + i=1 (1 + Taux (vi )) Given an evaluation context, T computes the number of edges traversed to go from the root to the hole in a depth-first left-to-right traversal, counting the edges to values both on the way down and on the way up. This traversal is illustrated by the following picture. cO @ ~~ > @@@ ~ @@ ~~ @@ ~ ~~~  v v c ~ ? >>> ~ ~ >> ~~ >>  ~~~  v e c> ~ >> ~ ~ >> ~~ >>  ~~~ e e c> @ >> >> >> >  v e [] As for Taux , it computes the number of edges followed by a full traversal of a composite value. 17 All fully traversed sub-terms must be values. With this definition, the result of T (E) is at most twice the number of nodes visited on such a traversal. We use the T function to measure how far the refocus function has progressed in its traversal of its argument. Lemma 2 (Termination) If E[e] = E ′ [r] then refocus(e, E) terminates after T (E ′ ) + Taux (r) − T (E) tail-recursive calls. Proof: Let us use T to define a progress measure on all the left-hand sides and right-hand sides of the definition of refocus and refocus aux : Progress(refocus(e, E))) Progress(refocus aux (E, v)) Progress((E, r)) Progress(v) = = = = T (E) T (E) + Taux (v) T (E) + Taux (r) Taux (v) With this definition, all choices of arguments to refocus and refocus aux give a right-hand side with a progress value one greater than the left-hand side. We omit the details and give only one case as example. If c needs to evaluate m sub-terms to become a value and 0 < k < m, then refocus aux contains the following case: refocus aux (E ◦ c(v1 , . . . , vk−1 , [ ], ek+1 , ek+2 , . . . , en ), vk ) = refocus(ek+1 , E ◦ c(v1 , . . . , vk−1 , vk , [ ], ek+2 , . . . , en )) Taking the progress measure on both the left-hand side and the right-hand side gives the expected one-greater value on the right-hand side: Progress(refocus aux (E ◦ c(v1 , . . . , vk−1 , [ ], ek+1 , . . . , en ), vi )) = T (E ◦ c(v1 , . . . , vk−1 , [ ], ek+1 , . . . , en )) + Taux (vk ) P = 1 + T (E) + k−1 i=1 (1 + Taux (vi )) + Taux (vk ) and Progress(refocus(ek+1 , E ◦ c(v1 , . . . , vk , [ ], ek+2 , . . . , en ))) = T (E ◦ c(v1 , . . . , vk , [ ], ek+2 , . . . , en )) Pk = 1 + T (E) + i=1 (1 + Taux (vi )) Pk−1 = 1 + T (E) + i=1 (1 + Taux (vi )) + Taux (vk ) + 1 Since each recursive call increases the progress measure by one, and there is only a finite possible number of different such calls (each call is to one of two functions on a decomposition of the same term, and a term has only finitely many decompositions), the recursion must eventually end. When this happens, the cumulative increase in the progress measure is also the number of recursive calls performed to reach it. This number is exactly Progress((E ′ , r)) − Progress(refocus(e, E)) = T (E ′ ) + Taux (r) − T (E).  Lemma 2 tells us both that refocus is a total function and how many calls it takes to find the result. In Section 3.7, we use it to show that refocus is always at least as efficient as using plug and decompose. 18 3.7 Complexity In order to show that refocus is more efficient than the composition of plug and decompose, we must first decide on a relevant complexity measure, and we must find the complexity of plug and decompose. For measuring the time complexity of refocus we use the number of recursive calls as in Section 3.6. Each call performs a bounded number of basic operations on the syntax (either constructing a term or deconstructing a term while naming the sub-terms—the latter is performed implicitly since the function is defined by cases). If each such operation takes constant time, the time taken to compute the result of refocus is proportional to the number of recursive calls (i.e., bounded by a constant factor times the number of calls). Let us informally argue that any implementation of a function decompose : Exp → Val + (EvCont × PotRedex) finding the unique complete decomposition of terms (in a syntactic category satisfying our requirements) must perform at least (T (E) + Taux (r))/2 basic operations on the syntax to compute decompose(E[r]) = (E, r). If it uses fewer operations, it cannot inspect all the values of the evaluation context and of the potential redex. If we changed such a value to a non-value, then the hole of the evaluation context should be in the new non-value term, but the function cannot see this without inspecting the value and thus it must generate an erroneous result, contradicting the assumption that it computes the unique complete decomposition. The plug function is implemented in time proportional to the depth of the hole in the evaluation context. Since the time complexity of decompose is always greater, we ignore plug in our comparison. In summary, the complexity of computing decompose(E[e]) = E ′ [r] is at least proportional to T (E ′ ) + Taux (r), whereas the complexity of computing refocus(e, E) = E ′ [r] is proportional to T (E ′ ) + Taux (r) − T (E). For syntactic theories where rewrites are often local, the difference between T (E) and T (E ′ ) is often significantly smaller than the value of T (E ′ ) itself. For example, in Example 1 of Section 1.3 the difference between T (E) and T (E ′ ) is constant whereas T (E ′ ) alone is proportional to the size of the program. In some cases, refocus does no better than decompose, e.g., when the context argument to refocus is always empty (e.g., for tail-recursive or continuationpassing style λ-terms, and for trampoline-like programs with control operators where the context is discarded in each step). Likewise, if the size of the context is bounded, the saving is at most a constant factor. 3.8 Summary We have presented a few mild requirements to construct a refocus function for a syntactic theory. We have also proven the correctness of this construction and the complexity of the constructed function. Refocusing is at least as efficient as first plugging and then decomposing, and it is often significantly more efficient. 19 4 Conclusion and issues We have presented a general result about syntactic theories with context-free grammars of values, evaluation contexts, and redexes, and satisfying a uniquedecomposition property. This result enables one to mechanically derive an interpreter that does not incur a linear-time overhead for each reduction step. We have illustrated the result with two interpreters, one for the call-by-value λ-calculus and one for arithmetic expressions with precedence, and by mechanically turning a quadratic-time program transformation into a program transformation that operates in one pass. Acknowledgments: We are grateful to Daniel Damian, Bernd Grobauer, Fritz Henglein, and Julia Lawall for commenting a previous version of this article. Thanks are also due to the anonymous RULE’01 referees. This work is supported by the ESPRIT Working Group APPSEM (http: //www.md.chalmers.se/Cs/Research/Semantics/APPSEM/). A Arithmetic expressions with precedence This section treats a comprehensive example illustrating all the cases of the construction of a refocus function as presented in Section 3.5. We consider arithmetic expressions with additions, multiplications, conditional expressions checking whether their first argument is zero, parenthesized expressions, literals, and oracles returning 0 or 1. (The oracles are only there to illustrate a case of the construction.) The grammar is unambiguous and hierarchic, as often given in compiler courses. It specifies expressions, terms, and factors, and reads as follows. e∈ t∈ f∈ n∈ Expr Term Fact Lit e ::= t + e | Ifz e e e | t t ::= f × t | f f ::= n | flip | (e) A program is an expression. A.1 A syntactic theory We define the syntactic theory of arithmetic expressions by specifying its values, its computations, its evaluation contexts, its redexes and its reduction rules. Values (trivial terms): ve ∈ ExprVal vt ∈ TermVal vf ∈ FactVal 20 ve ::= vt vt ::= vf vf ::= n Computations (serious terms): ce ∈ ExprComp ct ∈ TermComp cf ∈ FactComp ce ::= t + e | Ifz e e e | ct ct ::= f × t | cf cf ::= flip | (e) Evaluation contexts: Ee ∈ ExprEvCont Et ∈ TermEvCont Ef ∈ FactEvCont Ee ::= [ ]e | Et + e | vt + Ee | Ifz Ee e e | Et Et ::= [ ]t | Ef × t | vf × Et | Ef Ef ::= [ ]f | (Ee ) Redexes: re ∈ ExprRedex rt ∈ TermRedex rf ∈ FactRedex re ::= vt + ve | Ifz ve e e | rt rt ::= vf × vt | rf rf ::= flip | (ve ) Reduction rules: Ee [n1 + n2 ] Ee [Ifz n e1 e2 ] Ee [Ifz n e1 e2 ] Ee [n1 × n2 ] Ee [flip] Ee [flip] Ee [(n)] → → → → → → → Ee [n3 ] Ee [e1 ] Ee [e2 ] Ee [n3 ] Ee [0] Ee [1] Ee [n] where n3 is the sum of n1 and n2 if n = 0 if n 6= 0 where n3 is the product of n1 and n2 The reduction rules are defined only on decompositions that “make sense”, i.e., only an expression is plugged into [ ]e , only a term into [ ]t and a factor into [ ]f . The result of a reduction is an expression. This syntactic theory satisfies three unique-decomposition lemmas, i.e., decomposing an expression (resp. a term and a factor) into an evaluation context and a redex yields a unique result. We prove these lemmas by structural induction on the syntax. The language is simple enough that the number of cases is manageable. There are no stuck terms, and reduction always yields an expression. The reduction relation, however, is not a function from expressions to expressions, even though decompositions are unique, because of the oracles. 21 A.2 Evaluation contexts with right-composition In Section A.1, the grammar of evaluation contexts embodies left composition, i.e., the construction of contexts by composition with an elementary context on the left. In other words, each production except for the empty context, e.g., Ee ::= vt + Ee , can be written as Ee ::=(vt + [ ]e ) ◦ Ee . Since composition of contexts is associative, we can define the same contexts using right composition, giving a production of the form Ee ::= Ee ◦(vt + [ ]e ). We write it, however, in the more common way: Ee ::= Ee [vt + [ ]e ]. We transform the grammar of contexts into one representing composition on the right, and we do this in a completely mechanical way. The example above shows the general idea, though it does not illustrate the case where the sub-context is not of the same type as the generated context. We have three groups of evaluation contexts, grouped by the syntactic category they produce when plugged. Take the elementary context [ ]t + e. If we left-compose another evaluation context with this elementary context, we require that the other context produces terms. If we right-compose this elementary context, however, we require the other context to accept expressions in the hole. To capture this restriction in the grammar, we group the productions by the type of the hole instead. So for example, the production Ee ::= Et + e is transformed into Et ::= Ee [[ ]t + e]. In general, the transformation rewrites a production of the form Ea ::= c(x1 , . . . , Eb , . . . , xn ) into Eb ::= Ea [c(x1 , . . . , [ ]b , . . . , xn )], and it keeps the productions of empty contexts. Performing this transformation on the grammar of evaluation contexts above gives the following grammar. Evaluation contexts: Ee ∈ ExprEvCont Et ∈ TermEvCont Ef ∈ FactEvCont Ee ::= [ ]e | Ee [vt + [ ]e ] | Ee [Ifz [ ]e e e] | Ef [([ ])] Et ::= [ ]t | Ee [[ ]t + e] | Et [vf × [ ]t ] | Ee Ef ::= [ ]f | Et [[ ]f × t] | Et In the original representation, Ee ranged over all contexts that generated expression, but made no restriction on the type of the hole. The second representation likewise lets Ee range over all contexts with a hole accepting an expression, but makes no restriction on the type of the result of a plug. Using this second representation, we must require that the evaluation contexts appearing in the reduction rules must output expressions, i.e., we rule out the empty contexts from Et and Ef in the grammar of evaluation contexts. The resulting evaluation contexts and reduction rules read as follows. Evaluation contexts: Ee ∈ ExprEvCont Et ∈ TermEvCont Ef ∈ FactEvCont Ee ::= [ ]e | Ee [vt + [ ]e ] | Ee [Ifz [ ]e e e] | Ef [([ ])] Et ::= Ee [[ ]t + e] | Et [vf × [ ]t ] | Ee Ef ::= Et [[ ]f × t] | Et 22 Reduction rules: Ee [n1 + n2 ] Ee [Ifz n e1 e2 ] Ee [Ifz n e1 e2 ] Et [n1 × n2 ] Ef [flip] Ef [flip] Ef [(n)] datatype expr = | and expr_val = and expr_comp = | | and term = | and term_val = and term_comp = | and fact = | and fact_val = and fact_comp = | → → → → → → → Ee [n3 ] Ee [e1 ] Ee [e2 ] Et [n3 ] Ef [0] Ef [1] Ef [n] where n3 is the sum of n1 and n2 if n = 0 if n 6= 0 where n3 is the product of n1 and n2 EXPR_VAL of expr_val EXPR_COMP of expr_comp TERM_VAL’ of term_val ADD of term * expr IFZ of expr * expr * expr TERM_COMP’ of term_comp TERM_VAL of term_val TERM_COMP of term_comp FACT_VAL’ of fact_val MUL of fact * term FACT_COMP’ of fact_comp FACT_VAL of fact_val FACT_COMP of fact_comp INT of int FLIP PARENS of expr Figure 2: Abstract syntax of arithmetic expressions A.3 Implementation Figure 2 displays the BNF of arithmetic expressions in Standard ML. As usual with syntactic theories, we distinguish between values (trivial terms) and computations (serious terms). An expression (expr) is thus trivial (expr val) or serious (expr comp); a term (term) is trivial (term val) or serious (term comp); and a factor (fact) is trivial (fact val) or serious (fact comp). The only values are integers, hence values are defined with the hierarchy of types expr val, term val, fact val, and int. Computations are similarly embedded, hence the hierarchy of types expr comp, term comp, and fact comp. Figure 3 displays the evaluation contexts and Figure 4, the corresponding plug functions. Figure 5 displays the implementation of redexes and the result of decomposition, and Figure 6, the corresponding decomposition functions. 23 datatype expr_evcont = | | | and term_evcont = | | and fact_evcont = | EEC0 EEC1 EEC2 EEC3 TEC1 TEC2 TEC3 FEC1 FEC2 of of of of of of of of term_val * expr_evcont expr * expr * expr_evcont fact_evcont expr * expr_evcont fact_val * term_evcont expr_evcont term * term_evcont term_evcont Figure 3: Evaluation contexts for arithmetic expressions (* plug_expr : expr * expr_evcont -> expr *) (* plug_term : term * term_evcont -> expr *) (* plug_fact : fact * fact_evcont -> expr *) fun plug_expr (e, EEC0) = e | plug_expr (e, EEC1 (tt, eec)) = plug_expr (EXPR_COMP (ADD (TERM_VAL tt, e)), eec) | plug_expr (e, EEC2 (e1, e2, eec)) = plug_expr (EXPR_COMP (IFZ (e, e1, e2)), eec) | plug_expr (e, EEC3 fec) = plug_fact (FACT_COMP (PARENS e), fec) and plug_term (t, TEC1 (e, eec)) = plug_expr (EXPR_COMP (ADD (t, e)), eec) | plug_term (t, TEC2 (tf, tec)) = plug_term (TERM_COMP (MUL (FACT_VAL tf, t)), tec) | plug_term (TERM_VAL tt, TEC3 eec) = plug_expr (EXPR_VAL (TERM_VAL’ tt), eec) | plug_term (TERM_COMP st, TEC3 eec) = plug_expr (EXPR_COMP (TERM_COMP’ st), eec) and plug_fact (f, FEC1 (t, tec)) = plug_term (TERM_COMP (MUL (f, t)), tec) | plug_fact (FACT_VAL tf, FEC2 tec) = plug_term (TERM_VAL (FACT_VAL’ tf), tec) | plug_fact (FACT_COMP sf, FEC2 tec) = plug_term (TERM_COMP (FACT_COMP’ sf), tec) Figure 4: Plug functions for arithmetic expressions 24 datatype expr_redex = ADD_REDEX of term_val * expr_val | IFZ_REDEX of expr_val * expr * expr datatype term_redex = MUL_REDEX of fact_val * term_val datatype fact_redex = FLIP_REDEX | PARENS_REDEX of expr_val datatype decomposed = | | | VALUE of expr_val EXPR_DECOMPOSITION of expr_evcont * expr_redex TERM_DECOMPOSITION of term_evcont * term_redex FACT_DECOMPOSITION of fact_evcont * fact_redex Figure 5: Redexes and the result of decomposition for arithmetic expressions 25 (* (* (* (* fun decompose_expr decompose_expr_comp decompose_term_comp decompose_fact_comp : : : : expr -> expr_comp * expr_evcont -> term_comp * term_evcont -> fact_comp * fact_evcont -> decomposed decomposed decomposed decomposed decompose_expr (EXPR_VAL te) = VALUE te | decompose_expr (EXPR_COMP se) = decompose_expr_comp (se, EEC0) and decompose_expr_comp (ADD (TERM_VAL tt, EXPR_VAL te), eec) = EXPR_DECOMPOSITION (eec, ADD_REDEX (tt, te)) | decompose_expr_comp (ADD (TERM_VAL tt, EXPR_COMP se), eec) = decompose_expr_comp (se, EEC1 (tt, eec)) | decompose_expr_comp (ADD (TERM_COMP st, e), eec) = decompose_term_comp (st, TEC1 (e, eec)) | decompose_expr_comp (IFZ (EXPR_VAL te, e1, e2), eec) = EXPR_DECOMPOSITION (eec, IFZ_REDEX (te, e1, e2)) | decompose_expr_comp (IFZ (EXPR_COMP se, e1, e2), eec) = decompose_expr_comp (se, EEC2 (e1, e2, eec)) | decompose_expr_comp (TERM_COMP’ st, eec) = decompose_term_comp (st, TEC3 eec) and decompose_term_comp (MUL (FACT_VAL tf, TERM_VAL tt), tec) = TERM_DECOMPOSITION (tec, MUL_REDEX (tf, tt)) | decompose_term_comp (MUL (FACT_VAL tf, TERM_COMP st), tec) = decompose_term_comp (st, TEC2 (tf, tec)) | decompose_term_comp (MUL (FACT_COMP sf, t), tec) = decompose_fact_comp (sf, FEC1 (t, tec)) | decompose_term_comp (FACT_COMP’ sf, tec) = decompose_fact_comp (sf, FEC2 tec) and decompose_fact_comp (FLIP, fec) = FACT_DECOMPOSITION (fec, FLIP_REDEX) | decompose_fact_comp (PARENS (EXPR_VAL te), fec) = FACT_DECOMPOSITION (fec, PARENS_REDEX te) | decompose_fact_comp (PARENS (EXPR_COMP se), fec) = decompose_expr_comp (se, EEC3 fec) Figure 6: Decomposition of an arithmetic expression 26 *) *) *) *) (* eval : expr -> expr_val *) (* eval’ : decomposed -> expr_val *) fun eval e = eval’ (decompose_expr e) and eval’ (VALUE te) = te | eval’ (EXPR_DECOMPOSITION (eec, ADD_REDEX (FACT_VAL’ (INT n1), TERM_VAL’ (FACT_VAL’ (INT n2))))) = eval (plug_expr (EXPR_VAL (TERM_VAL’ (FACT_VAL’ (INT (n1 + n2)))), eec)) | eval’ (EXPR_DECOMPOSITION (eec, IFZ_REDEX (TERM_VAL’ (FACT_VAL’ (INT 0)), e1, e2))) = eval (plug_expr (e1, eec)) | eval’ (EXPR_DECOMPOSITION (eec, IFZ_REDEX (TERM_VAL’ (FACT_VAL’ (INT n)), e1, e2))) = eval (plug_expr (e2, eec)) | eval’ (TERM_DECOMPOSITION (tec, MUL_REDEX (INT n1, FACT_VAL’ (INT n2)))) = eval (plug_term (TERM_VAL (FACT_VAL’ (INT (n1 * n2))), tec)) | eval’ (FACT_DECOMPOSITION (fec, FLIP_REDEX)) = eval (plug_fact (FACT_VAL (INT (Oracle.flip ())), fec)) | eval’ (FACT_DECOMPOSITION (fec, PARENS_REDEX (TERM_VAL’ (FACT_VAL’ (INT n))))) = eval (plug_fact (FACT_VAL (INT n), fec)) Figure 7: Evaluation of an arithmetic expression Figure 7 displays the evaluation function, which attempts to decompose its input expression into an evaluation context and a redex. If the expression is a value (and thus cannot be decomposed), then the result is found. Otherwise, the redex is contracted, the contractum is plugged into the context, and evaluation is iterated. Contracting a redex amounts to adding two integers, selecting between two expressions, multiplying two integers, calling an oracle (which is unspecified here; our implementation is state-based), or skipping parentheses. No terms are stuck. A.4 Refocusing We now construct the refocus functions. We start from the ML definition of the grammar, in Figure 2, which fits the format expected for the construction. Let us determine the reduction sequence of each syntactic construct. 27 The main syntactic constructs: Both addition and multiplication evaluate their arguments from left to right. The conditional expression is special in that it only evaluates its first argument. The oracle has no argument. The parentheses have one argument and evaluate it. All of these syntactic constructs create redexes. The auxiliary syntactic constructs: The coercions between terms and expressions and between factors and expressions have one argument. This argument is evaluated. The refocus functions follow the grammatical structure of the source language. Their skeleton is thus as follows. fun refocus_expr (EXPR_VAL te, eec) = refocus_expr_val (te, eec) | refocus_expr (EXPR_COMP se, eec) = refocus_expr_comp (se, eec) and refocus_expr_val (te, eec) = ... and refocus_expr_comp (ADD (t, e), eec) = ... | refocus_expr_comp (IFZ (e0, e1, e2), eec) = ... | refocus_expr_comp (TERM_COMP’ st, eec) = ... and refocus_term (TERM_VAL tt, tec) = refocus_term_val (tt, tec) | refocus_term (TERM_COMP st, tec) = refocus_term_comp (st, tec) and refocus_term_val (tt, tec) = ... and refocus_term_comp (MUL (f, t), tec) = ... | refocus_term_comp (FACT_COMP’ sf, tec) = ... and refocus_fact (FACT_VAL tf, fec) = refocus_fact_val (tf, fec) | refocus_fact (FACT_COMP sf, fec) = refocus_fact_comp (sf, fec) and refocus_fact_val (tf, fec) = ... and refocus_fact_comp (FLIP, fec) = ... | refocus_fact_comp (PARENS e, fec) = ... In the rest of this section, we complete each missing case in this definition. 1. If a constructor needs to evaluate zero sub-terms, then the result of the production is a value or a redex. 28 (a) The case of values is already taken care of because the grammar differentiates between values and computations. All functions refocus x call refocus x val if given a value, and call refocus x comp if given a computation. (b) If the result of a production is a redex, then we have found a decomposition. Here, only the oracle fits the bill. We thus return a decomposition. refocus_fact_comp (FLIP, fec) = FACT_DECOMP (fec, FLIP_REDEX) 2. If the constructor needs to evaluate more than zero sub-terms then the first sub-expression must be evaluated. refocus_expr_comp (ADD (t, e), eec) = refocus_term (t, TEC1 (e, eec)) | refocus_expr_comp (IFZ (e0, e1, e2), eec) = refocus_expr (e0, EEC2 (e1, e2, eec)) | refocus_expr_comp (TERM_COMP’ st, eec) = refocus_term_comp (st, TEC3 eec) refocus_term_comp (MUL (f, t), tec) = refocus_fact (f, FEC1 (t, tec)) | refocus_term_comp (FACT_COMP’ sf, tec) = refocus_fact_comp (sf, FEC2 tec) | refocus_fact_comp (PARENS e, fec) = refocus_expr (e, EEC3 fec) 3. Likewise, each refocus x val is defined by cases on the inner elementary evaluation context. refocus_expr_val = ... | refocus_expr_val = ... | refocus_expr_val = ... | refocus_expr_val = ... (te, EEC0) (te, EEC1 (tt, eec)) (te, EEC2 (e1, e2, eec)) (te, EEC3 fec) refocus_term_val (tt, TEC1 (e, eec)) = ... | refocus_term_val (tt, TEC2 (tf, tec)) = ... | refocus_term_val (tt, TEC3 eec) = ... 29 refocus_fact_val (tf, FEC1 (t, tec)) = ... | refocus_fact_val (tf, FEC2 tec) = ... (a) If all the sub-terms needed by a constructor are evaluated and the production constructs a value, we refocus on this value. | refocus_term_val (tt, TEC3 eec) = refocus_expr_val (TERM_VAL’ tt, eec) | refocus_fact_val (tf, FEC2 tec) = refocus_term_val (FACT_VAL’ tf, tec) (b) If all the sub-terms needed by a constructor are evaluated and the production constructs a redex, we have found a decomposition. | refocus_expr_val (te, EEC1 (tt, eec)) = EXPR_DECOMP (eec, ADD_REDEX (tt, te)) | refocus_expr_val (te, EEC2 (e1, e2, eec)) = EXPR_DECOMP (eec, IFZ_REDEX (te, e1, e2)) | refocus_expr_val (te, EEC3 fec) = FACT_DECOMP (fec, PARENS_REDEX te) | refocus_term_val (tt, TEC2 (tf, tec)) = TERM_DECOMP (tec, MUL_REDEX (tf, tt)) (c) If the constructor needs to evaluate more sub-terms, we refocus on the next sub-expression to be reduced. refocus_term_val (tt, TEC1 (e, eec)) = refocus_expr (e, EEC1 (tt, eec)) refocus_fact_val (tf, FEC1 (t, tec)) = refocus_term (t, TEC2 (tf, tec)) 4. Finally, we turn to the empty context: refocus_expr_val (te, EEC0) = VALUE te The refocus functions are now complete (see Figure 8). The main evaluation function is displayed in Figure 9. Rather than decomposing the result of the plug functions, eval’ refocuses from a contractum to the next decomposition and iterates. 30 fun refocus_expr (EXPR_VAL te, eec) = refocus_expr_val (te, eec) | refocus_expr (EXPR_COMP se, eec) = refocus_expr_comp (se, eec) and refocus_expr_val (te, EEC0) = VALUE te | refocus_expr_val (te, EEC1 (tt, eec)) = EXPR_DECOMPOSITION (eec, ADD_REDEX (tt, te)) | refocus_expr_val (te, EEC2 (e1, e2, eec)) = EXPR_DECOMPOSITION (eec, IFZ_REDEX (te, e1, e2)) | refocus_expr_val (te, EEC3 fec) = FACT_DECOMPOSITION (fec, PARENS_REDEX te) and refocus_expr_comp (ADD (t, e), eec) = refocus_term (t, TEC1 (e, eec)) | refocus_expr_comp (IFZ (e0, e1, e2), eec) = refocus_expr (e0, EEC2 (e1, e2, eec)) | refocus_expr_comp (TERM_COMP’ st, eec) = refocus_term_comp (st, TEC3 eec) and refocus_term (TERM_VAL tt, tec) = refocus_term_val (tt, tec) | refocus_term (TERM_COMP st, tec) = refocus_term_comp (st, tec) and refocus_term_val (tt, TEC1 (e, eec)) = refocus_expr (e, EEC1 (tt, eec)) | refocus_term_val (tt, TEC2 (tf, tec)) = TERM_DECOMPOSITION (tec, MUL_REDEX (tf, tt)) | refocus_term_val (tt, TEC3 eec) = refocus_expr_val (TERM_VAL’ tt, eec) and refocus_term_comp (MUL (f, t), tec) = refocus_fact (f, FEC1 (t, tec)) | refocus_term_comp (FACT_COMP’ sf, tec) = refocus_fact_comp (sf, FEC2 tec) and refocus_fact (FACT_VAL tf, fec) = refocus_fact_val (tf, fec) | refocus_fact (FACT_COMP sf, fec) = refocus_fact_comp (sf, fec) and refocus_fact_val (tf, FEC1 (t, tec)) = refocus_term (t, TEC2 (tf, tec)) | refocus_fact_val (tf, FEC2 tec) = refocus_term_val (FACT_VAL’ tf, tec) and refocus_fact_comp (FLIP, fec) = FACT_DECOMPOSITION (fec, FLIP_REDEX) | refocus_fact_comp (PARENS e, fec) = refocus_expr (e, EEC3 fec) Figure 8: Refocusing over an arithmetic expression 31 (* (* (* (* (* (* (* (* (* refocus_expr refocus_expr_val refocus_expr_comp refocus_term refocus_term_val refocus_term_comp refocus_fact refocus_fact_val refocus_fact_comp : : : : : : : : : expr expr_val expr_comp term term_val term_comp fact fact_val fact_comp * * * * * * * * * expr_evcont expr_evcont expr_evcont term_evcont term_evcont term_evcont fact_evcont fact_evcont fact_evcont -> -> -> -> -> -> -> -> -> decomposed decomposed decomposed decomposed decomposed decomposed decomposed decomposed decomposed *) *) *) *) *) *) *) *) *) (* eval : expr -> expr_val *) (* eval’ : decomposed -> expr_val *) fun eval e = eval’ (refocus_expr (e, EEC0)) and eval’ (VALUE te) = te | eval’ (EXPR_DECOMPOSITION (eec, ADD_REDEX (FACT_VAL’ (INT n1), TERM_VAL’ (FACT_VAL’ (INT n2))))) = eval’ (refocus_expr (EXPR_VAL (TERM_VAL’ (FACT_VAL’ (INT (n1 + n2)))), eec)) | eval’ (EXPR_DECOMPOSITION (eec, IFZ_REDEX (TERM_VAL’ (FACT_VAL’ (INT 0)), e1, e2))) = eval’ (refocus_expr (e1, eec)) | eval’ (EXPR_DECOMPOSITION (eec, IFZ_REDEX (TERM_VAL’ (FACT_VAL’ (INT n)), e1, e2))) = eval’ (refocus_expr (e2, eec)) | eval’ (TERM_DECOMPOSITION (tec, MUL_REDEX (INT n1, FACT_VAL’ (INT n2)))) = eval’ (refocus_term (TERM_VAL (FACT_VAL’ (INT (n1 * n2))), tec)) | eval’ (FACT_DECOMPOSITION (fec, FLIP_REDEX)) = eval’ (refocus_fact (FACT_VAL (INT (Oracle.flip ())), fec)) | eval’ (FACT_DECOMPOSITION (fec, PARENS_REDEX (TERM_VAL’ (FACT_VAL’ (INT n))))) = eval’ (refocus_fact (FACT_VAL (INT n), fec)) Figure 9: Evaluation of an arithmetic expression, refocused 32 A.5 Summary We have considered arithmetic expressions with precedence and we have specified their meaning with a syntactic theory. We then applied the algorithm of Section 3.5 to write refocusing functions that map an evaluation context and a contractum into either a value or a decomposition into an evaluation context and a new redex. The result is a compositional evaluator that operates in one pass over its input. References [1] Rod M. Burstall and John Darlington. A transformational system for developing recursive programs. Journal of ACM, 24(1):44–67, 1977. [2] Olivier Danvy and Andrzej Filinski. Representing control, a study of the CPS transformation. Mathematical Structures in Computer Science, 2(4):361–391, 1992. [3] Olivier Danvy and Lasse R. Nielsen. CPS transformation of beta-redexes. In Amr Sabry, editor, Proceedings of the Third ACM SIGPLAN Workshop on Continuations, Technical report 545, Computer Science Department, Indiana University, pages 35–39, London, England, January 2001. Also available as the technical report BRICS RS-00-35. [4] Olivier Danvy and Lasse R. Nielsen. Syntactic theories in practice. In Mark van den Brand and Rakesh M. Verma, editors, Informal proceedings of the Second International Workshop on Rule-Based Programming (RULE 2001), volume 59.4 of Electronic Notes in Theoretical Computer Science, Firenze, Italy, September 2001. Extended version available as the technical report BRICS RS-02-04. [5] Matthias Felleisen. The Calculi of λ-v-CS Conversion: A Syntactic Theory of Control and State in Imperative Higher-Order Programming Languages. PhD thesis, Department of Computer Science, Indiana University, Bloomington, Indiana, August 1987. [6] Matthias Felleisen and Matthew Flatt. Programming languages and lambda calculi. Unpublished lecture notes. http://www.ccs.neu.edu/ home/matthias/3810-w02/readings.html, 1989-2001. [7] Matthias Felleisen and Daniel P. Friedman. Control operators, the SECD machine, and the λ-calculus. In Martin Wirsing, editor, Formal Description of Programming Concepts III, pages 193–217. Elsevier Science Publishers B.V. (North-Holland), Amsterdam, 1986. [8] John Greiner. Semantics-based parallel cost models and their use in provably efficient implementations. PhD thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, April 1997. Technical Report CMU-CS-97-113. 33 [9] Gilles Kahn. Natural semantics. Technical Report 601, INRIA, Sophia Antipolis, France, February 1987. [10] Lasse R. Nielsen. A study of defunctionalization and continuation-passing style. PhD thesis, BRICS PhD School, University of Aarhus, Aarhus, Denmark, July 2001. BRICS DS-01-7. [11] Gordon D. Plotkin. Call-by-name, call-by-value and the λ-calculus. Theoretical Computer Science, 1:125–159, 1975. [12] Amr Sabry and Matthias Felleisen. Reasoning about programs in continuation-passing style. Lisp and Symbolic Computation, 6(3/4):289–360, 1993. [13] Joseph Stoy. Some mathematical aspects of functional programming. In John Darlington, Peter Henderson, and David A. Turner, editors, Functional Programming and its Applications. Cambridge University Press, 1982. [14] Philip Wadler. Deforestation: Transforming programs to eliminate trees. Theoretical Computer Science, 73(2):231–248, 1989. [15] Philip Wadler. The essence of functional programming (invited talk). In Andrew W. Appel, editor, Proceedings of the Nineteenth Annual ACM Symposium on Principles of Programming Languages, pages 1–14, Albuquerque, New Mexico, January 1992. ACM Press. [16] Mitchell Wand. Correctness of procedure representations in higher-order assembly language. In Stephen Brookes, Michael Main, Austin Melton, Michael Mislove, and David Schmidt, editors, Proceedings of the 7th International Conference on Mathematical Foundations of Programming Semantics, number 598 in Lecture Notes in Computer Science, pages 294–311, Pittsburgh, Pennsylvania, March 1991. Springer-Verlag. [17] Yong Xiao, Zena M. Ariola, and Michel Mauny. From syntactic theories to interpreters: A specification language and its compilation. In Nachum Dershowitz and Claude Kirchner, editors, Informal proceedings of the First International Workshop on Rule-Based Programming (RULE 2000), Montréal, Canada, September 2000. Available online at http: //www.loria.fr/~ckirchne/=rule2000/proceedings/. [18] Yong Xiao, Amr Sabry, and Zena M. Ariola. From syntactic theories to interpreters: Automating proofs of decomposition lemma. Higher-Order and Symbolic Computation, 14(4):387–409, 2001. 34 Recent BRICS Report Series Publications RS-02-4 Olivier Danvy and Lasse R. Nielsen. Syntactic Theories in Practice. January 2002. 34 pp. This revised report supersedes the earlier BRICS report RS-01-31. RS-02-3 Olivier Danvy and Lasse R. Nielsen. On One-Pass CPS Transformations. January 2002. 18 pp. RS-02-2 Lasse R. Nielsen. A Simple Correctness Proof of the Direct-Style Transformation. January 2002. RS-02-1 Claus Brabrand, Anders Møller, and Michael I. Schwartzbach. The <bigwig> Project. January 2002. 36 pp. This revised report supersedes the earlier BRICS report RS-00-42. RS-01-55 Daniel Damian and Olivier Danvy. A Simple CPS Transformation of Control-Flow Information. December 2001. RS-01-54 Daniel Damian and Olivier Danvy. Syntactic Accidents in Program Analysis: On the Impact of the CPS Transformation. December 2001. 41 pp. To appear in the Journal of Functional Programming. This report supersedes the earlier BRICS report RS-00-15. RS-01-53 Zoltán Ésik and Masami Ito. Temporal Logic with Cyclic Counting and the Degree of Aperiodicity of Finite Automata. December 2001. 31 pp. RS-01-52 Jens Groth. Extracting Witnesses from Proofs of Knowledge in the Random Oracle Model. December 2001. 23 pp. RS-01-51 Ulrich Kohlenbach. On Weak Markov’s Principle. December 2001. 10 pp. RS-01-50 Jiřı́ Srba. Note on the Tableau Technique for Commutative Transition Systems. December 2001. 19 pp. To appear in Nielsen and Engberg, editors, Foundations of Software Science and Computation Structures, FoSSaCS ’02 Proceedings, LNCS 2303, 2002. RS-01-49 Olivier Danvy and Lasse R. Nielsen. A First-Order One-Pass CPS Transformation. December 2001. 21 pp. Extended version of a paper to appear in Nielsen and Engberg, editors, Foundations of Software Science and Computation Structures, FoSSaCS ’02 Proceedings, LNCS 2303, 2002.