Foundations of Speech and Language Technology

Discourse Foundations of Speech and Language Technology: = a unit of language (language use) consisting of more than one utterance – Utterance = the use of a NL expression (sentence, …) speaker S, to hearer(s) H, at time t, in situation s Discourse and Dialogue • Monologue vs. dialogue • Written vs. spoken, or multimodal Ivana Kruijff-Korbayová [email protected] • Characteristics: www.coli.uni-sb.de/~korbay/ ! Teaching 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) – Purpose; Collaboration; Coherence; Cohesion 1 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) Purpose Collaboration • Collaboration: • Purpose: – There is a reason why S and H interact: they have some goal(s) they want to achieve – There is a reason why any part of the discourse is there: it contributes to achieving some goal(s) • S and H may have joint (shared) goals or individual (different) goals – Communication is inherently a collaborative activity: S and H work together to establish and achieve their goals • Cooperative Principle (Grice) – Make your contribution such as is required, at the stage at which it occurs, by the accepted purpose or direction of the talk exchange in which you are engaged • Maxims of Conversation – – – – – Cooking dinner together vs. Getting someone to come to a surprise party 2/15/06 2 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 3 2/15/06 Maxim of quality Maxim of quantity Maxim of relevance Maxim of manner Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) Cf. [1]: Chapter 19 4 Cohesion Coherence • Cohesive devices: linguistic means that make a discourse stick together – anaphoric expressions, discourse connectives, lexical chains … Two guys were working for the city. One would furiously dig a hole, then the other would come behind him and quickly fill the hole. They were drenched in sweat. Two guys were working for the city. One would furiously dig a hole, then the other would come behind him and quickly fill the hole. They were drenched in sweat. Two guys were working for the city. He likes cake. A townhall is near a river. 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) = Making sense together, as a whole: the parts contribute in a meaningful way 5 Two guys were working for the city. He likes cake. A townhall is near a river. 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 6 Outline • • • • • Anaphoric reference Discourse relations Discourse structure Speech acts Grounding Anaphoric Reference Basic reading: [1]: Chapter 18, Section 18.1 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 7 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 8 Reference Discourse Model • Universe of discourse entities introduced as “referents” of linguistic expressions Objects in reality – Operations: • Evoke (new) discourse entity • Access (old) discourse entity Entities in a discourse model – Discourse entity status: • New, old, inferable – Basic types of discourse entities: • Objects (concrete/abstract) vs. events (states) • Basic types of referring expressions: – Noun phrases, pronouns – Temporal and spatial expressions Referring expressions A guy just finished running. 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 9 2/15/06 Anaphoric Reference Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 10 Anaphoric Reference Objects in reality • Coreference – Anaphoric expression refers to the same entity as its antecedent (identity of referent) Entities in discourse model • Bridging evoke access evoke A guy just finished running. He is tired. The sweat is dripping from his body. Antecedent 2/15/06 Anaphoric expressions Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 11 – Anaphoric expression refers to a different entity than the antecedent there is an association relationship between the referents, e.g., part-whole, set-member, entity-attribute… (the entity is inferable) 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 12 Exercise Anaphora Resolution Two guys were working for the city. One would furiously dig a hole, then the other would come behind him and quickly fill the hole. They were drenched in sweat. A man watching from the sidewalk couldn't believe how hard they were working, but also couldn't understand what they were doing. Finally he said: "I'm confused. You dig a hole and then your partner comes behind you and fills it up again!" The digger leaned on his shovel and replied, "Oh yeah, it must look funny. You see, the lazy jackass who plants the trees is sick again today!" 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 13 • Anaphora is a device of language economy, it’s natural and mostly easy for humans • Why is it a problem for NLP? – – – – Information extraction (topic segmentation) Summarization Machine translation Dialogue systems: (1) U: Do any samples contain magnesium? S: Yes. R560 and R668. U: And do they contain ruthenium? (2) S: Do any samples contain magnesium? U: No. S: And do they contain ruthenium? 2/15/06 Anaphora Resolution • Criteria on antecedent/anaphor pairs: – Determine referents (= for each referring expression, determine how the discourse model is to be updated) • Task of anaphora resolution: – Identify anaphors – Identify antecedents – Identify relationships Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 15 Anaphora Resolution • Task of reference resolution: 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 16 – – – – – Agreement (person, gender, number) Syntactic relationships (binding) Lexical repetition (edit distance) Selectional restrictions on arguments Salience: recency, grammatical role, semantic orientation, etc. – Repeated mention count – Parallelism – World knowledge Cf. [1]: Chapter 18, pp. 678-684 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 17 Coreference Resolution Algorithms Anaphora Resolution • Search: • Simplest: recency-based: – Given an anaphor, systematically consider one potential antecedent after another • Best-first (in structured search-space) • Hard to optimize multiple decisions • Classification: – Given all potential anaphor-antecedent pairs, decide yes/no (and optionally assign score) • Compute&evaluate all pairs • Local and global optimization: machine learning 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 18 – Look back & rule out incompatible candidates • There are various other approaches: – – – – – Hobb’s 1978: syntactic search Brennan et al. 1987: Centering-based Lappin&Leass 1994: weighted salience factors Baldwin 1995: specialized high precision rules Recent machine learning methods Cf. [1]: Chapter 18: pp. 684-694 but do not need to know for exam 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 19 Exercise Two guys were working for the city. One would furiously dig a hole, then the other would come behind him and quickly fill the hole. They were drenched in sweat. A man watching from the sidewalk couldn't believe how hard they were working, but also couldn't understand what they were doing. Finally he said: "I'm confused. You dig a hole and then your partner comes behind you and fills it up again!" The digger leaned on his shovel and replied, "Oh yeah, it must look funny. You see, the lazy jackass who plants the trees is sick again today!" 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 20 Discourse Relations (coherence/ rhetorical relations) Basic reading: [1] Chapter 18, Section 18.2, 18.3; [1] Chapter 19, Section 19.4 [2] 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 23 Discourse Relations Discourse Relations • Various specific types of connections relating utterances in discourse • These connections add meaning beyond the propositional content of each of the segments alone • Sometimes they are explicitly signaled by discourse connectives and other markers 1. Two guys were working for the city. Elaboration 2. One would furiously dig a hole, Occasion 3. then the other would come behind him Occasion 4. and quickly fill the hole. Result 5. They were drenched in sweat. 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 24 2/15/06 Discourse Relations Two guys were working for the city. One would furiously dig a hole, then the other would come behind him and quickly fill the hole. (As a result) They were drenched in sweat. A man watching from the sidewalk couldn't believe how hard they were working, but also couldn't understand what they were doing. Finally he said: "I'm confused. You dig a hole and then your partner comes behind you and fills it up again!" The digger leaned on his shovel and replied, "Oh yeah, it must look funny. You see, the lazy jackass who plants the trees is sick again today!" – Peter eats vegetables. He is healthy. • Explanation(b,a): eb causes ea – Peter is healthy. He eats vegetables. • Elaboration(b,a): eb included in ea – Peter went to the mountains. He skied every day. • Occasion: ea before eb – Peter bought skis. He went to the mountains. • Parallel(a,b): ea and eb are similar – Peter eats vegetables. Paul regularly sports. Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 25 Exercise • Result(b,a): ea causes eb 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 26 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 27 Discourse Relation Resolution Discourse Relation Recognition • Knowledge-intensive (suitable in limited domains): • Why? – Information extraction (topic segmentation) – Summarization – Dialogue systems: e.g., revision vs. occasion U: OK. That’s good. Now I’d like you to find and show the song Jingle Bells. Please search for the song Jingle Bells. – Inference-based: • Encode discourse relations as axioms • Construct a proof for discourse (abduction) – Plan-based • Encode discourse relations as plan operators • Instantiate plan for discourse goal • Knowledge-poor (suitable on large scale): – Discourse grammar-based (brittle) • Encode discourse relations as structural rules • Parse discourse – Shallow use of various surface clues (robust) • Supervised machine learning (needs annotated data) (cf. work by Daniel Marcu or Simone Teufel) 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 28 2/15/06 Discourse Structure 1. Two guys were working for the city. 2. One would furiously dig a hole, 3. then the other would come behind him 4. and quickly fill the hole. 5. They were drenched in sweat. – Subordination (embedding) e.g., elaboration, result, explanation – Coordination (linear precedence) e.g., parallel, occasion Elaboration Result • Recursively built discourse segments • Each segment has a purpose Occasion 1 Cf. [1]: 18.3, 19.4 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 29 Discourse Structure • Discourse is not just a flat linear sequence of utterances, but has hierarchical structure 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 30 2/15/06 2 3 4 5 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 31 Three Layers of Discourse Structure Discourse Structure • Discourse segment recognition Grosz and Sidner 1986: • Linguistic structure – Discourse markers (cue phrases), e.g., now, well, so – Prosodic indicators: – Segments marked by linguistic means • Change of pitch range – decreasing within segment, reset at boundary • Intentional structure • Speed – Faster for embedded segment – Hierarchically related discourse purposes • Pauses at boundaries • Attentional structure – Tense and mood changes – Topic changes tend to correlate with segment boundaries – Use of anaphoric expressions – Stack of “focus spaces” (accessible entities) • Antecedents accessible within segment • Antecedent inaccessible across segment boundaries Cf. [2]: Section 11.3 but need not know for exam 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 32 Cf. [2]: p. 442 but need not know for exam The levels are mutually co-constraining. 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 33 Global Discourse Structure • Particular discourse genres typically exhibit regular structural patterns – Scientific paper: abstract, introduction, body sections, related work, conclusions – Story: introduction, climax, ending – Recipe: ingredients, procedural steps, serving suggestions – News article: summary, detailed story – Telephone call: greetings, body section(s), closing –… 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 34 Speech Acts (Dialogue Acts/Moves) Basic Reading: [1] Chapter 19 (need know know details of 19.3 and 19.5) [3] (need not know details) 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 35 Speech Acts Speech Act Types Speech act theory [Austin, Searle] Assertive S commits to sth being the case Comment, suggest, swear, boast, conclude Directive S attempts to get H do sth Ask, order, request, beg, invite, advise Commissive S commits to future course of action Promise, plan, vow, bet, oppose Expressive S expresses psychological state Thank, apologize, welcome, deplore Declarations S changes world Resign, name, fire – how to do things with words • Utterances bring about acts on context – Locutionary act: the act of uttering the words with their semantic content – Illocutionary act: the communicative act the speaker intends to perform by saying the words = speech act – Perlocutionary act: the act that occurs as a result/effect of the utterance, intended or unintended (e.g., making someone laugh, scared…) 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 36 2/15/06 Conversation Structure • Why? – Expected SAs – To determine what user wants – And to decide about an appropriate response, e.g., accept/reject statement vs. fulfill/turn-down request • E.g., Opening, body, closing of telephone call(s) • Some SA sequences occur regularly, are even conventionalized (obligation to respond, preferred responses) • How do we decide what DA a user input is, e.g., statement vs. info-request Greeting-greeting Question-answer Compliment-downplayer Accusation-denial Offer-acceptance Request-grant … 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 37 Speech Act Recognition • Common overall organization – – – – – – – Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) – At first glance, this looks simple: different syntax: • Yes-no-questions have subj-verb inversion • Statements have declarative syntax • Commands have imperative syntax • However, the mapping between surface form and illocutionary act is not one-to-one 38 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 39 Speech Act Recognition Speech Act Recognition • For example, what looks like a yes/no question • Another example of “indirectness”: Can you give me a list of the flights from A to B A: That’s the telephone. B: I’m in the bath. A: OK. Can be a polite form of directive or request Please give me a list of flights from A to B • What looks like a statement And you said you wanted to travel next week • Can be paraphrased as follows: Can actually be a question, used to verify sth. (intonation?) 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) A requests B to perform action (answer phone) B states reason why he cannot comply (in bath) A undertakes to perform action (answer phone) 40 Speech Act Recognition 41 • Plan-based interpretation – Literal meaning (direct speech act) – Idiomatic meaning (indirect speech act) the grammar lists idiomatic meanings for each construction, e.g., “Can you X?” has request as one possible meaning • Inferential model: indirect speech acts arrived at by inference Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) Automatic SA Recognition • Idiom-based model: 2/15/06 2/15/06 42 – Essentially the inference model, differences lie in amount and depth of actual reasoning – Symbolic – Requires hand-coding and domain-knowledge • Cue-based recognition – Essentially derived from the idiom model – Using a combination of utterance features and context features (supervised machine learning methods) – Requires hand-annotated data 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 43 Example/Exercise Grounding Basic reading: [1] Chapter 19 [4] Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 44 Grounding Intention S proposes project w H considers project w Proposition S signals that p H recognizes that p Signal S presents signal s H identifies signal s Channel S executes behavior t H attends to behavior t Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 45 Positive Grounding Feedback • Establishing common ground • Levels of interpretation (Clark 1996): 2/15/06 2/15/06 46 • Continued attention • Relevant next contribution • Acknowledgement (nod or “continuer”, e.g., uhhuh, yeah; or assessment, e.g., that’s great) • Demonstration (by paraphrasing, reformulating or cooperatively completing) • Display (verbatim repetition) 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 47 weaker 2/15/06 Positive Grounding Feedback A: I'm confused. You dig a hole and then your partner comes behind you and fills it up again! B: Oh yeah, it must look funny. You see, the lazy jackass who plants the trees is sick again today! 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 48 Grounding Problems • Grounding problems are due to – – – – Lack of perception or understanding Ambiguity Conflicts (differences in beliefs) Misunderstanding (misinterpretation) • Clarification and repair strategies, e.g., ask for clarification, repetition, rephrase 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) Modeling Grounding Acts Grounding Acts • Traum (1999): grounding acts • Traum (1999) Cont(I, Uk ,DUi ) | SelfRepair(I, Uk, DUi) DUi: S Init(I, Uk ,DUi ) 1 Ack(R, Uk ,DUi ) Cancel(I, Uk ,DUi ) Cont(I, Uk ,DUi ) | SelfRepair(I, Uk, DUi) DUi: F S Cancel(I, Uk ,DUi ) Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) Init(I, Uk ,DUi ) 1 Ack(R, Uk ,DUi ) Cancel(I, Uk ,DUi ) D (1) 1:A: Move the boxcar to Corning Init(A,1,DU1) 2:A: and load it with oranges Cont(A,2,DU1) 3:B: OK Ack(B,3,DU1) 2/15/06 49 50 F Cancel(I, Uk ,DUi ) D (1) 1:A: Move the boxcar to Corning Init(A,1,DU1) 2:: and load it with oranges Cont(A,2,DU1) 3:B: OK Ack(B,3,DU1) 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 51 DUi: S Grounding Acts Grounding Acts Cont(I, Uk ,DUi ) | SelfRepair(I, Uk, DUi) Cont(I, Uk ,DUi ) | SelfRepair(I, Uk, DUi) Init(I, Uk ,DUi ) 1 Ack(R, Uk ,DUi ) DUi: F S [REPAIR(R, Uk ,DUi )] [REQREPAIR(R, Uk, DUi)] Cancel(I, Uk ,DUi ) (5) 1:A: Move the boxcar to Bath 2:B: Bath? 3:A: Oh, Corning. 4:B: OK 2/15/06 Init(I, Uk ,DUi ) 1 Ack(R, Uk ,DUi ) F [REPAIR(R, Uk ,DUi )] [REQREPAIR(R, Uk, DUi)] D Cancel(I, Uk ,DUi ) Cancel(I, Uk ,DUi ) Init(A,1,DU1) ReqRepr(B,2,DU1) ! Init(B,2,DU2) Ack(A,3,DU2) ! Repair(A,3,DU1) Ack (R,4,DU1) Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 52 D (4) 1:A: Move the boxcar to Bath 2:B: To Corning 3:A: Oh, sure. 2/15/06 Example/Exercise Cancel(I, Uk ,DUi ) Init(A,1,DU1) Repair(B,2,DU1) ! Init(B,2,DU2) Ack (A,3,DU2) Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 53 Wrap-Up • Language use is rife with challenging discourse level phenomena: – – – – Anaphoric reference Discourse relations Speech acts Grounding acts • Co-constraining aspects: structure, attention, intention • Interpretation – Ultimately requires inference and world knowledge • Possible in limited domains – Can be approximated using surface clues (robust, large scale) • Generation (see [1]: Chapter 20, Section 20.4) – Naturalness, economy --> easy to understand for users – Approximation according to available resources/information 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 54 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 55 Basic Reading [1] D. Jurafsky and J. Martin (2000): Speech and Language Processing Chapters 18, 19, 20. pp. 669-798. [2] B. Grosz, M. Pollack and C. Sidner (1989): Discourse. In Foundations of Cognitive Science. M. Posner (ed.). MIT Press. pp. 437-468. [3] D. Jurafsky (2006): Pragmatics and Computational Linguistics. In: Handbook of Pragmatics, L.R. Horn and G. Ward (eds.). Oxfrod: Blackwell. http://www.stanford.edu/~jurafsky/prag.pdf (draft v.) [4] D. Traum (1999): Computational Models of Grounding in Collaborative Systems. AAAI Fall Symposium on Psychological Models of Communication. Pp. 124-131. http://people.ict.usc.edu/~traum/Papers/psych.ps 2/15/06 Ivana Kruijff-Korbayová: Discourse&Dialogue (Foundations of Speech and Language Technology) 56