Lexical Analyser Parser

LEXICAL ANALYZER AND PARSER
COMPILER
compiler is a program takes a program written in a source language and translates it into an equivalent program in a target language. source program COMPILER target ( Normally a program written in ( Normally the equivalent program in program a high-level programming language) machine code relocatable object file)
A
error messages
2
PHASES OF COMPILER
PHASES OF A COMPILER
Source Program
Lexical Analyzer
Syntax Semantic Intermediate Code Code Analyzer Analyzer Code Generator Optimizer Generator
Target Program
Each phase transforms the source program from one representation into another representation.
They communicate with error handlers. They communicate with the symbol table.
LEXICAL ANALYZER
INTRODUCTION
A lexical analyzer breaks an input stream of characters into tokens. Programs performing lexical analysis are called lexical analyzer or lexer. A lexer consists of scanner and tokenizer.
Writing lexical analyzers by hand can be a tedious process, so software tools have been developed to ease this task. Perhaps the best known such utility is Lex. Lex is a lexical analyzer generator for the UNIX operating system, targeted to the C programming language
ROLE OF THE LEXICAL ANALYZER
INTRODUCING BASIC TERMINOLOGY

What
are Major Terms for Lexical Analysis?
TOKEN A classification for a common set of strings Examples Include <Identifier>, <number>, etc. PATTERN The rules which characterize the set of strings for a token Recall File and OS Wildcards ([A-Z]*.*) LEXEME Actual sequence of characters that matches pattern and is classified by a token Identifiers: x, count, name, etc
8
The input program as you see it. main () { int i, sum; sum = 0; for (i=1; i<=10; i++); sum = sum + i; printf("%d\n",sum); }
10
11
LEXICAL ANALYZER RESPONSIBILITIES

Lexical
Scan
analyzer [Scanner]
input
Remove
white spaces,tabs,new line characters Remove comments Manufacture tokens Generate lexical errors Pass token to parser
13
14
15
LEX INTRODUCTION
Lex is one of the compiler writing tools, that is used to generate a lexical analyzer or scanner from description of tokens of programming language to be implemented. Lex takes a specially-formatted specification file containing the details of a lexical analyzer. This tool then creates a C source file for the associated tabledriven lexer.
LEX SPECIFICATION
Input
to the Lex is a text file containing regular expression along with the actions to be taken by the generated scanner when each regular expression is matched.
The
output is a file that contains C source code defining procedure yylex(),which implements DFA corresponding to regular expression given in input file.
The
output file is usually called lex.yy.c or lexyy.c, which when compiled linked to the main program acts as a scanner or lexical analyzer recognizing tokens specified by regular expression of the input file.
LEX SPECIFICATIONS
A Lex input file is consists of three parts, a collection of definitions, a collection of rules, and a collection of user subroutines. These three sections are separated by double-percent directives (``%%'').
A proper Lex specification has the following format.
LEX SPECIFICATIONS
{definition} %% {rules} %% {user subroutines}
Where the definition & the user subroutines are often omitted. The second %% is optional, but the first is required to mark the beginning of rules.
The input program as you see it. main () { int i, sum; sum = 0; for (i=1; i<=10; i++); sum = sum + i; printf("%d\n",sum); }
21
22
23
24
25
26
27
28
29
MAIN FEATURES
Simple
implementation. Fast lexical analysis. Efficient resource utilization. Portable.
APPLICATIONS AND FUTURE WORK

Text
Editing Text Processing Pattern Matching File Searching
PARSER
PARSING
Parsing
(syntactic analysis) is the process of analyzing a sequence of tokens to determine their grammatical structure with respect to a given (more or less) formal grammar.
YACC SPECIFICATION
Yacc
(yet another compiler compiler) is a parser generator, which is a program that takes as its input a specification of syntax of the programming language, and produces as its output a parse procedure for that language whose name is yyparse().
The
notation used for preparing this specification is a grammer(CFG).

Input
to yacc is a specification file usually with .y suffix, containing the rules of grammar specifying the structure of language to be implemented. The output is C source code for parser, usually in a file y.tab.c or ytab.c.
FORMAT OF SPECIFICATION FILE

{ definition } %% { rules } %% { programs } The definition section contains information about tokens, data types, and grammar rules. It also includes any C code that must go directly into the output file at its beginning.
CREDITS
Credits goes out to
A special thanks goes out to
THANK YOU!

Lexical Analyser Parser

Uploaded by

Copyright:

Available Formats

Lexical Analyser Parser

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lexical Analyser Parser

Uploaded by

Copyright:

Available Formats

LEXICAL ANALYZER AND PARSER

ROLE OF THE LEXICAL ANALYZER

INTRODUCING BASIC TERMINOLOGY

are Major Terms for Lexical Analysis?

LEXICAL ANALYZER RESPONSIBILITIES

implementation. Fast lexical analysis. Efficient resource utilization. Portable.

APPLICATIONS AND FUTURE WORK

Editing Text Processing Pattern Matching File Searching

notation used for preparing this specification is a grammer(CFG).

FORMAT OF SPECIFICATION FILE

Credits goes out to

A special thanks goes out to

You might also like