Description

(CS 753)
Samit Biswas
samit@cs.iiests.ac.in
Department of Computer Science and Technology,
Indian Institute of Engineering Science and Technology, Shibpur
lex/flex
I We refer to the tool as Lex compiler , and to its input specification as the Lex language.
flex (and lex): Overview
Scanner generator
I Helps write programs whose control flow is directed by instances of regular expressions in the input stream.

flex input format
A Lex program (the lex.l file ) consists of three parts:
definitions required %%
rules optional
%% user code
I Shortest possible legal flex input:
%%
Definitions
I name definitions, each of the form
name definition
e.g.:
I name definitions, each of the form
DIGIT [0-9]
CommentStart “/*”
ID [a-zA-Z][a-zA-Z0-9]* I stuff to be copied verbatim into the flex output enclosed in %{ ….. }%, or (e.g., declarations, #includes):
Lex Rules
I The rules portion of the input contains a sequence of rules.
I Each rule has the following form Regular Expression Action where:
I Regular Expression describes a pattern to be matched on the input.
I action must begin on the same line.
Example of Lex Rules
< Reg.Exp > < action >
Pattern Action
int printf(“Keyword: INTEGER ”);
[0-9]+ printf(“Number ”);
Patterns
I Essentially, extended regular expressions.
I Syntax: similar to grep (see man page). I << EOF >> to match “end of file”.
I Character classes:
[:alpha:], [:digit:], [:alnum:], [:space:], etc. (see man page).
I {name} where name was defined earlier.

A flex program to read a file of (positive) integers and compute the average:

A flex program to read a file of (positive) integers and compute the average:

A flex program to read a file of (positive) integers and compute the average:

Example
A flex program to read a file of (positive) integers and compute the average:

Lex library routines
I yylex()
The default main() contains a call of yylex() I yymore() return the next token. I yyless(n) retain the first n characters in yytext.
I yywrap()
I is called whenever Lex reaches an end-of-file.
I The default yywarp() always returns 1.
Lex Predefined Variables
I yytext – a string containing the lexeme
I yyleng – the length of the lexeme
I yyin – the input stream pointer
– the default input of default main() is stdin
I yyout – the output stream pointer
-the default output of default main() is stdout
In lex program, a main() function is generally included as:
main(){
yyin = fopen(filename,“r00); while(yylex());
}
I Here filename corresponds to input file and the yylex() routine is called which returns the tokens. yyin is FILE pointer declared by Lex part.
Lexical Analyser Generators — Lex

Sequence input stream of Tokens
Assignment
Implement a lexical analyzer using the tool: lex/flex for the following types of tokens:
I Arithmetic, Relational, Logical, Bitwise and Assignment Operators of C.
I Reserved words: int, float, char, for, while, if and else I Identifier.
I Integer Constants.
I Parentheses, Curly braces
Take a complete C program as input and generate the above-mentioned tokens.
tokendef LEX.h