17 StringTokenizer

Download as ppt, pdf, or txt
Download as ppt, pdf, or txt
You are on page 1of 6

String Tokenizer

Dividing an input string line into words


• In some applications, the program may need to divide the
input line into words and then process each word separately.
• Although it is not hard to write code that divides a string into
words, it is easier still to make use of existing facilities in the
Java library to perform this task. One strategy is to use the
StringTokenizer class in the java.util package, which
divides a string into independent units called tokens. The
client then reads these tokens one at a time. The set of tokens
delivered by the tokenizer is called the token stream.
• The precise definition of what constitutes a token depends on
the application. Tokens are either words or the characters that
separate words, which are called delimiters.
The StringTokenizer Class
• The constructor for the StringTokenizer class takes three
arguments, where the last two are optional:
– A string indicating the source of the tokens.
– A string which specifies the delimiter characters to use. By
default, the delimiter characters are set to the whitespace
characters.
– A flag indicating whether the tokenizer should return delimiters
as part of the token stream. By default, a StringTokenizer
ignores the delimiters.

• Once you have created a StringTokenizer, you use it by


setting up a loop with the following general form:
while (tokenizer.hasMoreTokens()) {
String token = tokenizer.nextToken();
code to process the token
}
Example
• The following code fragment prints the words of a given
string, separated by whitespace characters:
String line = ”java is great";
StringTokenizer tokenizer = new StringTokenizer(line);
while (tokenizer.hasMoreTokens()) {
String token = tokenizer.nextToken();
println(token);
}

• Hence the output is:


java
is
great
Example
• The following code fragment prints the words of a given
string, separated by ‘-’:

String line = ”java-is-great";


StringTokenizer tokenizer = new StringTokenizer(line,”-”);
while (tokenizer.hasMoreTokens()) {
String token = tokenizer.nextToken();
println(token);
}

• Hence the output is:


java
is
great
Example
• The following code fragment prints the words of a given
string, separated by ‘-’ or ‘/’:

String line = ”java-is/great";


StringTokenizer tokenizer = new StringTokenizer(line,”-/”);
while (tokenizer.hasMoreTokens()) {
String token = tokenizer.nextToken();
println(token);
}

• Hence the output is:


java
is
great

You might also like