The StringTokenizer class in Java divides a string into tokens by delimiters and allows iterating through the tokens. It takes a string, optional delimiter string, and flag. In a loop, hasMoreTokens() checks for more tokens and nextToken() returns each one for processing. Examples show tokenizing strings by whitespace, "-", and "-/" and printing the tokens.
The StringTokenizer class in Java divides a string into tokens by delimiters and allows iterating through the tokens. It takes a string, optional delimiter string, and flag. In a loop, hasMoreTokens() checks for more tokens and nextToken() returns each one for processing. Examples show tokenizing strings by whitespace, "-", and "-/" and printing the tokens.
The StringTokenizer class in Java divides a string into tokens by delimiters and allows iterating through the tokens. It takes a string, optional delimiter string, and flag. In a loop, hasMoreTokens() checks for more tokens and nextToken() returns each one for processing. Examples show tokenizing strings by whitespace, "-", and "-/" and printing the tokens.
The StringTokenizer class in Java divides a string into tokens by delimiters and allows iterating through the tokens. It takes a string, optional delimiter string, and flag. In a loop, hasMoreTokens() checks for more tokens and nextToken() returns each one for processing. Examples show tokenizing strings by whitespace, "-", and "-/" and printing the tokens.
Download as PPT, PDF, TXT or read online from Scribd
Download as ppt, pdf, or txt
You are on page 1of 6
String Tokenizer
Dividing an input string line into words
• In some applications, the program may need to divide the input line into words and then process each word separately. • Although it is not hard to write code that divides a string into words, it is easier still to make use of existing facilities in the Java library to perform this task. One strategy is to use the StringTokenizer class in the java.util package, which divides a string into independent units called tokens. The client then reads these tokens one at a time. The set of tokens delivered by the tokenizer is called the token stream. • The precise definition of what constitutes a token depends on the application. Tokens are either words or the characters that separate words, which are called delimiters. The StringTokenizer Class • The constructor for the StringTokenizer class takes three arguments, where the last two are optional: – A string indicating the source of the tokens. – A string which specifies the delimiter characters to use. By default, the delimiter characters are set to the whitespace characters. – A flag indicating whether the tokenizer should return delimiters as part of the token stream. By default, a StringTokenizer ignores the delimiters.
• Once you have created a StringTokenizer, you use it by
setting up a loop with the following general form: while (tokenizer.hasMoreTokens()) { String token = tokenizer.nextToken(); code to process the token } Example • The following code fragment prints the words of a given string, separated by whitespace characters: String line = ”java is great"; StringTokenizer tokenizer = new StringTokenizer(line); while (tokenizer.hasMoreTokens()) { String token = tokenizer.nextToken(); println(token); }
• Hence the output is:
java is great Example • The following code fragment prints the words of a given string, separated by ‘-’:
String line = ”java-is-great";
StringTokenizer tokenizer = new StringTokenizer(line,”-”); while (tokenizer.hasMoreTokens()) { String token = tokenizer.nextToken(); println(token); }
• Hence the output is:
java is great Example • The following code fragment prints the words of a given string, separated by ‘-’ or ‘/’:
String line = ”java-is/great";
StringTokenizer tokenizer = new StringTokenizer(line,”-/”); while (tokenizer.hasMoreTokens()) { String token = tokenizer.nextToken(); println(token); }