All Packages  Class Hierarchy  This Package  Previous  Next  Index

Class sdsu.util.TokenCharacters

java.lang.Object
   |
   +----sdsu.util.TokenCharacters

public class TokenCharacters
extends Object
This class maintains special characters used in parsing strings into tokens. It keeps track of whitespace, characters to indicate the start of a comment, quote characters used to quote tokens that contain special characters, an escape character, and characters that separate tokens.

Author:
Roger Whitney (whitney@cs.sdsu.edu)
See Also:
SimpleTokenizer, Stringizer

Variable Index

 o COMMENT_CHAR
Default character (#) used to indicate start of comment.
 o ESCAPE_CHAR
Character (/) used to preceed a quote character or escape character in a quoted token.
 o QUOTE_CHAR
Default character (') used to delineate the start and end of a quoted token.
 o WHITESPACE
Default characters treated as whitespace.

Constructor Index

 o TokenCharacters()
Create TokenCharacters with default values.
 o TokenCharacters(String)
Create TokenCharacters with given characters for token separators and default values for the rest of parameters.
 o TokenCharacters(String, String, char, char, String)
Create a TokenCharacters object with given values

Method Index

 o addQuoteChars(char, char)
Add the quote pair beginQuote-endQuote to the pairs recognized as char pairs to quote a token.
 o containsEscapeableChar(String)
Returns true if c needs to be escaped in a quoted token.
 o containsTokenTerminator(String)
Returns true if contains a character that indicates the end of a token.
 o escapeToken(String)
Places escape character before any quote character or the escape character Returns the modified token
 o getCommentChar()
Returns a character that indicates start of a comment.
 o isBeginQuote(char)
Returns true if c indicates the start of a quoted token
 o isComment(char)
Returns true if c indicates the start of a comment
 o isEndQuote(char)
Returns true if c indicates the end of a quoted token
 o isEOL(char)
Returns true if c is Mac, Unix, or PC EOL character
 o isEscape(char)
Returns true if c is an escape character
 o isQuotePair(char, char)
Returns true if beginQuote and endQuote are matching begin ending quote characters
 o isSeparator(char)
Returns true if c is a separator character
 o isTokenTerminator(char)
Returns true if c indicates the end of an unquoted token IE c is a whitespace, separator or comment character
 o isWhitespace(char)
Returns true if c is a whitespace character
 o quoteToken(String)
Surrounds token with begin-end quote pair Returns the quoted token
 o requiresEscaping(char)
Returns true if c needs to be escaped in a quoted token.
 o setSeparatorChars(String)
Set the current set of separators to newSeparators

Variables

 o WHITESPACE
 public static final String WHITESPACE
Default characters treated as whitespace. Default is space, tab, line feed (ascii 10, \n in unix) and carriage return ( ascii 13, usd as newline char on PC and Macs )

 o ESCAPE_CHAR
 public static final char ESCAPE_CHAR
Character (/) used to preceed a quote character or escape character in a quoted token.

 o COMMENT_CHAR
 public static final String COMMENT_CHAR
Default character (#) used to indicate start of comment.

 o QUOTE_CHAR
 public static final char QUOTE_CHAR
Default character (') used to delineate the start and end of a quoted token. Tokens are quoted when they contain special characters.

Constructors

 o TokenCharacters
 public TokenCharacters()
Create TokenCharacters with default values. You must set separators before using the new object.

 o TokenCharacters
 public TokenCharacters(String separators)
Create TokenCharacters with given characters for token separators and default values for the rest of parameters.

 o TokenCharacters
 public TokenCharacters(String separators,
                        String commentChars,
                        char beginQuoteChar,
                        char endQuoteChar,
                        String whitespace)
Create a TokenCharacters object with given values

Parameters:
commentChar - character used to indicate start of a comment
beginQuoteChar - character used to start a quote of a string containing special characters
endQuoteChar - character used to end a quote of a string containing special characters
whitespace - characters used for whitespace. Use null or empty string for no whitespace characters

Methods

 o addQuoteChars
 public void addQuoteChars(char beginQuote,
                           char endQuote)
Add the quote pair beginQuote-endQuote to the pairs recognized as char pairs to quote a token.

 o getCommentChar
 public char getCommentChar()
Returns a character that indicates start of a comment.

 o setSeparatorChars
 public void setSeparatorChars(String newSeparators)
Set the current set of separators to newSeparators

 o isEOL
 public boolean isEOL(char c)
Returns true if c is Mac, Unix, or PC EOL character

 o isEscape
 public boolean isEscape(char c)
Returns true if c is an escape character

 o isWhitespace
 public boolean isWhitespace(char c)
Returns true if c is a whitespace character

 o isSeparator
 public boolean isSeparator(char c)
Returns true if c is a separator character

 o isBeginQuote
 public boolean isBeginQuote(char c)
Returns true if c indicates the start of a quoted token

 o isEndQuote
 public boolean isEndQuote(char c)
Returns true if c indicates the end of a quoted token

 o isComment
 public boolean isComment(char c)
Returns true if c indicates the start of a comment

 o isTokenTerminator
 public boolean isTokenTerminator(char c)
Returns true if c indicates the end of an unquoted token IE c is a whitespace, separator or comment character

 o isQuotePair
 public boolean isQuotePair(char beginQuote,
                            char endQuote)
Returns true if beginQuote and endQuote are matching begin ending quote characters

 o requiresEscaping
 public boolean requiresEscaping(char c)
Returns true if c needs to be escaped in a quoted token. That is if c is a quote character or the escape character

 o containsEscapeableChar
 public boolean containsEscapeableChar(String token)
Returns true if c needs to be escaped in a quoted token. That is if c is a quote character or the escape character

 o escapeToken
 public String escapeToken(String token)
Places escape character before any quote character or the escape character Returns the modified token

 o containsTokenTerminator
 public boolean containsTokenTerminator(String token)
Returns true if contains a character that indicates the end of a token. That is whitespace, comment char or a separator

 o quoteToken
 public String quoteToken(String token)
Surrounds token with begin-end quote pair Returns the quoted token


All Packages  Class Hierarchy  This Package  Previous  Next  Index