Module com.maroontress.clione
Package com.maroontress.clione

Interface LexicalParser

All Superinterfaces:
AutoCloseable
All Known Implementing Classes:
DefaultLexicalParser
public interface LexicalParser extends AutoCloseable
The lexical parser.

The LexicalParser object creates and returns a token from the stream of the source file. It often extracts the ones from the source file, but trigraph and digraph substitution and line concatenation may result in tokens that are not in the source file. It returns an empty token when it finally reaches the end of the source file.

The Token objects that the next() method returns are the preprocessing tokens. So, the evaluation is necessary before using their content. In other words, they can be incomplete according to the token type. For example, the string literal or comment may not terminate, the preprocessing number may not represent valid integer and floating-point constants, and so on.

  • Method Details

    • close

      void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Throws:
      IOException

    • getEof

      Optional<SourceChar> getEof() throws IOException
      Returns the character representing EOF.

      Note that there is no need for this method in most cases. If you want to detect when the line concatenation (a backslash followed by a newline character) is immediately followed by EOF, you can do so as follows:

              void parse(LexicalParser parser) {
            for (;;) {
                var maybeToken = parser.next();
                if (maybeToken.isEmpty()) {
                    var maybeEof = parser.getEof();
                    assert(maybeEof.isPresent());
                    var eof = maybeEof.get();
                    assert(eof.isEof());
                    var list = eof.getChildren();
                    if (list.size() > 0) {
                        var m = "backslash-newline at end of file";
                        System.err.println(eof.getSpan() + ": warning: " + m);
                    }
                    break;
                }
                ...
            }
        }
      Returns:
      The character representing EOF. Or Optional.empty() if this parser has not yet reached EOF.
      Throws:
      IOException - If an I/O error occurs.

    • getLocation

      SourceLocation getLocation()
      Returns the current location of the source file.
      Returns:
      The current location.

    • next

      Optional<Token> next() throws IOException
      Returns the next token.
      Returns:
      The next token. Or Optional.empty() if this parser reaches EOF.
      Throws:
      IOException - If an I/O error occurs.

    • of

      static LexicalParser of(Reader reader)
      Returns a new LexicalParser object.

      The instance considers Keywords.C11 as reserved keywords.

      Parameters:
      reader - The reader that provides the stream of the source file.
      Returns:
      The new LexicalParser object.

    • of

      static LexicalParser of(Reader reader, Collection<String> reservedWords)
      Returns a new LexicalParser object with the specified reserved words.
      Parameters:
      reader - The reader that provides the stream of the source file.
      reservedWords - The collection that contains reserved words. Note that the constructor copies the collection, so changes to the collection do not affect this instance.
      Returns:
      The new LexicalParser object.