Ignoring Whitespace
FrontEnd, Iteration 2

In the previous iteration, the CIAT tests forced our hand in dealing with whitespace. Let's embrace this hand in this iteration and deal with it completely.

5s with Whitespace

Use only the number 5 in these programs so that you can reuse them for Schobbes tests. Use a variety of spaces, tabs, and newlines in the source programs.

The key question, though, is what should the output be? Hint: the Hobbes front end ignores all whitespace.

You should get complaints that spaces and tabs aren't recognized. But all newlines are okay.

Fixing the Grammar

First, a unit test with some assertions.

Hint: use \n for newlines, an actual space character for the space character, and \t for a tab.

You're already ignoring newlines from this definition:

WHITESPACE
  : '\n'
    { skip(); }
  ;

When you ignore input with the skip() method, it skips all of the input that matches. So all newlines in your input is ignored. It seems logical, then, to add the other characters to this lexer rule.

Expressed in English, what you want is this: if the input character is a space or a tab or a newline, skip it. The "or" relationship in an ANTLR lexer is accomplished with the alternation operator, |. If I wanted to ignore the letters a, b, and c, I'd write ('a' | 'b' | 'c').

Make sure you put parentheses around the alternation. This is needed in this definition because you want the Java code to be attached to the whole rule (not just the last option in the alternation).