There will be subsequent commits to refine semantics, esp. version-specific semantics,
as well as I/O functionality and restrictions.
Note: I'm getting white-space differences in the preprocessor test results,
which I'm not checking in. I think they need to be tagged as binary or something.
Added some const as well. This will remove camouflage of the next commit,
which will add the bulk of Array of Array semantics and functionality.
(Note the basic grammar and data structure is already in place.)
Now extensions required by preprocessor should be checked via
the ppRequireExtensions method. This is more clear and coherent
with the rest of the code.
After parsing a #include directive, we push a TokenizableString
which contains the content of the included file into the input
stack. Henceforth, tokens will be read from the newly pushed
TokenizableString. However, the scanner in TParseContext still
points to the previous input stream. We need to update the scanner
to point to the new input stream inside TokenizableString. Thus,
the setCurrent{String|Line|..} method in TParseContext updates
the status of the correct input stream. After finishing the newly
pushed TokenizableString, we need to restore the scanner to the
previous input stream.
This patch introduces a new extension, GL_GOOGLE_include_directive,
to enable support #include directives. It depends on the extension
GL_GOOGLE_cpp_style_line_directive.
When an include directive is recognized by the preprocessor, it
executes a callback on the filepath argument to obtain the file
contents. That way the compilation client can deal with the file
system, include paths, etc.
Currently only accepts quoted filepaths -- no angle brackets yet.
Expose a new method setStringsWithLengthsAndNames() in the interface
which allows the caller to set descriptive names for source strings.
These names can be used in error messages.
According to the GLSL spec, the second parameter to #line should be
an integer source string number and __FILE__ will be substituted
with the integer source string number currently processed. This
patch extends the syntax of #line and __FILE__. Now #line accepts
as the second parameter a filename string quoted by double quotation
marks. And if such a #line is set, __FILE__ will be substituted with
the currently set filename string. The implementation is done via
introducing a new extension GL_GOOGLE_cpp_style_line_directive using
the extension framework.
The purpose is to support cpp-style #line directives, which is
required by #include.
Fixes issue #25. (char 255 aliased to -1 and missing tests for end of input).
1) All layers of input scanning now share a single EndOfInput value.
This avoids translation of it across layers of encapsulation.
2) Some places looking for end of line were not stopping on EndOfInput.
3) Use of "char" for the input made char values > 127 be negative numbers.
This allowed for aliasing of 255 to -1, etc. This is fixed by using
unsigned char.
This is just for '\' that's not before a new line.
Note the specification says it has no use other than as line continuation,
but #error is a grey area. (There are no escape sequences.)
SourceLineSynchronizer is added for keeping track of the last
line number and output newlines appropriately if we switch to
a new line in the preprocessor.
It also removes some old code that ancient compilers used to need.
However, the main issue is getting access to hash functions for
unordered_map in portable way.
This simplification is a prelude to eliminating what I appear unnecessary
symbol inserts into tables when tokenizing in the preprecessor, which
show up as taking notable time. (Performance issue.) It also simply makes
the preprocessor easier to understand, which it is badly in need of.
TInputScanner advances its internal indices to the next character at
the end of get(), which means, after reading in the last character
in the user-provided shader string, internal index (currentSource)
will point to the next shader string (currentSource == numSources),
which doesn't exist. Then if a location setting method is called,
we will write to some out-of-bound memory.
A test case for this is "#line 10000\n". The eval() method in CPPline()
will evaluate 10000, but at the same time it reads in the next
token, '\n', and the currentSource will be numSources in TInputScanner.
Then a parseContext.setCurrentLine() is called, we are writing to
out-of-bound memory. Another test case will be "#line 10000 0\n".
