85
85
* Backslashes themselves must also be backslashed for consistency.
86
86
* Any other character can be, but need not be, backslashed as well.
87
87
* * If the resulting token is '<>' (with no backslash), it is returned
88
* as a non-NULL pointer to the token but with length == 0. Note that
88
* as a non-NULL pointer to the token but with length == 0. Note that
89
89
* there is no other way to get a zero-length token.
91
91
* Returns a pointer to the start of the next token, and the length of the
92
* token (including any embedded backslashes!) in *length. If there are
92
* token (including any embedded backslashes!) in *length. If there are
93
93
* no more tokens, NULL and 0 are returned.
95
95
* NOTE: this routine doesn't remove backslashes; the caller must do so
96
96
* if necessary (see "debackslash").
98
98
* NOTE: prior to release 7.0, this routine also had a special case to treat
99
* a token starting with '"' as extending to the next '"'. This code was
99
* a token starting with '"' as extending to the next '"'. This code was
100
100
* broken, however, since it would fail to cope with a string containing an
101
101
* embedded '"'. I have therefore removed this special case, and instead
102
* introduced rules for using backslashes to quote characters. Higher-level
102
* introduced rules for using backslashes to quote characters. Higher-level
103
103
* code should add backslashes to a string constant to ensure it is treated
104
104
* as a single token.
259
259
* Slightly higher-level reader.
261
261
* This routine applies some semantic knowledge on top of the purely
262
* lexical tokenizer pg_strtok(). It can read
262
* lexical tokenizer pg_strtok(). It can read
263
263
* * Value token nodes (integers, floats, or strings);
264
264
* * General nodes (via parseNodeString() from readfuncs.c);
265
265
* * Lists of the above;