variable_grammar | ![]() |
![]() |
![]() |
![]() |
Consider what happens when the parser encounters a variable declaration such as x=2. Using Spirit, the parser for this statement can be written as:
spirit::rule<ScannerT> var_assignment;
expression_grammar expression;
name_grammar name;
var_assign = name >> '=' >> expression;
Having parsed this statement, the code should do two things:
0 variable_node x
1 const_value_node 2
2 assign_node
In order for expression_grammar to parse an expression successfully, it must have access to lists of all existing variables and functions. If successful, it should make available a stack representation of the data. If we use closures, then 'make available' becomes 'return'. The grammar's public interface follows naturally:
using namespace boost::spirit;
using boost::shared_ptr;
namespace yac {
struct stack_closure : closure<stack_closure, stack>
{
member1 stk;
};
struct expression_grammar
: spirit::grammar<expression_grammar, stack_closure::context_t>
{
typedef spirit::symbols<boost::shared_ptr<function> > function_table_t;
typedef spirit::symbols<double> var_table_t;
function_table_t const & functions;
var_table_t const & global_vars;
expression_grammar(function_table_t const & funcs,
var_table_t const & gvars)
: functions(funcs), global_vars(gvars)
{}
};
} // namespace yac
That is, the grammar receives const references to symbols tables for both functions and variables. These tables are not altered during the parsing of the expression but are used to ascertain whether a given symbol is allowable.
In fact, the code above does not tell quite the whole story; expression_grammar has a second constructor also that takes an additional var_table_t reference. This second constructor is used when parsing a function body, the second var_table_t parameter being a symbol table of variables local to the function.
YAC variable and function names must conform to the same naming rules, so name_grammar will be used by both. The semantic actions that are performed by the variable and function parsers are, however, rather different.
In the case in question, the name_grammar variable name will be used to update the global_vars symbol table that is accessed by the expression_grammar variable expression. However, the symbol table should not be updated until after expression itself is parsed. Otherwise definitions such as x = x + 1 would be legal. (OK as an assignment but not as the original declaration of x.)
This contrasts with the behaviour of the function_grammar which should add a function name to the symbol table of available functions before attempting to parse the defining expression. (To enable recursive functions to be defined.)
Clearly, therefore, these semantic actions should not be factored into the name_grammar class itself. It's public interface is, therefore, trivial:
struct name_grammar : spirit::grammar<name_grammar>
{
name_grammar() {}
};
Spirit will return a pair of iterators to the beginning and end of the variable or function name. Thereafter, it's up to the parent grammar to use these data appropriately.
The public interface of the variable_grammar class should fulfill the requirements outlined at the top of this page. It should 'return' a stack wrapped up as a closure. It's constructor should receive a non-const reference to the global variables symbol table. Finally, this same constructor should also receive a const reference to the functions symbol table to enable the grammar to parse expression:
using namespace boost::spirit;
using boost::shared_ptr;
namespace yac {
struct stack_closure : closure<stack_closure, stack>
{
member1 stk;
};
struct variable_grammar
: grammar<variable_grammar, stack_closure::context_t>
{
typedef symbols<shared_ptr<function_node> > function_table_t;
typedef symbols<double> var_table_t;
function_table_t const & functions;
var_table_t & vars;
variable_grammar(function_table_t const & f, var_table_t & v)
: functions(f), vars(v)
{}
template <typename ScannerT>
struct definition;
};
} // namespace yac
We know that variable_grammar::definition will use name_grammar and expression_grammar variables to parse the input data and we know what arguments these grammars' constructors require:
template <typename ScannerT>
struct definition
{
definition(variable_grammar const & self)
: name(self.vars),
expression(self.functions, self.vars)
{
...
}
private:
name_grammar name;
expression_grammar expression;
};
Two sets of semantic actions should be invoked when parsing a variable declaration. First, the parsed data should be stored locally. Thereafter, the global symbol tables and expression stack should be updated. The first step is encoded quite simply:
namespace yac {
struct var_closure
: spirit::closure<var_closure, std::string, stack>
{
member1 name;
member2 stk;
};
typename spirit::rule<scanner_t, var_closure::context_t> step1;
typename spirit::rule<ScannerT> step2;
...
step2
= step1
[
// semantic actions to fill the
// world-visible data structures.
]
;
step1
= name
[
step2.name = construct_<std::string>(arg1, arg2)
]
>> '='
>> expression
[
step2.stk = arg1
]
;
This code snippet is a perfect example of Phoenix's lambda facilities in action. They can lead to extremely clear code. Note, however, that the semantic actions attached to step1 are updating the closure variables attached to step2. The code was split into step1 and step2 simply to ease readability. There are no problems with doing so because this grammar is not recursive. If it were, then the closures would have to be attached to step1 and these data would then be returned to step2. Note, however, that it is the member1 closure variable that is returned in such cases, so we would need a compound data structure to store the data. In fact, it would be simpler to attach all the semantic actions to the step1 rule, discarding step2 entirely.
Having generated the local data stores, the second stage of the process is to use this stored data to update the global variables symbol table self.global_vars and the closure self.stk that is 'returned' by the variable_grammar:
using phoenix::arg1;
using phoenix::if_;
using phoenix::var;
step2
= step1
[
if_(!find_symbol(var(self.vars), step2.name))
[
add_symbol(var(self.vars), step2.name)
]
]
[
push_back(self.stk, find_symbol(var(self.vars),
step2.name)),
self.stk += step2.stk,
push_back(self.stk, new_<assign_node>())
]
;
Whoaa! What's going on here?
The first semantic action, if_(!var(name_return.already_present)) [...], is an example of a Phoenix lazy statement. If the symbol is not already present in self.global_vars then we add it. find_symbol and add_symbol are — you've guessed it — lazy functions invoking spirit::symbols::find and spirit::symbols::add, respectively. The difference is that we wrote these lazy functions ourselves.
Writing a lazy function is as easy as:
struct add_symbol_impl {
template <typename T, typename Arg>
struct result
{
typedef void type;
};
template <typename SymbolT>
void operator()(SymbolT & symbols, string const & str) const
{
symbols.add(str.c_str());
}
};
phoenix::function<add_symbol_impl> const
add_symbol = add_symbol_impl();
That is, the phoenix::function<> instance wraps a struct that contains a nested result class template and a template operator() that invokes the desired member function/variable. In this case, we're not interested in chaining multiple adds together so result::type is a typedef for void. Note that the result template has the same number of template parameters as the operator() has arguments, here two. The lazy functions specific to YAC can be found here.
Having ensured that the vars symbol table contains the just-parsed symbol, we're now in a position to construct the self.stk closure 'returned' by variable_grammar. The lazy functions push_back and find_symbol wrap stack::push_back and spirit::symbols::find, respectively. += is a lazy operator invoking stack::operator+= and new_ is Phoenix's lazy function wrapper for operator new — more on this later.
The actual code for variable_grammar can be found here. I won't try and convince you that this example has been anything less than complicated. I would say, however, that it is trying to do something that is itself complicated. It is my belief that Spirit and Phoenix have enabled me to do so in a transparent manner. I can only hope that you believe so too.
![]() |
![]() |
![]() |
Copyright © 2004 Angus Leeming
Distributed under the Boost Software License,
Version 1.0. (See accompanying file
LICENSE_1_0.txt or copy at
http://www.boost.org/LICENSE_1_0.txt
)