3
PDL::Dataflow -- description of the dataflow philosophy
7
perldl> $a = zeroes(10);
8
perldl> $b = $a->slice("2:4:2");
16
Dataflow is very experimental. Many features of it are disabled
17
for 2.0, particularly families for one-directional
18
dataflow. If you wish to use one-directional dataflow for
19
something, please contact the author first and we'll work out
20
how to make it functional again.
22
Two-directional dataflow (which implements ->slice() etc.)
23
is fully functional, however. Just about any function which
24
returns some subset of the values in some piddle will make a binding
28
$b = $a->slice("some parts");
31
also changes the corresponding element in $a. $b has become effectively
32
a window to some subelements of $a. You can also define your own routines
33
that do different types of subsets. If you don't want $b to be a window
36
$b = $a->slice("some parts")->copy;
38
The copying turns off all dataflow between the two piddles.
40
The difficulties with one-directional
41
dataflow are related to sequences like
46
where there are several possible outcomes and the semantics get a little
51
Dataflow is new to PDL2.0. The basic philosophy
52
behind dataflow is that
62
should work. It doesn't. It was considered that doing this
63
might be too confusing for novices and occasional users of the language.
64
Therefore, you need to explicitly turn on dataflow, so
71
produces the (un)expected result. The rest of this documents
72
explains various features and details of the dataflow implementation.
74
=head1 Lazy evaluation
76
When you calculate something like the above
82
nothing will have been calculated at this point. Even the memory for
83
the contents of $b has not been allocated. Only the command
87
will actually cause $b to be calculated. This is important to bear
88
in mind when doing performance measurements and benchmarks as well
89
as when tracking errors.
91
There is an explanation for this behaviour: it may save cycles
92
but more importantly, imagine the following:
102
Now, if $c were evaluated between the two resizes, an error condition
103
of incompatible sizes would occur.
105
What happens in the current version is that resizing $a raises
106
a flag in $c: "PDL_PARENTDIMSCHANGED" and $b just raises the same flag
107
again. When $c is next evaluated, the flags are checked and it is found
108
that a recalculation is needed.
110
Of course, lazy evaluation can sometimes make debugging more painful
111
because errors may occur somewhere where you'd not expect them.
112
A better stack trace for errors is in the works for PDL, probably
113
so that you can toggle a switch $PDL::traceevals and get a good trace
114
of where the error actually was.
118
This is one of the more intricate concepts of one-directional dataflow.
119
Consider the following code ($a and $b are pdls that have dataflow enabled):
127
What should $e and $f contain now? What about when $a is changed
128
and a recalculation is triggered.
130
In order to make dataflow work like you'd expect, a rather strange
131
concept must be introduced: families. Let us make a diagram:
140
This is what PDL actually has in memory after the first three lines.
141
When $d is changed, we want $c to change but we don't want $e to change
142
because it already is on the graph. It may not be clear now why you don't
143
want it to change but if there were 40 lines of code between the 2nd
144
and 4th lines, you would. So we need to make a copy of $c and $d:
153
Notice that we primed the original c and d, because they do not correspond
154
to the objects in $c and $d any more. Also, notice the dotted lines
155
between the two objects: when $a is changed and this diagram is re-evaluated,
156
$c really does get the value of c' with the diagonal incremented.
158
To generalize on the above, whenever a piddle is mutated i.e.
159
when its actual *value* is forcibly changed (not just the reference:
163
would produce a completely different result ($c and $d would not be bound
168
would yield the same as $d++), a "family" consisting of all other piddles
169
joined to the mutated piddle by a two-way transformation is created
170
and all those are copied.
172
All slices or transformations that simply select a subset of the original
173
pdl are two-way. Matrix inverse should be. No arithmetic
178
What you were told in the previous section is not quite true:
179
the behaviour described is not *always* what you want. Sometimes you
180
would probably like to have a data "source":
182
$a = pdl 2,3,4; $b = pdl 5,6,7;
186
Now, if you know that $a is going to change and that you want
187
its children to change with it, you can declare it into a data source
188
(XXX unimplemented in current version):
192
After this, $a++ or $a .= something will not create a new family
193
but will alter $a and cut its relation with its previous parents.
194
All its children will follow its current value.
196
So if $c in the previous section had been declared as a source,
197
$e and $f would remain equal.
201
A dataflow mechanism would not be very useful without the ability
202
to bind events onto changed data. Therefore, we provide such a mechanism:
207
> $c->bind( sub { print "A now: $a, C now: $c\n" } )
209
A now: [2,3,4], C now: [6 8 10]
213
A now: [1,1,4], C now: [4 4 10]
215
Notice how the callbacks only get called during PDL::dowhenidle.
216
An easy way to interface this to Perl event loop mechanisms
217
(such as Tk) is being planned.
219
There are many kinds of uses for this feature: self-updating graphs,
222
Bla bla bla XXX more explanation
226
Dataflow as such is a fairly limited addition on top of Perl.
227
To get a more refined addition, the internals of perl need to be
228
hacked a little. A true implementation would enable flow of everything,
243
At the moment we only have the first two (hey, 50% in a couple of months
244
is not bad ;) but even this is useful by itself. However, especially
245
the last one is desirable since it would add the possibility
246
of flowing closures from place to place and would make many things
249
To get the rest working, the internals of dataflow probably need to
250
be changed to be a more general framework.
252
Additionally, it would be nice to be able to flow data in time,
253
lucid-like (so you could easily define all kinds of signal processing
258
Copyright(C) 1997 Tuomas J. Lukka (lukka@fas.harvard.edu).
259
Redistribution in the same form is allowed provided that the copyright
260
notice stays intact but reprinting requires
261
a permission from the author.
added
removed
Basic/Pod/Dataflow.pod
