4
Sikuli Script can be used to perform **desktop surveillance**. In this tutorial, we will
5
go through a number of exercises to create scripts that can monitor the screen and
6
notify us when certain interesting visual event happens.
12
The first exercise is to create a Facebook app to periodically check our Facebook
13
homepage visually and see if a particular friend has recently updated the status
14
message. One easy way to detect this event is to look for the friend's face image on
15
our Facebook homepage. If found, the friend must have posted a new status message.
16
If not found, we should check back again in a few moments.
18
Let's implement this operation using a ``while:`` loop. First we need to capture a
19
screenshot of the friend's face image.
21
.. image:: facebook_capture.png
23
To check if the friend's face can be seen on the screen, we use :py:meth:`exists()
25
which returns True when the face image is found. We set the looping condition to Not
26
Found so that the while loop will terminate only when the face image is found. We
27
add a :py:func:`sleep(n) <sleep>` statement in the body of the loop to introduce a 5
28
second interval between attempts to look for the face image on the screen.
32
while not exists("obama.png"):
35
Alternatively, Sikuli provides a convenient :py:meth:`wait() <Region.wait>` function
36
that periodically checks the screen to wait for a given image pattern to appear.
37
Using wait(), the above code can be rewritten as:
41
wait("obama.png", FOREVER)
43
The constant *FOREVER* means we want Sikuli to wait indefinitely. If we do not want
44
to wait forever, we can replace *FOREVER* with a number to indicate the number of
45
seconds to wait until Sikuli should giveup.
49
wait("obama.png", 3600) # wait for an hour
51
After the while loop exits or the wait function returns, we can call
52
:py:func:`popup` to display a notification message.
56
popup("Obama has updated his message")
58
This will display a popup message that looks like below:
60
.. image:: facebook_popupmsg.png
62
Now we can run this Sikuli Facebook App, sit back and relax, and get notified when
63
our friend updates his message.
68
In the previous exercise, we wrote a script to detect an image's appearance. In this
69
exercise, we will do the opposite --- *detecting the disappearance of a visual
72
Skype is a great tool that allows us to stay in close contact with our friends even
73
if they are in remote parts of the world. However, there might be some unfortunate
74
circumstances we may want to avoid being seen online by a certain individual.
75
Perhaps the individual talks too much. Perhaps we owe the individual some money. It
76
would be nice to know when the individual is offline so that it is safe to get
77
online. While Skype does provide a feature to notify us when an individual is
78
online, there is no notification feature when the opposite happens.
80
An automatic logoff notifier would be desirable to deal with this situation. Let us
81
build this tool using Sikuli Script. Notice that if an individual is no longer
82
online, the combined visual pattern of the green status icon and the individual's
83
screen name will disappear. Thus, we can take a screenshot that includes both the
84
green icon and the screen name as follows.
86
.. image:: skype_capture.png
88
Then, we can write a Sikuli Script to watch for the disappearance of the screenshot
89
image we just captured.
93
while exists("vgod.png"):
96
popup("vgod just logged off")
98
This script is very similar to the one in the previous exercise. The only difference
99
is the removal of the NOT operator from the condition statement for the while loop,
100
since we are trying to do the opposite.
102
Another way to wait for the disappearance of an image is to use the
103
:py:meth:`waitVanish() <Region.waitVanish>` function. The above script can be
104
rewritten as follows:
108
waitVanish("vgod.png", FOREVER)
109
popup("vgod just logged off")
115
The third exercise is to build a bus arrival notification tool. For many bus riders,
116
online GPS-based tracking services are very useful. Instead of patiently standing
117
outside at a bust stop, braving the freezing wind in the winter or scorching sun in
118
the summer, riders can sit comfortably inside in front of their computers, checking
119
emails, updating Facebook status, or watching YouTube? videos, or what have you.
120
They only need to look at the map every few moments to check the location of the bus
121
symbol on the map. Only when the bus is close enough do they have to finally get out
122
and walk to the bus top.
124
Since we care about whether the bus is getting close to the stop, we only need to
125
look at the neighborhood around the stop. Thus, we can resize the browser to show
126
just that portion of the map, while leaving a lot of screen space to do something
127
else, in this case, reading CNN news.
129
.. image:: bus_fullscreen.png
131
Let us write a Sikuli Script to do the bus tracking for us. It is possible to define
132
a region and ask Sikuli Script to focus only on that region to search for a
133
particular visual pattern. This way, Sikuli don't have to waste precious computing
134
cycles scanning the whole screen. To do so, let us click on the "select a region"
135
button in the toolbar as indicated below.
137
.. image:: bus_toolbar_region.png
139
The entire screen will freeze and turn darker, similar to what happen in the screen
140
capture mode. Simply draw a rectangle to cover the entire neighborhood map. The
141
region covered by this rectangle is visually represented as a thumbnail image of the
142
entire desktop where the region is shaded in red.
144
.. image:: bus_region_thumbnail.png
146
Next, we capture the screenshot image of the bus symbol so that we can tell Sikuli
147
Script to watch for its appearance.
149
.. image:: bus_capture_bus_symbol.png
152
Now. we can write the following script to check the map and watch for the appearance
157
while not "region.png".inside().exists("bus.png"):
159
popup("bus has arrived")
161
Interpreting the meaning of the while loop is straightforward---while it is not the
162
case that a bus symbol can be found inside the region, sleep for 5 seconds. In other
163
words, the while loop will exit only when the bus symbol is found inside the region.
164
Then, the popup statement will be executed to notify us the bus has arrived.
166
Again, the same effect can be achived using :py:meth:`wait() <Region.wait>`. The
167
script can be rewritten as:
171
"region.png".inside().wait("bus.png",FOREVER)
172
popup("bus has arrived")
added
removed
docs/source/tutorials/surveillance/surveillance.rst
