6
Class Screen is there, to have a representation for a pysical monitor where the
7
capturing process (grabbing a rectangle from a screenshot, to be used for further
8
processing with find operations is implemented. For :ref:`Multi Monitor Environments
9
<MultimonitorEnvironments>` it contains features to map to the relevant monitor.
11
Since Screen extends class :py:class:`Region`, all methods of
12
class Region can be used with a screen object.
14
Of special interest might be the grouping of region method calls using ``with:`` in Multi Monitor
15
Environments: use it for other screens, than the default/primary screen, where
16
you have this feature by default.
18
Be aware, that using the whole screen for find
19
operations may have an impact on performance. So if possible either use ``setROI()`` or
20
restrict a find operation to a smaller region object (e.g. ``reg.find()``) to speed up
24
Screen: Setting, Getting Attributes and Information
25
---------------------------------------------------
29
.. py:method:: Screen([id])
31
Create a new Screen object
33
:param id: an integer number indicating which monitor in a multi-monitor
36
:return: a new screen object.
38
It creates a new screen object, that represents the default/primary monitor
39
(whose id is 0), if id is omitted. Numbers 1 and higher represent additional
40
monitors that are available at the time, the script is running (read for
43
Using numbers, that do not represent an existing monitor, will stop the
44
script with an error. So you may either use getNumberScreens() or exception
45
handling, to avoid this.
47
Note: If you want to access the default/primary monitor ( Screen(0) )
48
without creating a new screen object, use the constant reference SCREEN,
49
that is initiated when your script starts: SCREEN=Screen(0).
52
.. py:method:: getNumberScreens()
54
Get the number of screens in a multi-monitor environment at the time the
57
.. py:method:: getBounds()
59
Get the dimensions of monitor represented by the screen object.
61
:return: a rectangle object
63
The width and height of the rectangle denote the dimensions of the monitor
64
represented by the screen object. These attributes are obtained from the
65
operating system. They can not be modified using Sikuli script.
69
Screen as (Default) Region
70
--------------------------
72
Normally all region methods are used as reg.find(PS), where reg is a region object
73
(or a screen or a match object). If written as find(PS) it acts on the default
74
screen being the implicit region in this case (mapped to the constant reference
75
SCREEN). In Multi Monitor Environments this is the primary monitor (use the
76
constant reference SCREEN, to access it all the time), that normally is Screen(0),
77
but might be another Screen() object depending on your platform.
79
So its a convenience feature, that can be seen as an implicit use of the python
80
construct '''with object:'''.
82
On the other hand this may slow down processing speed, because of time consuming
83
searches. So to speed up processing, saying region.find() will restrict the search
84
to the specified rectangle. Another possibility is to say setROI() to restrict the
85
search for all following find operations to a smaller region than the whole screen.
86
This will speed up processing, if the region is significantly smaller than the whole
92
Capturing is the feature, that allows to grab a rectangle from a screenshot, to save
93
it for later use. At each time, a capturing is initiated, a new screenshot is taken.
95
There are two different versions: the first one :py:meth:`Screen.capture` saves the
96
content of the selected rectangle in a file and returns its file name, whereas the
97
second one :py:meth:`Screen.selectRegion` just returns the position and dimension of
98
the selected rectangle.
100
Both features are available in the IDE via the buttons in the toolbar.
104
.. py:method:: capture([region | rectangle | text])
107
:param region: an existing region object.
108
:param rectangle: an existing rectangle object (e.g., as a return value of
109
another region method).
110
:param text: text to display in the middle of the screen in the interactive
112
:param x: x position of the rectangle to capture
113
:param y: y position of the rectangle to capture
114
:param w: width of the rectangle to capture
115
:param h: height of the rectangle to capture
117
:return: the path to the file, where the captured image was saved. In
118
interactive mode, the user may cancel the capturing, in which case
121
**Interactive Mode:** The script enters the screen-capture mode like when
122
clicking the button in the IDE, enabling the user to capture a rectangle on
123
the screen. If no *text* is given, the default "Select a region on the screen"
126
If any arguments other than text are specified, capture() automatically
127
captures the given rectangle of the screen. In any case, a new screenshot is
128
taken, the content of the selected rectangle is saved in a temporary file.
129
The file name is returned and can be used later in the script as a reference
130
to this image. It can be used directly in cases, where a parameter PS is
131
allowed (e.g. :py:meth:`Region.find`, :py:meth:`Region.click`, ...).
133
.. py:method:: selectRegion([text])
135
Select a region on the screen interactively
137
:param text: Text to display in the middle of the screen.
138
:return: a new :py:class:`Region` object or None, if the user cancels the capturing process.
140
**text** is displayed for about 2 seconds in the middle of the screen.
141
If **text** is omitted, the default "Select a region on the screen" is
144
The interactive capture mode is entered and allows the user to select a
145
region the same way as using the selection tool in the IDE.
147
**Note:** You should check the result, since the user may cancel the capturing.
149
.. _MultimonitorEnvironments:
151
Multi-Monitor Environments
152
--------------------------
154
If more than one monitor is available, Sikuli is able to manage regions and click
155
points on these monitors.
159
The base is the coordinate system (picture above), that positions the primary
160
monitor with its upper left corner at (0,0) extending the x-direction to the right
161
and the y-direction towards the lower boundary of the screen. The position of
162
additional monitors can be configured in the operating system to be on either side
163
of the primary monitor, with different positions and sizes. So monitors left of the
164
primary will have pixels with negative x-values and monitors above will have
165
negative y-values (left and above both x and y are negative).
167
At script start, Sikuli gets the relevant information from the operating system and
168
creates respective screen objects, that have an ID (0 for the first or primary
169
monitor, 1 and higher for additional monitors with a maximum of one less than the
170
number of screens) and know the rectangle, they cover in the coordinate system.
171
These informations are readonly for a script.
173
These predefined screen objects can be accessed with Screen(0), Screen(1), ... and
174
are normally used to create your own screen objects. The possibility to use the
175
region methods on a default region mapped to the primary monitor is implemented with
176
the constant reference SCREEN. This concept is only available for the primary
179
How to get the relevant information:
181
* :py:func:`getNumberScreens() <Screen.getNumberScreens>` returns the number of available screens.
182
* :py:func:`getBounds() <Screen.getBounds>` returns the rectangle covered by the default/primary
184
* :py:meth:`Screen.getBounds` returns the rectangle covered by a screen object
185
created using :py:meth:`Screen(id) <Screen.Screen>`.
187
Be aware: Changes in your system settings are only recognized by the IDE, when it is
190
**Windows:** The monitor, that is the first one based on hardware mapping (e.g. the
191
laptop monitor), will always be Screen(0). In the Windows settings it is possible to
192
place the taskbar on one of the secondary monitors, which makes it the primary
193
monitor getting the base coordinates (0,0). The other available monitors are mapped
194
around based on your settings. But the Sikuli internal mapping is not changed, so the primary
195
monitor might be any of your Screen() objects. Sikuli takes care for that and maps
196
SCREEN always to the primary monitor (the one with the (0,0) coordinates).
197
So for example you have a laptop with an external monitor, that shows the taskbar
198
(is primary monitor):
200
* SCREEN maps to Screen(1)
201
* Screen(0) is your laptop monitor
203
**Mac:** The monitor, that has the System Menu Bar, is always Screen(0) and mapped
204
to the default SCREEN.
206
**Linux** (Under construction)
208
With its rectangle, a screen object is always identical with the monitor
209
it was created using :py:meth:`Screen(id) <Screen.Screen>`. Using :py:meth:`Region.setROI` to restrict
210
the region of interest for find operations has no effect on the base rectangle of
213
On the other hand region objects and location objects can be positioned anywhere in
214
the coordinate system. Only when a find operation or a click action has to be
215
performed, the objects rectangle or point has to be inside the rectangle of an
216
existing monitor (basically repersented by Screen(0), Screen(1), ...). When
217
this condition is met, everything works as expected and known from a single monitor
220
With finding and acting there are the following exceptions:
222
* **Point Outside:** a click point is outside any monitor rectangle. The
223
clickpoint will be mapped to the edges or corners of the primary monitor
224
according to the relative position:
226
* to the edges if its x or y value is in the range of the respective edge
227
(right, left, above, below)i
228
* to the corners, if x and y are outside any range of any edge (left/above ->
229
upper left corner, ...)
231
* **Region Outside:** a region is completely outside any monitor
233
* a click action is handled in thesame way as **Point Outside**
234
* a find operation will always fail
236
* **Region Partially Outside:** a region is partially outside a monitor but
237
not overlapping another monitor
239
* a click action is handled in the same way as **Point Outside**
240
* a find operation will be carried out only in the part of region within the
241
bounds of the monitor, excluding the area outside the monitor.
243
* **Region Across Monitors:** a region lies across multiple monitors:
245
* a click action is handled in the same way as **Point Outside**
246
* a find operation will be restricted to the region within the bounds of the
247
monitor that has a smaller *id*.
250
An interactive capture (the user is asked to select an image or a rectangle via
251
:py:meth:`Screen.capture` or :py:meth:`Screen.selectRegion`) will automatically be
252
restricted to the monitor, where it was started.
254
A scripted capture using a rectangle or a region
255
(i.e. :py:meth:`Screen.capture( region | rectangle ) <Screen.capture>`),
256
will be handled accordingly:
258
* **Region Outside:** no image is captured, *None* is returned
259
* **Region Partially Outside:** the returned image will only cover the part
261
* **Region Across Monitors:** the returned image will only cover the part
262
inside the monitor with the smallest id.
264
Based on the knowledge of your monitor configuration, you can now start some further
265
evaluations using e.g. :py:meth:`Region.hover` together with
266
:py:func:`setShowActions(True) <setShowActions>` and highlighting using :py:meth:`Region.highlight`.
added
removed
docs/source/screen.rst
