1
.. TUTORIAL:Camera Module Introduction
3
.. include:: common.txt
5
*************************************************
6
Pygame Tutorials - Camera Module Introduction
7
*************************************************
10
Camera Module Introduction
11
==========================
13
.. rst-class:: docinfo
15
:Author: by Nirav Patel
16
:Contact: nrp@eclecti.cc
17
:Revision: 1.0, May 25th, 2009 (Updated October 21, 2016)
20
Pygame 1.9 comes with support for interfacing cameras, allowing you to capture
21
still images, watch live streams, and do some simple computer vision. This
22
tutorial will cover all of those use cases, providing code samples you can base
23
your app or game on. You can refer to the :mod:`reference documentation <pygame.camera>`
28
As of Pygame 1.9, the camera module offers native support for cameras
29
that use v4l2 on Linux. There is support for other platforms via Videocapture
30
or OpenCV, but this guide will focus on the native module. Most of the code
31
will be valid for other platforms, but certain things like controls will not
32
work. The module is also marked as **EXPERIMENTAL**, meaning the API could
33
change in subsequent versions.
43
from pygame.locals import *
48
As the camera module is optional, it needs to be imported and initialized
49
manually as shown above.
52
Capturing a Single Image
53
------------------------
55
Now we will go over the simplest case of opening a camera and capturing a frame
56
as a surface. In the below example, we assume that there is a camera at
57
/dev/video0 on the computer, and initialize it with a size of 640 by 480.
58
The surface called image is whatever the camera was seeing when get_image() was
61
cam = pygame.camera.Camera("/dev/video0",(640,480))
63
image = cam.get_image()
66
Listing Connected Cameras
67
^^^^^^^^^^^^^^^^^^^^^^^^^
69
You may be wondering, what if we don't know the exact path of the camera?
70
We can ask the module to provide a list of cameras attached to the
71
computer and initialize the first camera in the list. ::
73
camlist = pygame.camera.list_cameras()
75
cam = pygame.caemra.Camera(camlist[0],(640,480))
81
Most cameras support controls like flipping the image and changing brightness.
82
set_controls() and get_controls() can be used at any point after using start(). ::
84
cam.set_controls(hflip = True, vflip = False)
85
print camera.get_controls()
88
Capturing a Live Stream
89
-----------------------
91
The rest of this tutorial will be based around capturing a live stream of
92
images. For this, we will be using the class below. As described, it will
93
simply blit a constant stream of camera frames to the screen, effectively
94
showing live video. It is basically what you would expect, looping get_image(),
95
blitting to the display surface, and flipping it. For performance reasons,
96
we will be supplying the camera with the same surface to use each time. ::
98
class Capture(object):
100
self.size = (640,480)
101
# create a display surface. standard pygame stuff
102
self.display = pygame.display.set_mode(self.size, 0)
104
# this is the same as what we saw before
105
self.clist = pygame.camera.list_cameras()
107
raise ValueError("Sorry, no cameras detected.")
108
self.cam = pygame.camera.Camera(self.clist[0], self.size)
111
# create a surface to capture to. for performance purposes
112
# bit depth is the same as that of the display surface.
113
self.snapshot = pygame.surface.Surface(self.size, 0, self.display)
115
def get_and_flip(self):
116
# if you don't want to tie the framerate to the camera, you can check
117
# if the camera has an image ready. note that while this works
118
# on most cameras, some will never return true.
119
if self.cam.query_image():
120
self.snapshot = self.cam.get_image(self.snapshot)
122
# blit it to the display surface. simple!
123
self.display.blit(self.snapshot, (0,0))
124
pygame.display.flip()
129
events = pygame.event.get()
131
if e.type == QUIT or (e.type == KEYDOWN and e.key == K_ESCAPE):
132
# close the camera safely
139
Since get_image() is a blocking call that could take quite a bit of time on a
140
slow camera, this example uses query_image() to see if the camera is ready.
141
This allows you to separate the framerate of your game from that of your camera.
142
It is also possible to have the camera capturing images in a separate thread,
143
for approximately the same performance gain, if you find that your camera does
144
not support the query_image() function correctly.
147
Basic Computer Vision
148
---------------------
150
By using the camera, transform, and mask modules, pygame can do some basic
157
When initializing a camera, colorspace is an optional parameter, with 'RGB',
158
'YUV', and 'HSV' as the possible choices. YUV and HSV are both generally more
159
useful for computer vision than RGB, and allow you to more easily threshold by
160
color, something we will look at later in the tutorial.
164
self.cam = pygame.camera.Camera(self.clist[0], self.size, "RGB")
166
.. image:: camera_rgb.jpg
171
self.cam = pygame.camera.Camera(self.clist[0], self.size, "YUV")
173
.. image:: camera_yuv.jpg
178
self.cam = pygame.camera.Camera(self.clist[0], self.size, "HSV")
180
.. image:: camera_hsv.jpg
187
Using the threshold() function from the transform module, one can do simple
188
green screen like effects, or isolate specifically colored objects in a scene.
189
In the below example, we threshold out just the green tree and make the rest
190
of the image black. Check the reference documentation for details on the
191
:func:`threshold function <pygame.transform.threshold>`\ .
195
self.thresholded = pygame.surface.Surface(self.size, 0, self.display)
196
self.snapshot = self.cam.get_image(self.snapshot)
197
pygame.transform.threshold(self.thresholded,self.snapshot,(0,255,0),(90,170,170),(0,0,0),2)
199
.. image:: camera_thresholded.jpg
203
Of course, this is only useful if you already know the exact color of the object
204
you are looking for. To get around this and make thresholding usable in the
205
real world, we need to add a calibration stage where we identify the color of an
206
object and use it to threshold against. We will be using the average_color()
207
function of the transform module to do this. Below is an example calibration
208
function that you could loop until an event like a key press, and an image of
209
what it would look like. The color inside the box will be the one that is
210
used for the threshold. Note that we are using the HSV colorspace in the below
217
self.snapshot = self.cam.get_image(self.snapshot)
218
# blit it to the display surface
219
self.display.blit(self.snapshot, (0,0))
220
# make a rect in the middle of the screen
221
crect = pygame.draw.rect(self.display, (255,0,0), (145,105,30,30), 4)
222
# get the average color of the area inside the rect
223
self.ccolor = pygame.transform.average_color(self.snapshot, crect)
224
# fill the upper left corner with that color
225
self.display.fill(self.ccolor, (0,0,50,50))
226
pygame.display.flip()
228
.. image:: camera_average.jpg
233
pygame.transform.threshold(self.thresholded,self.snapshot,self.ccolor,(30,30,30),(0,0,0),2)
235
.. image:: camera_thresh.jpg
239
You can use the same idea to do a simple green screen/blue screen, by first
240
getting a background image and then thresholding against it. The below example
241
just has the camera pointed at a blank white wall in HSV colorspace.
246
# capture a bunch of background images
249
bg.append(self.cam.get_image(self.background))
250
# average them down to one to get rid of some noise
251
pygame.transform.average_surfaces(bg,self.background)
252
# blit it to the display surface
253
self.display.blit(self.background, (0,0))
254
pygame.display.flip()
256
.. image:: camera_background.jpg
261
pygame.transform.threshold(self.thresholded,self.snapshot,(0,255,0),(30,30,30),(0,0,0),1,self.background)
263
.. image:: camera_green.jpg
267
Using the Mask Module
268
^^^^^^^^^^^^^^^^^^^^^
270
The stuff above is great if you just want to display images, but with the
271
:mod:`mask module <pygame.mask>`, you can also use a camera as an
272
input device for a game. For example, going back to the example of
273
thresholding out a specific object, we can find the position of that object and
274
use it to control an on screen object.
278
def get_and_flip(self):
279
self.snapshot = self.cam.get_image(self.snapshot)
280
# threshold against the color we got before
281
mask = pygame.mask.from_threshold(self.snapshot, self.ccolor, (30, 30, 30))
282
self.display.blit(self.snapshot,(0,0))
283
# keep only the largest blob of that color
284
connected = mask.connected_component()
285
# make sure the blob is big enough that it isn't just noise
286
if mask.count() > 100:
287
# find the center of the blob
288
coord = mask.centroid()
289
# draw a circle with size variable on the size of the blob
290
pygame.draw.circle(self.display, (0,255,0), coord, max(min(50,mask.count()/400),5))
291
pygame.display.flip()
293
.. image:: camera_mask.jpg
297
This is just the most basic example. You can track multiple different colored
298
blobs, find the outlines of objects, have collision detection between real life
299
and in game objects, get the angle of an object to allow for even finer control,
added
removed
docs/reST/tut/CameraIntro.rst
