The Linux kernel has a built-in API for video capture devices called Video for Linux (V4L2).  When using this API, video capture devices appear as /dev nodes that can be opened and controlled using standard function calls (open, ioctl, close).  For video capture, the user application will generally select a camera device to use and then configure it to run at a certain resolution and framerate.  This is done by first opening the device node, then using ioctls to check its capabilities (VIDIOC_QUERYCAP) and set a format (VIDIOC_S_FMT).

Video Capture in Linux

The V4L2 API supports a number of ways to get captured image data to the user application.  One method uses buffers allocated by the capture driver.  First the user application requests a number of buffers (VIDIOC_REQBUFS), then it requests a pointer for each buffer that can be mapped into the application’s memory space (VIDIOC_QUERYBUF).  After this the buffers can be queued so the capture driver can use them (VIDIOC_QBUF).  Then the user application can start the stream (VIDIOC_STREAMON).

While streaming, the capture driver will notify the user application when a frame is ready by signaling the file handle for the /dev node.  The user application gets the next frame by dequeuing a buffer (VIDIOC_DQBUF).  The mapped memory associated with the buffer will contain the captured video data, and the user application can process it.  When the user application is finished processing, the buffer should be queued again.  This happens repeatedly until the user application is ready to stop the stream (VIDIOC_STREAMOFF).

More Information