# Camera Sensor

## Hardware

Elcano system uses a Pi Cam version 2 camera designed to connect to the Raspberry Pi CSI camera port. The Raspberry Pi connected to the camera is the vision processor in the Elcano system and communicates with other processors and boards using the Elcano Serial library. The Pi Cam mounts on the front of the Elcano trike and software on the Raspberry Pi performs visual functions like lane detection and cone detection.

## Functions

### Visual Traffic Cone Ranging

The Camera sensor is an essential part of optical object detection and ranging. In order to know the distance to an object like a traffic cone, traffic cone detection software running on a Raspberry Pi can identify the outline of a traffic cone and determine distance of the cone using the cone outline's height in the image. It is possible to know the range to an object of known height using the camera's known characteristics.

 *Camera Model* *Focal length (mm)* *Pixel dimensions (um)* Pi Cam v1.x 3.5 1.4 x 1.4 Pi Cam v2.x 3.0 1.12 x 1.12

These are values provided in Pi Cam documentation. These are designed dimensions and may not be accurate from camera to camera.

The vision processor is a Raspberry PI, and the camera is the Pi camera. Raspberry Pi v1.x camera has focal length of 3.6 mm and pixel size of 1.4 x 1.4 um. Cone height is 0.46 m. If cone height in image is N pixels, then by similar triangles, 3.6 mm / (N * 0.0014 mm) = range / 0.46 m.

range = 3.6 * 0.46 m / (N * 0.0014)

### Visual Traffic Cone Angle

Calculating angle relies on similar principles as calculating range. Before converting horizontal or vertical position into angle, we have to know the camera's real focal length and pixel size. From this we can use trigonometry to calculate degrees per pixel.

Optional: first estimate degrees per pixel by dividing the horizontal field of view (often available in the device's documentation) by the camera's horizontal pixel count:

Degrees / pixel = (horizontal field of view) / (horizontal pixel count)

Example: 62.2 degrees / 3280 pixels = 0.0190 degrees per pixel, equivalent to 52.7 pixels per degree.

Now calculate the degrees per pixel using the focal length, pixel size, and data from the range calculations described in "Visual Traffic Cone Ranging":

Tan(degrees) = (Object height in meters) / (Object range in meters)

Degrees = Tanh( height / range)

Pixels = (focal length * height) / (pixel size * range)

Degrees / pixel = Degrees / pixel

Compare your calculated deg / px to the estimated deg / px. The numbers should be similar but will not be the same unless your image is corrected for lens distortion.

To find the horizontal degree offset of an object using its center point, first find the object's offset from center:

Horizontal offset in pixels= (object horizontal center in image) - (image width / 2)

Horizontal offset in degrees = (Horizontal Offset) * (degrees / pixel)

The horizontal offset in degrees will be positive when the object is right of center and negative when the object is left of center.

#### Experimentally finding focal length and pixel size

Pi Cam and other similar products have designed focal length and pixel size dimensions in their respective specification sheets. These documented numbers are nominal (official) values that may be different from real (actual) values. If these nominal values are used in range calculations, the returned range values can be bigger or smaller than in reality. To correct for this, we can calculate a camera's ratio of focal length to pixel size using a formula derived from the similar triangles ratios.

The ratio of a Camera's focal length to its pixel size is part of the similar triangles formula:

focal_length / (N * pixel_size) = range / 0.46 m

We can characterize the ratio of focal length to pixel size by taking a picture of a traffic cone (or other object of known height) at a known distance. Using the known distance, height, and number of pixels in our calibration image, we can calculate focal length and pixel size as a ratio:

focal_length / pixel_size = N * range / 0.46 m

A camera with a focal length of 3.5mm and pixel size of 1.4um has a focal_length / pixel_size of 2500. This is unit-less because we divide meters by meters. We used the method described above to determine an actual Pi Cam version 1.x camera had a focal_length / pixel_size ratio of ~1900, equal to a 1.4um pixel size and 2.7mm focal length. This real value allows accurate range calculations with a specific camera and does not transfer to other hardware of the same version.

### Expected world coordinates

The C6 Arduino communicates with the Raspberry Pi over serial lines. The Arduino sends the expected world coordinate position of the cone in the format

G {n Num} {p EPosMeters,NPosMeters} {b Deg}

and the current position of the trike as

S {p EPosMeters,NPosMeters} {b Deg}

The Pi returns the position of the cone as

G {n Num} {p EPosMeters,NPosMeters} {b Deg} {r Probability}

These formats are documented in GitHub under Documentation / SerialCmd.html.

The Pi needs to translate world coordinates to pixel coordinates (and back) using the 3D matrix transformations given in Computer Graphics: Principles and Practice (2nd ed. Foley, Van Dam, Feiner, Hughes; or 3rd ed. Hughes et al.)

-- Main.TylerCFolsom - 2017-06-16

ALSO > VisionPage

NEXT > ActuatorPage