Camera Calibration with Python – OpenCV

Camera calibration is a fundamental process in computer vision that allows us to understand and correct for the distortions introduced by a camera's lens. By estimating the camera's internal and external parameters, we can achieve a more accurate representation of the real world from captured images.

What is Camera Calibration?

Camera calibration is the process of determining the parameters of a camera to correct for lens distortions and to accurately map 2D image points to 3D world coordinates.

Purpose of Camera Calibration

• Remove Lens Distortion: Correct radial and tangential distortions caused by the camera's optics.
• Recover Camera Parameters: Determine the camera's intrinsic and extrinsic parameters.
• Accurate 2D-to-3D Mapping: Enable precise mapping of points from the 2D image plane to their corresponding 3D locations in the real world.

Key Camera Parameters

1. Intrinsic Parameters

These parameters describe the internal characteristics of the camera, defining how it projects 3D points onto the 2D image plane. They are typically represented by the intrinsic camera matrix, often denoted as K.

• Focal Lengths (fx, fy): The distance from the optical center to the image plane, measured in pixels, along the x and y axes respectively.
• Optical Center (cx, cy): The principal point, which is the intersection of the optical axis with the image plane.
• Skew Coefficient (s): This parameter accounts for any non-orthogonality between the x and y sensor axes. In most modern cameras, this value is close to zero.

The intrinsic camera matrix K is represented as:

K = | fx  s  cx |
    |  0  fy cy |
    |  0  0  1  |

2. Extrinsic Parameters

These parameters define the camera's position and orientation in the 3D world coordinate system. They consist of:

• Rotation Matrix (R): A 3x3 matrix that describes the camera's orientation (rotation) relative to the world coordinate system.
• Translation Vector (t): A 3x1 vector that describes the camera's position (translation) relative to the world coordinate system.

These parameters are used to transform points from the world coordinate system to the camera coordinate system.

3. Distortion Coefficients

Real-world lenses are not perfect and introduce distortions into images. OpenCV models these distortions using the following coefficients:

• Radial Distortion: This is the most common type of distortion, causing straight lines to appear curved.
  • k1, k2, k3: Coefficients for radial distortion. Higher values mean more distortion.
• Tangential Distortion: This occurs when the lens is not perfectly aligned with the image sensor.
  • p1, p2: Coefficients for tangential distortion.

The distortion model typically involves these coefficients to map ideal image points to their distorted counterparts.

Required Setup for Calibration

To perform camera calibration, you will need:

1. A Known Calibration Pattern: A chessboard pattern is commonly used due to its easily detectable corners. Typical sizes include 9x6 or 7x7 internal corners.
2. Multiple Images of the Pattern: Capture several images of the chessboard from different angles and positions. This allows the calibration algorithm to accurately estimate the camera's parameters.
3. OpenCV Installation: Ensure you have OpenCV installed in your Python environment.

pip install opencv-python opencv-contrib-python

Step-by-Step: Camera Calibration Using OpenCV

Step 1: Import Libraries

Import the necessary libraries for image processing and numerical operations.

import cv2
import numpy as np
import glob

Step 2: Prepare Object Points and Image Points

Before calibrating, you need to define the 3D coordinates of the corners of your calibration pattern in the real world (objp) and their corresponding 2D pixel coordinates in the captured images (imgpoints).

# Define dimensions of the chessboard (number of inner corners)
chessboard_size = (9, 6)  # Example: 9x6 inner corners

# Prepare object points, like (0,0,0), (1,0,0), (2,0,0), ... in the real world
# We assume the chessboard is on the Z=0 plane
objp = np.zeros((np.prod(chessboard_size), 3), np.float32)
objp[:, :2] = np.mgrid[0:chessboard_size[0], 0:chessboard_size[1]].T.reshape(-1, 2)

# Lists to store the 3D object points and 2D image points from each image
objpoints = []  # 3D points in real world
imgpoints = []  # 2D points in image plane

Step 3: Load Chessboard Images and Detect Corners

Iterate through your captured images, convert them to grayscale, and find the chessboard corners.

# Path to your calibration images
images = glob.glob('calibration_images/*.jpg') # Make sure to replace with your image path

for fname in images:
    img = cv2.imread(fname)
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

    # Find the chessboard corners
    # `ret` is True if corners are found, `corners` are the detected corner coordinates
    ret, corners = cv2.findChessboardCorners(gray, chessboard_size, None)

    # If corners are found, add object points and image points to our lists
    if ret:
        objpoints.append(objp)
        imgpoints.append(corners)

        # Draw and display the corners for visual verification
        cv2.drawChessboardCorners(img, chessboard_size, corners, ret)
        cv2.imshow('Chessboard Corners', img)
        cv2.waitKey(100) # Wait for 100ms to see the corners

cv2.destroyAllWindows()

Step 4: Calibrate the Camera

Use the collected objpoints and imgpoints to compute the camera matrix (mtx), distortion coefficients (dist), rotation vectors (rvecs), and translation vectors (tvecs).

# Calibrate the camera
# gray.shape[::-1] provides the image dimensions (height, width)
# The last two arguments (None, None) are for initial guesses for camera matrix and distortion coefficients
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1], None, None)

print("Camera matrix (Intrinsic Parameters):\n", mtx)
print("\nDistortion coefficients:\n", dist)

Output Parameters Explained:

• Camera Matrix (mtx): Contains the intrinsic parameters (fx, fy, cx, cy, and skew).

  [[fx 0 cx]
   [ 0 fy cy]
   [ 0 0  1]]

• Distortion Coefficients (dist): A list of coefficients [k1, k2, p1, p2, k3] representing radial and tangential distortion.
• Rotation Vectors (rvecs): A list of rotation vectors, one for each image, describing the orientation of the chessboard relative to the camera for that specific image.
• Translation Vectors (tvecs): A list of translation vectors, one for each image, describing the position of the chessboard relative to the camera for that specific image.

Step 5: Undistort an Image

Once calibrated, you can use the computed mtx and dist to undistort new images. It's often beneficial to refine the camera matrix to get an optimal view after undistortion.

# Load an image to undistort
img = cv2.imread('test_image.jpg') # Replace with the path to an image you want to undistort
h, w = img.shape[:2]

# Refine the camera matrix for optimal viewing
# `1` is the focal length scaling factor. It can be adjusted to zoom in/out.
newcameramtx, roi = cv2.getOptimalNewCameraMatrix(mtx, dist, (w, h), 1, (w, h))

# Undistort the image
dst = cv2.undistort(img, mtx, dist, None, newcameramtx)

# Crop the image to remove black borders that might appear after undistortion
x, y, w, h = roi
dst = dst[y:y+h, x:x+w]

cv2.imshow('Undistorted Image', dst)
cv2.waitKey(0)
cv2.destroyAllWindows()

Summary of Key OpenCV Functions for Calibration

Conclusion

Camera calibration is an essential step for any computer vision task that requires accurate spatial understanding or geometric transformations. By following this process with OpenCV, you can effectively correct for lens distortions and enable precise 2D-to-3D mapping, paving the way for applications like 3D reconstruction, augmented reality, and accurate measurements.

SEO Keywords

Camera calibration OpenCV, Intrinsic and extrinsic camera parameters, Lens distortion correction, Chessboard pattern camera calibration, Calibrate camera Python tutorial, OpenCV findChessboardCorners usage, Camera matrix and distortion coefficients, Undistort images OpenCV, Camera calibration step-by-step guide, 3D to 2D image mapping camera.

Interview Questions

• What is camera calibration and why is it important in computer vision?
• What are intrinsic and extrinsic parameters of a camera?
• How does lens distortion affect images and how can it be corrected?
• What role does a chessboard pattern play in camera calibration?
• How do you detect chessboard corners in an image using OpenCV?
• What does the camera matrix represent?
• What are distortion coefficients and what types of distortion do they model?
• How do you undistort an image after calibration?
• Explain the steps involved in camera calibration using OpenCV.
• What is the difference between rotation vectors and translation vectors in camera calibration?