Identifying Number Types and Geometric Shapes in Images using CV2 and Numpy

Identifying number types and geometric shapes in images using OpenCV (cv2) and NumPy involves converting the image to grayscale, applying thresholding to create a binary mask, finding contours with cv2.findContours, approximating polygonal curves to determine shape properties, and finally classifying objects based on vertex count and aspect ratio.

Preprocessing the Image for Clarity

The first step in any computer vision pipeline is ensuring the input data is clean and standardized. Raw images often contain noise, varying lighting conditions, or color information that complicates shape detection. To simplify the problem, we convert the image into a format that highlights structural boundaries rather than color variations. Start by loading the image using cv2.imread. It is crucial to convert this BGR image to grayscale immediately. Grayscale conversion reduces the data from three channels (Red, Green, Blue) to a single intensity channel, which significantly speeds up subsequent processing steps. You can achieve this using cv2.cvtColor(image, cv2.COLOR_BGR2GRAY). Once the image is grayscale, you must separate the foreground objects (the shapes or numbers) from the background. This is typically done using thresholding. If the background is uniform and light while the shapes are dark, use simple binary thresholding with cv2.threshold. For more complex scenarios where lighting is uneven, consider using adaptive thresholding, which calculates thresholds for smaller regions of the image. This ensures that faded shapes remain detectable even if their contrast varies across the frame. After thresholding, you will have a binary image where pixels are either pure black or pure white, creating distinct edges for contour detection.

Finding and Approximating Contours

With a clean binary image, the next phase is identifying the actual objects. OpenCV provides the cv2.findContours function, which locates continuous curves that bound regions of identical color. In OpenCV versions 4.x, this function returns three values: the hierarchy, the contours themselves, and the retrieval mode. The contours are stored as a list of points defining the boundary of each object. However, raw contours are often jagged due to pixel-level noise. To classify shapes accurately, we need smooth, simplified polygons. This is where cv2.approxPolyDP comes into play. This function approximates a curve shape with another curve shape of fewer vertices. It requires two main arguments: the contour to approximate and an epsilon value. Epsilon defines the maximum distance between the original curve and its approximation. Choosing the right epsilon is critical. If it is too small, the approximation retains too many vertices, making a circle look like a hexagon. If it is too large, distinct shapes might merge. A common practice is to set epsilon as a percentage of the contour's perimeter. Calculate the perimeter using cv2.arcLength(contour, True) and then set epsilon to approximately 0.02 times that perimeter. This dynamic scaling ensures consistent approximation regardless of the object's size. The result is a simplified polygon represented by a few key vertices, which serves as the foundation for shape classification. Enhancing Crack Detection with ImageJ Pre-processing demonstrates similar pre-processing rigor, emphasizing how cleaning data before analysis leads to more reliable outcomes in structural identification tasks.

Classifying Geometric Shapes

Now that you have simplified polygons, you can classify them based on their geometric properties. The most robust method is counting the number of vertices in the approximated polygon. For triangles, the vertex count will be 3. Quadrilaterals, such as squares and rectangles, will have 4 vertices. Pentagons have 5, and so on. However, circles pose a unique challenge because they have infinite vertices. To detect circles, you can check the circularity of the shape. Circularity is calculated using the formula: $4 \times \pi \times Area / Perimeter^2$. A perfect circle has a circularity of 1. Values close to 1 indicate a circular shape, while lower values suggest irregular polygons. Additionally, you can distinguish between squares and rectangles by analyzing the aspect ratio. Get the bounding rectangle using cv2.boundingRect and calculate the width divided by height. A square will have a ratio very close to 1.0, whereas a rectangle will deviate from this value. When handling faded shapes, standard thresholding might fail to capture the entire object. In such cases, morphological operations like dilation can help connect broken edges. Use cv2.morphologyEx with a kernel to expand the white regions, effectively filling gaps in faded lines before re-running contour detection. This preprocessing step ensures that fragmented parts of a shape are treated as a single entity.

Identifying Number Types via Template Matching or OCR

While geometric shapes are defined by their boundaries, numbers are often identified by their internal structure or visual similarity to known templates. For simple digit recognition without heavy machine learning models, template matching is effective. First, extract individual character contours from the image. Sort these contours from left to right to reconstruct the sequence of digits. For each isolated character contour, crop the region of interest (ROI) from the original binary image. Resize this ROI to a fixed standard size, such as 20x20 pixels, to ensure consistency. Compare this resized ROI against a library of pre-defined digit templates (e.g., images of '0' through '9'). Use cv2.matchTemplate to find the best match. The template with the highest correlation score identifies the digit. For higher accuracy, especially with handwritten numbers, consider combining this with optical character recognition libraries like Tesseract, feeding it the cleaned binary segments. This hybrid approach leverages OpenCV's speed for segmentation and specialized OCR engines for complex character recognition.

Handling Edge Cases and Performance Optimization

Real-world images rarely behave perfectly. Shadows, reflections, and overlapping objects can confuse contour detectors. Always filter contours by area. Small noise artifacts often appear as tiny contours; ignoring contours below a certain area threshold (e.g., 100 pixels) cleans up the results significantly. Furthermore, performance matters when processing video streams or high-resolution images. Limit the search for contours to specific regions of interest (ROIs) if you know where the shapes will appear. This reduces the computational load on findContours. Also, ensure that your image data type is compatible with OpenCV functions; most CV2 functions expect uint8 images, so verify your data types before passing them to processing routines. By systematically preprocessing, approximating, and classifying, you build a robust system capable of distinguishing between basic geometric forms and numerical characters, laying the groundwork for more advanced automated inspection systems.

Frequently Asked Questions

What happens if the image has a noisy background?

Noisy backgrounds create false contours that clutter your results. Apply Gaussian blur using cv2.GaussianBlur before thresholding to smooth out high-frequency noise. This helps the thresholding algorithm focus on significant edges rather than random pixel variations.

How do I distinguish between a circle and a hexagon?

Use the circularity metric derived from area and perimeter. A hexagon will have a circularity significantly less than 1.0, typically around 0.8-0.9 depending on the approximation. Circles consistently yield values very close to 1.0, allowing for clear differentiation even with slight imperfections.

Why is my contour detection returning multiple pieces for one shape?

This usually occurs due to faint internal lines or holes within the shape. Use morphological closing operations to fill small gaps inside the object. Alternatively, adjust your thresholding method to ensure the interior is fully filled, preventing the detector from seeing separate inner and outer boundaries.

Can I detect rotated squares?

Yes, standard bounding boxes may not align with rotated shapes. Use cv2.minAreaRect instead of cv2.boundingRect. This function returns a rotated rectangle that tightly fits the object, allowing you to calculate accurate dimensions and aspect ratios regardless of rotation angle.

Is template matching suitable for all fonts?

Template matching works best for uniform, printed fonts. Handwritten or highly stylized fonts vary too much for simple pixel-based matching. For varied handwriting, integrate OCR tools like Tesseract or train a simple neural network using frameworks like TensorFlow or PyTorch for better generalization.