Image Processing With Focus Stacking and Image Stitching


Image Processing With Focus Stacking and Image Stitching


Using imaging software, multiple images can be combined in a way that shows more detail. Two popular methods are image stitching and focus stacking. Image stitching is the processes of adding images together to form a larger image. This type of image is more commonly known as a mosaic or panorama. Focus stacking uses a set of images of the same scene and stacks them on top of each other. This process is used to keep the in-focus regions of each individual image to create a more overall in-focus image. This blog covers the methods for how these types of image processing are achieved and what is required to begin this process.

Image Stitching 

There are two main ways to consolidate image data into a larger image. The first is by using a single camera and combining images taken over time. The second is a combination of cameras that take images together, ideally at the same time, and then merging these images into a larger image. This combination of images is called a mosaic and gets its name from an artistic picture that is assembled with multiple materials. In the case of digital imaging, a mosaic refers to a set of images that are stitched together using software.

Mosaic with 6 images

Overlap 

For image stitching, at least two images are required to create a mosaic. Image stitching works like a jig saw puzzle, where two images can be connected if they have a matching side. The amount of image data that matches both images is referred to as the overlap or overlapping image data. Overlap refers to the number of pixels in each image that match. Therefore, each image will have a certain area that overlaps with another image and reduces the unique pixels within each image. In older solutions, up to 80% of images need to be overlapped to achieve usable results. In Figure 2, the two stitched images have some overlap with part of the landscape that was imaged. The two images are stitched together and there is a clear indication of what areas do not contain unique information. This illustrates the issues that can come from improper calibration. Using real-time sensor data combined with deterministic triggering ensures minimal overlap can augment the maximum amount of unique data that can be collected from each image within a mosaic.

Image stitching with 2 images

Why Image Stitching Can Improve Aerial Imaging? 

Due to the significance of the pixel width, it can be advantageous to combine cameras together and effectively double the pixel width. When mounting two cameras side by side, a high level of accuracy is required to minimize the overlap of the images that make up the mosaic. The difference between a 20MP camera and a properly synchronized set of 9MP cameras can be seen in Figure 3.

Image comparison

To understand how overlap can play an important role in aerial imaging, read the whitepaper, “Using a Single Versus Multiple Cameras for Aerial Imaging.” This whitepaper covers the usage of multiple cameras in an aerial imaging system by using stereo imaging to combine multiple images. This type of imaging strategy can multiply the effective pixel width that a vision system can capture, but ensuring correct alignment, synchronization, and image overlap can be crucial to achieving a proper mosaic.

Alignment 

The position of cameras used together for multiple camera imaging is crucial to the overall vision system. In applications such as aerial imaging where adding a second or third camera can greatly increase the resolution of the overall vision system, it necessary that the cameras are aligned to produce just the right amount of overlap between the cameras. Too much overlap and a vison system will suffer from a reduced resolution and effectively shrinking the mosaic.

Focus Stacking 

Instead of taking images as a camera or a target moves to create a larger image, focus stacking keeps the same resolution and size, but improves the image data. This upgrade comes from a deeper focus that can be achieved through taking a series of images and algorithmically or manually taking the regions from each image that are in focus.

By focus stacking, an entire image can seem in focus. This type of image processing can be vital for images that have a lot of depth but need the full contents to be analyzed. The raw ingredients for focus stacking come from the series of images taken by a camera at different focus points. This can be any number of images. Generally, focus stacking requires approximately six images on average to get a good range of focus points for a target to be captured in focus from back to front. However, this will vary depending on the actual setting on the camera, aperture (most important), and the actual depth of the target being imaged. Figure 4 shows a set of images taken at different focus points and are merged to produce an entirely in focus image.

Focus Stacking

The importance of image quality 

The best way keep image data from losing quality is to capture images in RAW format rather than using files such as .JPEG. RAW images use a lossless data compression. This way, they keep a higher amount of detail at the cost of much larger file sizes (typically 2-6 times larger). This type of file can have a range of file types with extensions such as, .DNG, .TIF, and many more. The standard raw image format is TIF, but many camera manufactures use their own property software format. Teledyne Imaging offers software with support for TIF and other lossless compression formats such as windows bitmap.

The bit-depth of the images can also be increased to 12 or 14 bits from a typical camera output of 8 bits. For more information on how bit-depth affects image data, visit our blog post, “An In-Depth Look At Bit Depth.” The higher bit-depth allows image processing to have more control over image data such as noise reduction and sharpening. This greater control in image post processing can aid in applications such as image stitching and focus stacking.

More Information 

For more information, please contact Teledyne Lumenera’s imaging experts. They can also help with selecting the cameras to best meet your requirements. Reach out at [email protected].

And, sign up for our newsletter to automatically receive regular updates from Teledyne Lumenera.