The new technique extracts more detail from raw sensor data and creates fewer distracting artefacts ... [+]
Samsung researchers have developed a new image processing technique which could result in a significant improvement in image quality from digital cameras including smartphones.
The research team, from the Samsung R&D Institute in Bengaluru India, claims to outperform current image processing algorithms in several key areas resulting in visibly sharper, more detailed images from existing sensors.
The technique is a new method of ‘demosaicing’ the raw image data captured by camera sensors. This is a critical step in converting image sensor data into viewable pictures
Unlike digital photos, image sensors typically come with an array of individual red, green and blue pixels. ‘Demosaicing’ is the name given to the process of combining the original single-color pixels into the multi-colored ones found in photos. This process is an inexact science, so existing techniques are prone to unwanted artifacts and color errors which the best algorithms strive to reduce.
The team uses deep learning to improve the quality of the demosaicing process as detailed in the paper Deep Demosaicing using ResNet-Bottleneck Architecture. The advantage of this new method is in its ability to work well across all types of photos without specific training on different types of images as previously required.
When compared to state-of-the-art methods, the new technique demonstrates greater clarity in detailed portions of the image while unwanted effects such as color fringing are reduced. In examples based on smartphone imagery, the results are clearly less ‘pixelated’ and more detailed.
Earlier this year Adobe launched Photoshop's ‘Enhance Details’ demosaicing feature, also based on machine learning, which leverages the power of macOS or Windows PCs to deliver picture quality improvements ranging from the significant to barely perceptible. However, Samsung’s team is working on putting the new technology directly into smartphones.
Relatively small quality enhancements at the start of the image processing pipeline can have far-reaching effects on smartphone photography as many advanced functions and camera modes rely on getting the best input possible.
These include digital zoom and super-resolution effects that magnify small imperfections in any photograph as well as features such as artificial bokeh which relies on accurate edge detection. Extreme low-light photography should also see improvements as this is greatly impacted by pixel-level quality.
Additionally, all of these small improvements are the type likely to affect camera benchmarks such as those carried out by DxOMark, leading to potentially higher scores for Samsung smartphones in the future.
