Machine Vision is an important branch developing rapidly in the field of artificial intelligence. At present, it is in the stage of continuous breakthrough and maturity. It is generally believed that machine vision "is through the optical device and non-contact sensor automatically accept and process the image of a real scene, by analyzing the image to obtain the required information or used to control the machine motion device", it can be seen that intelligent image processing technology plays a pivotal role in machine vision.

Intelligent image processing refers to a kind of computer based image processing and analysis technology adaptive to various applications, itself is an independent theoretical and technical field, but at the same time is a very important technical support in machine vision. The relationship between artificial intelligence, machine vision and intelligent image processing technology is shown in Figure 1.

FIG. 1 The supporting role of intelligent image processing

Machine vision with intelligent image processing function, equivalent to people in the machine intelligence at the same time for the machine to press the eyes, so that the machine can "see", "see accurately", can replace or even better than the human eye to do measurement and judgment, so that the machine vision system can achieve high resolution and high speed control. Moreover, the machine vision system has no contact with the detected object, which is safe and reliable.

1. Machine vision technology

The origin of machine vision can be traced back to the image processing research of the polyhedral building block world by American scholar L.R. Roberts in the 1960s, and the opening of the "Machine vision" course in the Artificial Intelligence laboratory of Massachusetts Institute of Technology (MIT) in the 1970s. In the 1980s, the global machine vision research boom began to rise, and some application systems based on machine vision appeared. After the 1990s, with the rapid development of computer and semiconductor technology, the theory and application of machine vision have been further developed.

After entering the 21st century, the development of machine vision technology is faster, and it has been widely used in many fields, such as intelligent manufacturing, intelligent transportation, medical and health care, security monitoring and other fields. At present, with the rise of the wave of artificial intelligence, machine vision technology is in a new stage of continuous breakthrough and maturity.

In China, the research and application of machine vision began in the 1990s. Starting from tracking foreign brand products, after more than 20 years of efforts, the domestic machine vision from scratch, from weak to strong, not only the rapid progress of theoretical research, and there have been some highly competitive companies and products. It is estimated that with the continuous deepening of domestic research, development and promotion of machine vision, catching up with and surpassing the world level is not a far-fetched thing.

Common machine vision systems can be divided into two main categories, one is computer based, such as industrial computer or PC, and the other is more compact embedded devices. Typical IPC based machine vision system mainly includes: optical system, camera and IPC (including image acquisition, image processing and analysis, control/communication) and other units, as shown in Figure 2. Machine vision system for the core image processing algorithm requires accurate, fast and stable, but also requires the implementation of the system cost is low, convenient upgrading.

FIG. 2 Case of machine vision system

2. Intelligent image processing technology

The image processing system of machine vision carries out operation and analysis on the digital image signal in the field according to the specific application requirements, and controls the action of the field equipment according to the processing results obtained. Its common functions are as follows:

(1) Image acquisition

Image acquisition is the process of obtaining scene images from the work site, which is the first step of machine vision. Most of the acquisition tools are CCD or CMOS cameras or cameras. The camera takes a single image, and the camera can take a continuous scene image. In terms of an image, it is actually the projection of a three-dimensional scene on the two-dimensional image plane, and the color of a point in the image (luminance and chromaticity) is the reflection of the color of the corresponding point in the scene. This is the fundamental basis that we can use images to replace the real scene.

If the camera is an analog signal output, it is necessary to digitize the analog image signal and send it to a computer (including an embedded system) for processing. Now most cameras can output the digital image signal directly, which can avoid the step of analog-to-digital conversion. Not only that, now the digital output interface of the camera is also standardized, such as USB, VGA, 1394, HDMI, WiFi, Blue Tooth interface, can be directly sent to the computer for processing, in order to avoid the trouble of adding a piece of image acquisition card between the image output and the computer. The subsequent image processing work is often carried out by computer or embedded system in the way of software.

(2) Image preprocessing

Due to the influence of equipment and environmental factors, the acquired digital field images are often affected by different degrees of interference, such as noise, geometric deformation, color imbalance, etc., which will hinder the following processing links. Therefore, the acquired image must be preprocessed. Common preprocessing includes noise cancellation, geometric correction, histogram equalization and so on.

Usually the time domain or frequency domain filtering method is used to remove the noise in the image. Geometric transformation is used to correct the geometric distortion of the image. Histogram equalization and homomorphic filtering are used to reduce color deviation. In short, through this series of image preprocessing technology, the acquisition of images for "processing", for body machine vision applications to provide "better", "more useful" images.

(3) Image segmentation

Image segmentation is according to the application requirements, the image is divided into each characteristic area, from which the object of interest is extracted. The common features in the image are gray, color, texture, edge, corner and so on. For example, the image of automobile assembly line is segmented into background area and workpiece area, which can be provided for subsequent processing unit to process the workpiece installation part.

Image segmentation has been a difficult problem in image processing for many years. There are a variety of segmentation algorithms, but the effect is often not ideal. Recently, deep learning methods based on neural networks have been used for image segmentation, which outperform traditional algorithms.

(4) Target recognition and classification

In manufacturing or security and other industries, machine vision is inseparable from the input image of the target recognition and classification processing, in order to complete the subsequent judgment and operation on this basis. Recognition and classification techniques have a lot in common, often after the target recognition is completed, the target category is also clear. Recent image recognition technology is leapfrog traditional methods, forming intelligent image recognition methods with neural network as the mainstream, such as convolutional neural network (CNN), regression neural network (RNN) and other superior performance methods.

(5) Target positioning and measurement

In intelligent manufacturing, the most common work is to install the target workpiece, but it is often necessary to locate the target before installation, and measure the target after installation. Both installation and measurement need to maintain high accuracy and speed, such as millimeter accuracy (or even less), millisecond speed. Such high precision, high speed positioning and measurement, relying on the usual mechanical or manual methods is difficult to do. In machine vision, image processing method is used to process the image on the installation site, according to the complex mapping between the target and the image, so as to complete the positioning and measurement task quickly and accurately.

(6) Target detection and tracking

Moving object detection and tracking in image processing is to detect whether there is a moving object in the scene image captured by the camera in real time, and predict its next moving direction and trend, that is, tracking. And timely submit these motion data to the subsequent analysis and control processing to form the corresponding control action. Image acquisition generally uses a single camera, if necessary, can also use two cameras to imitate the binocular vision and obtain the stereo information of the scene, which is more conducive to target detection and tracking processing.

3. Application of machine vision

As shown in Figure 3, machine vision is widely used, such as security, manufacturing, education, publishing, medical, transportation, military fields, etc. In the applications of these machines, intelligent image processing is indispensable. Here only a few of the applications are briefly introduced.

FIG. 3 Common machine vision applications

(1) Intelligent manufacturing

In order to achieve the ambitious goal of China's intelligent Manufacturing 2025, machine vision is indispensable. For example, Guangdong Xuntong Technology Co., LTD. (hereinafter referred to as "Xuntong Technology"), which has always been a leader in intelligent image processing, has developed a machine vision analyzer platform to meet this demand, as shown in Figure 4. The automatic positioning, detection and recognition system of door limiter developed by Xuntong Technology for the assembly line of a well-known automobile manufacturer is shown in Figure 5. By means of intelligent image recognition, the system automatically detects whether the model is correct and the location is accurate, completely replaces the manual operation, and the detection accuracy reaches 100%. Previously, each station required four workers to check and locate 16 types of limiter with their eyes, which not only made employees tired easily, but also made mistakes frequently.

(2) Educational examination

Examination papers are often found to be affected by typesetting or printing errors. By using intelligent image processing technology, the machine automatically compares the printed papers with the original papers, and automatically prompts and alarms when inconsistences are found. Before complete replacement, the papers can only be verified manually.

(3) Publication and printing

Similar to educational examinations, professional publishing and printing factories often make mistakes in typesetting and printing due to the variety and quantity of books, newspapers and magazines printed, as well as the product packaging and publicity materials from enterprises. To this end, need to arrange a lot of professional personnel to proofread, spend a lot of money and time. Through the use of intelligent image processing technology for automatic proofreading, not only improve the accuracy of proofreading, but also shorten the time of proofreading, reduce the printing cost, shorten the delivery cycle of publications.

(4) Security monitoring

This is an area of current attention of machine vision. Machine vision breaks the limitation of the traditional video surveillance system, increases the intelligence of the system, and makes the intelligent video analysis can be gradually realized. Taking video surveillance in public places as an example, by using machine vision technology, it can realize automatic detection, face recognition, real-time tracking of suspicious people, and when necessary, it can also realize continuous tracking of multiple cameras, send alarms at the same time, and store on-site information.

(5) Intelligent transportation

Machine vision is widely used in the field of transportation. For example, on the highway and at the bayonet, the vehicle type, license plate and other identification, and even the violation of the driving vehicle to identify. The driver's face image is analyzed on the car to determine whether the driver is in the state of fatigue driving. For example, with the help of machine vision technology, driverless cars use cameras, laser/millimeter wave/ultrasonic radar, GPS and other senses of road environment information to automatically plan and control the safe driving of vehicles.

Data show that the global market size of machine vision system is about $4.6 billion in 2016, about $5 billion in 2017, and is expected to reach $5.5 billion in 2018, with an annual growth rate of about 10%. The growth of the machine vision market in China started in 2010, and the market size was about 6.8 billion yuan in 2017. It is estimated that the market growth rate will exceed 100% by 2020 or reach 78 billion yuan.

4. Technical bottleneck and future development

In the development of intelligent image processing technology of machine vision, there are still many technical bottlenecks, such as:

1) Stability: a certain treatment method often performs well in the research and development, but in the complex and changeable application environment, but from time to time problems appear. For example, face recognition system, when the target with the recognition rate can be as high as 95%, but in the actual monitoring environment, the recognition rate will be greatly reduced.

2) Real-time performance: If the image acquisition speed and processing speed are slow, coupled with the newly introduced deep learning algorithm, the difficulty of real-time processing of the system will be increased, and the system cannot keep up with the pace of machine operation and control.

3) Accuracy: Machine vision system requires the accuracy of image recognition and measurement to be close to 100%, and any small error may bring unpredictable consequences. For example, the error of target positioning will make the assembled equipment not meet the requirements.

4) System capability: The current embedded image processing system has the problems of insufficient computing power of chip and limited storage space, which often cannot meet the image processing operation with large computation, such as iterative operation of neural network and large-scale matrix operation.

The development of intelligent image processing in machine vision in the future is mainly reflected in the following aspects:

1) Algorithms: Traditional algorithms continue to make breakthroughs. The new wave of artificial intelligence has brought many new image processing algorithms with excellent performance, such as deep learning (DL), convolutional neural network (CNN), Generative adversarial network (GAN), and so on.

2) Real-time performance: more hardware platforms with novel structures, sufficient resources and fast computing will be provided, such as computers with parallel processing structures based on multi-CPU and multi-GPU, massive storage units, etc.

3) Embedded: new high-speed signal processor array, large scale FPGA chip.

4) Fusion processing: from the development of single image sensor to multi-sensor (multi-view) fusion processing, can more fully obtain field information. It can also integrate multiple sensors, such as image sensors, sound sensors, temperature sensors, etc., to complete the location, identification and measurement of the field target.

In short, whether "Made in China 2025" or "Industry 4.0" are inseparable from artificial intelligence, inseparable from computer vision, and intelligent image processing is the core technology of machine vision, with the continuous improvement of image processing level, will strongly promote the rapid development of machine vision.