By Stephen Kress
Machine vision is hardly an emerging technology. Users in a variety of industries have been effectively putting vision systems to work in machine guidance and quality control applications for more than a decade. But until recently, the systems have required a high level of technical skill and programming expertise that has limited applications to manufacturers with in-house expertise or large projects involving qualified system integrators.
What has changed? Most importantly, the systems themselves have become more powerful while gaining much greater ease of use. Thanks to advances in several related technologies—especially computer interfaces and microprocessors—machine vision has evolved into a highly capable and flexible tool for both mass-production and custom manufacturing of suppliers and OEMs alike.
The Basics of Vision Technology
The primary goal of most machine vision systems is to improve productivity and quality in the manufacturing process. On a typical production line, a sensor detects the presence of a part and signals the vision system to activate a video camera, positioned above or to the side of the inspection point, to capture an image of a part or subassembly and send it to a machine vision processor. This processor is usually configured as either a box-level unit or a plug-in board for the PC or the VMEbus.
Using a combination of hardware and machine vision software, the vision system analyzes the image, usually in a fraction of a second. In this stage, it might determine where the part is located, analyze its orientation, measure critical dimensions, read or verify an identification code, or verify correct assembly. At the completion of each image analysis, the machine vision communicates the results to other factory equipment—such as programmable logic controllers (PLCs), robots or data collection computers.
Advanced vision systems each have their own application development software. Traditionally, the development tools for these systems have required extensive programming expertise—usually in the "C" computer language. However, the newest generation of systems makes it relatively easy to take advantage of powerful machine vision capabilities through easy "point and click" graphical environments.
On a typical automotive production line two important factors can affect the performance of a machine vision system: materials handling and lighting. Typically, most lines use conveyors, indexing tables, or robotic arms to move the parts into position for inspection. Lighting requirements vary from one application to another—ranging from the simple use of ambient room illumination to filters, backlights and strobes that "freeze" images for vision processing on high-speed lines.
How Vision Applications Work
Most vision applications in the automotive industry are for machine guidance or quality inspections. In quality control inspections, the vision system determines whether parts or subassemblies are acceptable or defective and then directs motion control equipment to reject or accept them. Machine guidance applications use vision systems to improve the accuracy and speed of assembly robots and automated material handling equipment.
Although they can vary in a number of ways, the applications are usually in one of several general categories.
Robotics. The most advanced machine vision systems enable a robot to locate the part or subassembly on which it is working, regardless of rotation or scale. In most applications, machine vision systems provide real-time data and live feedback to guide robots as they go through programmed sequences of operations. To perform this level of machine guidance, a vision system usually locates parts for the robot to pick up, identifies the correct locations at which to place or fasten the parts, and sends this information to the robot for the assembly procedure.
Dimensional Gaging. With their precise recognition capabilities and easy programmability, the new generation of machine vision systems excels at ensuring that those measurements are correct. Dimensional gaging by machine vision often involves a variety of odd lines, angles, arcs, diameters, and tolerances. Almost without exception, the systems can measure them much more quickly, and with far greater reliability and accuracy than would be possible with even the most sophisticated manual methods.
Assembly Verification. Once again, here is an area in which the new generation of machine vision is proving its worth. Users easily "train" vision systems to look for detailed patterns and shapes that match templates for correctly made subassemblies. The systems accomplish those inspections better than virtually any other quality control method—manual or automated.
Flaw Detection. This has become a primary mission for many machine vision systems on automotive industry production lines. These vision systems use powerful pattern recognition capabilities to find missing material, chips, scratches, dents, misplaced markings, and a wide variety of other flaws. In addition to ensuring the quality of finished parts and products, they also enable manufacturers to reduce costs by eliminating defective pieces before wasting additional material and production time on them.
Print Verification. Using various optical character verification (OCV) methods, machine vision systems inspect parts, components and labels to make sure that they are labeled and marked correctly. Due to various types of marking methods commonly used on automotive parts, this task is not always as simple as it may initially seem. The vision systems must often learn to deal with variations in character density, inking and shapes, as well as the secondary effects of laser etching, stamping and engraving.
Code Reading. The use of 2D Data Matrix code reading is a true cutting-edge machine vision application with enormous potential for the automotive industry. As manufacturers make greater use of sophisticated 2D Data Matrix codes, they are employing vision systems to read them and then use the detailed information for tracking, verification, and statistical quality control.
Data Matrix symbology has many advantages such as compact size and large data capacity. But experience shows that conventional automated recognition systems often have a hard time locating and reading the codes accurately. Typically laser etched on silicon, metal, or glass, Data Matrix markings have low contrast and can wear down or degrade during the production process—factors which typically cause traditional recognition systems to misread or miss the codes entirely.
Some systems use gray scale searching technology to overcome these problems. The systems first locate the matrix codes on the parts and then apply image processing and analysis tools, along with decoding software calibrated to the appropriate ECC algorithm, to gain an accurate reading of the symbol. The result is that manufacturers can gain all the informational advantages of 2D Data Matrix coding at speeds that match virtually any high-volume production line.
Case Examples on the Production Line
Engines and Power Trains. With a comparatively modest investment in machine vision, a major European automobile maker has solved a critical quality challenge in its gear assembly bearings. Before installing the vision system on the production line, the company found it nearly impossible to ensure 100% accuracy—an especially thorny problem because of the safety implications. The machine vision system has now remedied the situation by automatically verifying the presence of all bearings in each assembly and gaging their diameters to make certain they are correctly shaped and undamaged.
The company is also taking advantage of the vision system's "point & click" simplicity with a customzied interface that makes it easy for operators to use the system correctly with a minimum of training.
Other companies around the world are similarly using machine vision for engine and power train applications such as:
- Gaging nozzle holes on oil jet pumps
- Inspecting adhesives applied to gear boxes
- Verifying the presence and correct shape of gear teeth
- Reading characters on engine blocks with OCR functions
- Detecting defective seals on engine blocks
- Inspecting engine castings for
- Checking and identifying engine block types before assembly
- Gaging the gaps on spark plugs
- Verifying the presence of critical stoppers on fuel injection pumps.
Electronic controls. Machine vision can have a dramatic impact on both production and quality control for automotive electronics. In one such application, a maker of anti-lock brake sensors has greatly improved its product quality by teaming a vision system with an assembly robot.
The problem in this case was that human operators previously had to grasp electrical lead wires and tuck them into place around the sensor body. A tedious and repetitive task, this often resulted in tool slips which scraped through the insulation skeins, subsequently causing shorts and component failures.
Working with an experienced system integrator, the company installed a powerful vision system to guide an industrial robot. Using critical data from the vision system, the robot now automatically tucks each lead into place, greatly improving quality and throughput.
This application features some technical innovations that showcase the power of advanced machine vision. To find and identify the shiny wire leads, the system integrator designed the turnkey system to use three cameras with telecentric optics that eliminate the perspective distortion of conventional lenses, along with fiber optic backlighting for effective illumination. In addition, the integrator also created an extremely simple interface that allows an operator to work the vision-guided robot with the click of a single button.
Using the system, the manufacturer is now meeting its quota of high-quality parts with no overtime—and has very satisfied customers.
Applications like this are now quite common in the industry. Some real-world examples include:
- Controlling the application of die-bonding onto electronic modules
- Inspecting air bag controls
- Verifying the correct assembly of fuse blocks
- Inspecting the soldering on electronic modules
- Verifying the presence of fasteners and labels on machine controls.
Brakes. There is no room for error in automotive brakes and their components; absolute quality is required for each and every piece. To achieve this level of quality, a leading U.S. producer of disk brake pads had previously been forced to live with the costs of high scrap rates. But using machine vision, it has now automated the inspection process to achieve zero defects while also providing documentation that proves its quality levels to key customers.
Using powerful software, the vision system identifies and measures a set of critical lines, slots and curves on the brake shoes. To gain these measurements, the company's system integrator developed special part manipulation equipment that positions the brake shoes precisely for three individual inspections at different angles.
At the end of each three-step inspection, the vision system directs a PLC to reject bad parts and move good parts onto an accumulating conveyor for packaging and shipping. By enabling the manufacturer to identify recurring production flaws, it has significantly reduced scrap rates—effectively achieving a full return on investment after only two months of operation.
Vehicle interiors. Many companies have found that insuring the fit and finish of automobile interiors has often been much more difficult than they initially thought.
A maker of plastic radio bezels (faceplates) and buttons learned this when it first tried to inspect laser-marked characters and icons. After deciding that manual inspections could not do the job, the company first installed an older-design machine vision system using simple recognition technology. But the system could not handle normal variations in ambient light and thus produced inconsistent and inaccurate results.
However, those same variations proved to be no problem for a new-generation machine vision system based on more powerful 256-shade grey scale image recognition. That system now runs three shifts a day, totally unaffected by the real-world changes in ambient light. In addition, the system has been exceptionally easy to set up and run. Using simple point and click programming, the company's engineers configured the system themselves and developed the inspection application with no outside help.
Vision systems are now handling a wide range of similar interior related inspections throughout the industry. Some notable examples:
- Guidance for robotic assembly of instrument panel illumination systems
- Calibration of instrument readouts
- Verification of instrument assemblies
- Testing for LEDs and lamps
- Inspecting logo quality
- Verifying the installation of seat adjustment levers.
Until recently, some manufacturers in the automotive industry have been reluctant to invest intensively in machine vision—largely due to the limitations of earlier generations of the technology.
But with the simplicity and power built into the new generation of systems, the industry has now widely embraced vision as a dependable and useful tool for all sorts of automated applications.
The Keys to a Successful Machine
To implement a machine vision application successfully, a developer needs to know precisely what it needs to achieve. That makes it essential to understand the characteristics of the parts and subassemblies which the vision system will examine, as well as the specifications of the production line itself. The key characteristics include:
- Specific part dimensions
- Part tolerances
- Needed level of measurement precision
- Minimum size of defects to be detected
- Speed of the production line.
Experience shows that one of the most important considerations for automotive applications is potential variation in the parts or subassemblies. Virtually all manufacturing processes will produce some degree of variability. While the best machine vision technology is robust enough to compensate automatically for minor differences over time, the application itself needs to take major changes into account. These variations usually fall into several categories:
- Parts produced in distinctly different colors, textures, shapes and sizes
- Process tolerances that allow noticeable variations in color, texture, shape and size
- Variations caused by on-going modifications to the production process
- Substitutions of components and materials
- Different suppliers for a single part
- The presence of oil, paint, cleaning solvents and other substances which might obscure a part or change its appearance. (Note: these may be accidentally introduced, or they may be a known result of the production process.)
- Imprecise control over the part's orientation.
Even though machine vision systems have become much simpler to use, the applications themselves can still be extremely complex. In some cases, the best way to ensure success is to rely on the experience and know-how of the vision system vendor or a qualified system integrator. This is especially true in applications with demanding inspections, complex lighting and optical strategies, or unusual material handling logistics.
The Advantage of Vision Tools
To achieve high levels of precision and accuracy, the most effective machine vision applications typically use software-based vision tools that perform sub-pixel level, image analysis. These tools work with the system's image processing capabilities to perform specific types of recognition and analysis, effectively streamlining the application development cycle. Here are some examples of vision tools now commonly used in automotive applications.
Search. Search is a pattern recognition tool that scans an image in order to locate the best match with a pre-defined model of the part. It enables vision systems to measure the position of an object and to determine whether certain parts or elements of the object are present or missing. A "search" step often precedes processing by other tools so that the system can handle variations in the positions of objects to be inspected.
Blob. A blob tool is especially useful in area searches to locate and identify specific features on a part. It measures the size, shape, position, and "connectivity" in defined pixel areas.
Caliper. This tool works in much the same way as a mechanical caliper; it measures the dimensions of an object by locking onto a specific pair of parallel edges. Using this tool, an application developer specifies the required separation between the "jaws" of the caliper, which represent the desired distance between the object edges. In action, the tool locates those same edges on each inspected object and determines whether they match the specified distance.
Calibration. Just as the name implies, the calibration tool automatically calibrates the vision system's optical parameters. It serves two practical purposes. First, it enables the system to make highly accurate measurements by compensating for optical distortion due to parallax, camera misalignment or lens imperfections. Secondly, it converts pixel units in the image into "real-world" units, such as millimeters or inches, thus allowing the vision system to report results in meaningful formats.
Fixturing. This tool is extremely useful for examining parts that may move around on the production line. Using the results from other tools—such as search, edge detection, or blob—it establishes a "fixture" on a specific feature on the part and compensates for any movement, rotation or changes in scale.
Arc/Circle Gage. This is a special edge detection tool for curved measurements and defection on round parts. It enables an application developer to define each curve as a connected series of points which together form an arc.
Lightmeter. A high-speed vision tool, the lightmeter is most useful for quick identification of well-defined features or markings on a part. It works by analyzing the pixel values in rectangular or curved regions of an acquired image.
2D Data Matrix. This special tool enables a vision system to locate and read AIM-standard Data Matrix codes under a variety of conditions. It is especially useful in automotive parts manufacturing because it is designed to recognize Data Matrix symbols despite confusing backgrounds, low contrast and degraded markings.
OCV—Optical Character Verification. There are a number of approaches to optical character verification, but the most effective vision tools provide "true verification" capabilities in addition to the ability to verify that markings are correct. This allows a vision system to distinguish between confusing pairs of characters—such as "G" and "6" or "B" and "8"—thus ensuring far more reliable performance.