SPIE Membership Get updates from SPIE Newsroom
  • Newsroom Home
  • Astronomy
  • Biomedical Optics & Medical Imaging
  • Defense & Security
  • Electronic Imaging & Signal Processing
  • Illumination & Displays
  • Lasers & Sources
  • Micro/Nano Lithography
  • Nanotechnology
  • Optical Design & Engineering
  • Optoelectronics & Communications
  • Remote Sensing
  • Sensing & Measurement
  • Solar & Alternative Energy
  • Sign up for Newsroom E-Alerts
  • Information for:
SPIE Photonics West 2018 | Call for Papers

SPIE Defense + Commercial Sensing 2018 | Call for Papers




Print PageEmail PageView PDF

Sensing & Measurement

Automated inspection applications for smart robot cameras

An intelligent robot camera using so-called ‘creative control’ can solve a set of problems in dynamic programming and optimal control.
15 May 2006, SPIE Newsroom. DOI: 10.1117/2.1200604.0144

In a recent paper,1 we described a theory of learning called creative control (CC) and suggested it as a possible solution to camera setup and calibration problems that are encountered in a variety of automated measurement applications. In general, CC can be used in applications that are not necessarily repetitive, but more for plug-and-play and unstructured environments. The CC learning architecture does this by integrating a task control center (TCC) and a dynamic database (DD) that feed into ‘adaptive critic’ learning algorithms. Creative control can also enable a machine to adapt to, learn from, and predict changes in its environment using both perception and action. This could lead to a new generation of smart robotic cameras that automate and simplify industrial sensing by making setup and use easier.

Dynamic programming

In CC, both decision and estimation theory are used and modeled by neural networks. The intelligent system acts as the decision maker for the dynamic system that, in turn, makes decisions in stages. The outcome of each decision may not be fully predictable but can be anticipated or estimated to some extent before the next decision is made. Furthermore, an objective or cost function can be defined for the system. Generally, the goal is to minimize this cost function over some decision space subject to a set of constraints, which may include natural ones. With this definition, the intelligent robot camera can be considered as solving a set of problems in dynamic programming and optimal control.

Dynamic programming (DP) is the only approach for sequential optimization applicable to general nonlinear, stochastic environments. However, DP needs efficient approximate methods to overcome its dimensionality problems. It is only with the presence of artificial neural networks (ANN) and the invention of back propagation that such a powerful and universal approximate method has become a reality.

The essence of dynamic programming is contained in Bellman's Principle of Optimality. The original Bellman equation, used for the adaptive critic algorithm, may be written as:


Dynamic programming gives the exact solution to the problem of how to maximize a utility function U(R(t)) over the future times, t, in a nonlinear stochastic environment where the vector R(t) represents the state of the environment at time t. Dynamic programming converts a difficult long-term problem in optimization over time (< U(R(t)) >, the expected value of U(R(t)) over all the future times), into a much more straightforward problem in simple, short-term function maximization—after we know the function J. Thus, all of the approximate dynamic programming methods discussed here use some kind of general-purpose nonlinear approximation to the J function, the value function in the Bellman equation, or something closely related to it.

In most forms of adaptive critic designs, we approximate J using a neural network. Therefore, we approximate J(R) by some function Ĵ(R,W), where W is a set of weights or parameters. Ĵ is called a critic network.

If the weights W are adapted or iteratively solved for in real-time learning or offline iteration, we call it an adaptive critic. An adaptive critic design (ACD) is any system which includes a component of this type; a critic, in turn, is a neural net or other nonlinear function approximation that is trained to converge to the function J(X). In adaptive-critic learning designs, the critic network learns to approximate the cost-to-go or strategic utility function J and uses the output of an action network as one of its' inputs, either directly or indirectly. When the critic network learns, back propagation of error signals is possible along its feedback path to the action network.

Adaptive critic and creative control

Most advanced methods in neurocontrol are based on adaptive-critic learning techniques consisting of an action network, an adaptive critic network, and a model or identification network, as shown in Figures 1 and 2. These methods are able to control processes in a way that is approximately optimal with respect to any given criteria.

Figure 1. Proposed creative learning algorithm structure.

Figure 2. Composition of the creative learning structure.

The use of the creative learning architecture in automated inspection systems promises to revolutionize the design of smart robotic camera inspection systems.

Ernie Hall, Xiaoqun Liao
Center for Robotics, Univ of Cincinatti
Cincinnati, Ohio
Prof. Ernie Hall is the Director and founder of the Center for Robotics at the University of Cincinnati. He also holds the Paul E. Geier Chair in Robotics. He is interested in machine vision and industrial robotics and merging these two disciplines to design smart robotic vision systems. In addition, he is a Fellow of SPIE and was a founder and has been co-chair of the Intelligent Robotics and Computer Vision conference for more than 20 years.
Xiaoqun (Sherry) Liao is a Ph.D. student in the Department of Mechanical, Industrial, and Nuclear Engineering at the University of Cincinnati. She has developed the theory of creative control and applied it to several applications including the Bearcat mobile robot as part of her dissertation. In addition, Ms. Liao is a frequent author and participant at the SPIE Intelligent Robotics and Computer Vision conference.