Automated inspection applications for smart robot cameras
In a recent paper,1 we described a theory of learning called creative control (CC) and suggested it as a possible solution to camera setup and calibration problems that are encountered in a variety of automated measurement applications. In general, CC can be used in applications that are not necessarily repetitive, but more for plug-and-play and unstructured environments. The CC learning architecture does this by integrating a task control center (TCC) and a dynamic database (DD) that feed into ‘adaptive critic’ learning algorithms. Creative control can also enable a machine to adapt to, learn from, and predict changes in its environment using both perception and action. This could lead to a new generation of smart robotic cameras that automate and simplify industrial sensing by making setup and use easier.
In CC, both decision and estimation theory are used and modeled by neural networks. The intelligent system acts as the decision maker for the dynamic system that, in turn, makes decisions in stages. The outcome of each decision may not be fully predictable but can be anticipated or estimated to some extent before the next decision is made. Furthermore, an objective or cost function can be defined for the system. Generally, the goal is to minimize this cost function over some decision space subject to a set of constraints, which may include natural ones. With this definition, the intelligent robot camera can be considered as solving a set of problems in dynamic programming and optimal control.
Dynamic programming (DP) is the only approach for sequential optimization applicable to general nonlinear, stochastic environments. However, DP needs efficient approximate methods to overcome its dimensionality problems. It is only with the presence of artificial neural networks (ANN) and the invention of back propagation that such a powerful and universal approximate method has become a reality.
The essence of dynamic programming is contained in Bellman's Principle of Optimality. The original Bellman equation, used for the adaptive critic algorithm, may be written as:
Dynamic programming gives the exact solution to the problem of how to maximize a utility function U(R(t)) over the future times, t, in a nonlinear stochastic environment where the vector R(t) represents the state of the environment at time t. Dynamic programming converts a difficult long-term problem in optimization over time (< U(R(t)) >, the expected value of U(R(t)) over all the future times), into a much more straightforward problem in simple, short-term function maximization—after we know the function J. Thus, all of the approximate dynamic programming methods discussed here use some kind of general-purpose nonlinear approximation to the J function, the value function in the Bellman equation, or something closely related to it.
In most forms of adaptive critic designs, we approximate J using a neural network. Therefore, we approximate J(R) by some function Ĵ(R,W), where W is a set of weights or parameters. Ĵ is called a critic network.
If the weights W are adapted or iteratively solved for in real-time learning or offline iteration, we call it an adaptive critic. An adaptive critic design (ACD) is any system which includes a component of this type; a critic, in turn, is a neural net or other nonlinear function approximation that is trained to converge to the function J(X). In adaptive-critic learning designs, the critic network learns to approximate the cost-to-go or strategic utility function J and uses the output of an action network as one of its' inputs, either directly or indirectly. When the critic network learns, back propagation of error signals is possible along its feedback path to the action network.
Most advanced methods in neurocontrol are based on adaptive-critic learning techniques consisting of an action network, an adaptive critic network, and a model or identification network, as shown in Figures 1 and 2. These methods are able to control processes in a way that is approximately optimal with respect to any given criteria.
The use of the creative learning architecture in automated inspection systems promises to revolutionize the design of smart robotic camera inspection systems.
