Fuzzy Dynamic Programming: A Step Towards Cognitive Dynamic Programming

This presentation concerns dynamic programming which is, roughly speaking, an optimal procedure to solve problems that have a multistage character, that is, when decisions over some, finite or infinite horizon, are to be taken at some consecutive stages, notably time related. It has been developed just after World War II by Richard E. Bellman, one of the most famous control and systems theorists, and�more generally�one of the most famous applied mathematicians. Since its inception, dynamic programming has been extensively applied to solve a multitude of problems in a wide array of areas, exemplified by operations research, mathematical economics, control, systems analysis, etc. Basically, the essence of dynamic programming, which boils down to finding an optimal policy, refers to simplifying a complicated problem by breaking it down into simpler subproblems in a recursive manner, all that in an extremely simple and effective way, though not always efficient (the infamous curse of dimensionality). This is clearly closely related to how humans approach problem solving, and also how they evaluate control trajectories among which an optimal one is to be found. That power of dynamic programming and its universal importance and applicability has inspired Ballman and Zadeh to propose in their 1970 article the use of fuzzy sets, notably via fuzzy constraints on controls and fuzzy goals on states/outputs, to extend the basic dynamic programming model into fuzzy dynamic programming. That foundational model has then been extended to, e.g., fuzzy termination time, fuzzy systems under control, infinite termination time, etc. by Kacprzyk, summarized in his books of 1983, 1997 and 2001. Moreover, the use of neural networks as models of such multistage decision/control processes has been proposed by Francelin, Gomide and Kacprzyk, and then in a more comprehensive modeling and optimization setting by Jun Wang. Kacprzyk�s models of fuzzy dynamic programming have been applied to solve many real life problems, notably related to the modeling and planning/programming of sustainable regional development, and it turned out that an inherent �softness� and human consistency of fuzzy dynamic programming, which results just from a softer definition of constraints, goals and a (separable, i.e. via some proper aggregation operators) performance function is not enough. In particular, the human planners or decision makers have tended to perform a much more sophisticated assessment of the development trajectory (the consecutive decisions/controls and outputs) that by its simple efficiency (a high value). In particular, they have advocated a more sophisticated assessment of variability, stability, balancedness of consecutive controls (expenditures) over time and subject area, etc. Clearly, this all was imprecisely specified and could be well represented by fuzzy sets. This all has implied a necessity to reformulate the basic fuzzy dynamic programming model, and not all resulting performance function have had a separable form to use fuzzy dynamic programming. In fact, the model developed could have been better termed a fuzzy modeling based dynamic programming, and actually its first version was quoted as one most successful examples of fuzzy modeling in L.C. Thomas (Ed.): Golden developments in operational research, Pergamon Press, 1987. The model has been extended along that line. In the presentation, we will first present that comprehensive fuzzy dynamic programming based model, emphasizing the role of human specific judgments, assessments, perceptions, etc. We will basically follow the human centric system paradigm in the spirit of MIT (notably advocated by the late Michael Dertouzos), though in decision/control perspective. Then, we will show that it would be beneficial for the field in question, i.e. fuzzy dynamic programming, or more generally the formulation and solution of multistage decision and control models meant for human centric systems, to find some more general paradigm. It should make it possible to better reflect the very process of developing a multistage model by the human, but even more so a better way to evaluate trajectories (decisions/controls and outputs), and find best (possibly optimal) solutions. As a first alternative in the above search for a proper paradigm, we will consider as a potential candidate cognitive informatics introduced by Yingxu Wang. It is basically a multidisciplinary field within informatics, or computer science, that is based on results of cognitive and information sciences, and which deals with human information processing mechanisms and processes and their decision theoretic, engineering, applications in broadly perceived computing, including our recursive fuzzy dynamic programming scheme. The purpose is to develop and implement technologies to facilitate and extend the information acquisition, comprehension and processing capacity of humans to overcome some cognitive difficulties related to the presence of the human being as a crucial part of the system (in our case, an inherently optimization focused problem), exemplified by abilities like a limited comprehension, memorizing, learning, choice and decision making, satisfaction with partial truth, allowing for not perfect solutions, etc. These issues are considered and solved using tools and techniques derived from many areas like psychology, behavioral science, neuroscience, artificial intelligence, linguistics, etc. In our case, we will concentrate on some cognitive informatics type elements that mostly have been inspired by psychology and behavioral sciences, as our problem is inherently related to human judgments and perceptions, but we will mention some inspirations from neuroscience, notably along the lines of neuroeconomics. The new fuzzy dynamic programming models presented, in which the above human specific aspects will be shown and analyzed, will be illustrated for clarity on a sustainable regional development considered in terms of expenditures, subsidies, life qualities, etc. We hope that our proposal can be a first step towards what might be termed cognitive fuzzy dynamic programming which might both shed a new light on the very essence of how multistage decisions are made in human centric problems, how inherently imprecise human judgments should be reflected, and how to reflect human evaluations of time evolution of performance indicators so that a "meaningful" optimal solution be obtained.