Monday 16 July 2012

Notes about complexity - some theoretical background


We live in a boom period of both technology and communication, which provide us with more convenience and opportunities. However, what inherent in this boom is the inevitably increasing complexity in product development. More diverse demands from customers and advanced technologies drive enterprises to provide products with high variety. And the growing cooperation among organizations generates a more complex global network. Also, the environments, such as market, technology and policy, are changing more frequently because of the fast development. Great product variety, complex dependencies among organizations and unpredictable environment all contribute to continuously increasing complexity.
But are we prepared for this increasing complexity? A CEO study conducted by IBM in 2010 showed that Canadian CEOs anticipate much more complexity than they feel confident about handling. Seventy-eight percent expect the level of complexity to grow significantly over the next five years, but only 36 percent believe they know how to deal with it successfully. In this situation, it is really necessary to find some way to mitigate complexity.

It is pretty interesting to see complexity in the perspective of “pure science”. Until the early 20th century, classical mechanics, as first formulated by Newton and further developed by others, was seen as the foundation for science. Not only physics, but also other disciplines adopted the Newtonian worldview. So the traditional science is often referred to as “Newtonian science”.
One important principle in Newtonian science is reductionism, meaning that to understand any complex phenomenon, you need to take it apart, reduce it to its individual components. For example, people learnt about life by observing cells through microscope, learnt about some properties of water by analyzing the molecules. Despite the success, this approach is one-sided. We can ask questions like how these cells organize themselves into something that is alive, and how these atoms go together into a complex whole with new properties emerged. These cannot be answered by reductionism. An opposite idea “holism” was proposed in 1926, indicating that a whole is more than the sum of its parts. There are not only parts, but also interactions and relationships between parts in a system. And a holistic view should be adopted to understand complex systems.
Another implication of Newtonian science is determinism.  Because it was believed that the nature is governed by deterministic laws of cause and effect, so if you know the initial positions and velocities of the particles constituting a system together with the forces acting on those particles, then you can predict the further evolution of the system with complete certainty and accuracy. The famous quote of Einstein “I am convinced that God does not play dice” reflected the concept of determinism. However, at the beginning of 20th century, determinism was challenged by quantum mechanics, as it implied the unpredictable properties of particles. Also, chaos theory stated tiny differences in initial conditions   yield widely diverging outcomes for chaotic systems, making long-term prediction impossible. This is well known as the “butterfly effect”.
These developments such as holism and chaos theory are being integrated into complexity science. So we can see complexity science emphases wholeness, dependencies, uncertainty, etc. It shows a new way of thinking, a different perception of the laws of nature and changes the approaches required to understand the world.

The changes also happened in product development (PD). Research on PD has emphasized the dependencies between system parts, showing a holistic and systematic view; and much work has been done on handling the uncertainty, improving flexibility in PD, showing the realisation that the evolution of system is unpredictable. With this new way of thinking, many researchers have contributed to the field of PD to define, to measure complexity, and to propose strategies to manage it.
 As to the measurement of complexity, some considered complexity in multiple attributes such as structural, design, production and aggregate them to measure the total complexity. There are also researchers looking into functions. Some considered the number of functions and others also considered the level of functions in a functional hierarchy. Besides, many authors regarded uncertainty in information as equivalent to entropy in statistical mechanics, so they used entropy as a measure of complexity.
Many strategies have been introduced to mitigate complexity. One approach that has been successfully applied is scrum. Scrum is an agile software development method. It acknowledges that the development processes are incompletely defined. It uses more frequent inspection to perform a prompt adaption to the constantly changing environment. Some matrix methods were also used to manage complexity. These approaches provide insights into the relationships of components within or between complex systems, and thereby facilitate the reduction of uncertainty in PD. Matrices are graphs in another form, so graph theory here provides the mathematical basics for matrix method. Another way to reduce complexity is modularity. Modules  are  units  in  a  larger  system  that are structurally  independent  of one another,  but work together.  The system provides a framework that allows for both independence of structure and integration of function. Modularity reduces diversity in variants from design as well as from customers. Multi agent control was also applied to reduce complexity. A multiagent system (MAS) consists of a collection of individual agents. Each agent displays a certain amount of autonomy so multiagent systems were able to adapt to changing environments and more flexible.

To summarise, globalisation and fast-paced development of technology have caused a steady increase in complexity in products and thus in product development, which calls for measures to be taken to manage this complexity. In contrast to classical science, complexity science holds a different perception of the world; it emphases relationships and wholeness, and admits that there is uncertainty in a system. Complexity research in product development also adopts this new perspective, so much effort has been done to measure and manage complexity based on dependencies, network, uncertainty, etc.

However, I find many metrics and management method are mostly based on experience or intuition, without a rigorous validation.Some questions:
How to define complexity?
How to validate a management method?
How to measure the effectiveness of a method?

P.S.
The Mandelbrot fractal is so amazing. Mathematics is a beautiful language as well as delicate art.

No comments:

Post a Comment