In the first comment to this post below (first comment ever, truth be told) Tom questioned my assertion that the patterns that emerge from lower level interactions are essentially illusory. He points out that our use of technology has been developed using higher level patterns observed without our awareness of the lower levels.
Considering his argument, I have to admit that was overbroad in my statements. There are many cases where one does not need to invoke the lower levels to describe the behavior of an entire population. In some cases, the behavior "flattens out", the chaotic effects cancel leaving an easily figured linear system. For example, one would not need to calculate the quantum paths of every quark in a ball to deduce the ball's trajectory. Another case is where a large number of "random" events form patterns that reflect the probabilities of those events. Insurance actuarial tables and radioactive half-lives are both cases where one can make predictions on the large scale but is denied predictive power over individuals.
Tom made a very good point about how those who understand the patterns that emerge have an advantage in increasing the value in their lives. I fully agree with that sentiment. The brain in a very complex system, and one of the ways that it is truly amazing is that it can create models of other complex systems. A person who can create a more accurate model of a prey animal's behavior will be a better hunter. If a person has a better sense of the behavior of the market, then that person would be the one I'd want managing my retirement fund. This process is not entirely conscious, I believe that this is from where feelings of intuition and hunches arise.
While there are many cases where complexity in nature has denied science predictability (weather, turbulent flow, etc.) I intended in the post below to apply the concept of chaotic unpredictability only to social systems. The simple systems that I listed previously are simple because there is relatively little feedback of information in those systems. I can say that a flipped coin will land on Heads 50% of the time precisely because each flip is not "aware" of what previous flips have done.
I changed the tag line of this blog just before posting this entry. It means that those who are aware of the rules and all of their implications tend to come out better. It is incomplete (I was going for pithy) because one would also need to know the conditions in which the rules are being applied. This is where chaos in social systems really get out of hand. The possession of information changes the system. This includes the degree to which patterns are recognized. There is an old schtick that is illustrative: I know that he knows that I usually go right, so I'll go left. But he knows that I know the he knows the I usually go right, so I can't go left. etc., etc., ad nauseum. The degree to which information is fed back into the system is huge, and any change made to the rules or knowledge gleaned about patterns resulting from the rules will have far reaching effects.
Have you ever read those "Investing Secrets from the Pros" in financial magazines? Don't bother with them. Those are patterns that have emerged from the rules of the market, and the only reason that they might have worked is that they were secret. Once a large number of investors start pursuing those strategies, the relative advantage decreases and there comes a counteracting force applied by those who wish to profit from the behavior of the first group.
I had meant the first post of this line to be a critique of social engineering in light of chaos theory and emergent phenomena. Much of social science in the past decades has been to develop rules that will provide predictive power over the actions of groups of people, an Ideal Gas Law for people or Laws of Mechanics for social institutions. The problem is that should those laws ever be discovered, we enter into a feedback loop (He knows that I know the he knows that I know...) where those laws themselves become causes generating unpredictable effects because no law can describe the repercussions of its own knowledge.
The feedback loop ensures that System I is not the same as System I + New Law or System I + Knowledge of Pattern. One might argue that such change would be minimal, but please recall that the Butterfly Effect tells us that there is no such thing as a negligible change. The systems may remain similar for a period of time (the period of time where the parties with the knowledge of the pattern are profitting), but after that period the systems diverge to where the Law or Knowledge are operating in conditions that they were not created to address.