The structure of the global catastrophe

45. Uncertainty of values of new terms

Describing threats of global catastrophe, we use the terms which values for the present have no unequivocal interpretation, at least owing to that they describe the events which have not happened yet and not created technologies. Sharing of several such terms increases «люфт» possible values and leads to misunderstanding. If we knew, that such "artificial intellect", we already could create it. It creates basic uncertainty of value of this term before AI will be created.

Chapter 3. As когнитивные the distortions, able to concern any risks, influence an estimation of global risks
1. A principal cause of human errors is the superconfidence

The superconfidence means the raised conviction in correctness of the picture the worlds and its inability it is essential to change at receipt of the new information. In other words, the superconfidence means inability of human to assume, that he at the moment is mistaken. Therefore in any sense the superconfidence concerns errors about the nature of errors. The superconfidence is fixed by several psychological mechanisms, and it is probable, had evolutionary value. Therefore to find out it in itself and to eradicate it very difficultly. Human showing the big confidence, can apply for the big power in a society. And, possibly, itself devices of human thinking contradicts idea of consideration of set equiprobable будущ: to think that something or is much more habitually, or it is not present. The word "future" is used in a singular, as though it one. At last, once the made choice towards one interpretation becomes the filter which selects the information so that she confirmed this idea.

2. Excessive attention to slowly developing processes and underestimation of the fast

Slow processes are more convenient for analyzing, and on them it is saved up more than data. However systems adapt for slow changes is easier and perish often from the fast. Catastrophe it is more dangerous than fading. Thus slow fading does system vulnerable to fast catastrophes. Working out of new technologies - нано, био, AI, когнитивные sciences - concerns fast processes. Their prompt development eclipses any processes which will prove after 2030. However it does not mean, that it is necessary to refuse the analysis of slow century changes absolutely.

3. Age features of perception of global risks

The youth is inclined to underestimate global risks as they in большей to a measure are biologically aimed at a gain of new spaces, instead of on safety. More older humans are inclined to give enough great value of safety, but it is more difficult to them to accept possibility of essentially new technologies.

4. Disputes do not give rise to true about global risks

Discussions between people usually lead to polarisation of opinions. Human who had in the beginning two hypotheses which attributed equal probabilities, reduces the position to one hypothesis opposite to a hypothesis of the opponent. Thus, it narrows the representation about the possible future. See article Yudkowsky about an estimation of risks more in detail.

5. Skill of conducting disputes is harmful

As I already spoke in the foreword, Yudkowsky underlines, that skill of conducting disputes is harmful, as allows to out-argue any opponent, not penetrating in essence in its position. Application of the given list когнитивных distortions to accuse of them of the opponent is especially dangerous. It can lead to intellectual blindness.

6. Desire of death

Freud has stated idea about Танатосе, aspiration to death which is at everyone. Similar ideas expressed also other scientists. This desire can неосознанно influence human, forcing it to underestimate or overestimate risks or to aspire to them. It is possible to assume, that any who rejects idea to live 1000, tests certain aspiration to death. It can unconsciously push human to choose the strategy conducting to global risks.

7. The conservatism of thinking connected with natural selection of the steadiest systems of outlook

C.Doukins, the author of the book «Selfish gene», considers each separate idea circulating in a society, as a replicator, capable to different speed of self-distribution and names such objects мемам (meme). That it will be protected from them, the human consciousness is compelled to develop original «immune system», one of which most effective variants is the system «reject all new». The history is full примерками of how obviously useful new ideas were rejected without the visible reasons during the long period. For example, from the offer to use закись nitrogen for anaesthesia in surgery in the end of XVIII century prior to the beginning of real application in 1840th years there have passed almost 50 years, the same concerns also rules to wash hands before operations. However in our century of changes very flexible and free thinking is necessary to consider and accept seriously all variants of global risks.

8. Detection of errors in a reasoning on possibility of a certain concrete catastrophe is not way of strengthening of safety

There are two kinds of reasonings - proofs of safety of a certain system and the proof of its danger. These reasonings неравноценны logically - in the first case it is a question of all chances, whereas in the second - at least about one case. To deny the general statement, enough one counterexample. However the refutation of one counterexample almost does not add the validity to the general statement.

For example: to prove danger of a certain plane, it is enough to specify of that in some experiments the covering material has shown propensity to effect of "weariness of metal». However to prove safety of the plane, it is absolutely not enough to find out an incorrectness in carrying out of these experiments on measurement of weariness of metal. Instead it is necessary to prove, that the chosen material will really sustain the given mode of loadings.

In other words, if to direct all intellectual efforts to a refutation of various catastrophic scenarios, not giving equal attention to their possibilities and errors in safety system, - that total safety of system will go down. All listed rules should be applied to search of errors in reasonings that a certain catastrophe is impossible, - then it promotes safety. In projects of difficult technical systems always there are «technical substantiations of safety» where «the maximum design failure» and ways of its localisation is described. The real proof of safety is strict the proof of that something cannot happen under no circumstances plus - practical experience of use of the given device in all possible modes for a long time.

From the point of view of a science we should prove, object existence, and from the point of view of safety ­­ - to prove, that something does not exist. The duty of the proof of safety lays on designers of the plane, instead of on passengers. Therefore the requirement of developers to extraneous experts: «prove, that the given risk is real» - the damage of the general safety causes.

9. Any of directions of researches in new technologies cannot provide the safety in itself

Because each system aspires to self-preservation, and conversations on danger of new technologies can lead to closing of new projects. For example, the site «Commercial biotechnology» reprints basically those articles which deny bioterrorism threats though then itself prints on them refutations. Or the same in a case with reactors. Supporters of the atomic power station will spend the efforts not for search уязвимостей in safety of stations, and on disputes with ecologists and attempts to prove, that the existing design is safe, is can reach attempts to stop up opponents.

10. Erroneous representation that when the problem will ripen then it is possible to start to prepare for it

The most serious problems arise suddenly. The more seriously a problem, the more its energy and - it is possible - faster rate of its development. And the more difficultly to it to prepare. Global catastrophes are powerful problems. Therefore they can develop too quickly that to them to have time to be prepared in process. Besides, we do not have experience which would allow to define harbingers of global catastrophe in advance. For example, catastrophes develop suddenly.

11. Concrete risks are perceived as more dangerous, than described in the general words

For example, "mutiny on a nuclear submarine" looks more устрашающе, than "large sea catastrophe". Yudkowsky writes: «From the point of view of probability theory, addition of an additional detail to history does by its less probable … But from the point of view of human psychology addition of each new detail does history more and more authentic».

12. Representations that the thinking about global risks - is pessimistic

It leads to that the people thinking of "doomsday", condemn - so, and deny their ideas. But on a minefield it is necessary to go it is realised: to dance on it blindly is not optimism.

13. «Plot theories» as an obstacle for the scientific analysis of global risks

Circulating in a society various «theories of plots» (like Fomenko's new chronology) have set the teeth on edge. As the majority if not all from them, false, and their predictions almost never come true. Often «theories of plots» too predict certain risks. But they structurally differ from the scientific analysis of risks. The plot theory usually asserts, that the mankind is threatened only with one risk, and this risk is realised by concrete image during the concrete moment of time: For example, «the dollar will fail in the autumn 2007». As a rule, the author also knows the recipe as with this risk to struggle. Whereas more correctly to assert, that the mankind is threatened with cascade reaction from several risks, and we do not know, when and that happens.

The is worse we predict the future, the it is more dangerous. The main danger of the future - its unpredictability. «Plot theories» are harmful to a future prediction as narrow representation about set of the future possibilities. Thus they assume superconfidence in own прогностических abilities. The good prediction of the future does not predict the concrete facts, and describes space of possible scenarios. And on the basis of this knowledge it is possible to allocate central points of this space and to protect them.

Moreover, such "predictions" undermine trust to the sensible ideas laying in their basis, for example, that large act of terrorism can weaken dollar and cause collapse chain reaction. Also that Ben Laden too understands it, and on it, probably, counts. «Plot theories» always mean, that there are certain THEY which with us do something, hide etc. It undermines comprehension of the responsibility for an event in the world and that is not less important, rejects an catastrophe role, as important factor of catastrophic process. Besides, «plot theories» are incapable to be joined with each other, formulating space of possibilities. And any theory of plot does not recognise itself as that. These theories extend in a society as мемы, self-copied information units.

At the same time from this, that the principle of the theory of plots is compromised also the majority of them false, does not follow, that some of such theories, nevertheless, cannot turn out to be true. «Even if you cannot catch a black cat in a black room is does not mean yet, that it there is not present».

14. The errors connected with mess of short-term, intermediate term and long-term forecasts

The short-term forecast considers a current condition of system, the majority of discussions concerns that on a theme of a policy. Intermediate term considers possibilities of system and current tendencies. Long-term considers only development of possibilities. I will exemplify it the following:

Let's admit, we have a ship with gunpowder on which sailors go and smoke makhorka. It is in short term possible to argue so: one sailors stand highly on a yard, and others sleep, therefore today explosion will not be. But in intermediate term prospect the quantity of gunpowder and quantity of smoking sailors which define probability of explosion because sooner or later any smoking sailor will appear in a wrong place are important only. And in дальнесрочной to prospect in the account there is only a quantity of gunpowder, and fire somehow yes will be. In the same way and with threat of nuclear war. When we discuss its probability the next two months, for us the concrete behaviour of world powers matters. When we speak about the next five years, to the account there is a quantity of nuclear powers and rockets. When we speak about prospect on tens years, to the account there is only a quantity of the turned out plutonium.

Thus in different areas of knowledge the time scale of brevity of the forecast can differ. For example, in the field of coal output of 25 years is a short-term forecast. And in the field of manufacture of microprocessors - 1 year.

15. Features of human emotion of fear

Ability of human to be afraid joins in reply to concrete stimulus in a concrete situation. Our emotion of fear is not intended for an estimation of the remote risks. It is expressed in Russian proverb: «While the thunder will not burst, the muzhik will not cross». On the other hand, the fear can suppress thoughts on possible dangers. For example, when human refuses to hand over analyses because is afraid, that at it something will find.

Concerning global catastrophes the vicious circle turns out: that the probability of a certain approaching global catastrophe became conventional - should burst "thunder" - that is there should be a certain event, is unequivocal its defining, and, probably, doing it practically inevitable.

16. Underestimation of value of remote events (discount rate)

Natural property of human is that it gives smaller value to the events which removed in space and have been kept away in time. For example, the flooding which has happened in an antiquity which have ruined one million human, will be equivalent on the importance to modern catastrophe with тысячью victims in the remote country or to a fire in the next house with several victims. It is known as «discount rate» - «discount level». This discount operates in the relation an estimation of utility and risk of the future events. Thus it has both rational, and irrational character. Rational level of the discount is a discount which is done by the economic subject, preferring to receive, say, 100 dollars today, instead of 105 in a year, - and this the discount is slowly decreasing function from time. On the contrary, the emotional estimation of a situation includes very quickly decreasing level of the discount which decreases гиперболически in time. It leads to that global catastrophe, отстоящая for years from the present moment, gets almost zero weight.

17. Conscious unwillingness to know the unpleasant facts

It is shown in a situation, when human postpones a campaign to the doctor not to know the unpleasant diagnosis. Though it reduces its chances to survive in the long-term future, but he wins in the quiet near future - that is here discount level too is shown.

18. Effect of displacement of attention.

The more a certain human gives attention to one possible reason of global catastrophe, the less it gives to another and as a result its knowledge gets certain shift towards its specialisation. Therefore revaluation of any one global risk conducts to underestimation of others and also is harmful.

19. The Internet as a source of possible errors

The Internet as sir Martin Rees underlines, I allow to create the newspaper «today», by selection only those sites which support a starting point of sight of the subject which then strengthens itself in the chosen position, constantly reading only the adherents. Not saying that on the Internet low level of reliability of the information as the considerable part of a good trustworthy information is in paid editions, for example, articles in Nature on 30 dollars for article, and any human can freely lay out the texts in a network, creating high information noise. Thus more sensational materials extend faster, than less sensational. In other words, if earlier the printed matter promoted faster distribution of better texts now the Internet promotes faster distribution of less qualitative, but more importunate in the advertising of sources. On the other hand, the Internet accelerates access to the information and accelerates its search.

20. Beliefs

Difficultly отрефлексировать the beliefs as they it is perceived as authentic knowledge or as an incompetent imperative, but it is easy to notice, how those or other superstitions influence an estimation of risks other people. For example, high breakdown susceptibility in Turkey is connected, in particular, with a principle «иншала» - on all will of the Allah: as destinies of all people are written already down in the book at the Allah it is not important, that you do - day of your death is already appointed. Therefore it is possible to risk somehow. Sir Martin Rees writes, that in Reagan's administration for environment religious fundamentalist James Uatt who believed answered, that the Apocalypse will come earlier, than oil stocks will be settled, woods will be lost and there will come global warming so to waste resources are almost that our duty. Many beliefs can carry more thin, pseudo-scientific or "scientific" character, as belief in correctness or Einstein's wrongfulness, danger or safety of this or that risk etc. Beliefs do not mean falsification possibility. To reveal belief it is useful to ask a question: «What event or a reasoning would force me to change my point of view to this question?»

21. Congenital fears

Many people have congenital fears - a dragon, heights, waters, illnesses. It is not difficult to assume, that they will overestimate the events reminding these fears, and to underestimate what are unlike them. An estimation of risks can influence and a posttraumatic syndrome when a certain strong fright was remembered and now all similar things frighten. On the contrary, strong desires can suppress fears, in the spirit of a saying: «If it is impossible, but very much it would be desirable, it is possible». For example, human with congenital fastidiousness can overestimate risk of biological war and underestimate risks of falling of an asteroid, and, on the contrary, human, gone through catastrophe, will exaggerate risks of falling of an asteroid, and to underestimate biological risks.

22. The error resulting because of struggle not with a source of risk, and with messages on risk

The discontent is transferred not on a source of risk, and on human who has informed on it, in the spirit of east rule: «to Kill the messenger who has brought a bad message». I repeatedly faced the rigid criticism directed on mine «humanal lacks», and charges in "paranoia" when suggested to discuss those or other risks.

23. Difficulty in delimitation of own knowledge

I do not know what I do not know. It creates sensation of that I know everything because I know everything, that I know. That is creates false sensation всеведения, conducting to intellectual blindness and unwillingness to accept new data. A.Kamju: «the Genius is the mind which has realised the limits». But while we completely do not learn the world, we cannot settle the list of global risks.

24. Humour as the factor of possible errors

The joke gives the right to human lawfully to speak a lie. Accordingly, it creates expectation at those who hears unexpected news, that it is a joke. For example, the security guard of embassy in Kenya has called the partner and has informed, that terrorists have approached on collars with a pistol; that has not believed and has switched off communication; the bomb has blown up. As we do not know, in what form to us global threat can come, we can apprehend the message on it, as a joke. It is possible to recollect and Reagan's joke that nuclear attack to the USSR will begin in 5 minutes which he has told for microphone check before performance that has led to reduction of the Soviet army in alertness.

25. A panic

The hypertrophied reaction to stress leads erroneous and dangerous actions. For example, human can jump out of a window at a fire though fire has not reached yet it. It is obvious, that the panic influences and thoughts of human in a stress condition. For example, one human, in a peace time adhering ахимсы (a yoga principle about неубиенAI live beings), during war developed the plan of explosion of dams in германAI to flood cities. That is the panic can be long enough condition essentially changing models of behaviour. But also the short-term panic is dangerous, as the situation of global risk can develop very quickly, in hours and even minutes, and strategic decisions should be accepted for this time.

26. Drowsiness and other factors of natural instability of the human consciousness, influencing occurrence of errors

On one of versions, Napoleon has lost Waterloo because has been chilled. How it is possible to expect, what the president will make the true decision, being woken among night? Here it is possible to add and basic inability of human precisely to follow instructions, and finiteness of length of instructions which it can understand and execute. Though it concerns first of all errors of operators, it is possible to present, that a condition of time turbidity of consciousness will affect and conclusions of the designer or the composer of a certain instruction, having led is admissible, to an error in the drawing.

27. Propensity of people to struggle with dangers which already were in the past

For example, was a tsunami of 2004, and now all began to build systems of preventions of a tsunami. And next time it will be not a tsunami. Thus with a current time the alarm of people decreases, and the probability of repeated strong earthquake (but not афтершока) - increases.

28. Weariness from catastrophe expectation

The error, consisting that after some catastrophe happens is typical, all start to expect repetition in the near future the second precisely same and after this expectation is not executed, translate this catastrophe in the category «it was for a long time and a lie». So was after act of terrorism on September, 11th. At first all waited for repeated attacks to skyscrapers, and building of skyscrapers in the world was braked. Now all about it have forgotten, and building of skyscrapers goes by accelerated tempo. It contradicts that in a reality of catastrophe of such scale can occur to periodicity in many years and consequently after a long time interval their probability really increases. The weariness from catastrophe expectation is expressed and in loss of sensitivity of a society to preventions.

29. The expert estimations which have been not based on strict calculations, cannot serve as a measure of real probability

Unlike a situation in the share markets where the average estimation of the best experts is used for a prediction of the future result, we cannot estimate and select our experts on global risks by quantity of the catastrophes guessed by them. Besides, that the rate in an estimation of global risks is very high - survival of mankind is does not lead automatically to that predictions become more exact.

In experiments on a prediction the following statistics is found out: «Only 73 % of answers on which have made rates 100:1, were true (instead of 99.1 %). Accuracy has increased to 81 % at rates 1000:1 and to 87 % at 10.000:1. For answers on which put 1.000.000:1, accuracy made 90 %, that is corresponding level of trust should generate rates 9:1. As a result, examinees often were mistaken even at the highest levels of rates. Moreover, they were inclined to do very high rates. More than half of their rates was more than 50:1».

«Similar levels of errors have been found out and in experts. Hynes and Vanmarke (1976) seven world famous geotechnicians about height of a dam which will cause destruction of the base from clay breeds have interrogated, and have asked to estimate an interval of 50 % of confidence round this estimation. It has appeared, that any of the offered intervals did not include correct height». The reason of this error is «the superconfidence of experts» - for example because the expert is afraid to lose the status of the expert if will doubt too the opinions.