Swarm Intelligence, Emergent Behavior

A curious quirk of contemporary discourse is the resolute refusal to reason about ourselves in the same way we reason about other animals, despite the fact that humans are – for all practical purposes – highly advanced animals.

Natural scientists have spent many years documenting the mechanics of swarms in various animal species:

  • Bird flocking appears to require an extreme level of coordination, but in practice it is simply an outcome of individual birds following a few very simple rules for motion. Schooling fish move in a similar way. This is considered an emergent behavior – an accidental collective behavior that is not conscious or deliberate on behalf of individuals.
  • Foraging ants tend to go off in completely random directions, but leave invisible chemical trails (pheromones) that are further reinforced when food is detected/transported. This can be thought of as swarm intelligence – creatures without much individual intelligence solving complex problems by each solving a tiny part of that problem, using some form of communication and social organization to ensure a reasonable division of labor.

You’ll never hear humans described in terms of emergent behavior or swarm intelligence (well, almost never), despite the fact that all of our individual brains could be described as exactly this type of system: 100 billion neurons each without any real smarts of its own, all firing according to simple rules to create the impression of “intelligence”. It is as if the idea of a swarm is somehow beneath us as a species, or perhaps an honest and in-depth study would reveal some unpleasant truths about the so-called “wisdom of crowds”.

My guess as to why this topic doesn’t come up much in human biology: the results would be hard to understand. Flocking and foraging are complex behaviors for birds and ants, but still appear simplistic to our massively-evolved human brains. However, a human swarm intelligence necessarily implies something much more intelligent than any individual human, and we can’t understand higher intelligence.

Emergent behaviors among groups of humans surely exist, and yet we struggle to see the pattern. We only seem to become aware of their existence through the proxy of easy-to-measure collective outcomes: income, crime, voting, health. As per today’s fashion, we paper over these inexplicable collective differences as being rooted entirely in geography or family history. Some of that is probably true; we adapt to our environment, and epigenetics can transfer some adaptations to descendants. Clearly the equation is more complex, given that these conditions persist after decades of deliberate screwing with the geographical and socioeconomic equation.

It’s also become fashionable to describe ideologies (including religion) as a kind of “mind virus” that infects individuals and spreads to entire groups and societies, more popularly described by the term memetics. However, given what we know of swarm intelligence and emergent behavior, we should consider the alternative: perhaps these fashionable thinkers are reversing cause and effect.

Perhaps ideas, memes, and culture are not independent entities, even in the abstract. Perhaps these constructs are simply the outcome of certain groups under certain conditions. Perhaps these transcendent properties are simply the emergent behavior of groups. Perhaps memes do not transmit culture, but are simply an expression of culture. Perhaps the Great Meme War is just real-life emergent gameplay.

I’m not going to set out to prove my hypothesis. I don’t have a massive research team or several years to conduct experiments. Should you choose to consider it, however, the hypothesis has a lot of explanatory power. It enables us to make useful distinctions between individual behavior and group behavior. There may even be some mathematical law that predicts a likelihood of certain behaviors emerging based on the number of members of a particular group, with additional second-order effects from extremely large groups.

There could be consequences for social groups and working groups, in terms of both their total size and their distribution. It might demonstrate some unintended negative consequences for dense urban development, immigration, diversity programs, and even social media organization.

Hm… probably best not to investigate, then. We can’t afford to have unpleasant facts disturbing the order imposed by our new state religion.

We Can’t Understand Higher Intelligence

I’m always amused when I see someone pronounce on social media that they’ve “solved” the problem of artificial superintelligence, or insist that they have a 100% ACCURATE! prediction of where it will lead, often used as flimsy pretext to justify some awful idea like Universal Basic Income. This, despite the fact that some of the brightest minds alive today have been working on the Friendly AI problem for over a decade and still aren’t even confident in their predictions, let alone their solutions.

Too much has already been written on why we should or shouldn’t be worried about ASI. If you’re unfamiliar with the debate, there’s a good summary and great infographic at Future of Life. I won’t rehash that here. Instead, I want to explain why there are so many terrible ideas and predictions floating around the “I F***ING LOVE SCIENCE!” crowd (i.e. not scientists and certainly not AI researchers). And indeed, how this very same problem applies to human intelligence and infects every aspect of social and political thinking.

A good starting point is the Dunning-Kruger Effect. The least-able are most likely to overestimate their ability. Even those who know they are below average tend to be way off in their estimation of how far below average they are, and cannot even conceive of the different levels of mastery. Ironically, knowing about Dunning-Kruger does not make one immune to it, leading to some embarrassingly cringey articles from self-important journalists. (I’m sure that conservative writers have done this too, I just… can’t seem to find any.)

Dunning-Kruger explains why, as a middling chess player, I can predict who will win in a game between amateurs, but have no clue what’s going to happen next in a grandmaster game. It also explains why many new business owners have very high turnover; they’re still learning the trade and can’t tell good from bad, and have to use a trial-and-error approach to hiring. Rating systems address this; in chess, it’s completely objective, and on Yelp it’s very subjective but still a decent predictor of outcome. With intelligence, the objective rating is IQ.

Despite appearances, I’m not an IQ-ist. I have never asked anyone for their IQ, nor told anyone mine without having explicitly been asked. You don’t need to be smart to be successful, or even to master a particular trade. IQ is not a reliable individual predictor of life outcomes. At an aggregate level, however, it informs us of certain social outcomes. A phenomenon called assortative mating explains why successful relationships tend to involve partners of similar IQ, which itself explains why marriage is for the rich. It also explains why high-IQ nations have more economic output than low-IQ nations. A lot of people know this, but what they do not realize is that the relationship between average IQ and collective outcome is not linear, it’s exponential.

The exponential relationship is important. We measure IQ on a bell curve, but the measurement itself is more like a decibel of sound than, say, a degree on a thermometer. Various high-IQ societies have each done their own analyses, concluding that an approximate 5-point increase is equivalent to double the actual intellectual performance (i.e. problem-solving speed). So, on average, a 150-IQ individual can solve problems about 60 times faster than a 120-IQ individual, and more than 1000 times faster than a typical 100-IQ individual.

Those numbers are insane to think about. Try to imagine driving your car, on the same roads you’ve always driven on, but at 3000 mph. Or 50,000 mph. It’s all just a blur at that point, and the 3000 mph blur doesn’t feel much different from the 50,000 mph blur; either way you’d probably crash instantly. An X-15 pilot could relate to 3000 mph in the wide-open skies, but navigating ground traffic over short distances at that speed would still be inconceivable.

But now imagine that you can drive at a normal speed of 50 mph, and everyone else around you is limited to 1 mph. A few thoughts might cross your mind:

  • Your commute time would be way shorter than everyone else’s.
  • Being stuck behind a 1 mph vehicle would drive you crazy.
  • Anyone else going much faster than 1 mph would stand out. A lot.
  • You still wouldn’t be able to see a car going by at 3000 mph.

It’s not too difficult to imagine other people being slower than you – either physically or intellectually. You won’t really understand or empathize with their experience, but you can interact with them, and you can predict their behavior. However, none of us – not even the smartest of us – are capable of even imagining higher intelligence than our own, because if we could, then we’d be more intelligent ourselves. We can imagine the outcomes of being super-smart, like having a dozen Ph.Ds and starting 50 wildly successful companies, but not the actual process of getting from here to there.

The exponential relationship between ability and outcome is described by a Pareto distribution or power law:

ability-achievement2

Ability can be intelligence, or anything you can observe or measure. These distributions pop up everywhere, by the way, as the fabled “80/20 rule”, although in reality it’s often more like 90/10, or even 99/1. It all depends on how far right the x-axis goes. In the above example, more than 4-5 standard deviations above average ability is literally off the chart for achievement. Not every field of human endeavor will have this exact scale, but almost all have this general shape.

If you equate “achievement” to “wealth”, and you imagine (incorrectly) that the amount of wealth in the world is fixed, then this graph looks terrifying. However, if achievement represents the production of wealth (or other resources), all of history starts to make sense. The poorest family in America today lives better than the richest kings and aristocrats of Europe in the middle ages, and it’s all because of the achievements of a very small number of inventors, entrepreneurs, artists, military generals, and so on.

As historical figures, we hold prodigies like Rembrandt and Edison in high respect, even reverence. They advanced civilization by leaps and bounds. Yet today, the trend seems to be fear and jealousy, as though these “1 percenters” are vampires feeding off us plebs. The reality is, if I, not Steve Jobs, had been the CEO of Apple, you wouldn’t have your iPhone, and Apple probably wouldn’t exist anymore. If you, not Lincoln, had been president during the Civil War, America wouldn’t be a single country. These outcomes required unique individuals.

Maybe the resentment was always there, and just omitted in the history books. Either way, the more heterogeneous a group, the more resentment you seem to get. Every identifiable subgroup seems to be equally hypocritical, believing that the lower-achieving subgroups simply don’t have the same ambition or ability (which is mostly true), but at the same time deluding themselves into believing that higher-achieving groups got there by cheating. This is essentially the basis for all collectivism and identitarian beliefs, which are best described as weaponized intellectual laziness rather than coherent ideologies.

An artificial superintelligence would be way past the edge of today’s Pareto distribution. The ASIs would become responsible for nearly all “human” achievement, unless we could keep up via genetic enhancement and technological augmentation. If we lag behind, then we would all become insignificant underachievers compared to the intellectual and creative marvels produced by the supers.

What I wonder is: are we ready? Assuming, hypothetically, that ASI is Friendly, are we emotionally and intellectually mature enough to deal with a social class over and above the current billionaires? Machines that we can’t even begin to understand, but are nevertheless responsible for managing vast amounts of resources and producing almost all of the new goods and employment opportunities? I’m not really worried about superintelligence destroying jobs or culture, because that’s not what actually happens when you add super-producers to a society. What I wonder about is whether we would be able to accept the new reality, or whether humans would collectively become so bitter that they’d immediately try to destroy it.

Futurists believe that ASI will save us and deliver a post-scarcity economy. I’m not sure if we could handle it. My hunch is, the only way we’ll be able to truly advance beyond General AI is by improving ourselves, not our machines.

Success = Intelligence × Liberty

In knowledge industries like engineering and finance, debates rage on endlessly about “best practices”. Should you do pair programming, or rely on thorough code review? Is market research valuable, or should companies exclusively do A/B testing? Will bilingual education improve overall student performance, or is it a waste of time?

Ask any expert who does not have a conflict of interest – i.e. they’re not selling their own “system” – and they’ll eventually admit that these best practices aren’t nearly as important as the people on your team. Process can help at the margins; for example, I’d probably be nervous flying with any pilot who didn’t use a pre-flight checklist. However, given the choice between a pilot without a checklist and a pilot who does not know the definition of altitude, I’ll pick the former every time, no matter how much training the second guy supposedly has.

There has been much hand-wringing of late by social activists desperate to salvage the “blank slate” theory that literally everything is socially constructed, and with the right education and environment (and rules – lots of rules!) we can prepare anyone for any kind of work. They then push insane solutions, making the bizarre claim that adding more non-whites racial diversity and women gender diversity will improve performance, apparently oblivious to the inherent contradiction. The reality is as brutal as it is obvious: individual IQ determines group IQ. All prior “studies” that claim otherwise have failed to control for individual IQ. You can’t fix stupid.

Aside: All of the above refers to g-loaded tasks (AKA “knowledge work”). g is the symbol for general intelligence, and IQ is an indirect but highly accurate measurement of g in terms of how well it predicts performance on a wide range of cognitive tasks. Given our knowledge that females cluster more tightly around the mean, there is in fact a real scenario where hiring as many women as possible is the optimal choice: jobs that are moderately g-loaded, where the avoidance of low-g employees is more important than the recruitment of high-g employees. Consciously or unconsciously, society knows this, which is why women dominate the service sector. Average is good when it comes to social interaction and tasks that are mostly repetitive with a small degree of improvisation. I’m not mad that men are over-represented in prison, and women shouldn’t be mad that men are over-represented in STEM; in both cases, we’re dealing with people at the extreme end of the g-curve.

Returning to the central thesis: It should therefore be no surprise that high-IQ societies overwhelmingly fare better than low-IQ societies:

iq_by_countryGdp_per_capita_ppp_world_map_2005HDIdemic1

The correlation is near-perfect aside from some anomalous results in China, which I’ll get to in a moment. It’s actually astonishing how, despite the massive cultural, economic and political differences between the USA, Argentina, Russia, Australia, and the various European nations, they have all done pretty well. It seems that a nation built on petty criminals can thrive if they are smart criminals, and once-great nations will eventually bounce back even after decades of oppression by mass-murdering Communist dictators. Governance is a heavily g-loaded task; highly-intelligent groups have inherent capacity for self-governance and will naturally organize themselves into some sort of hierarchy (the specifics of which are culturally-dependent), whereas low-IQ groups will govern like children, subject to laziness, paranoia, and tribal violence.

Some may be tempted to cry cultural bias or assert that the causative effect is backwards – that high development leads to high IQ, not the other way around. These things are not mutually exclusive, and causation can actually go both ways in a virtuous cycle. At least, that seems to be the only explanation why average IQ for a race residing in the USA or Canada is about 10-20 points higher than the exact same race elsewhere in the world; Ashkenazi Jews average 110-115 in the USA but only about 100 in Israel, and Africans average around 85 in the USA vs. 70-75 in Africa. It’s hard to pinpoint the exact cause of this, as it still seems to be a taboo subject for researchers, but I will use some logical inference to make an educated guess that the mystery ingredient is liberty.

The combination of libertarian economics and traditional marriage (monogamous, but not arranged) are inherently eugenic. Societies that uphold these values also embrace the inverted-pyramid model of human development – the reality that phenomena like Moore’s Law require continued breakthroughs in scientific, mathematical, technological and business (entrepreneurship) fields, and that these breakthroughs are the result of exceptionally-talented individuals. 100 average scientists will ultimately contribute less than a single Newton or Einstein. These individuals create entirely new branches of science and industry. The trouble is, we can’t simply make more Einsteins or even identify them in advance. The best system we have at present is one that supports free inquiry by as many smart individuals as possible (libertarian), and also tries to make as many new smart people as possible by implicitly linking material success (which correlates highly with intelligence) with genetic success and providing stable two-parent homes. If a better system exists, no one has found it yet.

By contrast, Marxist/Leninist systems are inherently dysgenic. From a purely economic point of view, if you subsidize low intelligence, you’ll get more of it. Restricting free association and free speech makes scientific inquiry very difficult, so the most talented people will flee on principle alone. Once the economic downturn of late-stage Communism hits, the marginally-talented people will start to flee for economic reasons. Any talented people who remain will tend to be persecuted (c.f. the Kulaks, or more recently, Venezuelan bakers) if not killed outright. As the saying goes, collectivism is always one execution away from utopia. It is through this ruthless suppression of individual success – and by consequence, individual intelligence – that entire societies can not only halt economic and scientific progress but also wipe out their entire gifted-to-genius population for several generations.

Which brings us back to China. They don’t have Chairman Mao anymore, and have moved from pure Communism to a strange mixed economic model, but compared to Russia, they took a very long time to recover. This explanation for their comparatively slower development is corroborated by U.S. census data, which shows that Asians (predominantly Chinese) in America are more successful than Caucasians by a wide margin. Ethnic IQ would predict this outcome on a local level, but it also rules out the possibility that ethnic Chinese have some other genetic disadvantage that cancels out their higher IQ. In other words, the problem is China itself, not the Chinese people. And it’s surely not geography, because China has plenty of arable land and access to waterways. It’s the Chinese government and national culture. That is gradually changing, and if those changes continue, they will catch up to the west.

Technology is getting smarter and will continue to replace low-g tasks with automation. It’s more important than ever that we find some new breakthroughs and create new industries, and that means pumping out as many geniuses and entrepreneurs as possible while maximizing their opportunities for success. That’s going to require significant changes to the way we think. If we want to come out of this century alive, we can’t afford to make sympathy our first priority.

For those of us still living in an “average” (>= 100) IQ society, our first priority must be keeping western society above the breaking point of IQ 97, which means severe restrictions on immigration and a permanent end to the welfare state. Our second priority should be throwing every available resource at finding a reliable and repeatable way to raise the average IQ by even a tiny amount, and exporting it across the globe. We can’t be greedy here; boosting global IQ by even a few points will dramatically lower global conflict and bring many new research partners to the table for the third priority: finding some way, whether biological or technological, of massively raising general intelligence. Most of the world needs at least a 10-point gain, and if we want to fix the problems in Africa, we need a 30-point gain.

Ignoring or denying the reality is civilizational suicide. Whether by war, invasion, civil strife, or plain old overpopulation, western society is vulnerable. This is more important than climate change, and it’s damn well more important than manspreading. We have the knowledge, talent, and technology in the west to make real progress on this issue, without cruelty or violence, if we could only break through the social taboos and get serious about it. Our future depends on it.

The National Review Has Lost The Plot

Remember when the National Review offered high-quality writing and analysis and a great alternative to the mainstream media? Those days are officially over.

lalball-nr2

Behold their review of the Second Battle of Berkeley, where they have some whining strong words for the authorities:

Urban and academic progressive leaders can respond to violence with all the scolding tweets, sternly worded statements, and calls for calm they want. But until those who break the law and violate university policies are aggressively brought to justice, it won’t matter.

To summarize: For a solid 6 months, the American left has been refusing to accept or even acknowledge the results of the 2016 national election. They’ve been destroying property in the capital, planning acid bombs at conservative events, busing paid protestors to heckle town halls, beating up every Trump supporter they can find, sucker-punching anyone they can manage to associate with the dreaded “alt-right” and then bragging about it on social media, all while the university faculty clearly side with the Bolsheviks or even join them, and the police are visibly standing down. The National Review has apparently just now realized that the left has gone feral, and their response is to politely demand that the authorities step in.

Unfortunately, this seems to be what mainstream conservatism has become. It’s a literal mirror image of the intellectually lazy limousine liberal solution to every social ill: Point at it, clutch pearls, shriek “somebody should do something!” then move on, secure in the knowledge that it is Somebody Else’s Problem. At the National Review, we’re always just one step away from the edge of the cliff but somehow never fall and can always easily turn back.

Meanwhile, out in the real world, the right wing has been predicting this ever since Hillary’s “we go high” speech, and the emergence of characters like Based Stickman merely reflect a decision to start fighting back. Even many classical liberals have figured it out by now: we’re not “risking” a civil war, the war has already started, and the evidence is literally staring us in the face. The black blocs aren’t going to just calm down, crawl back into their dorm rooms and transfer from White Privilege Theory into Applied Mathematics. They are going to keep escalating until either they get everything they want or they are physically incapacitated. Weimerica isn’t just some weird neoreactionary meme anymore, it’s officially arrived at Berkeley and probably coming to a campus near you.

Not content with the level of cuckoldry already on display, French wraps up with this virtue-signaling howler:

We are now teetering on the edge of a truly terrifying incident, one trigger-pull away from a slaughter. Campus and urban progressives have a choice to make. Is this a nation of laws? If it is, then it’s time to grow a backbone, protect free speech, punish rioters, and expel those who disrupt the educational environment regardless of ideology. There should be no more sympathy or leniency for the lawless social-justice warrior than there is for the lawless neo-Nazi.

“Remember, all violence is bad, and both sides are equally guilty. Did everybody hear that? Please don’t hurt me.”

Clearly, he knows who the real instigators are, but like a good little housebroken fakeservative, he wraps it up in layer after layer of obfuscation, making sure to differentiate himself from those dastardly “neo-Nazis” whom we keep hearing about but no one can actually seem to find. Grow a backbone – not by fighting back, of course, but by taking to social media and shaming the thugs who kicked your face in. Everything will sort itself out, if you just cower in the corner and pray for the Rule of Law to return.

The National Review has been going downhill for a long time, but if they can’t even take a firm stand in the face of bona fide domestic terrorism, then they have crossed the final bridge into total irrelevance.

It’s a good thing that the Old Right is a dying breed. With friends like this, who needs enemies?

The Weakest Link: Transitive Trust

Raise your hand if you remember going through this sequence of events at least once in your life:

  1. You have a favorite doctor, hairdresser, babysitter, or some other professional whom you trust.
  2. One day, that person isn’t available. Maybe they are sick, vacationing, retiring, or moving away.
  3. “But don’t worry!” they say. “I have a friend/colleague who will take great care of you. Here, I’ll introduce you.”
  4. You follow their advice… and immediately regret it. Their replacement is awful. Incompetent. Maybe even rude.
  5. It takes several days or weeks to clean up the mess, and then you have to start from scratch on finding a new replacement.

If this story sounds familiar, then you’ve experienced firsthand the problem of transitive trust: You trust person X, exclusively. Person X trusts people Y and Z and chooses to delegate your trust to them. Depending on what they are delegating, the consequences can range from minor nuisance to life-threatening disaster.

During my Infosec days, I was introduced to this problem when my small employer was bought by a larger corporation. We had our network, and they had theirs. Our employees needed access to their network, and vice versa, but one of us had a much stricter security policy. That meant several months of periodic outages, lockouts, general confusion, and painful arguments about the policy itself.

Another example in the tech world is ads; even if the folks who run your favorite news site are squeaky-clean, they may choose an ad network that does not properly vet its advertisers, thus exposing readers to malware. Ad blockers are becoming a security feature.

Later on, I learned that this problem doesn’t only apply to systems; it also applies to people, as in the first example above. The difference is, in meatspace, we usually see it as an isolated incident and fail to identify the earliest point when our decision-making process started to go wrong. Many if not most of us will even continue to make this mistake over and over again, assuming that trustworthy or competent people can reliably deliver us other trustworthy or competent people, even though events rarely play out that way.

The human version of the problem is simple: Doing is not the same as delegating. They are different skill sets, but for some bizarre reason, our brains seem hardwired to believe that they are the same. It’s true that those with a certain skill are better at recognizing that skill in others via the quality of their work or technique; for example, I can easily identify a great coder by looking at their code, but can’t evaluate a football quarterback very well by watching them play. However, even if my skill assessment were perfect, trust in a person depends on much more than their ability to perform a task. There are social factors involved; work ethic; general intelligence; verbal/language skills; etc. I might, for example, have a high tolerance for tardiness in my colleagues, but if I’m not tardy myself, then a client of mine might be angry if I referred them to colleagues who were.

Smart companies who depend on highly-specialized skill sets recognize that this problem also applies to hiring. We hire people, who will eventually hire other people, who hire more people, and so on. The success and reputation of the company depends heavily on employee competence, so by necessity, we have a very difficult and complex hiring process that requires significant training and vetting for interviewers, involves several non-interviewers, and is extremely biased toward no hire. The consequences of taking on even a single unqualified hire are potentially catastrophic, if they manage to hire others who are worse than themselves. Despite all of these safeguards, bad hires have started to creep in, due mainly to constant pressure to “streamline” the process (AKA: lower the bar).

On the flip side, companies that don’t understand the Do vs. Delegate distinction can get into a lot of trouble, which is why many startups fail during the initial growth phase. At one startup where I worked, the founders were sales guys. They were phenomenal at sales, racking up huge contracts every few weeks. Eventually, however, they needed to dial down their sales activity to actually run the business; that meant hiring other salespeople. Surprise: those people were mostly awful. One simply underperformed; another was a blowhard who annoyed everyone else in the office. Then they took on a sales manager, who hired three other salesbots who were all fired on the same day for some serious ethical violation that we weren’t allowed to know the details of. Growth stalled out, because the customer trust that the founders had built up was subsequently broken by the founders’ delegates.

In systems and in life, it’s best to avoid second- or third-degree trust. Be suspicious when a person or institution you trust is telling you that you can trust a third party whom you’ve never met nor heard of. They may have good intentions, and there’s nothing wrong with following up on a referral, but treat that referral as you would any other unknown, starting with zero trust and plenty of skepticism, and keeping your options wide open. This applies to everything from your professional contacts to family friends to the news you read.

If you ever need to inflict this situation on someone else, be transparent about your actual level of trust and familiarity with the third party, and expect a rough transition. Like it or not, your own credibility will take a hit if your surrogate causes a trainwreck.

Bonus round: Consider how the transitive trust problem might affect larger institutions, including governments and entire nations.