The back-channel culture, Silicon Valley’s war on privacy, and the juvenility of all of this.

One of the more execrable Silicon Valley institutions (and it’s not like there’s a shortage of moral failures in the contemporary Silicon Valley) is the “back channel” reference call. This is when a prospective employer or investor circumvents the candidate’s provided reference list and calls people who weren’t volunteered. While it’s morally acceptable for certain kinds of government jobs (e.g. in a security clearance) because national security is stake (and because back-channel reference checking is a well-published part of the clearance process) this is just plain obnoxious, unprofessional, and often unethical when done for regular office jobs, where human lives aren’t at stake. It’s bad for job seekers, but also bad for the people being called, who may have never volunteered to give references in the first place.

Unfortunately, the technology industry is full of unprofessional, juvenile man-children who don’t seem to know, or care, about professional protocols. So this conversation actually has to happen, and it will happen here. But, for us as a community, it’s an embarrassment that I’m writing this. It’s like when tech conferences have to publish anti-harassment policies. Shan’t we be embarrassed, as a community, that not all of our members know that groping strangers is not OK? We should, and for this issue, likewise.

Why is back-channel reference checking so bad? I can think of four reasons to despise this practice.

It violates an existing and important social contract.

When someone applies for a job, there’s a social contract between the candidate and the company. The candidate is, under this contract, expected to represent her qualifications truthfully, and the company is expected to evaluate her in good faith.

A violation of this contract would be a company that has no open positions, but holds interviews to get proprietary information about its competitors. That’s not “good faith” because the candidate has no chance of being hired. Her time is being wasted, in order for the company to get information. That does happen, but it’s generally considered to be a slimy practice, and it’s hard enough for a company to keep the secret that it’s uncommon. Back-channel reference calling is another, similar, violation of the social contract. A company that extends an interview is representing that (a) it has the resources to hire, and that (b) it will hire the employee if the employee’s total packet (CV, interview performance, and furnished references) performs sufficiently well. To do otherwise is to show a complete lack of respect for the employee’s time. This implies that if a candidate is rejected, it ought to be something in the official “front channel” package that was the reason.

How much feedback should be offered to rejected candidates is, ethically, an open question. I doubt that it’s reasonable to expect an employer to take time to explain exactly where in the interview a candidate failed, because that can lead to fruitless and mutually demoralizing discussions. Many companies refuse to provide explanations, for that reason. I will maintain the ethical obligation of the employer to communicate (sometimes, passively) what stage the failure occurred at. If the candidate isn’t called back, it was the CV. If the candidate gets an interview but nothing else, it was interview performance. If the candidate is asked for references but doesn’t get an offer, then he needs to consider a different set of people the next time he gives references. Injecting other “secret” stages into this process just adds noise to the feedback. While I don’t consider companies responsible to communicate the exact reasoning behind their decision, using a process that obfuscates existing feedback is a breach of professional ethics.

For a concrete example, let’s say that a candidate gets to the stage of furnishing references and volunteers a few, and they come up positive. Then a few back-channel references are called, and something negative comes back. It doesn’t matter if it’s untrue or if the person isn’t a credible source; the candidate’s probably sunk and, of course, he probably won’t be told that it was a back-channel reference that did him in. Now his relationship with three of his closest professional colleagues is needlessly and wrongly complicated.

Back-channel reference checking also has a way of getting back to the candidate’s current employer. Plenty of defenders of this practice say, “Oh, I’d never do that.” Bullshit. If you’d reject an otherwise stellar candidate based on unreliable back-channel feedback, then you’ve already proven that you can’t be trusted to be “careful” with peoples’ careers. Back-channeling publishes a possibly private job search (yet another violation of the social contract) and word travels fast.

I think that, in the long run, back-channel reference checking is actually quite expensive for companies. Savvy candidates, when dealing with companies that use the practice, are going to fake competing offers in order to put time pressure on employers and prevent the back-channel cavity search from happening. (It violates the social contract for a candidate to lie like that, but if the contract’s already broken, why not?) That will lead to hasty decision-making, compromise the existing hiring practices, and result in costly mistakes.

It’s a show of power and aggression.

It takes social access to get into a stranger’s past at any level of depth. People don’t like giving references unless they’ve agreed to be a reference for someone, and back-channel references never knew that they were references (and may take personal offense to not having been asked first, not knowing that their names weren’t volunteered). HR officials at companies, often, will only verify basic details about previous employees, knowing the legal risks of giving anything more. Likewise, most people who are asked out of the blue for a reference aren’t going to give one to just anyone. They have to trust the person asking. Back-channel cavity searches require knowing a lot of people. They’re easier for large corporations which can involve a lot of people, or for venture capitalists who’ve been buying and selling influence for decades, but pretty much impossible for the little guy to use.

When VCs claim that back-channel reference checks (currently legal, but let’s hope that Washington becomes aware of the issue and does the right thing) are critical to their business, what they’re actually doing is gloating about having the social resources necessary to conduct such investigations. It’s hard to get people to volunteer information that is often inappropriate for them to share. “Do you feel like fucking over a random stranger?” “I really want to know if Sue is of the future-pregnant persuasion; does she talk about kids a lot?” “Tom didn’t put dates on his CV; can you tell me his approximate ‘graduation year’?” “Give me a rundown of Mark’s health-problems-I-mean-‘performance’-reviews from 2008 to 2013.” “Is Angela one of the ‘political’ Native Americans or is she just like anyone else?” People don’t answer these questions, asked cold by strangers with no skill at interrogation. It takes resources (mostly, trust and contacts) to get it.

Often, the person who does the back-channel reference check will admit to doing it. When it results in rejecting the candidate, the failure is more silent, but often it results in further “conversation”, the purpose of which is to humiliate the candidate (reducing her likelihood of negotiating for a higher salary or better job role than the company is prepared to give her) under the guise of addressing “concerns”. At that point, it’s about showing utter dominance by waltzing into that person’s career, turning over all the furniture, and using the toilet without flushing. It’s to impress the target with the godlike ability to get access to all sorts of inappropriate information. It’s a way of saying, I don’t have to play by the rules, because I’m powerful enough to get away with anything.

It’s invasive.

That Silicon Valley’s back-channeling is invasive hardly needs explanation. There’s a general protocol around what is and is not appropriate for a prospective employer to research. Running a background check to make sure the person worked at the companies, and attended the schools, that she said she did? Totally OK. Finding out that she has kids, showing up at their elementary school unannounced to observe them, and bribing an unscrupulous principal into getting their academic records, in order to find out if they’re special-needs kids who might be more demanding of the mother’s time than average kids? That’s not OK. There’s a lot of information that is arguably potentially relevant to someone’s future job performance that we, as a society, have rightly decided to be off-limits in making decisions about whether to hire someone.

The “front channel” employment process, at least, imposes some accountability on both sides. The employer communicates its priorities through the questions that it asks, and thereby puts credibility at risk if those priorities are unreasonable or, worse yet, illegally discriminatory. Volunteered references are provided so the employer can validate that the candidate actually worked at the companies claimed and isn’t completely off-base about previous roles and functions within those companies. Using back-channel references is, however, about the more powerful party’s escape from accountability. To ask for information communicates that there is interest in it. To surreptitiously acquire it does not, which means that there’s plenty of room for impropriety and invasion.

It’s also uselessly invasive. The feedback is noisy. For every person with knowledge about someone and his work, there are ten with opinions. Venture capitalists and CEOs who perform these back-channel inquiries may think they’re sharpshooters who can quickly get to credible sources, but they’re not that good. They just never get feedback on their failures, because they’ll reject anyone who doesn’t come up with a perfect bill and never see that person again.

One of the reasons why 2-3 volunteered references (and, at absolute most, 5) has been the standard in employment for so long is that, perhaps counterintuitively, the quality of employee hired doesn’t improve beyond that number. The main reason to check references is to filter out unethical people who interview well, but past 7, you are empirically more likely to hire an unethical psychopath. Why so? Among unethical people, you tend to have two kinds: the petty, untalented ones who make annoying messes, and the talented, dangerous ones (psychopaths, usually) who can take down a whole company. The first category can’t get references; they burn bridges, leave messes in their wake, and are generally disliked by everyone who knows them well. The second category always have glowing references. They have no qualms about making friends pose as bosses, buying references off-the-shelf from made-up companies (yes, this service exists) and “coaching” people into telling exactly the story they want. You can’t actually filter out the second category of psychopath through any social proof system, which means that, after a certain point, your best odds are with not excluding normal people.

If you ask for 10 references, the average, basically ethical, person with a normal career has to dig into his third string and at least one of them is going to be less impressed with him than he expected, so he’ll fail. The psychopath, however, will always pass the 10-ply reference check. That applies even moreso to back-channel references, because psychopaths hide in plain sight and great at intimidating other people into acquiescence. The psychopath might have enemies and detractors, but it’s not being disliked that ruins a person’s career, but low social status (by the way, “performance” at 95% of jobs is social status). Psychopaths make sure that any slight against their social status is swiftly punished, often having loyalists in every social sphere they’ve inhabited. So the back-channel reference check, counter-intuitively, strengthens the psychopath’s power. Some of the people called will dislike him, but not a single one will diminish his social status in any way, and this will strengthen his image as a powerful, “high-performing” person and deliver him the job. Psychopaths really are like cancer cells, able not just to evade human “immune systems” built on social proof and reputation, but often to co-opt those for their own purposes and weaponize them against the good. How to beat psychopaths is a complicated topic for another essay; the best strategy is not to attract them. One of the major reasons I champion open allocation is the specific fact that such an environment is unpalatable to the workplace psychopath.

The invasiveness of the back-channel reference check, empirically, delivers no investigative value about what actually matters (ethical character). It drags up a lot of “juicy” (meaning “inappropriate”) gossip, though. That is why, in a world of oppressive, inane juvenility like Silicon Valley, they’ll probably never go away entirely. It’s too much “fun”, for a certain species of manchild that the Valley has given undue power, to invade a stranger’s personal and professional lives.

At any rate, reference checks aren’t actually investigative in purpose. The real purpose of reference checking is to keep the moral middle classes– people who aren’t unethical psychopaths, but probably would lie a bit to improve their careers, if they weren’t afraid of getting caught– honest. At that, it’s probably a necessary evil, and I think it’s fine for employees to ask for 2-3 (hell, even 5) references to validate that the employee’s represented career history and qualifications are correct. If references were never checked, then people would inflate their qualifications more than they already do, and that would add noise to the job-search process. The reference check has legitimate value in verifying the correctness of a candidate’s claims. This doesn’t justify an adversarial invasion of privacy.

It’s discriminatory.

For those who rely on back-channel references, one of their favorite reasons for doing so is access to all sorts of information that can’t be requested in the “front channel” process, relating to age and health issues and pregnancy history and socioeconomic status. Off the bat, the discriminatory intent is obvious.

There’s more to it, though. Right now, in 2014, venture capitalists and technology employers have an almost pedophiliac attraction to youth, especially when it comes in the package of a sociopathic frat boy. It’s not really about chronological age. Rather, they like people who haven’t been challenged yet, with stars in their eyes and a general cluelessness about the world. They champion “failure” because it benefits the VC for the founders to take risks that would be considered irresponsible by anyone who’s had enough life experience to see what actually happens when one fails. Also, there’s a vicarious nostalgia in play: VCs want to be reminded of the time when all they worried about was drinking and getting laid, before the jobs and the kids and aging parents. Instead of being honest about their midlife crises and buying Ferraris or boats or marrying trophy wives, they’ve taken the midlife crisis in the form of a hilariously underqualified protégé (like Lucas Duplan or Evan Spiegel) made “startup CEO” by their own largesse (and, given the Valley’s culture of co-funding, connections to other investors). This, dear reader, is what “culture fit” at startups is really about. It’s about socioeconomic and cultural homogeneity, and isolation from the challenges of the real world (kids, aging parents, health issues). It’s college for people who were too socially incompetent in their late adolescence to make the most of that stage of life when they were in it, and who want a retry in their young adult (or, for investors, midlife) years.

People who’ve been challenged, and how know that there are actual stakes in life, don’t like back-channel reference checking for the same reason that they don’t like open-plan offices. If you’ve ever had a serious health problem, the added stress of having it in front of 50 other people is just intolerable. With age and challenge, people become far more competent in general but lose a bit of endurance with respect to the weird and mostly minor (but cumulative) cultural insults of the open-plan, juvenile, “startup” culture with its lousy health benefits, blurred lines between personal and professional life, and general lack of respect for established professional protocols that form out of decades of experience. If this heightened desire for privacy doesn’t happen for any other reason, it certainly happens when people have kids. Why so? The college bubble is an artificial world whose socioeconomic heterogeneity is enforced in order to create a culture of ubiquitous trust. You don’t need to worry about each person individually, because the “adult supervision” of the admissions office has already done that. The “hip” tech culture is all about preserving that youthful attitude (“I don’t need privacy because there are no really bad people around”) toward life. However, in reality, the world is dangerous and has lots of evil people in it (even at colleges, but that’s swept under the rug!) When people have children, the biological and emotional need to protect another creature makes it impossible to harbor that prevailing, universal trust. If you have to protect someone else’s life, you can’t trust everyone. And open-plan offices (more specifically, visibility from behind, or “open-back visibility”) are about forcing the employee to render trust to the whole office. The same holds for back-channel references, but in that case, there’s even less consent. If you think back-channel reference checking is morally acceptable, then you’re arguing that people should be mandated to trust complete strangers in their own careers.

The truth about privacy and protocol is that they’re not just there to protect “the weak” or people “who have something to hide”. To want privacy doesn’t mean you’re doing anything wrong. It just means that you’ve had enough life experience to know that not everyone can be trusted with all information. I’m well aware of the fact that my tone, on issues like this one, sounds unduly adversarial to many people. I don’t actually see the world in adversarial, zero-sum terms. The world ought to be mostly cooperative, and I think that it is. However, I recognize that a large number of interactions and transactions are innately adversarial, and I’m old enough to know that even people who’ve done nothing wrong have a desire for (and a right to) privacy.

Privacy and the lack of it

College is a time of life in which people relinquish some privacy because there is “adult supervision” that is supposed to prevent things from getting too far out of hand, and because people of low socioeconomic status (presumed to be where criminals come from) are generally excluded, unless they’ve been vetted heavily for intellectual ability as well as good behavior. In the college bubble, most students don’t have to work outside of their studies, and most students’ parents are in their 50s and at the peak of their careers (not 35-year-old single mothers who gave birth to them at 17, not 82 and dying). Excluding managed intellectual challenges in coursework, most of these people have never been challenged… and those who have either don’t fit in or become others’ “diversity experiences”. If this is an uncharitable depiction, let me admit: this isn’t entirely bad. It’s the “magic” of socioeconomic protection and age heterogeneity that enables people who met in September to be “best friends” by October, and feeling safe to discover alcohol and sex and psychoactive drugs and politics and computer science around such people– in a world where privacy is relaxed and people get to know each other quickly– is a big part of that. The suspicion and chaos and status-assessment and busyness that characterize “the Real World” haven’t set in yet, in the sunny college bubble, and that allows deep friendships to form in a month instead of over years. Yes, I can see the appeal of being 18-22 forever.

“Culture fit” and the Valley’s worship of youth are outgrowths of this desire, which many share: to create a world in which it’s possible not to grow up. (If one wonders how the adult supervisors, like VCs, benefit by running a silly camp for overgrown adolescents, the answer is that people who aren’t expected to act like adults won’t demand to be paid like adults, and the VCs can make out like bandits. Considering that, they actually charge a bit more than college bureaucracies.) One of the reasons why the consumer web contingent now dominating the Valley simply doesn’t get peoples’ need for privacy is that it, collectively, is still stuck at a mental age around 23 and, more specifically, in the mindset of a certain type of 23-year-old who’s never been challenged or tested. (Obviously, I do not intend to apply the “never been tested” label to all people at that age, or at any age, since it wouldn’t be accurate or fair.) They don’t have to be white, male, young, heterosexual, childless and from upper-middle- or upper-class backgrounds but, for statistical reasons, they usually are.

Back-channel reference checking becomes, if not morally acceptable, more understandable when one realizes how juvenile private-sector technology has become. I’ve lived in the Real World and I’ve definitely had legitimate challenges: deaths in the family, personal health issues, lost jobs and even a couple 3-4 month spells of unemployment. I’ve seen enough to know that the stakes in this life are fucking real. There’s no Dean of Students who sits down and talks the bad things out of happening. You don’t go in front of a Financial Standing Committee if you lose all sources of income; you actually suffer. As such a person who has actually lived in the world, and seen what it is, and learned that it is pure idiocy to trust literally anyone into one’s career via “back channel references”, I am vehemently against the practice. And I am morally right. However, there are exactly two types of people who can be ethically OK with the increasing prevalence of back-channeling in technology: (1) powerful sociopaths who’ve decided that the rules no longer apply to them, because their getting away with something is proof that it’s OK for them to do it, and (2) clueless naifs who’ve never suffered or been challenged… yet. To people in college mode, a back-channel reference check is no different from asking, “Hey bro, how many guys do you think Monica has slept with?” That is, it’s very inappropriate, but not the sort of thing that would lead to a seven-figure lawsuit or a jail sentence.

More generally than this, I think that people become more private and discerning as they grow up. When you’re six, you’ll be friends with that “funny” kid who puts dog poop on a stick, holds it front of his face, and laughs at it. When you’re 31, if you’re like me, you’ll have a hard time making conversation with people of average (or even 90th percentile!) taste and cultural awareness. This isn’t all good. I spend a lot of time trying to figure out how to crack barriers in a meaningful way and, given my average-at-best social ability, don’t always know how to do it. I’m glad that there are people like the 20-year-old (as of 2004) Harvard student Mark Zuckerberg who can get insights into these problems and, at least in some partial way, solve them. College is too “open” for that mentality to work outside of a socioeconomically heterogeneous bubble– we’d have to become a Scandinavian socialist country for the college mentality to work outside of an academic bubble, and I don’t see that as politically palatable in this country– but the “adult world” is a bit too closed, cynical, and cold. In many ways, I see the appeal of the former, and I think the ideal point is somewhere in the middle. (The world would be less closed and frigid with less socioeconomic inequality, but I’m not anywhere close to having control over that variable.) That said, I’m a realist. Trusting strangers unconditionally in one’s career is just plain stupid at any age. Not everyone in this world is good, it doesn’t take much effort or luck for the bad people to make themselves dangerous, and the stakes are too fucking real to pretend they don’t exist.

We, the fully-fledged adults, might seem cynical, stodgy, and adversarial when we tell Silicon Valley man-children that their fratty back-channel reference calls aren’t OK, or that they should stop putting sexist humor in slides about their products, or that open-plan offices are a back-door age and health discrimination that we find crass, or that they need to stop betting their companies on the (extremely rare) clueless young thing that gives his all to a company that gives him only 0.05% of the equity (because, honestly, most of those people aren’t talented, just young and eager). We’re not. We’re just experienced. We know that privacy, protocol, and propriety are actually important. We’ve seen people suffer needlessly due to others’ stupidity, and we’ve learned that the world is a complicated and difficult place, and we’re trying to defend the good, not just against the evil, but against the much larger threat presented by the mindless and immature.

So…

The predominant culture in Silicon Valley has moved against privacy, with personal and professional lives bleeding together, frat culture (and its general disregard for propriety) invading San Francisco, and back-channel communication becoming part of the hiring process, all in the name of “culture fit” (preserving the college-like bubble). This is not the only culture in technology, and it’s certainly not moving in unopposed. Sadly, it does seem to be winning. Demanding privacy at a level previously taken for granted (even asking for quiet working conditions and a barrier at one’s back) has become unusual, isolating, and embarrassing. The attitude that it often meets is, “Why do you need privacy if you’re not doing anything wrong?” Only political naifs consider that question to be remotely reasonable to ask. Everyone needs privacy, because the world is complicated and dangerous and trusting the whole world with all of one’s information is just reckless. This isn’t Stanford. This is real fucking life.

The end result of this is an exclusionary, insular culture of an especially pernicious sort. Silicon Valley’s oppressive mandatory optimism and its contempt for privacy and those who demand it aren’t just classist and sexist and racist and ageist. In fact, Silicon Valley doesn’t have a coherent desire to be any of those things. It’s about a rejection of experience. To live in that sort of college-like bubble, you have to reject the knowledge that not all people in the world are good. You have to accept intrusions against your privacy and person like open-back visibility at work, micromanagement in the name of “Agile”, and back-channel reference calls. You have to have never been challenged or tested, or at least seem like that’s the case. However, that puts us, as an industry and community, far away from the realities of human existence. It makes us, just as we are ethically and professionally reckless as shown in our use of back-channel references, out-of-touch and dangerously oblivious to what we are actually doing to the world.

We have to take stock of this and change course. No one else is going to do it for us. It’s up to us to lead and, to do that, we have to grow the fuck up.

An insight on how to fix technology’s Damaso Effect

I’m going to start this analysis by focusing on a negative pattern of behavior that seems unrelated to technology’s Damaso Effect.

The Misogyny Loop

I know someone (I’ll call him “Stan”) who’s about 30 and has never been in a relationship that lasted longer than about six months. He’s not unattractive and, while his opinions on many topics are wrong, he’s intelligent and well-employed. He’s even cordial. It’s pretty easy to see what’s wrong with the guy: he’s openly misogynistic. He believes that women are petty, irrational, capricious, emotionally dysregulated and have poor values, and it doesn’t take much to get him to share that opinion.

On one hand, he’s completely wrong. I don’t mean that it’s morally abhorrent (although it is) so much as that it’s just incorrect. Virtually everything Stan believes (or, at least, says he believes) about women and men and the relative value of each is total horseshit. You can’t take it seriously; it will just make you angry. Unfortunately, men who think this way are not uncommon, either in the technology industry or the world at large. Some learn canned social skills to trick shallow, confused women into going to bed with them and call themselves “pickup artists”. Others call themselves “incels” (involuntarily celibate) and stew about their predicament on the Internet. They’re awful to listen to, repetitive in their whining, and generally bad at seeing the real problem. “Hookup culture” is revolting, but pinning it on the women alone is misguided. The general pattern, for these men, is that they take traits of the worst people and project them on to “women”. I wouldn’t be surprised if female misandrists didn’t do the same to “men”. It’s a gender-neutral fact that the sexually and socially “loudest” people are often the worst ones, but they’re also a small fraction of the population. If you let them bias you, you’ll reach bad conclusions.

Stan, when he generalizes to “women”, sounds like an idiot. (If women are irrational and emotional, then why are most crimes committed by men?) Yet, I will give him this: his judgment is correct, over most of the women he’s dated. I’ve met a few of them, and they generally treat him poorly. He dates stupid, shallow, capricious, and damaged women, and this generates negative experiences that reinforce his blighted worldview. Comparing his small group of male friends to an adversely selected set of women, he concludes that men are better people and that women are unfair, capricious, and slutty creatures to be used only for sex. His negative views of woman keep him from meeting normal women (they don’t want anything to do with a guy who thinks women are innately inferior, and who can blame them?) and forming healthy relationships, and thus he falls into a Misogyny Loop. Because his attitudes toward women are abhorrent, he meets and dates damaged women who confirm his biases, and becomes even more entrenched in this negative view of the female sex.

Technology and the Business

I’ve written about the Damaso Effect. Programmers have a tribal dislike for “The Business”, for HR, and for the managers and executives (“pointy-haired bosses”) who pay us. As with Stan’s misogyny, these feelings didn’t emerge out of nothing. The sampling may be adverse and biased (and I’ll get to that) but the experiences are real. For most of us, the businesspeople who manage us are incompetent fuckups. Stack ranking, a textbook example of bad HR that is despised by competent business leaders, is still quite common in technology companies, and that’s because our biggest companies actually are run by a bunch of fucking idiots.

From Harvard Business School, the good students go on to start hedge funds or work on billion-dollar private equity deals, the middling ones get partner-track slots at McKinsey or end up directly report to CEOs of large corporations, and the leftovers get passed to California and boss nerds around. The best of our tribe (software engineers and lifelong technologists) answer to the worst of theirs. It sucks. It’s hard to ignore the tribal hatred that comes out of the humiliation inherent in a skilled programmer being told, by a fresh-out-of-college management consultant, that he has to attach future code-change commits to “user stories”. But let’s try to put any hatred and history aside, and step back. By any definition of the concept of a “business person”, there are competent, smart, ethical ones out there. I’ve met quite a few of them. They exist. They just… don’t come anywhere near our industry. Can you blame them?

Could we be like Stan? Is it possible that our blanket negative view of “business people” makes our industry unattractive to all but the hangers-on in their tribe? I think so. I think that it’s likely.

The screwy art of making exceptions

There’s something I should mention about Stan. Because he’s socially inept and (in his own mind) a tad bit desperate, he fabricates relationships that exist only in his head. His bitterness toward women in general leads him to erroneously up-regulate a woman’s signals. Thinking that women are horrible creatures who are predisposed to despise him, he tends to overreact to basic decency (real and superficial) in them. So he will mistake politeness for niceness, niceness for friendship, and friendship for sexual attraction. Also, his extreme negativity makes him a terrible judge of character. If a woman is basically decent to him, he puts her on a pedestal. She becomes one of those ultra-rare “good ones”, untouched by “feminism” and “frat boys” and the filth of the world. After this happens, she can treat him like dirt and he’ll make excuses for her. He’ll see the light (on her, but not in general) eventually, but his worldview becomes even more depraved as he learns the wrong lesson, that even his “perfect woman” turned out to be awful.

Similar to Stan’s misogyny, software engineers also harbor a generalized dislike for “business people”, whom we tend to view as stupid, emotional, childish, petty, and short-sighted. And about the business people who run our industry, we’re not wrong. We’re just failing to see the whole set of them. (See a pattern, here?) This bitterness doesn’t make us keen observers or tough judges of character. It makes us fucking marks. A naive engineer, whose model of a slimy businessman is a 40-year-old Harvard graduate in a suit, is underprepared when the 25-year-old Stanford “bro”, in jeans and sandals, cuts him out of the startup he built.

In relationships and business, deeply bitter people are often clingy. Whether it’s “women” or “business”, there is some Other that they despise but also depend on, and they tend to caricature it in the most vicious way. Anyone sending signals that negate this stereotype can break through the “bitter shield”, win undeserved trust, and eventually have license to treat a person badly while that person makes excuses for the awful behavior. In dating, the Stans of the world fall for “quirky” girls who’ve put on glasses to send “I’m not a slut” signals without improving their moral character. In business, the bitter engineers fall head-over-heels for talentless young things who read enough Hacker News to know that professing fandom for Postgres and OCaml will make them seem technical and smart (even while they continue to force their engineers to use PHP and work 18-hour days). In all of this confusion, we get cloudy but stable result where most of the clueless young engineers, even if they despise “managers” and “executives” in the abstract, like their supervisors. This is analogous to the common American attitude toward elected officials, and why there is so much incumbency that bad politicians are almost never voted out. Most Americans dislike “politicians” who make backhanded deals “in Washington”, but they love their politicians. “George Starr fixed the pothole on my street!” Well, yeah, that’s what he’s fucking supposed to do (maybe not directly, but you get the idea). These people fail to recognize that their own charismatic local favorites, more often than not, are part of what’s wrong with “politicians”. They also tend, because they’ve put a specific person (the “alpha male” to whom they’ve clung for protection) on a pedestal, to react with undue emotion (a sense of “betrayal”) when the relationship goes sour.

Good and bad tribalism

The truth about us, as software engineers, is that we get our tribalism completely-the-fuck wrong. We can be immensely tribal about stupid shit. If a person doesn’t have an active Github profile, we “flip the switch” (bozo bit) and assume that he’s an idiot, even if he has a completely different job. People who don’t “look like” programmers (and there shouldn’t fucking be a “look”, because technology is too fucking important to push talent out for stupid reasons) face an uphill battle at getting basic respect. We’re quite superficial, and I’ve been guilty of this in the past, too, such as by overvaluing technical preferences as an index of value or intelligence (e.g. “he’s a Java developer, he must be an idiot.”) Our superficiality, however, makes us really easy to confuse and hack.

We tend to think of “non-technical people”, in the business world, as idiots. It doesn’t help fight this stereotype that many of them are idiots, especially in technology, due to the Damaso Effect (which, as I’ve indicated, might be partly our fault).

Before going further, let me give a working definition of idiot for the business world: an idiot is either (a) a person doing an important job poorly, or (b) one doing an unimportant (or counterproductive) job aggressively enough to cause problems. There are many causes for idiocy, and while the cerebral narcissists among us tend to jump to a lack of innate intelligence, that’s actually one of the less common ones. Other issues are (and these will overlap) willful ignorance, lack of care, poor interpersonal skills, dislike of one’s job, and many external ones such as inept (or malevolent) supervision, bad strategic leadership, or environmental incoherency. Over 90% of non-technical people (and probably, to be honest, more than 70% of programmers) are idiots at work and a lack of innate intelligence isn’t the problem. We err when we assume that it is. Our adversaries (who are not always idiots themselves, but who spread and encourage idiocy for their own benefit) can, usually, flash enough cognitive muscle to break our stereotype of “the business idiot” and get us to take them seriously– to our detriment.

For all our superficial tribalism, software engineers are shockingly quick to sell their colleagues out to management, and to do exactly what their bosses want. Let’s use stack ranking as an example. When a company puts forced ranking in place, every savvy manager will hire an “insurance incompetent” or two, and put them on inconsequential work. It’s not a good thing for the company or the world, but it’s a way to put the team at ease because their jobs aren’t at risk; when the stack-rank gods demand human sacrifice, the insurance incompetents go while everyone capable stays employed. This allows members of the core team to focus on their jobs rather than politics. Typical software engineers are too clueless to realize the value in this and, further yet, will vehemently oppose it. “I can’t believe I’m working at the same company as a guy who mixes tabs and spaces!” (“I can’t believe someone else’s money is being used to hire an incompetent and protect my job!”) Many engineers (without political benefit in doing so) turn on their own weak, and will sell out their entire teams if given time in the executive sun. This is because they’re easy to confuse by invoking superficial tribal color. “Don’t worry, Tom’s not an asshole manager. He plays video games and used to code, ten years ago. He’s one of us.” Because engineers are a low-status tribe in the business world, it only takes a few flashes of tribal empathy, from The Business, to compromise the weakest of us. Instead of falling for this nonsense, we need to stop judging people based on tribal identity and start judging them based on what they actually do.

Perhaps it’s counter-intuitive, but managers don’t really suffer from engineers’ tribal dislike of them. We still need people with at least some of these skills. The tribalism makes engineers easy to manipulate. Here’s one place where the two cases (of Stan’s misogyny, and engineers’ dislike of business people) depart. Misogyny actually hurts women. Our dislike of “slimy suit-wearing businessmen” just makes us easier to hack by slimy businessmen who don’t wear suits– because they’re 23-year-old Stanford grads. Businesspeople don’t give a shit whether we like or hate or love or despise “businesspeople”; they just care about making money off of us. And while we should be gladly helping the best of them make money (so long as they don’t harm in the world in doing so, and they share the wealth with us) we should not be allowing them to drive us into subordination.

The right kind of tribalism

The current software tribalism is mean-spirited, exclusionary, and privileged, but it’s also ineffective at getting us what we actually want. For an analogue, peruse Quinn Norton’s notion of white privilege as, in reality, a ruse that convinces disaffected whites to oppose their own economic interests; because, at least, they have it better than the blacks. We’ve created a culture of subordinate and pointless privilege. In software, we now have a world in which well-educated, white men never have to grow up, and this suits the venture capitalists and “founders” because, so long as we aren’t required to turn into adults, we won’t expect to be paid or treated like adults.

I’ve written at length about the value of having a professional guild or collective bargaining. Respectfully negotiating on our behalf, toward our genuine shared interests, will get us a lot further than tribal shit flinging. Rather than having this unfocused dislike toward a large set of people whose skills we barely understand, we should figure out how, with focus and respect, to reach equality. We need them and they need us.

It’s going to be hard to reach common ground with the business elites. There’s much in the business world that is archaic, anti-meritocratic, classist, anti-feminist, irrational, mean-spirited, status-driven, imperialistic, and just plain broken. A world in which “pedigree” and connections matter so much more than substance, drive, and talent is a hard one to respect, and that’s what the business mainstream, at the highest levels, is. (It’s less that way amid the small, local businesses that tech companies, more often than not, blow away.) But what world are we creating, and have we created? The noxious miasma in VC-funded Silicon Valley is not superior to the corporate mainstream “establishment”; it is the establishment. So maybe it’s time to just forget about “worlds” and start talking to individuals as we figure out how to do things better. We had our chance to build a better world, in Northern California, back when housing was cheap and the air was fresh and the word “traffic” referred to telecommunications rather than automotive congestion, and we fucking blew it. Maybe those people in Washington and New York (“the paper belt”) were worth listening to, after all. Maybe people who’ve spent as much time amid bureaucracy and human politics and finance have a thing or few they can teach us.

There are certain tribal values among us, as technologists, that are worth preserving. What makes us unusual is that our discipline is progressive in nature. Code, well written, can be used forever. While most of the world is mired in zero-sum power struggles and territorial squabbling, we get a chance to add, even if in a small way, to the sum of human knowledge. That’s pretty damn cool, and the best things about us as a culture, I believe, derive from the progressive and collaborative nature of what we do. At least on paper, we create new wealth. We solve new problems, and the nature of code is that a solution once devised can be used anywhere.

Unfortunately, there are facts that break against us: our leadership is absolutely excremental, as Valleywag gleefully (and very competently) shows the world on a regular basis. “We” (meaning the executives injected into companies where we do most of the actual fucking work) destroyed San Francisco, and now the world is bracing for our “disruption”, just hoping that a 2000-style dot-bomb crash will prevent us “techies” (meaning the slimy proto-executives whom many of us blindly follow) from ruining everything. We have to fix “ourselves”, and fast. We have to organize so we can choose better leaders. Instead of having puppet leaders shoved into our top ranks from the refuse of the business world, we have to prove to ourselves (and to them) that we can do better when we select leaders who respect the best of our values, while diverting us away from our worst tendencies. They don’t have to be full-time coders. They probably won’t be. They have to be technologists in ethics; being so in craft is important, but secondary.

Right now, as a tribe, we’re far from self-sufficient. Indeed, in the 21st century, self-sufficiency doesn’t really exist. That ship sailed (or, more accurately, that container carrier motored away) a long time ago, and the global economy is far too interconnected. There’s a lot of knowledge that we, as a tribe of a couple million people with elite technical skills, just don’t have. We should be meeting with union organizers in order to learn the diverse forms of collective bargaining, and in order to find an arrangement that prevents the negatives (wage normalization, tyranny of seniority, mediocrity) associated with unions from setting in. We should be venturing deeper into “the business world” to find better (and, most likely, more experienced and older) leadership. There’s a whole slew of successes and failures that the rest of the business and governmental world has seen over the past few centuries, and throwing that knowledge out because we think we’re above “paper belt” politics (and we’re obviously not) does no good to anyone. We’ve got to do two things. First, we have to end our own tribal bigotry and reach out. We must vehemently oppose assaults (external and internal) on our values; but, at the same time, welcome other kinds of people. It’s not just our game, and they know how to do things (such as lead and build organizations) that we haven’t learned to do for ourselves. Second, we need to develop the ability to manage our own affairs. We have to step up, from the inside, and lead. If we don’t, then we’ll continue to answer to the mainstream business culture’s fail-outs, and stack ranking will never go away.

The core of the problem

It’s going to be hard to fix our affairs. To illustrate this, let’s take note of Stan and his dating woes. The transactional, superficial view of dating is that people match up based on attractiveness, whether superficial and physical or total and holistic: 10’s pair up with 10’s, 3’s pair up with 3’s, and so on, and it’s rare that a person gets to go into a higher “league”. Some people find this reductive and offensive, but there’s no question that the early stages of dating often are that way. Of course, as the Stans of the world will endlessly complain, there are skewing factors. Age is one, because men tend to prefer younger women and women prefer older men. Thus, 25-year-old men will readily date 20-year-old women, but the reverse is uncommon. Among college students, a male “7” is unlikely to find a female “7” in his age group, because the women have more options and some are “taken off the market” by older men. He’ll have to settle for a “4” or “5” if he wants relational market equality. Among 40-year-olds, it’s the reverse; average-looking men, in that pool, become highly desirable. Most men who fall into the Misogyny Loop do so in high school or college, when they get shafted by age-skew and there really is a shortage of available decent women (relative to the number of decent men) in the same-age dating pool. It’s not that available decent women (at any age) don’t exist. There are a lot of them, but (among 20-year-olds) they are popped off the market faster than men.

I don’t care to analyze dating, because I’m a 31-year-old married man who believes (and hopes) he is done with that nonsense, for life. It should be obvious that I intend to apply this to business. Business has “marriage/matching problems” and skew issues as people with separate skill sets try to size each other up and find parity. What should the equity split be, say, between two Technology 7’s and a Business 8? How should decisions be reached, and salaries be computed?

Let’s talk about new business formation (startups). The only reason why software engineers are decently well-paid, compared to the rest of the U.S. increasingly-former middle class, is that (at least, in theory) we have an alternative: to do a startup. A good software engineer typically makes $140,000 per year in San Francisco, $110,000 in Chicago, $90,000 in London, and $60,000 in Paris or Madrid. Why? It’s not just cost of living: London’s more expensive than San Francisco, and Paris isn’t cheap either. It’s about competing alternatives. The Googles have to compete with the Facebooks and the Facebooks have to compete with companies that don’t exist yet; and, for all the Valley’s flaws, it’s still the easiest place in the world to raise venture capital. To start a business that is interesting in the technology space, two things (the “2 C’s”) matter: Code or Contacts. If someone’s not packing one or the other (or, very rarely, both) then you can’t afford to make that person a founder.

You’ll soon need accountants, attorneys, HR experts, economists, and sales managers. I don’t mean to denigrate those skills. They just don’t need to be baked in to a new company’s formation. They’ll typically be employees, not founders. You can start with a service like TriNet and “bootstrap” up to a full-fledged HR department, just as you can start with Amazon Web Services (“the cloud”) and build your own data centers later. Early on, however, you absolutely need Code and, even more desperately, you need Contacts. These two assets almost never occur in the same person (they both take years of full-time effort to build, except for the very wealthy are born into Contacts) so they’re often described as separate roles: the technology co-founder(s) and business co-founder(s). In order to have a healthy company without warring tribes, you need equality between the partners. So, at what point are they socially equal, in terms of leverage?

Of course, there’s a hell of a lot of skew. Is a Tech 8, like me, going to pair up with a Biz 8? Not a chance. The market value of a Tech 8 isn’t that much higher than that of a Tech 5. A Tech 5’s salary will be between $90,000 and $150,000 per year; the Tech 8 is at $120,000 to $200,000, depending on location. A Biz 8 can get a $500,000-per-year job out of a 5-minute phone call. They, to put it simply, have more options. Tech 10s have to stay in tech for their skills to be treated as meaningful. (I’d argue that a legit Technology 10 could kill it, with proper training, in business or law or medicine… but I won’t get into that here.) Business 10s can go anywhere in the global economy. There are two fundamental commodities in an economy: Past (property, reputation, connections, wealth) and Future (motivation, creativity, talent, grit) and the exchange rate between the two has always favored Past.

Even in the contemporary technology “startup” world, it matters more to have Contacts than Code. A Tech 8 can write a scalable, back-end recommendation engine in Haskell. She’s incredibly valuable. You can bet a company on her. On the other hand, a Biz 6 can get a TED invite and a Biz 8 can get into Davos. Even a Biz 7 can deliver Sequoia or Y Combinator in an afternoon with half an idea scribbled on a napkin, and can get a total pile of crap “acqui-hired” for $5 million per head by Google. The Biz 8 and Tech 8 “deserve” to be equal in social standing, but they’re not. The skew is enormous. It sucks and it’s frustrating, but it’s something we have to deal with.

If I (a Tech 8 without special connections) went to New York or San Francisco today and looked for a “business co-founder”, I’d have to sift through hundreds of Biz 3-5 who “just need a programmer”, offering 5% equity in companies around their lame ideas, not shared ideas that might emerge and be better than either of us would come up with individually. It’s a dreadful market. The exchange rate between Code and Contacts is morally unacceptable. Contacts is winning so handily that it’s creating tribal hatred and bad startups. It’s driving us, as programmers, toward defensive rejection. We loathe our (objectively unfair) low status relative to the business mainstream, and our loss of the world (Silicon Valley) we created. We should loathe it. We should be disgusted (even though a large part of it is our fault). Unfortunately, many of us overreact. Instead of hating arrogant individuals who lord their unreasonable, unjustified high status over us, we let it evolve into a generalized hatred of “business people” or “suits” or “MBAs”. I’ve been as guilty of this, in the past, as anyone, but we’ve got to fucking stop.

So why is it like this? Why is there so much skew? Part of it is that, sadly, society just has more options for Contacts than for Code. Jeff Dean (a Tech 10) would likely be an obscure programmer without Google. The proving ground of a large corporation that allowed him to hone and show his exceptional engineering ability; without his work at Google, we wouldn’t know that he’s a Tech 10. On the other hand, Biz 10’s aren’t anywhere near tech companies; they’re managers of billion-dollar hedge funds who turn away almost all investment offered to them. Hell, it’s rare that you find a Biz 6 willing to be a “business co-founder”. A Silicon Valley founder is a middling product manager, and the true executives are the investors, and savvy people spot the pattern quickly and head for the investor ranks if they’re going to be part of that game. Biz 5-6 are routinely offered entry-level (associate) positions at prestigious venture capital firms, and Biz 7-8 get partner-level jobs, offered on the spot. Yes, it’s unfair as hell. So, what are we going to do about it?

I’m coming to an answer that isn’t the forceful kind of solution that I tend to like. Slinging mud is a hell of a lot of fun. I won’t deny that. And there’s a world of deserving targets out there. However, is hating “The Business” getting us anywhere? Or might we do better to swallow our pride, and to replace unfocused tribal dislike with focused and deliberate organization around our own interests? I think the answers are obvious, here.

If we don’t want for the bulk of us to answer to Biz 3’s for the rest of our lives, then we need to start attracting the Biz 7-9’s who have other options. We have to convince them that what we can build is genuinely important (which means we need to stop it with the played-out social media nonsense). They don’t need us, but we (probably) need them. Their skills, we can learn and grow internally, and we’ll have to do some of that. Their contacts, at least for now, have to be drawn in from outside, since we (as engineers) are a deeply middle-class group. It’s going to be hard to do this. It’s going to be especially hard to convince them to form partnerships at the level of equality that we deserve. I don’t have all of the answers, certainly not yet. I do think that we’d have better odds if we took stock of, and reformed, our attitudes toward business people and what they do. This will also force us to acquire a sense of nuance that will enable us to push the actually scummy business leaders (who are, right now, most of them) out of our industry. It can’t hurt.

Silicon Valley may not be fixable

I’ve come to an upsetting conclusion about Silicon Valley. There’s always been a hold-out hope that I’ve had that it could be fixed, and the balance of power restored to where it belongs (lifelong technologists and engineers) rather than where it currently resides, and categorically does not belong (managers, investors, non-technologists). Traditional business culture, focused on power relationships and emotions and influence peddling, has invaded the Valley and created a rash of startups with mediocre products, terrible treatment of talent, and little respect for users, employees, or technology as a whole. The result has underperformed from a return-on-investment perspective, but remained in force because it makes the well-connected exceedingly rich (at the expense of average workers, and of passive investors).  

This is disturbing because, while there are many in Silicon Valley who think that they are ready to rule, that is proven false by some humiliating, and public, failures of that ecosystem. For one thing, the real winners, in the Bay Area, aren’t technology people but landlords. The housing situation in San Francisco alone is sufficient to prove that “nerds”, at least of that stripe, aren’t ready to rule. 

We’ve also seen a hard-core Damaso Effect. The highest-status people in the Valley aren’t lifelong technologists, but people who failed out of the corporate mainstream, and are bossing nerds around as a second act. Passed over for MD at Goldman Sachs? Go to California, become a VC partner, and boss nerds around. Bottom 5% of your class in business school, untouchable even by second-rate management consultancies? Congratulations, you’re now the VP of HR at a well-funded, 100-person startup.

If the highest positions in the Valley are reserved for people who failed out of the dominant, old-style, “paper belt” culture, then we’re not going to see much innovation or “nerdiness”. Indeed, most of the winners in this crop are well-connected, full-blooded douchebags like Evan Spiegel and Lucas Duplan, who couldn’t even code themselves a better haircut. The unsettling but faultless distinction of being the last genuine nerd to succeed in the Valley probably goes to Mark Zuckerberg.

The new Valley isn’t one where underdogs can succeed. It’s not a place where land is cheap and ideas flow freely. Instead, it’s where highly productive and often creative, but politically disorganized, people (“nerds”) are treated as a resource from which value is extracted. The dominating culture of the Valley now is that of resource extraction. Economically, the latest incarnation has more in common with Saudi Arabia than with what Silicon Valley used to be. The only difference is that, instead of oil being drilled out of the ground, it’s hours of effort and creative passion being mined from talented but politically clueless (and usually quite young) software professionals, who haven’t figured out yet that their work is worth several times what they’re paid to do it, and that the importance of their skills ought to command some basic respect and equity. 

All this said, I’d like to step away from the name-calling and mudslinging. This isn’t some high-minded rejection of those impulses, because I love name-calling and mudslinging. I just want to take a more technical tack. It is fun to pillory the worst people (and that’s why I’m a faithful reader of Valleywag) but, nonetheless, their mere existence shouldn’t prevent the good guys (“nerds”) from succeeding. And, of course, the tribalism that I sometimes invoke (good nerds versus evil businessmen) is, if taken literally, quite ridiculous. There are bad nerds and there are good business people, and I’ve never wanted to imply otherwise. What’s distressing about the Valley is that it so rarely attracts the good business people.

In fact, the bad reputation of business people in the Valley, I think, stems largely from the fact that the competent ones never go there, instead working on billion-dollar Wall Street deals. This makes it easy for a young programmer to forget (or to never know) that they exist. The competent, ethical business people tend either to stay in New York and climb the finance ladder and end up running hedge funds, or deliberately downshift and run smaller “lifestyle” businesses or elite consultancies. Either they win the corporate game, or they leave it, but they don’t half-ass it by playing a less competitive but equally soulless corporate game on the other coast. It’s the inept and malignant ones who tend to find their way out to the Valley, attracted by the enormous “Kick Me” sign that each generation of young software engineers has on its back. 

The degenerate balance of power in the Valley attracts the bad business people, those who can’t make it on their home turf. Meanwhile, the talented and scrupulous ones tend to avoid it, not wanting the association. If the worst students out of Harvard Business School are becoming startup founders and venture capitalists and building nerd armies to build toilet check-in apps, then the best students from those programs are going to stay far away from that ecosystem. 

So why is the balance of power so broken, in the Valley? I think the answer is its short-term focus. To put numbers to it, let’s note that a typical Valley startup requires a business co-founder (who’ll become CEO) and a technical one (who’ll become CTO). The business co-founder raises money, and the technology co-founder builds the product and team. The general rule with founder inclusion is “Code or Contacts”. If someone’s not packing either, you can’t afford the complexity and cost of making him a founder. Both seem important, so why does the “Contacts” side tend to win? Why can’t “Code” people catch a break?

Let’s look at it as a bilateral matching problem, like dating, and assign an attractiveness level to each. For the purposes of this essay, I’m using a 10-point rating scale. 0-2 are the unqualified, 3-4 are the hangers-on, and 5-6 are average among the set (a fairly small one) of people equipped to found startups at all. 7-8 are the solid players of evident high capability, and 9-10 occur at a rate of a few per generation, and people tend to make “Chuck Norris” jokes about them.

You’d expect 10s to match with 10s, 4s with 4s, and so on. Of course, externalities can skew the market and push one side down a notch. In a typical college environment, for an example, women are more attractive to older men (including graduate students and professors) and therefore have more options, and thus more power, than their same-age male counterparts. Mature women simply have no interest in 18-year-old men, while older men do frequently pair off with young women. That power dynamic tends to reverse with age, to the point that women have a justified grievance about the dating scene later in life; but in college, women hold all the cards. 

What does it look like in technology? I’m a Technology 8 and, realistically, I couldn’t expect to pair with a Business 8. It’s not that the Biz 8 is rarer or better than the Tech 8. It’s just that he has more options. Technology people are rated based on what they know and can do. Business people are rated based on connections and what they can squeeze out of their pre-existing relationships. A Business 5 would be a typical graduate of a top-10 MBA program with the connections that implies. A Business 6 has enough contacts to raise venture funding, on reasonable terms, with a product. A Business 7 can raise a Series-A without a product, is frequently invited to lunch with CEOs of large companies, and could become partner at Sequoia on a conversation. What about my at-level counterpart, the Business 8? He’s probably not even in venture-funded technology. He likely has a personal “search fund” and is doing something else. 

The Business 8s and 9s have so many options outside of the Valley that they’re almost never seen there. In fact, a genuine Business 8 who entered the Valley would likely become the most powerful person in it. On the other hand, Technology 8’s like me aren’t nearly as rare. In fact, and it seems almost tautological that it would be this way, Tech 10s and 9s and 8s are found predominantly in technology. Where else would they go?

In a weird way, the Tech 10s are in a disadvantageous position because they have to stay in the industry to remain Tech 10s. The typical Tech 10 works on R&D in a place like Google X, and there aren’t many jobs that offer that kind of autonomy. If he leaves that world, he’ll slide down to Tech 9 (or even 8) status in a few years. The technical co-founder’s major asset requires continual work to remain sharp, and at the 7+ level, it’s pretty damn competitive. The business co-founders asset (connection) is much less perishable. In fact, the weird benefit of connections is that they get more powerful with time. College connections are taken to be much deeper than professional ones, and prep-school connections run deeper still. Why? Because connections and pedigree are all about nostalgia. Blue blood runs deep and must be old; otherwise, it won’t be properly blue. For all of our mouth-honored belief in progress as a technological society, we’re still run by people who are inflexibly backward-looking and neophobic. It’s no wonder, one might remark, that Silicon Valley’s greatest contribution to the world over the past five years has been a slew of toilet check-in apps. 

The Tech 8s and 9s and 10s are generally funneled into technology, because few other environments will even let them maintain (much less improve) their skills. On the other hand, the Biz 8s and 9s and 10s are drawn away by other and better options. Alone, this wouldn’t be that devastating. On both scales, there are more people at each level than at the ones above it, so the absence of the Biz 8+ might have the Tech 8-10s pairing with Biz 7s and the Tech 7s pairing with Biz 6s. Fine; they’ll live. A Biz 6 or 7 can still open plenty of doors. Speaking as a Tech 8, I’d be happy to pair with a Biz 7 as an equal partner. 

Unfortunately, the advantage of business co-founders seems to be about 3-4 points, at any level. That’s what you way pay for their connections. If you’re a Tech 6 and you pair up with a Biz 2-3 co-founder, you’ll probably split the equity evenly (but raising money will be extremely difficult, if not impossible). Pair with a Biz 4, as a Tech 6, and you’ll probably get a third of what he does. If you pair with a Biz 6 co-founder, you’re likely to get 5%. It’s unfair, and unreasonable, but that’s where the market seems to have settled. Why? Removal of the Business 8+ from the market is not, alone, enough to explain it. 

I’ve explained that the business people have up-flight options into other industries. They also have down-flight options when it comes to picking co-founders. If a Tech 8 turns down a Biz 8 to work with a Biz 6, then he’s saying no to someone with millions in VC funding that is essentially already-in-hand, in order to work with someone who might be able to deliver Sequoia after 6 months of work on the product. The Biz 8 is so trusted by the investors that the Tech 8 will probably get a raise to take the founder job; if he works for the Biz 6, he’ll end up working for free for half a year. What, on the other hand, happens to a Biz 8 who turns down his Tech-8 counterpart for a Tech 6? 

The answer is… (crickets) nothing. Investors simply don’t care about the difference. The Biz 8 could pair with a Tech 3 or even another business guy and, while the technical infrastructure of company would be terrible, it wouldn’t really matter. In the Valley, technology is just made to be sold, not actually used. Why hire a Tech 7+ who’s going to make annoying sounds about “functional programming” and “deep neural nets” when you can just hire another VC-connected salesman? Why worry about building a company “for investors” when, with more-connected people involved, you can always just get more investors to pay back the early ones? 

At any chosen level, the business co-founder can choose a tech co-founder 2 to 4 points below him and not lose very much. Strong technical leadership matters if the business is going to last a long time, but these companies are built to be sold or shot dead within four years, so who cares if the technology sucks? To the Biz 8, a Tech 4 willing to take a terrible equity split is a good-enough substitute for the Tech 8. The same doesn’t hold for the Tech 8. Expecting a Biz 4 to handle the VCs is just not an option for him. Even if a Biz 4 or 5 is able to get venture capital, and that itself is unlikely, he’ll probably still be eaten alive by the investors, resulting in terms like “participating preferred” that will emasculate the entire company. 

What all this means is that the business co-founders are more essential because not only are they rarer, but because these businesses don’t last long enough– and aren’t built to last, either– for poor technical leadership to really matter. In such a climate, the leverage of the Tech 7+ is incredibly weak. The value offered by a genuine Tech 8 in the early stages is crucial to a company if one takes a 5- or 10- or 20-year-view, but no one does that anymore. No one cares beyond the sale. The perverse result of this is that technical talent has become non-essential on its home turf. As a result, more power accrues every month to the vision-less, egotistical business guys running the show in Silicon Valley, and lifelong technologists (“nerds”) are even more of a marginalized class. 

I’d like to say that I know how to fix this, how to kill the next-quarter mentality and drive the invaders back out of this wonderful territory (cutting-edge technology) where they are most unwelcome, but the truth is that I don’t know what to do. The short-term mentality seems to be a permanent fixture and, in that light, I don’t think that talented technologists are ever going to get a fair shake against influence peddlers, manipulators, and merchants of connections. I’m sorry, but I can’t fix it, and I don’t know whether it can be fixed. 

Why there are so few AI jobs

Something began in the 1970s that has been described as “the AI winter”, but to call it that is to miss the point, because the social illness it represents involves much more than artificial intelligence (AI). AI research was one of many casualties that came about as anti-intellectualism revived itself and society fell into a diseased state.

One might call the “AI winter” (which is still going on) an “interesting work winter” and it pertains to much more of technology than AI alone, because it represented a sea change in what it meant to be a programmer. Before the disaster, technology jobs had an R&D flavor, like academia but with better pay and less of the vicious politics. After the calamitous 1980s and the replacement of R&D by M&A, work in interesting fields (e.g. machine learning, information retrieval, language design) became scarce and over 90% of software development became mindless, line-of-business makework. At some point, technologists stopped being autonomous researchers and started being business subordinates and everything went to hell. What little interesting work remained was only available in geographic “super-hubs” (such as Silicon Valley) where housing prices are astronomical compared to the rest of the country. Due to the emasculation of technology research in the U.S., economic growth slowed to a crawl, and the focus of the nation’s brightest minds turned to creation of asset bubbles (seen in 1999, 2007, and 2014) rather than generating long-lasting value.

Why did this happen? Why did the entrenched public- and private-sector bureaucrats (with, even among them, the locus of power increasingly shifting to private-sector bureaucrats, who can’t be voted out of office) who run the world lose faith in the research being done by people much smarter, and who work much harder, than them? The answer is simple. It’s not even controversial. End of the Cold War? Nah, it began before that. At fault is the lowly perceptron.

Interlude: a geometric puzzle

This is a simple geometry puzzle. Below are four points at the corners of the square, colored (and numbered) like so:

0 1
1 0

Is it possible to draw a line that separates the red points (0’s) from the green points (1’s)?

The answer is that it’s not possible. Any separating line would have to separate two points from each other. Now draw a circle passing through all four points. Any line can intersect that circle at no more than two points. Therefore, a line separating two points from the other two would have to separate two adjacent points, which would be of opposing colors. It’s not possible. Another way to say this is that the classes (colors) aren’t linearly separable.

What is a perceptron?

“Perceptron” is a fancy name given to a mathematical function with a simple description. Let w be a known “weight” vector (if that’s an unfamiliar term, a list of numbers) and x be an input “data” vector of the same size, with the caveat that x[0] = 1 (a “bias” term) always. The perceptron, given w, is a virtual “machine” that computes, for any given input x, the following:

  • 1, if w[0]*x[0] + … + w[n]*x[n] > 0,
  • 0, if w[0]*x[0] + … + w[n]*x[n] < 0.

In machine learning terms, it’s a linear classifier. If there’s a linear function that cleanly separates the “Yes” class (the 1 values) from the “No” class (the 0 values) it can be expressed as a perceptron. There’s an elegant algorithm for, in that linearly separable case, finding a working weight vector. It always converges.

A mathematician might say, “What’s so interesting about that? It’s just a dot product being passed through a step function.” That’s true. Perceptrons are very simple. A single perceptron can solve more decision problems than one might initially think, but it can’t solve all of them. It’s too simple a model.

Limitations

Let’s say that you want to model an XOR (“exclusive or”) gate, corresponding to the following function:

| in_1 | in_2 | out |
+------+------+-----+
|   0  |   0  |  0  |
|   0  |   1  |  1  |
|   1  |   0  |  1  |
|   1  |   1  |  0  |
+------+------+-----+

One might recognize that this is identical to the “brainteaser” above, with in_1 and in_2 corresponding to the x- and y- dimensions in the coordinate plane. This is the same problem. This function is nonlinear; it could be expressed as f(x, y) = x + y – 2xy. and that’s arguably the simplest representation of it that works. A separating “plane” in the 2-dimensional space of the inputs would be a line, and there’s no line separating the two classes. It’s mathematically obvious that the perceptron can’t do it. I showed this, above, using high-school geometry.

To a mathematician, this isn’t surprising. Marvin Minsky pointed out the mathematically evident limitations of a single perceptron. One can model intricate mathematical functions with more complex networks of perceptrons and perceptron-like units, called artificial neural networks. They work well. One can also, using what are called “basis expansions”, generate further dimensions from existing data in order to create a higher-dimensional space in which linear classifiers still work. (That’s what people usually do with support vector machines, which provide the machinery to do so efficiently.) For example, adding xy as a third “derived” input dimension would make the classes (0’s and 1’s) linearly separable. There’s nothing mathematically wrong with doing that; it’s something that statisticians do when they want to build complex models but still have some of the analytic properties of simpler ones, like linear regression or nearest-neighbor modeling.

The limitations of the single perceptron do not invalidate AI. At least, they don’t if you’re a smart person. Everyone in the AI community could see the geometrically obvious limitation of a single perceptron, and not one of them believed that it came close to invalidating their work. It only proved that more complex models were needed for some problems, which surprised no one. Single-perceptron models might still be useful for computational efficiency (in the 1960s, computational power was about a billion times as expensive as now) or because the data don’t support a more complex model; they just couldn’t learn or model every pattern.

In the AI community, there was no scandal or surprise. That some problems aren’t linearly separable is not surprising. However, some nerd-hating non-scientists (especially in business upper management) took this finding to represent more than it actually did.

They fooled us! A brain with one neuron can’t have general intelligence!

The problem is that the world is not run, and most of the wealth in it is not controlled, by intelligent people. It’s run by social-climbing empty-suits who are itching for a fight and would love to take some “eggheads” down a notch. Insofar as an artificial neural network models a brain, a perceptron models a single neuron, which can’t be expected to “think” at all. Yet the fully admitted limitations of a single perceptron were taken, by the mouth-breathing muscleheads who run the world, as an excuse to shit on technology and pull research funding because “AI didn’t deliver”. That produced an academic job market that can only be described as a pogrom, but it didn’t stop there. Private-sector funding dried up as short-term, short-tempered management came into vogue.

To make it clear, no one ever said that a single perceptron can solve every decision problem. It’s a linear model. That means it’s restricted, intentionally, to a small subspace of possible models. Why would people work with a restricted model? Traditionally, it was for a lack of data. (We’re in the 1960s and ’70s, when data was contained on physical punch cards and a megabyte weighed something and a disk drive cost more than a car.) If you don’t have a lot of data, you can’t build complex models. For many decision problems, the humble perceptron (like its cousins, logistic regression and support vector machines) did well and, unlike other computationally intensive linear classification methods (such as logistic regression, which requires gradient descent, or a variant thereof, over the log-likelihood surface; or such as the support vector machine, which are a quadratic programming problem that we didn’t know how to solve efficiently until the 1990s) it could be trained with minimal computational expense, in a bounded amount of time. Even today, linear models are surprisingly effective for a large number of problems. For example, the first spam classifiers (Naive Bayes) operated using a linear model, and it worked well. No one was claiming that a single perceptron was the pinnacle of AI. It was something that we could build cheaply on 1970-era hardware and that could build a working model on many important datasets.

Winter war

Personally, I don’t think that the AI Winter was an impersonal, passive event like the changes of seasons. Rather, I think it was part of a deliberate resurgence of anti-intellectualism in a major cultural war– one which the smart people lost. The admitted limitations of one approach to automated decision-making gave the former high school bullies, now corporate fat cats, all the ammo they needed in order to argue that those “eggheads” weren’t as smart as they thought they were. None of them knew exactly what a perceptron or an “XOR gate” were, but the limitation that I’ve described was morphed into “neural networks can’t solve general mathematical problems” (arguably untrue) and that turned into “AI will never deliver”. In the mean-spirited and anti-liberal political climate of the 1980s, this was all that anyone needed as an excuse to cut public funding. The private sector not only followed suit, but amplified the trend. The public cuts were a mix of reasonable fiscal conservatism and mean-spirited anti-research sentiment, but the business elites responded strongly to (and took to a whole new level) the mean-spirited aspect, flexing their muscles as elitism (thought vanquished in the 1930s to ’50s) became “sexy” again in the Reagan Era. Basic research, which gave far too much autonomy and power to “eggheads”, was slashed, marginalized, and denigrated.

The claim that “AI didn’t deliver” was never true. What actually happened is that we solved a number of problems, once thought to require human intelligence, with a variety of advanced statistical means as well as some insights from fields like physics, linguistics, ecology and economics. Solving problems demystified them. Automated mail sorting, once called “artificial intelligence”, became optical character recognition. This, perhaps, was part of the problem. Successes in “AI” were quickly put into a new discipline. Even modern practitioners of statistical methods are quick to say that they do machine learning, not AI. What was actually happening is that, while we were solving specific computational problems once thought to require “intelligence”, we found that our highly specialized solutions did well on the problems they were designed for, and could be adapted to similar problems, but with very slow progress toward general intelligence. As it were, we’ve learned in recent decades that our brains are even more complicated than we thought, with a multitude of specialized modules. That no specific statistical algorithm can replicate all of them, working together in real time, shouldn’t surprise anyone. Is this an issue? Does it invalidate “AI” research? No, because most of those victories, while they fell short of replicating a human brain, still delivered immense economic value. Google, although it eventually succumbed to the sociological fragility and failure that inexorably follow closed allocation, began as an AI company. It’s now worth over $360 billion.

Also mixed in with the anti-AI sentiment is the religious aspect. It’s still an open and subjective question what human intelligence really is. The idea that human cognition could be replicated by a computer offended religious sentiments, even though few would consider automated mail sorting to bear on unanswerable questions about the soul. I’m not going to go deep into this philosophical rabbit hole, because I think it’s a waste of time to debate why people believe AI research (or, for a more popular example, evolution by natural selection) to offend their religious beliefs. We don’t know what qualia is or where it comes from. I’ll just leave it at this. If we can use advanced computational techniques to solve problems that were expensive, painful, or impossible given the limitations of human cognition, we should absolutely do it. Those who object to AI on religious grounds fear that advanced computational research will demystify cognition and bring about the end of religion. Ignoring the question of whether an “end of religion” is a bad thing, or what “religion” is, there are two problems with this. First, if there is something to us that is non-material, we won’t be able to replicate it mechanically and there is no harm, to the sacred, in any of this work. Second, computational victories in “AI” tend to demystify themselves and the subfield is no longer considered “AI”. Instead, it’s “optical character recognition” or “computer game-playing”. Most of what we use on a daily basis (often behind the scenes, such as in databases) comes from research that was originally considered “artificial intelligence”.

Artificial intelligence research has never told us, and will never tell us, whether it is more reasonable to believe in gods and religion or not to believe. Religion is often used by corrupt, anti-intellectual, politicians and clerics to rouse sentiment against scientific progress, as if automation of human grunt work were a modern-day Tower of Babel. Yet, to show what I mean by AI victories demystifying themselves, almost none would hesitate to use Google, a web-search service powered by AI-inspired algorithms.

Why do the anti-intellectuals in politics and business wish to scare the public with threats of AI-fueled irreligion and secularism (as if those were bad things)? Most of them are intelligent enough to realize that they’re making junk arguments. The answer, I think, is about raw political dominance. As they see it, the “nerds” with their “cushy” research jobs can’t be allowed to (gasp!) have good working conditions.

The sad news is that the anti-intellectuals are likely to take the economy and society down with them. In the 1960s, when we were putting billions of dollars into “wasteful” research spending, the economy grew at a record pace. The world economy was growing at 5.7 percent per year, and the U.S. economy was the envy of the world. Now, in our spartan time of anti-intellectualism, anti-science sentiment, and corporate elitism, the economy is sluggish and the society is stagnant– all because the people in charge can’t stand to see “eggheads” win.

Has AI “delivered”?

If you’re looking to rouse religious fear and fury, you might make a certain species of fantastic argument against “artificial intelligence”. The truth of the matter, however, is that while we’ve seen domain-specific superiority of machines over human intelligence in rote processes, we’re still far from creating an artificial general intelligence, i.e. a computational entity that can exhibit the general learning capability of a human. We might never do it. We might not need to and, I would argue, we should not if it is not useful.

In a way, “artificial intelligence” is a defined-by-exclusion category of “computational problems we haven’t solved yet”. Once we figure out how to make computers better at something than humans are, it becomes “just computation” and is taken for granted. Few believe they’re using “an AI” when they use Google for web search, because we’re now able to conceive of the computational work it does as mechanical rather than “intelligent”.

If you’re a business guy just looking to bully some nerds, however, you aren’t going to appeal to religion. You’re going to make the claim that all this work on “artificial intelligence” hasn’t “delivered”. (Side note: if someone uses “deliver” intransitively, as business bullies are wont to do, you should punch that person in the face.) Saying someone or something isn’t “delivering” is a way to put false objectivity behind a claim that means nothing other than “I don’t like that person”. As for AI, it’s true that artificial general intelligence has eluded us thus far, and continues to do so. It’s an extremely hard problem: far harder than the optimists among us thought it would be, fifty years ago. However, the CS research community has generated a hell of a lot of value along the way.

The disenchantment might be similar to the question about “flying cars”. We actually have them. They’re called small airplanes. In the developed world, a person of average means can learn how to fly one. They’re not even that much more expensive than cars. The reason so few people use airplanes for commuting is that it just doesn’t make economic sense for them: the savings of time don’t justify increased fuel and maintenance costs. But a middle-class American or European can, if she wants, have a “flying car” right now. It’s there. It’s just not as cheap or easy to use as we’d like. With artificial intelligence, that research has brought forth a ridiculous number of victories and massive economic growth. It just hasn’t brought forth an artificial general intelligence. That’s fine; it’s not clear that we need to build one in order to get the immense progress that technologists create when given the autonomy and support.

Back to the perceptron

One hard truth I’ve learned is that any industrial effort will have builders and politicians. It’s very rare that someone is good at both. In the business world, those unelected private-sector politicians are called “executives”. They tend, for a variety of reasons, to put themselves into pissing contests with the builders (“eggheads”) who are actually making stuff. One time-tested way to show up the builders is to take something that is obviously true (leading the builders to agree with the presentation) but present it out of context in a way that is misleading.

The incapacity of the single perceptron at general mathematical modeling is a prime example of this. Not one AI researcher was surprised that such a simple model couldn’t describe all patterns or equational relationships. The fact that can be proven (as I did) with high school geometry. That a single perceptron can’t model a key logical operation is, as above, obviously true. The builders knew it, and agree. Unfortunately, what the builders failed to see was that the anti-intellectual politicians were taking this fact way out of context, using the known limitations of a computational building block to ascribe limitations (that did not exist) to general structures. This led to the general dismantling of public, academic, and private support for technological research, an anti-intellectual and mean-spirited campaign that continues to this day.

That’s why there are so few AI jobs.

Technology’s Loser Problem

I’m angry. The full back story isn’t worth getting into, but there was a company where I applied for a job in the spring of 2013: to build a company’s machine learning infrastructure from scratch. It was a position of technical leadership (Director equivalent, but writing code with no reports) and I would have been able to use Clojure. As it were, I didn’t get it. They were looking for someone more experienced, who’d built those kinds of systems before, and wouldn’t take 6 months to train up to the job. That, itself, is not worth getting angry about. Being turned down happens, especially at high levels.

I found out, just now, that the position was not filled. Not then. Not 6 months later. Not to this day, more than a year later. It has taken them longer to fill the role than it would have taken for me to grow into it.

When they turned me down, it didn’t faze me. I thought they’d found a better candidate. That happens; only thing I can do is make myself better. I found myself, however, a bit irked when I found out that they hadn’t filled the position for longer than it would have taken me to gain the necessary experience. I lost, and so did they.

That’s not what makes me angry. Rationally, I realize that most companies aren’t going to call back a pretty-good candidate they rejected because they had just opened the position and they thought they could do better (if you’re the first 37% of candidates for a job, it makes sense for them not to choose you and, empirically, first and second applicants for a high-level position rarely get it). That’s the sort of potentially beneficial but extremely awkward social process that just won’t happen. What makes me angry is the realization of how common a certain sort of decision is in the technology world. We make a lot of lose-lose decisions that hurt all of us. Extremely specific hiring requirements (that, in bulk, cost the company more in waiting time than training a 90% match up to the role) are just the tip of the iceberg.

You know those people who complain about the lack of decent <gender of sexual interest> but (a) reject people for the shallowest, stupidest reasons, (b) aren’t much of a prize and don’t work to better themselves, and (c) generally refuse to acknowledge that the problem is rooted in their own inflated perception of their market value? That’s how I feel every time I hear some corporate asswipe complain about a “talent shortage” in technology. No, there isn’t one. You’re either too stingy or too picky or completely inept at recruiting, because there’s a ton of underemployed talent out there.

Few of us, as programmers, call the initial shots. We’ve done a poor job of making The Business listen to us. However, when we do have power, we tend to fuck it up. One of the problems is that we over-comply with what The Business tells us it whats. For example, when a nontechnical CEO says, “I only want you to hire absolute rock stars”, what he actually means is, “Don’t hire an idiot just to have a warm body or plug a hole”. However, because they tend to be literal, over-compliant, and suboptimal, programmers will interpret that to mean, “Reject any candidate who isn’t 3 standard deviations above the mean.” The leads to positions not being filled, because The Business is rarely willing to pay what one standard deviation above the mean costs, let alone three.

Both sides now

I’ve been on both sides of the interviewing and hiring process. I’ve seen programmers’ code samples described with the most vicious language over the most trivial mistakes, or even stylistic differences. I’ve seen job candidates rejected for the most god-awful stupid reasons. In one case, the interviewer clearly screwed up (he misstated the problem in a way that made it impossible) but, refusing to risk face by admitting the problem was on his end, he claimed the candidate failed the question. Another was dinged on a back-channel reference (don’t get me started on that sleazy practice, which ought to be illegal) claiming, without any evidence, that “he didn’t do much” on a notable project four years ago. I once saw an intern denied a full-time offer because he lived in an unstylish neighborhood. (The justification was that one had to be “hungry”, mandating Manhattan.) Many of us programmers are so butthurt about not being allowed to sit at the cool kids’ table that, when given the petty power associated with interviewing other programmers, the bitch-claws come out in a major way.

Having been involved in interviewing and recruiting, I’ll concur that there are a significant number of untalented applicants. If it’s 99.5 percent, you’re doing a lot of things wrong, but most resumes do come from people way out of their depth. Moreover, as with dating, there’s an adverse weighting in play. Most people aren’t broken, but broken people go on orders of magnitude more dates than everyone else, which is why most peoples’ dating histories have a disproportionate representation of horror stories, losers, and weirdos. It’s the same with hiring, but phone screening should filter against that. If you’re at all good at it, about half of the people brought in-office will be solid candidates.

Of course, each requirement cuts down the pool. Plenty of companies (in finance, some officially) have a “no job hopper” or “no unemployeds” rule. Many mandate high levels of experience in new technologies (even though learning new technologies is what we’re good at). Then, there are those who are hung up on reference checking in weird and creepy ways. I know of one person who proudly admits that his reference checking protocol is to cold-call a random person (again, back-channel) is the candidate’s network and ask the question, without context, “Who is the best person you’ve ever worked with?” If anyone other than the candidate is named, the candidate is rejected. That’s not being selective. That’s being an invasive, narcissistic idiot. Since each requirement reduces the size of qualified people, it doesn’t take long before the prejudices winnow an applicant pool down to zero.

Programmers? Let’s be real here, we kinda suck…

As programmers, we’re not very well-respected, and when we’re finally paid moderately well, we let useless business executives (who work 10-to-3 and think HashMap is a pot-finding app) claim that “programmer salaries are ridiculous”. (Not so.) Sometimes (to my horror) you’ll hear a programmer even agree that our salaries are “ridiculous”. Fuck that bullshit; it’s factually untrue. The Business is, in general, pretty horrible to us. We suffer under closed allocation, deal with arbitrary deadlines, and if we don’t answer to an idiot, we usually answer to someone else who does. Where does the low status of programmers come from? Why are we treated as cost centers instead of partners in the business? Honestly… much of the problem is us. We’ve failed to manage The Business, and the result is that it takes ownership of us.

Most of the time, when a group of people is disproportionately successful, the cause isn’t any superiority of the average individual, but a trait of the group: they help each other out. People tend to call these formations “<X> Mafia” where X might be an ethnicity, a school, or a company. Y Combinator is an explicit, pre-planned attempt to create a similar network; time will tell if it succeeds. True professions have it. Doctors look out for the profession. With programmers, we don’t see this. There isn’t a collective spirit: just long email flamewars about tabs versus spaces. We don’t look out for each other. We beat each other down. We sell each other out to non-technical management (outsiders) for a shockingly low bounty, or for no reason at all.

In many investment banks, there’s an established status hierarchy in which traders and soft-skills operators (“true bankers”) are at the top, quants are in the middle, and programmers (non-quant programmers are called “IT”) are even lower. I asked a high-ranking quant why it was this way, and he explained it in terms of the “360 degree” performance reviews. Bankers and traders all gave each other top ratings, and wrote glowing feedback for minor favors. They were savvy enough to figure out that it was best for them to give great reviews up, down, and sideways, regardless of their actual opinions. Quants tended to give above-average ratings and occasionally wrote positive feedback. IT gave average ratings for average work and plenty of negative feedback. The programmers were being the most honest, but hurting each other in the process. The bankers and traders were being political, and that’s a good thing. They were savvy enough to know that it didn’t benefit them to sell each other out to HR and upper management. Instead, they arranged it so they all got good ratings and the business had to, at baseline, appreciate and reward all of them. While it might seem that this hurt top performers, it had the opposite effect. If everyone got a 50 percent bonus and 20% raise, management had to give the top people (and, in trading, it’s pretty obvious who those are) even more.

Management loves to turn high performers against the weak, because this enables management to be stingy on both sides. The low performers are fired (they’re never mentored or reassigned) and the high performers can be paid a pittance and still have a huge bonus in relative terms (not being fired vs. being fired). What the bankers were smart enough to realize (and programmers, in general, are not) is that performance is highly context-driven. Put eight people of exactly equal ability on a team to do a task and there will be one leader, two or three contributors, and the rest will be marginal or stragglers. It’s just more efficient to have the key knowledge in a small number of heads. Open source projects work this way. What this means is that, even if you have excellent people and no bad hires, you’ll probably have some who end up with not much to show for their time (which is why open allocation is superior; they can reassign themselves until they end up in a high-impact role). If management can see who is in what role, it can fire the stragglers and under-reward the key players (who, because they’re already high performers, are probably motivated by things other than money… at least, for now). The bankers and traders (and, to a lesser extent, the quants) had the social savvy and sense to realize that it was best that upper management not know exactly who was doing what. They protected each other, and it worked for them. The programmers, on the other hand, did not, and this hurt top performers as well as those on the bottom.

Let’s say that an investment bank tried to impose tech-company stack ranking on its employees, associate level and higher. (Analyst programs are another matter, not to be discussed here.) Realizing the mutual benefit in protecting each other, the bankers would find a way to sabotage the process by giving everyone top ratings, ranking the worst employees highly, or simply refusing to do the paperwork. And good for them! Far from being unethical, this is what they should do: collectively work The Business to get what they’re actually worth. Only a programmer would be clueless enough to go along with that nonsense.

In my more pessimistic moods, I tend to think that we, as programmers, deserve our low status and subordinacy. As much as we love to hate those “business douchebags” there’s one thing I will say for them. They tend to help each other out a lot more than we do. Why is this? Because they’re more political and, again, that might not be a bad thing. Ask a programmer to rate the performance of a completely average colleague and you’ll get an honest answer: he was mediocre, we could have done without him. These are factual statements about average workers, but devastating when put into words. Ask a product manager or an executive about an average colleague and you’ll hear nothing but praise: he was indispensable, a world-class player, best hire in ten years. They realize that it’s politically better for them, individually and as a group, to keep their real opinions to themselves and never say anything that could remotely endanger another’s career. Even if that person’s performance was only average, why make an enemy when one can make a friend?

“Bad code”

Let’s get to another thing that we do, as programmers, that really keeps us down. We bash the shit out of each other’s code and technical decision-making, often over minutiae.

I hate bad code. I really do. I’ve seen plenty of it. (I’ve written some, but I won’t talk about that.) I understand why programmers complain about each other’s code. Everyone seems to have an independent (and poorly documented) in-head culture that informs how he or she writes code, and reading another person’s induces a certain “culture shock”. Even good code can be difficult to read, especially under time pressure. And yes, most large codebases have a lot of code in them that’s truly shitty, sometimes to the point of being nearly impossible to reason about. Businesses have failed because of code quality problems, although (to tell the whole story) it’s rare that one bad programmer can do that much damage. The worst software out there isn’t the result of one inept author, but the result of code having too many authors, often over years. It doesn’t help that most companies assign maintenance work to either to junior programmers, or demoted (and disengaged) senior ones, neither category having the power to do it right.

I’d be the last one to come out and defend bad code. That said, I think we spend too much time complaining about each other’s code– and, worse yet, we tend toward the unforgivable sin of complaining to the wrong people. A technical manager has, at least, the experience and perspective to know that, at some level, every programmer hates other peoples’ code. But if that programmer snitches to a non-technical manager and executive,  well… you’ve just invited a 5-year-old with a gun to the party. Someone might get fired because “tabs versus spaces” went telephone-game into “Tom does shoddy work and is going to destroy the business”. Because executives are politically savvy enough to protect the group, and only sell each other out in extreme circumstances, what started out as a stylistic disagreement sounds (to the executive ear) like Tom (who used his girlfriend’s computer to fix a production problem at 11:45 on a Friday night, the tabs/spaces issue being for want of an .emacs.d) is deliberately destroying the codebase and putting the whole company at risk.

As programmers, we sell each other out all the time. If we want to advance beyond reasonable but merely upper-working class salaries, and be more respected by The Business, we have to be more careful about this kind of shit. I’ve heard a great number of software engineers say things like, “Half of all programmers should just be fired.” Now, I’ll readily agree that there are a lot of badly-trained programmers out there whose lack of skill causes a lot of pain. But I’m old enough to know that people come to a specific point from a multitude of paths and that it’s not useful to personalize this sort of thing. Also, regardless of what we may think as individuals, almost no doctor or banker would ever say, to someone outside his profession, “half of us should be fired”. They’re savvy enough to realize the value of protecting the group, and handling competence and disciplinary matters internally. Whether to fire, censure, mentor or praise is too important a decision to let it happen outside of our walls.

There are two observations about low-quality code, one minor and one major. The minor one is that code has a “all of us is worse than any of us” dynamic. As more hands pass over code, it tends to get worse. People hack the code needing specific features, never tending to the slow growth of complexity, and the program evolves over time into something that nobody understands because too many people were involved in it. Most software systems fall to pieces not because of incompetent individuals, but because of unmanaged growth of complexity. The major point on code-quality is: it’s almost always management’s fault.

Bad code comes from a multitude of causes, only one of which is low skill in programmers. Others include unreasonable deadlines, unwillingness to attack technical debt (a poor metaphor, because the interest rate on technical “debt” is both usurious and unpredictable), bad architecture and tooling choices, and poor matching of programmers to projects. Being stingy, management wants to hire the cheapest people it can find and give them the least time possible in which to do the work. That produces a lot of awful code, even if the individual programmers are capable. Most of the things that would improve code quality (and, in the long term, the health and performance of the business) are things that management won’t let the programmers have: more competitive salaries, more autonomy, longer timeframes, time for refactoring. The only thing that management and the engineers can agree on is firing (or demoting, because their work is often still in use and The Business needs someone who understands it) those who wrote bad code in the past.

One thing I’ve noticed is that technology companies do a horrible job of internal promotion. Why is that? Because launching anything will typically involve compromises with the business on timeframe and headcount, resulting in bad code. Any internal candidate for a promotion has left too many angles for attack. Somewhere out there, someone dislikes a line of code he wrote (or, if he’s a manager, something about a project he oversaw). Unsullied external candidates win, because no one can say anything bad about them. Hence, programming has the culture of mandatory (but, still, somewhat stigmatized) job hopping we know and love.

What’s really at the heart of angry programmers and their raging against all that low-quality code? Dishonest attribution. The programmer can’t do shit about the dickhead executive who set the unreasonable deadlines, or the penny-pinching asswipe managers who wouldn’t allow enough salary to hire anyone good. Nor can he do much about the product managers or “architects” who sit above and make his life hell on a daily basis. But he can attack Tom, his same-rank colleague, over that commit that really should have been split into two. Because they’re socially unskilled and will generally gleefully swallow whatever ration of shit is fed to them by management, most programmers can very easily be made to blame each other for “bad code” before blaming the management that required them to use the bad code in the first place.

Losers

As a group, software engineers are losers. In this usage, I’m not using the MacLeod definition (which is more nuanced) and my usage is halfway pejorative. I generally dislike calling someone a loser, because the pejorative, colloquial meaning of that word conflates unfortunate circumstance (one who loses) with deserved failure. Here, however, it applies. Why do we lose? Because we play against each other, instead of working together to beat the outside world. As a group, we create our own source of loss.

Often, we engage in zero- or negative-sum plays just to beat the other guy. It’s stupid. It’s why we can’t have nice things. We slug each other in the office and wonder why external hires get placed over us. We get into flamewars about minutiae of programming languages, spread FUD, and eventually some snot-nosed dipshit gets the “brilliant” idea to invite nontechnical management to weigh in. The end result is that The Business comes in, mushroom stamps all participants, and says, “Everything has to be Java“.

Part of the problem is that we’re too honest, and we impute honesty in others when it isn’t there. We actually believe in the corporate meritocracy. When executives claim that “low performers” are more of a threat to the company than their astronomical, undeserved salaries and their doomed-from-the-start pet projects, programmers are the only people stupid enough to believe them, and will often gleefully implement those “performance-based” witch hunts that bankers would be smart enough to evade (by looking for better jobs, and arranging for axes to fall on people planning exits anyway). Programmers attempt to be apolitical, but that ends up being very political, because the stance of not getting political means that one accepts the status quo. That’s radically conservative, whether one admits it or not.

Of course, the bankers and traders realize the necessity of appearing to speak from a stance of professional apolitical-ness. Every corporation claims itself to be an apolitical meritocracy, and it’s not socially acceptable to admit otherwise. Only a software engineer would believe in that nonsense. Programmers hear “Tom’s not delivering” or “Andrea’s not a team player” and conceive of it as an objective fact, failing to recognize that, 99% of the time, it means absolutely nothing more or less than “I don’t like that person”.

Because we’re so easily swayed, misled, and divided, The Business can very easily take advantage of us. So, of course, it does. It knows that we’ll sell each other out for even a chance at a seat at the table. I know a software engineer who committed felony perjury against his colleagues just to get a middle-management position and the right to sit in on a couple of investor meetings. Given that this is how little we respect each other, ourselves, and our work, is it any wonder that software engineers have such low status?

Our gender issues

I’m going to talk, just briefly, about our issues with women. Whatever the ultimate cause of our lack of gender diversity– possibly sexism, possibly that the career ain’t so great– it’s a major indictment of us. My best guess? I think sexism is a part of it, but I think that most of it is general hostility. Women often enter programming and find their colleagues hostile, arrogant, and condescending. They attribute that to their gender, and I’m sure that it’s a small factor, but men experience all of that nonsense as well. To call it “professional hazing” would be too kind. There’s often nothing professional about it. I’ve dealt with rotten personalities, fanaticism about technical preference or style, and condescension and, honestly, don’t think there’s a programmer out there who hasn’t. When you get into private-sector technology, one of the first things you learn is that it’s full of assholes, especially at higher levels.

Women who are brave enough to get into this unfriendly industry take a look and, I would argue, most decide that it’s not worth it to put up with the bullshit. Law and medicine offer higher pay and status, more job security, fewer obnoxious colleagues, and enough professional structure in place that the guy who cracks rape jokes at work isn’t retained just because he’s a “rockstar ninja”.

“I thought we were the good guys?”

I’ve often written from a perspective that makes me seem pro-tech. Originally, I approached the satirical MacLeod pyramid with the belief that “Technocrat” should be used to distinguish positive high-performers (apart from Sociopaths). I’ve talked about how we are a colonized people, as technologists. It might seem that I’m making businesspeople out to be “the bad guys” and treating programmers as “the good guys”. Often, I’m biased in that very direction. But I also have to be objective. There are good business people out there, obviously. (They’re just rare in Silicon Valley, and I’ll get to that.) Likewise, software engineers aren’t all great people, either. I don’t think either “tribe” has a monopoly on moral superiority. As in Lost, “we’re the good guys” doesn’t mean much.

We do get the worst (in terms of ethics and competence) of the management/business tribe in the startup world. That’s been discussed at length, in the essay linked above. The people who run Silicon Valley aren’t technologists or “nerds” but machiavellian businessmen who’ve swooped in to the Valley to take advantage of said nerds. The appeal of the Valley, for the venture capitalists and non-technical bro executives who run it, isn’t technology or the creation of value, but the unparalleled opportunity to take advantage of too-smart, earnest hard workers (often foreign) who are so competent technically that they often unintentionally generate value, but don’t know the first thing about how to fight for their own interests.

It’s easy to think ourselves morally superior, just because the specific subset of business people who end up in our game tends to be the worst of that crowd. It’s also a trap. We have a lot to learn form the traders and bankers of the world about how to defend ourselves politically, how to stand a chance of capturing some of the value we create, and how to prevent ourselves from being robbed blind by people who may have lower IQs, but have been hacking humans for longer than we could have possibly been using computers. Besides, we’re not all good. Many of us aren’t much better than our non-technical overlords. Plenty of software engineers would gladly join the bad guys if invited to their table. The Valley is full of turncoat software engineers who don’t give a shit about the greater mission of technology (using knowledge to make peoples’ lives better) and who’d gladly sell their colleagues out to cost-cutting assholes in management.

Then there are the losers. Losers aren’t “the bad guys”. They don’t have the focus or originality that would enable them to pull off anything complicated. Their preferred sin is typically sloth. They’ll fail you when you need them the most, and that ‘s what makes them infuriating. They just want to put their heads down and work, and the problem is that they can’t be trusted to “get political” when that’s exactly what’s needed. The danger of losers is in numbers. The problem is that so many software engineers are clueless, willing losers who’ll gladly let political operators take everything from them.

When you’re young and don’t know any better, one of the appeals of software engineering is that it appears, superficially, to tolerate people of low social ability. To people used to artificial competition against their peers, this seems like an attractive trait of the industry; it’s not full of those “smooth assholes” and “alpha jocks”. After several years observing various industries, I’ve come to the conclusion that this attitude is not merely misguided, but counterproductive. You want socially skilled colleagues. Being the biggest fish in a small pond just means that there are no big fish to protect you when the sharks come in. Most of those “alpha jocks” aren’t assholes or idiots (talk to them, nerds; you’ll be surprised) and, when The Business comes in and is looking for a fight, it’s always best to have strong colleagues who’ve got your back.

Here’s an alternate, and quite possible hypothesis: maybe The Business isn’t actually full of bad guys. One thing that I’ve realized is that people tend to push blame upward. For example, the reputation of venture capitalists has been harmed by founders blaming “the VCs” for their own greed and mismanagement. It gives the grunt workers an external enemy, and the clueless can be tricked into working harder than they should (“they don’t really like us and haven’t given us much, but if we kill it on this project and prove them wrong, maybe they’ll change their minds!”). It actually often seems that most of the awfulness of the software industry doesn’t come directly from The Business, but from turncoat engineers (and ex-engineers) trying to impress The Business. In the same way that young gang members are more prone to violence than elder dons, the most creative forms of evil seem to come from ex-programmers who’ve changed their colors.

The common enemy

So long as software engineers can easily be divided against each other on trivial matters like tabs versus spaces and scrotum versus kanban, we’ll never get the respect (and, more importantly, the compensation) that we’re due. These issues distract us from what we really need to do, which is figure out how to work The Business. Clawing at each other, each trying to become the favored harem queen of the capitalist, is suboptimal compared to the higher goal of getting out of the harem.

I’ve spoken of “The Business” as if it were a faceless, malevolent entity. It might sound like I’m anti-business, and I’m not. Business is just a kind of process. Good people, and bad people, start businesses and some add great value to the world. The enemy isn’t private enterprise itself, but the short-term thinking and harem-queen politics of the established corporation. Business organizations get to a point where they cease having a real reason to exist, and all that’s left is the degenerate social contest for high-ranking positions. We, as programmers, seem to lack the skill to prevent that style of closed-allocation degeneracy from happening. In fact, we seem to unintentionally encourage it.

The evil isn’t that software is a business, but that technical excellence has long since been subordinated entirely to the effectively random emotional ups and downs of non-technical executives who lack the ability to evaluate our work. It’s that our weird ideology of “never get political” is actually intensely political and renders us easy to abuse. Business naturally seems to be at risk of anti-intellectual tendencies and, rather than fight back against this process, we’ve amplified it just to enjoy the illusion of being on the inside, among the “cool kids”, part of The Business. Not only does our lack of will to fight for our own interests leave us at the mercy of more skilled business operators, but it attracts an especially bad kind of them. Most business people, actually, aren’t the sorts of corporate assholes we’re used to seeing run companies. It’s just that our lack of social skill appeals to the worst of that set: people who come in to technology to take advantage of all the clueless, loser nerds who won’t fight for themselves. If we forced ourselves to be more discerning judges of character, and started focusing on ethics and creativity instead of fucking tabs-versus-spaces, we might attract a better sort of business person, and have an industry where stack ranking and bastardized-“Agile” micromanagement aren’t even considered.

If we want to improve our situation, we have to do the “unthinkable” (which is, as I’ve argued, actually quite thinkable). We have to get political.

What’s a mid-career software engineer actually worth? Try $779,000 per year as a lower bound.

Currently, people who either have bad intentions or a lack of knowledge are claiming that software engineer salaries are “ridiculous”. Now, I’ll readily admit that programmers are, relative the general population, quite well paid. I’m not about to complain about the money I make; I’m doing quite well, in a time and society where many people aren’t. The software industry has many problems, but low pay for engineers (at least, for junior and mid-career engineers; senior engineers are underpaid but that’s an issue for another time) doesn’t crack the top 5. Software engineers are underpaid, relative to the massive amount of value (if given proper projects, rather than mismanaged as is typical) they are capable of delivering. In comparison to the rest of the society, they do quite well.

So what should a software engineer be paid? There’s a wide disparity in skill level, so it’s hard to say. I’m going to focus on a competent, mid-career engineer. This is someone between 5 and 10 years of experience, with continual investment in skill, and probably around 1.6 on this software engineering scale. He’s not a hack or the stereotypical “5:01″ programmer who stopped learning new skills at 24, but he’s not a celebrity either. He’s good and persistent and experienced, but probably not an expert. In the late 1990s, that person was just cracking into six-figure territory: $100,000 per year. No one thought that number was “ridiculous”. Adjusted for inflation, that’s $142,300 per year today. That’s probably not far off what an engineer at that level actually makes, at least in New York and the Bay Area.

Software engineers look “ridiculous” to people who haven’t been software engineers in 20 years (or ever) and whose numbers are way out of date. If you’re a Baby Boomer whose last line of code was in 1985, you’re probably still thinking that $60,000 is a princely sum for a programmer to earn. When one factors inflation into the equation, programmer salaries are only “at record high” because inflation is an exponential process. Taking that out, they’re right about where history says they should be.

I would argue, even, that programmer salaries are low when taking a historical perspective. The trend is flat, adjusting for inflation, but the jobs are worse. Thirty years ago, programming was an R&D job. Programmers had a lot of autonomy: the kind of autonomy that it takes if one is going to invent C or Unix or the Internet or a new neural network architecture. Programmers controlled how they worked and what they worked on, and either answered to other programmers or to well-read scientists, rather than anti-intellectual businessmen who regard them as cost centers. Historically, companies sincerely committed to their employees’ careers and training. You didn’t have to change jobs every 2 years just to keep getting good projects and stay employable. The nature of the programming job, over the past couple decades, has become more stressful (open-plan offices) and careers have become shorter (ageism). Job volatility (unexpected layoffs and, even, phony “performance-based” firings in lieu of proper layoffs, in order to skimp on severance because that’s “the startup way”) has increased. With all the negatives associated with a programming job in 2014, that just didn’t exist in the 1970s to ’80s, flat performance on the salary curve is disappointing. Finally, salaries in the Bay Area and New York have kept abreast of general inflation, but the costs of living have skyrocketed in those “star cities”, while the economies of the still-affordable second-tier cities have declined. In the 1980s and ’90s, there were more locations in which a person could have a proper career, and that kept housing prices down. In 2014, that $142,000 doesn’t even enable one to buy a house in a place where there are jobs.

All of those factors are subjective, however, so I’ll discard them. We have sufficient data to know that $142,000 for a mid-career programmer is not ridiculous. It’s a lower bound for the business value of a software engineer (in 1999); we know that employers did pay that; they might have been willing to pay more. This information already gives us victory over the assclowns claiming that software engineer salaries are “ridiculous” right now.

Now, I’ll take it a step further and introduce Yannis’s Law: programmer productivity doubles every 6 years. Is it true? I would say that the answer is a resounding “yes”. For sure, there are plenty of mediocre programmers writing buggy, slow websites and abusing Javascript in truly awful ways. On the other hand, there is more recourse for a good programmer who find quality; rather than commit to commercial software, she can peruse the open-source world. There’s no evidence for a broad-based decline in programmer ability over the years. It’s also easy to claim that the software career “isn’t fun anymore” because so much time is spent gluing existing components together, and accounting for failures of legacy systems. I don’t think these gripes are new, and I think tools are improving, and a 12% per year rate sounds about right. Put another way, one who programs exactly as was done in 1999 is only about 18 percent as productive as one using modern tools. And yet that programmer, only 18% as productive as his counterpart today, was worth $142,000 (2014 dollars) back then!

Does this mean that we should throw old tools away (and older programmers under the bus)? Absolutely not. On the contrary, it’s the ability to stand on the shoulders of giants that makes us able to grow (as a class) at such a rate. Improved tools and accumulated knowledge deliver exponential value, but there’s a lot of knowledge that is rarely learned except over a decades-long career. Most fresh Stanford PhDs wouldn’t be able to implement a performant, scalable support vector machine from scratch, although they could recite the theory behind one. Your gray-haired badasses would be rusty on the theory but, with a quick refresh, stand a much greater chance of building it righjt. Moreover, the best old ideas tend to recur and long-standing familiarity is an advantage. The most exciting new programming language right now is Clojure, a Lisp that runs on the Java Virtual Machine. Lisp, as an idea, is over 50 years old. And Clojure simply couldn’t have been designed by a 25-year-old in Palo Alto. For programmers, the general trend is a 12% increase in productivity; but individuals can reliably do 30 percent or more, and for periods spanning over decades.

If the business value of a mid-level programmer in 1999 was $142,000 in today’s dollars, then one can argue that today, with programmers 5.7 times more productive, the true value is $779,000 per year at minimum. It might be more. For the highly competent and for more senior programmers, it certainly is higher. And here’s another thing: investors and managers and VPs of marketing didn’t create that surplus. We did. We are almost 6 times as productive as we were in the 1990s not because they got better at their jobs (they haven’t) but because we built the tools to make ourselves (and our successors) better at what we do. By rights, it’s ours.

Is it reasonable, or realistic, to argue that mid-career software engineers ought to be earning close to a million dollars per year? Probably not. It seems to be inevitable, and also better for society, that productivity gains are shared. We ought to meet in the middle. That we don’t capture all of the value we create is a good thing. It would be awful, for example, if sending an email cost as much as sending a letter by post or, worse yet, as much as using the 19th-century Pony Express, because the producers of progress had captured all of the value for themselves. So, although that $779,000 figure adequately represents the value of a decent mid-career engineer to the business, I wouldn’t go so far as to claim that we “deserve” to be paid that much. Most of us would ecstatic with real equity (not that 0.05% VC-istan bullshit) and a quarter of that number– and with the autonomy to deliver that kind of value.

What Silicon Valley’s ageism means

Computer programming shouldn’t be ageist. After all, it’s a deep discipline with a lot to learn. Peter Norvig says it takes 10 years, but I’d consider that number a minimum for most people. Ten years of high-quality, dedicated practice to the tune of 5-6 hours per day, 250 days per year, might be enough. For most people, it’s going to take longer, because few people can work only on the interesting problems that constitute dedicated practice. The fundamentals (computer science) alone take a few thousand hours of study, and then there’s the experience of programming itself, which one must do in order to learn how to do it well. Getting code to work is easy. Making it efficient, robust, and legible is hard. Then, there’s a panoply of languages, frameworks, paradigms, to learn and absorb and, for many, to reject. As an obstacle, there’s the day-to-day misery of a typical software day job, in which so much time is wasted on politics and meetings and pointless projects that an average engineer is lucky to have 5 hours per week for learning and growth. Ten years might be the ideal; I’d bet that 20 years is typical for the people who actually become great engineers and, sadly, the vast majority of professional programmers never get anywhere close.

It takes a long time to be actually good in software engineering. The precocious are outliers. More typically, people seem to peak after 40, as in all the other high-skill disciplines. It, then, seems that most of age-related decline in this field is externally enforced. Age discrimination is not an artifact of declining ability but changing perceptions.

It doesn’t make sense, but there it is.

Age discrimination has absolutely no place in technology. Yet it exists. After age 40, engineers find it increasingly difficult to get appropriate jobs. Startups are, in theory, supposed to “trade against” the inefficiencies and moral failures of other companies but, on this issue, the venture capital (VC) funded startups are the biggest source of the problem. Youth and inexperience have become virtues, while older people who push back against dysfunction (and, as well, outright exploitation) are cited as “resistant to change”.

There’s another issue that isn’t derived from explicit ageism, but might as well be. Because our colonizers (mainstream business culture) are superficial, they’ve turned programming into a celebrity economy. A programmer has two jobs. In addition to the work itself, which is highly technical and requires continual investment and learning, there’s a full-time reputation-management workload. If a machine learning engineer works at a startup and spends most of his time in operations, he’s at risk of being branded “an ops guy”, and may struggle to get high-quality projects in his specialty from that point on. He hasn’t actually lost anything– in fact, he’s become far more valuable– but the superficial, nontechnical idiots who evaluate us will view him as “rusty” in his specialty and, at the least, exploit his lack of leverage. All because he spent 2 years doing operations, because it needed to be done!

As we get older and more specialized, the employment minefield becomes only more complicated. We are more highly paid at that point, but not by enough of a margin to offset the increasing professional difficulties. Executives cite the complexity of high-end job searches when demanding high salaries and years-long severances. Programmers who are any good face the same, but get none of those protections. I would, in fact, say that any programmer who is at all good needs a private agent, just as actors do. The reputation management component of this career, which is supposed to be about technology and work and making the world better, but is actually about appeasing the nontechnical, drooling patron class, constitutes a full-time job that requires a specialist. Either we need unions, or we need an agent model like Hollywood, or perhaps we need both. That’s another essay, though.

The hypocrisy of the technology employer

Forty years ago, smart people left finance and the mainstream corporate ladder for technology, to move into the emerging R&D-driven guild culture that computing had at the time. Companies like Hewlett-Packard were legitimately progressive in how they treated their talent, and rewarded for it by their employees’ commitment to making great products. In this time, Silicon Valley represented, for the most technically adept people in the middle class, a genuine middle path. The middle path will require its own essay, but what I’m talking about here is a moderate alternative between the extremes of subordination and revolution. Back then, Silicon Valley was the middle path that, four decades later, it is eagerly closing.

Technology is no longer managed by “geeks” who love the work and what it can do, but by the worst kinds of business people who’ve come in to take advantage of said geeks. Upper management in the software industry is, in most cases, far more unethical and brazen than anywhere else. To them, a concentration of talented people who don’t have the inclination or cultural memory that would lead them to fight for themselves (labor unions, agent models, ruthlessness of their own) is an immense resource. Consequently, some of the most disgusting HR practices (e.g. stack ranking, blatant sexism) can be found in the technology industry.

There’s one really bad and technical trait of software employers that, I think, has damaged the industry immensely. Technology employers demand specialties when vetting people for jobs. General intelligence and proven ability to code isn’t enough; one has to have “production experience” in a wide array of technologies invented in the past five years. For all their faults, the previous regime of long-lasting corporations was not so bigoted when it came to past experience, trusting people to learn on the job, as needed. The new regime has no time for training or long-term investment, because all of these companies have been built to be flipped to a greater fool. In spite of their bigoted insistence on pre-existing specialties in hiring, they refuse to respect specialties once people are hired. Individual programmers who attempt to protect their specialties (and, thus, their careers) by refusing assignment to out-of-band or inferior grunt work are quickly fired. This is fundamentally hypocritical. In hiring, software companies refuse to look twice at someone without a yellow brick road of in-specialty accomplishments of increasing scope; yet, once employees are inside and fairly captive (due to the pernicious stigma against changing jobs quickly, even with good reason) they will gladly disregard that specialty, for any reason or no reason. Usually, this is framed as a business need (“we need you to work on this”) but it’s, more often, political and sometimes personal. Moving talent out of its specialty is a great way for insecure middle managers to neutralize overperformance threats. In a way, employers are like the pervert who chases far-too-young sexual partners (if “partner” is the right word here) for their innocence, simply to experience the thrill of destroying it. They want people who are unspoiled by the mediocrity and negativity of the corporate world, because they want to inflict the spoilage. The virginity of a fresh, not-yet-cynical graduate from a prestigious university is something they want all for themselves.

The depression factor

I’m not going to get personal here, but I’m bipolar so when I use words like “depression” and “hypomania” and “anxiety” I do, in fact, know what the fuck I am talking about.

A side effect of corporate capitalism, that I see, is that it has created a silent epidemic of middle-aged depression. The going assumption in technology that mental ability declines after age 25 is not well supported, and it is in fact contrary to what most cultures believe about intelligence and age. (In truth, various aspects of cognition peak at different ages– from language acquisition at age 5 to writing ability around 65– and there’s also so much individual variation that there’s no clear “peak” age.) For general, holistic intelligence, there’s no evidence of an age-bound peak in healthy people. While this risks sounding like a “No True Scotsman” claim, what I mean to say is that every meaningful age-related decline in cognition can be tracked to a physical health problem and not aging itself. Cardiovascular problems, physical pain and side effects of medication can impair cognition. I’m going to talk about the Big One, though, and that’s depression. Depression can cause cognitive decline. Most of that loss is reversible, but only if the person recovers from it and, in many cases, they never do.

In this case, I’m not talking about severe depression, the kind that would have a person considering electroconvulsive therapy or on suicide watch. I’m talking about mild depression that, depending on time of diagnosis, might be considered subclinical. People experiencing it in middle age are, one presumes, liable to attribute it to getting older rather than a real health problem. Given that middle-aged “invisibility” in youth-obsessed careers is, in truth, unjust and depressing, it seems likely that more than a few people would experience depression and fail to perceive it as a health issue. That’s one danger of depression that those who’ve never experienced it might not realize exists: when you’re depressed, you suffer from the delusion that (a) you’ve always been depressed, and (b) that no other outlook or mood makes sense. Depression is delusional, and it is a genuine illness, but it’s also dangerously self-consistent.

Contrary to the stereotype, people with depression aren’t always unhappy. In fact, people with mild depression can be happy quite often. It’s just easier to make them unhappy. Things that don’t faze normal people, like traffic jams or gloomy weather or long queues at the grocery store, are more likely to bother them. For some, there’s a constant but low-grade gloom and tendency to avoid making decisions. Others might experience 23 hours and 50 minutes per day of normal mood and 10 minutes of intense, debilitating, sadness: the kind that would force them to pull over to the side of the road and cry. There isn’t a template and, just as a variety of disparate diseases (some viral, some bacterial, and some behavioral) were once called “fevers”, I feel like “depression” is a cluster of about 20 different diseases that we just don’t have the tools to separate. Some depressions come without external cause. Others are clearly induced by environmental stresses. Some depressions impair cognition and probably constitute a (temporary) 30-IQ-point loss. Others (more commonly seen in artists than in technology workers) seem to induce no intellectual impairment at all; the person is miserable, but as sharp as ever.

Corporate workers do become less sharp, on average, with age. You don’t see that effect, at least not involuntarily so, in most intellectually intense fields. A 45-year-old artist or author or chess master has his best work ahead of him. True entrepreneurs (not dipshits who raise VC based on connections) also seem to peak in their 50s and, for some, even later. Most leaders hit their prime around 60. However, it’s observable that something happens in Corporate America that makes people more bitter, more passive, and slower to act over time, and that it starts around 40. Perhaps it’s an inverse of survivor bias, with the more talented people escaping the corporate racket (by becoming consultants, or entrepreneurs) before middle age. I don’t think so, though. There are plenty of highly talented people in their forties and fifties who’ve been in private-sector programming for a long time and just seem out of gas. I don’t blame them for this. With better jobs, I think they’d recover their power surprisingly quickly. I think they have a situationally-induced case of mild depression that, while it may not be the life-threatening illness we tend to associate with major depression, takes the edge off their abilities. It doesn’t make them unemployable. It makes them slow and bitter but, unlike aging itself, it’s very easily reversible: change the context.

Most of these slowed-down, middle-aged, private-sector programmers wouldn’t qualify for major depressive disorder. They’re not suicidal, don’t have debilitating panic attacks, and can attribute their losses of ability (however incorrectly) to age. Rather, I think that most of them are mildly but chronically depressed. To an individual, this is more of a deflation than a disability; to society, the costs are enormous, just because such a large number of people are affected, and because it disproportionately affects the most experienced people at a time when, in a healthier economic environment, they’d be in their prime.

The tournament of idiots

No one comes out of university wanting to be a private-sector social climber. There’s no “Office Politics” major. People see themselves as poets, economists, mathematicians, or entrepreneurs. They want to make, build, and do things. To their chagrin, most college graduates find that between them and any real work, there’s at least a decade of political positioning, jockeying for permissions and status, and associated nonsense that’s necessary if one intends to navigate the artificial scarcity of the corporate world.

The truth is that most of the nation’s most prized and powerful institutions (private-sector companies) have lost all purpose for existing. Ideals and missions are for slogans, but the organization’s true purpose is to line the pockets of those ranking high within it. There’s also no role or use for real leadership. Corporate executives are the farthest one gets from true leaders. Most are entrenched rent-seekers. With extreme economic inequality and a culture that worships consumption, it should surprise no one that our “leadership” class is a set of self-dealing parasites at a level that hasn’t been seen in an advanced economy since pre-Revolution France.

Leadership and talent have nothing to do with getting to the top. It’s the same game of backstabbing and political positioning that has been played in kings’ courts for millennia. The difference, in the modern corporation, is that there’s a pretense of meritocracy. People, at least, have to pretend to be working and leading to advance further. The work that is most congruent with social advancement, however, isn’t the creative work that begets innovation. Instead, it’s success in superficial reliability. Before you get permission to be creative, you have to show that you can suffer, and once you’ve won the suffering contest, it’s neither necessary nor worth it to take creative risks. Companies, therefore, pick leaders by loading people with unnecessary busywork that often won’t go anywhere, and putting intense but ultimately counterproductive demands on them. They generate superficial reliability contests. One person will outlast the others, who’ll fall along the way due to unexpected health problems, family emergencies, and other varieties of attrition (for the lucky few, getting better jobs elsewhere). One of the more common failure modes by which people lose this tournament of idiots is mild depression: not enough to have them hospitalized, but enough to pull them out of contention.

The corporate worker’s depression, especially in midlife, isn’t an unexpected side effect of economic growth or displacement or some other agent that might allow Silicon Valley’s leadership to sweep it under the rug of “unintended consequences”. Rather, it’s a primary landscape feature of the senseless competition that organizations create for “leadership” (rent-seeking) positions when they’ve run out of reasons to exist. At that level of decay, there is no meaningful definition of “merit” because the organization itself has turned pointless, and the only sensible way to allocate highly-paid positions is to create a tournament of idiots, in which people psychologically abuse each other (often subtly, in the form of microaggressions) until only a few remain healthy enough to function.

Here we arrive at a word I’ve come to dread: corporate culture. Every corporation has a culture, and 99% of those are utterly abortive. Generally, the more that a company’s true believers talk about “our culture”, the more fucked-up the place actually is. See, culture is often ugly. Foot binding, infantile genital mutilation (“female circumcision”), war, and animal torture are all aspects of human culture. Previous societies used supernatural appeal to defend inhumane practices, but modern corporations use “our culture” itself as a god. “Culture fit” is often cited to justify the otherwise inconsistent and, sometimes, unjustifiable. Why wasn’t the 55-year-old woman, a better coder than anyone else on the team, hired? “It wouldn’t look right.” Can’t say that! “A seasoned coder who isn’t rich would shatter the illusion that everyone good gets rich.” Less illegal, but far too honest. “She didn’t fit with the culture.” Bingo! Culture can always be used in this way, by an organization, because it’s a black box of blame, diffusing moral culpability about the group. Blaming an adverse decision on “the team” or “the culture” avoids individual risk for the blamer, but the culture itself can never be attacked as bad. Most people in most organizations actually know that the “leadership team” (career office politicians, also known as executives) of their firm is toxic and incompetent. When they aren’t around, the executives are attacked. But it’s rare that anyone ever attacks the culture because “the culture” is everyone. To indict it is to insult the people. In this way, “the culture” is like an unassailable god.

Full circle

We’ve traveled through some dark territory. The tournament of idiots that organizations construct to select leadership roles, once they’ve ceased to have a real purpose, causes depression. The ubiquity of such office cultures has created, I argue, a silent epidemic of mild, midlife depression that has led venture capitalists (situated at its periphery, their wealth buying them some exit from the vortex of mediocrity in which they must still work, but do not live) and privileged young psuedo-entrepreneurs (terrified of what awaits them when their family connections cool and they must actually work) to conclude that a general cognitive mediocrity awaits in midlife, even though there is no evidence to support this belief, and plenty of evidence from outside of corporate purgatory to contradict it.

What does all of this say? Personally, I think that, to the extent that large groups of individuals and organizations can collectively “know” things, the contemporary corporate world devalues experience because it knows that the experience it provides is of low value. It refuses to eat its own dogfood, knowing that it’s poisoned.

For example, software is sufficiently technical and complex that great engineers are invariably experienced ones. The reverse isn’t true. Much corporate experience is of negative value, at least if one includes the emotional side effects that can lead to depression. Median-case private-sector technology work isn’t sufficiently valuable to overcome the disadvantages associated with age, which is another way of saying that the labor market considers average-case corporate experience to have negative value. I’m not sure that I disagree. Do I think it’s right to write people off because of their age? Absolutely not. Do I agree with the labor market’s assessment that most corporate work rots the brain? Well, the answer is mostly “yes”. The corporate world turns smart people, over the years, into stupid ones. If I’m right that the cause of this is midlife depression, there’s good news. Much of that “brain rot” is contextual and reversible.

How do we fix this?

Biologists and gerontologists seeking insight into longevity have studied the genetics and diet of long-living groups of people, such as the Sardinians and the people of Okinawa. Luckily for us, midlife cognitive decline isn’t a landscape feature of most technical or creative fields. (In fact, it’s probably not present in ours; it’s just perceived that way.) There are plenty of places to look for higher cognitive longevity, because few industries are as toxic as the contemporary software industry. When there is an R&D flavor to the work, and when people have basic autonomy, people tend to peak around 50, and sometimes later. Of course, there’s a lot of individual variation, and some people voluntarily slow down before that age, in order to attend to health, family, spiritual, or personal concerns. The key word to that is voluntary

Modeling and professional athletics (in which there are physical reasons for decline) aside, a career in which people tend to peak early, or have an early peak forced upon them, is likely to be a toxic one that enervates them. Silicon Valley’s being a young man’s game (and the current incarnation of it, focused on VC-funded light tech, is exactly that) simply indicates that it’s so destructive to the players that only the hard-core psychopaths can survive in it for more than 10 years. It’s not a healthy place to spend a long period of time and develop an expertise. As discussed above, it will happily consume expertise but produces almost none of value (hence, its attraction to those with pre-existing specialties, despite failing to respect specialties once the employee is captive). This means, as we already see, that technical excellence will fall by the wayside, and positive-sum technological ambition will give way to the zero-sum personal ambitions of the major players.

We can’t fix the current system, in which the leading venture capitalists are striving for “exits” (greater fools). That economy has evolved from being a technology industry with some need for marketing, to a marketing industry with some need (and that bit declining) for technology. We can’t bring it back, because its entrenched players are too comfortable with it being the way it is. While the current approach provides mediocre returns on investment, hence the underperformance of the VC  asset class, the king-making and career-altering power that it affords the venture capitalist allows him to capture all kinds of benefits on the side, ranging from financial upside (“2 and 20″) to executive positions for talentless drinking buddies to, during brief bubbly episodes such as this one, “coolness”. They like it the way it is, and it won’t change. Rather than incrementally fix the current VC-funded Valley, it must be replaced outright.

The first step, one might say, is to revive R&D within technology companies. That’s a step in the right direction, but it doesn’t go far enough. Technology should be R&D, full stop. To get there, we need to assert ourselves. Rather than answering to non-technical businessmen, we need to learn how to manage our own affairs. We need to begin valuing the experience on which most R&D progress actually relies, rather than shutting the seasoned and cynical out. And, as a first step in this direction, we need to stop selling each other out to nontechnical management for stupid reasons, including but especially “culture fit”.