How the Other Half Works: an Adventure in the Low Status of Software Engineers

Bill (not his real name, and I’ve fuzzed some details to protect his identity) is a software engineer on the East Coast, who, at the time (between 2011 and 2014) of this story, had recently turned 30 and wanted to see if he could enter a higher weight class on the job market. In order to best assess this, he applied to two different levels of position at roughly equivalent companies: same size, same level of prestige, same U.S. city on the West Coast. To one company, he applied as a Senior Software Engineer. To the other, he applied for VP of Data Science.

Bill had been a Wall Street quant and had “Vice President” in his title, noting that VP is a mid-level and often not managerial position in an investment bank. His current title was Staff Software Engineer, which was roughly Director-equivalent. He’d taught a couple of courses and mentored a few interns, but he’d never been an official manager. So he came to me for advice on how to appear more “managerial” for the VP-level application.

The Experiment

His first question was what it would take to get “managerial experience” in his next job. I was at a loss, when it comes to direct experience, so my first thought was, “Fake it till you make it”. Looking at his résumé, the “experiment” formed in my mind. Could I make Bill, a strong but not exceptional data-scientist-slash-software-engineer, over into a manager? The first bit of good news was that we didn’t have to change much. Bill’s Vice President title (from the bank) could be kept as-is, and changing Staff Software Engineer to Director didn’t feel dishonest, because it was a lateral tweak. If anything, that’s a demotion because engineering ladders are so much harder to climb, in dual-track technology companies, than management ladders.

Everything in Bill’s “management résumé” was close enough to true that few would consider it unethical. We upgraded his social status and management-culture credibility– as one must, and is expected to, do in that world– but not his technical credentials. We turned technical leadership into “real”, power-to-fire leadership, but that was the only material change. We spent hours making sure we weren’t really lying, as neither Bill nor I was keen on damaging Bill’s career to carry out this experiment, and because the integrity of the experiment required it.

In fact, we kept the management résumé quite technical. Bill’s experience was mostly as implementor, and we wanted to stay truthful about that. I’ll get to the results of the experiment later on, but there were two positive side effects of his self-rebranding, as a “manager who implemented”. The first is that, because he didn’t have to get his hands dirty as a manager, he got a lot of praise for doing things that would just have been doing his job if he were a managed person. Second, and related to the first but far more powerful, is that he no longer had to excuse himself for menial projects or periods of low technical activity. As opposed to, “I was put on a crappy project”, which projects low status, his story evolved into “No one else could do it, so I had to get my hands dirty”, which is a high-status, managerial excuse for spending 6 months on an otherwise career-killing project. Instead of having to explain why he didn’t manage to get top-quality project allocation, as one would ask an engineer, he was able to give a truthful account of what he did but, because he didn’t have to do this gritty work, it made him look like a hero rather than a zero.

What was that project? It’s actually relevant to this story. Bill was maintaining a piece of old legacy code that took 40,000 lines to perform what is essentially a logistic regression. The reason for this custom module to exist, as opposed to using modern statistical software instead, was that a variety of requirements had come in from the business over the years, and while almost none of these custom tweaks were mathematically relevant, they all had to be included in the source code, and the program was on the brink of collapsing under the weight of its own complexity. These projects are career death for engineers, because one doesn’t learn transferrable skills by doing them, and because maintenance slogs don’t have a well-defined end or “point of victory”. For Bill’s technical résumé, we had to make this crappy maintenance project seem like real machine learning. (Do we call it a “single-layer neural network”? Do we call the nonsensical requirements “cutting-edge feature engineering”?) For his management résumé, the truth sufficed: “oversaw maintenance of a business-critical legacy module”.

In fact, one could argue that Bill’s management résumé, while less truthful on-paper, was more honest and ethical. Yes, we inflated his social status and gave him managerial titles. However, we didn’t have to inflate his technical accomplishments, or list technologies that he’d barely touched under his “Skills” section, to make a case for him. After a certain age, selling yourself as an engineer tends to require (excluding those in top-notch R&D departments or open-allocation shops) that you (a) only work on the fun stuff, rather than the career-killing dreck, and play the political games that requires, (b) mislead future employers about the quality of your work experience, or (c) spend a large portion of your time on side projects, which usually turns into a combination of (a) and (b).

Was this experiment ethical? I would say that it was. When people ask me if they should fudge their career histories or résumés, I always say this: it’s OK to fix prior social status because one’s present state (abilities, talents) is fully consistent with the altered past. It’s like formally changing a house’s address from 13 to 11 before selling it to a superstitious buyer: the fact being erased is that it was once called “13″, one that will never matter for any purpose or cause material harm to anyone. On the other hand, lying about skills is ethically wrong (it’s job fraud, because another person is deceived into making decisions that are inconsistent with the actual present state, and that are possibly harmful in that context) and detrimental, in the long term, to the person doing it. While I think it’s usually a bad idea to do so, I don’t really have a moral problem with people fudging dates or improving titles on their résumés, insofar as they’re lying about prior social status (a deception as old as humanity itself) rather than hard currencies like skills and abilities.

Now, let’s talk about how the experiment turned out.

Interview A: as Software Engineer

Bill faced five hour-long technical interviews. Three went well. One was so-so, because it focused on implementation details of the JVM, and Bill’s experience was almost entirely in C++, with a bit of hobbyist OCaml. The last interview sounds pretty hellish. It was with the VP of Data Science, Bill’s prospective boss, who showed up 20 minutes late and presented him with one of those interview questions where there’s “one right answer” that took months, if not years, of in-house trial and error to discover. It was one of those “I’m going to prove that I’m smarter than you” interviews.

In the post-mortem, I told Bill not to sweat that last interview. Often, companies will present a candidate with an unsolved or hard-to-solve problem and don’t expect a full solution in an hour. I was wrong on that count.

I know people at Company A, so I was able to get a sense of how things went down. Bill’s feedback was: 3 positive, 1 neutral, and 1 negative, exactly as might have been expected from his own account. Most damning were the VP’s comments: “good for another role, but not on my team“. Apparently the VP was incensed that he had to spend 39 and a half minutes talking to someone without a PhD and, because Bill didn’t have the advanced degree, the only way that that VP would have considered him good enough to join would be if he could reverse-engineer the firm’s “secret sauce” in 40 minutes, which I don’t think anyone could.

Let’s recap this. Bill passed three of his five interviews with flying colors. One of the interviewers, a few months later, tried to recruit Bill to his own startup. The fourth interview was so-so, because he wasn’t a Java expert, but came out neutral. The fifth, he failed because he didn’t know the in-house Golden Algorithm that took years of work to discover. When I asked that VP/Data Science directly why he didn’t hire Bill (and he did not know that I knew Bill, nor about this experiment) the response I got was “We need people who can hit the ground running.” Apparently, there’s only a “talent shortage” when startup people are trying to scam the government into changing immigration policy. The undertone of this is that “we don’t invest in people”.

Or, for a point that I’ll come back to, software engineers lack the social status necessary to make others invest in them.

Interview B: as Data Science manager.

A couple weeks later, Bill interviewed at a roughly equivalent company for the VP-level position, reporting directly to the CTO. 

Worth noting is that we did nothing to make Bill more technically impressive than for Company A. If anything, we made his technical story more honest, by modestly inflating his social status while telling a “straight shooter” story for his technical experience. We didn’t have to cover up periods of low technical activity; that he was a manager, alone, sufficed to explain those away.

Bill faced four interviews, and while the questions were behavioral and would be “hard” for many technical people, he found them rather easy to answer with composure. I gave him the Golden Answer, which is to revert to “There’s always a trade-off between wanting to do the work yourself, and knowing when to delegate.” It presents one as having managerial social status (the ability to delegate) but also a diligent interest in, and respect for, the work. It can be adapted to pretty much any “behavioral” interview question.

As a 6-foot-1, white male of better-than-average looks, Bill looked like an executive and the work we did appears to have paid off. In each of those interviews, it only took 10 minutes before Bill was the interviewer. By presenting himself as a manager, and looking the part, he just had an easier playing field than a lifelong engineer would ever get. Instead of being a programmer auditioning to sling code, he was already “part of the club” (management) and just engaging in a two-way discussion, as equals, on whether he was going to join that particular section of the club.

Bill passed. Unlike for a typical engineering position, there were no reference checks. The CEO said, “We know you’re a good guy, and we want to move fast on you”. As opposed tot he 7-day exploding offers typically served to engineers, Bill had 2 months in which to make his decision. He got a fourth week of vacation without even having to ask for it, and genuine equity (about 75% of a year’s salary vesting each year).

I sat in when Bill called to ask about relocation and, honestly, this is where I expected the deal to fall apart. Relocation is where so many offers fall to pieces. It’s a true test of whether a company actually sees someone as a key player, or is just trying to plug a hole with a warm body. The CEO began by saying, “Before getting into details, we are a startup…”

This was a company with over 100 employees, so not really a startup, but I’m going to set that aside for now. I was bracing for the “oh, shit” moment, because “we’re a startup” is usually a precursor to very bad news.

“… so we’ll cover the moving costs and two months of temporary housing, and a $10,000 airfare budget to see any family out East, but we can’t do loss-on-sale for the house, and we can’t cover realtor fees.”

Bill was getting an apology because the CEO couldn’t afford a full executive relocation workup. (“We’re just not there yet.”) For a software engineer, “relocation” is usually some shitty $3,000 lump-sum package, because “software engineer”, to executives, means “22-year-old clueless male with few possessions, and with free storage of the parental category”. On the other hand, if you’re a manager, you might be seen as a real human being with actual concerns about relocating to another part of the country.

It was really interesting, as I listened in, to see how different things are once you’re “in the club”. The CEO talked to Bill as an equal, not as a paternalistic, bullshitting, “this is good for your career” authority figure. There was a tone of equality that a software engineer would never get from the CEO of a 100-person tech company. 

Analysis

Bill has a superhuman memory and took a lot of notes after each interview, so there was plenty to analyze about this sociological experiment. It taught me a lot. At Company A, Bill was applying for a Senior Engineer position and his perceived “fit” seemed to start at 90. (Only 90, for his lack of PhD and Stanford pedigree.) But everything he didn’t know was points off. No experience with Spring and Struts? Minus 5. Not familiar with the firm’s Golden Algorithm? Not a real “data scientist”; minus 8. No Hadoop experience? Minus 6. Bill was judged on what he didn’t know– on how much work it would take to get him up to speed and have him serving as a reliable corporate subordinate.

Company B showed a different experience entirely. Bill started at 70, but everything he knew was a bonus. He could speak intelligently about logistic regression and maximum likelihood methods? Plus 5. He’s actually implemented them? Plus 6. He knows about OCaml? Plus 5. Everything he knew counted in his favor. I’d argue that he probably scored these “points” for irrelevant “interesting person” details, like his travel.  

When a programmer gets to a certain age, she knows a lot of stuff. But there’s a ton of stuff she doesn’t know, as well, because no one can know even a fraction of everything that’s going on in this industry. It’s far better, unless you’re applying for a junior position, to start at 70 and get credit for everything you do know, than to start at 90 (or even 100) and get debited for the things you don’t know.

This whole issue is about more than what one knows and doesn’t know about technology. As programmers, we’re used to picking up new skills. It’s something we’re good at (even if penny-shaving businessmen hate the idea of training us). This is all about social status, and why status is so fucking important when one is playing the work game– far more important than being loyal or competent or dedicated. 

Low and high status aren’t about being liked or disliked. Some people are liked but have low status, and some people are disliked but retain high status. In general, it’s more useful and important to have high status at work than to be well-liked. It’s obviously best to have both, but well-liked low-status people get crap projects and never advance. Disliked high-status people, at worst, get severance. As Machiavelli said, “it is far safer to be feared than loved if you cannot be both.” People’s likes and dislikes change with the seasons, but a high-status person is more unlikely to have others act against his interests.

Moreover, if you have low social status, people will eventually find reasons to dislike you unless you continually sacrifice yourself in order to be liked, and even that strategy runs out of time. At high social status, they’ll find reasons to like you. At low status, your flaws are given prime focus and your assets, while acknowledged, dismissed as unimportant or countered with “yes, buts” which turn any positive trait into a negative. (“Yes, he’s good in Clojure, but he’s might be one of those dynamic-typing cowboy coders!” “Yes, he’s good in Haskell, but that means he’s one of those static-typing hard-asses.” “Yes, he’s a good programmer, but he doesn’t seem like a team player.”) When you have low status, your best strategy is to be invisible and unremarkable, because even good distinctions will hurt you. You want to keep your slate ultra-clean and wait for mean-reversion to drift you into middling status, at which point being well-liked can assist you and, over some time– and it happens glacially– bring you upper-middle or high status.

When you have high status, it’s the reverse. Instead of fighting to keep your slate blank, it’s actually to your benefit to have things (good things) written about you on it. People will exaggerate your good traits and ignore the bad ones (unless they are egregious or dangerous). You start at 70 and people start looking for ways to give you the other 30 points.

The Passion of the Programmer

I’ve always felt that programmers had an undeserved low social status, and the experiment above supports that claim. Obviously, these are anecdotes rather than data, but I think that we can start to give a technical definition to the low social status of “software engineers”.

Whether programmers are over- or underpaid usually gets into debates about economics and market conditions and, because those variables fluctuate and can’t be measured precisely enough, the “are programmers (under|over)-paid?” debate usually ends up coming down to subjective feelings rather than anything technical. Using this technical notion of status– whether a person’s flaws or positive traits are given focus– we have the tools to assess the social status of programmers without comparing their salaries and work conditions to what we feel they “deserve”. If you are in a position where people emphasize your flaws and overlook your achievements, you have low social status (even if you make $200,000 per year, which only means efforts to cut your job will come faster). If the opposite is true, you have high social status. 

Using this lens, the case for the low social status of the programmer could not be any clearer. We’ll never agree on a “platonically correct” “fair value” for an engineer’s salary. What can see is that technologists’ achievements are usually under-reported by the businesses in which they work, while their mistakes are highlighted. I’ve worked in a company where the first thing said to me about a person was the production outage he caused 4 years ago, when he was an intern. (Why is nothing said about the manager who let an intern cause an outage? Because that manager was a high status person.) A big part of the problem is that programmers are constantly trying to one-up each other (see: feigned surprise) and prove their superior knowledge, drive, and intelligence. From the outside (that is, from the vantage point of the business operators we work for) these pissing contests make all sides look stupid and deficient. By lowering each others’ status so reliably, and when little to nothing is at stake, programmers lower their status as a group. 

There was a time, perhaps 20 years gone by now, when the Valley was different. Engineers ran the show. Technologists helped each other. Programmers worked in R&D environments with high levels of autonomy and encouragement. To paraphrase from one R&D shop’s internal slogan, bad ideas were good and good ideas were great. Silicon Valley was an underdog, a sideshow, an Ellis Island for misfits and led by “sheepdogs” intent on keeping mainstream MBA culture (which would destroy the creative capacity of that industry, for good) away. That period ended. San Francisco joined the “paper belt” (to use Balaji Srinivasan’s term) cities of Boston, New York, Washington and Los Angeles. Venture capital became Hollywood for Ugly People. The Valley became a victim of its own success. Bay Area landlords made it big. Fail-outs from MBA-culture strongholds like McKinsey and Goldman Sachs found a less competitive arena in which they could boss nerds around with impunity; if you weren’t good enough to make MD at the bank, you went West to become a VC-funded Founder. The one group of people that didn’t win out in this new Valley order were software engineers. Housing costs went up far faster than their salaries, and they were gradually moved from being partners in innovation to being implementors’ of well-connected MBA-culture fail-outs’ shitty ideas. That’s where we are now. 

So what happened? Was it inevitable that the Valley’s new wealth would attract malefactors, or could this have been prevented? I actually think that it could have been stopped, knowing what we know now. Would it be possible to replicate the Valley’s success in another geographical area (or, perhaps, in a fully distributed technical subculture) without losing our status and autonomy once the money spotted it and came in? I think so, but it’ll take another article to explain both the theoretical reasons why we can hold advantage, and the practical strategies for keeping the game fair, and on our terms. That’s a large topic, and it goes far beyond what I intend to do in this article. 

The loss of status is a sad thing, because technology is our home turf. We understand computers and software and the mathematical underpinnings of those, and our MBA-culture colonizers don’t. We ought to have the advantage and retain high status, but fail at doing so. Why? There are two reasons, and they’re related to each other.

The first is that we lack “sheep dogs”. A sheep dog, in this sense, is a pugnacious and potentially vicious person who protects the good. A sheep dog drives away predators and protects the herd. Sheep dogs don’t start fights, but they end many– on their terms. Programmers don’t like to “get political”, and they dislike it even when their own kind become involved in office politics, and the result is that we don’t have many sheep dogs guarding us from the MBA-culture wolves. People who learn the skills necessary to protect the good, far too often, end up on the other side. 

The second is that we allow “passion” to be used against us. When we like our work, we let it be known. We work extremely hard. That has two negative side effects. The first is that we don’t like our work and put in a half-assed effort like everyone else, it shows. Executives generally have the political aplomb not to show whether they enjoy what they’re doing, except to people they trust with that bit of information. Programmers, on the other hand, make it too obvious how they feel about their work. This means the happy ones don’t get the raises and promotions they deserve (because they’re working so hard) because management sees no need to reward them, and that the unhappy ones stand out to aggressive management as potential “performance issues”. The second is that we allow this “passion” to be used against us. Not to be passionate is almost a crime, especially in startups. We’re not allowed to treat it as “just a job” and put forward above-normal effort only when given above-normal consideration. We’re not allowed to “get political” and protect ourselves, or protect others, because we’re supposed to be so damn “passionate” that we’d do this work for free. 

What most of us don’t realize is that this culture of mandatory “passion” lowers our social status, because it encourages us to work unreasonably hard and irrespective of conditions. The fastest way to lose social status is to show acceptance of low social status. For example, programmers often make the mistake of overworking when understaffed, and this is a terrible idea. (“Those execs don’t believe in us, so let’s show them up by… working overtime on something they own!”) To do this validates the low status of the group that allows it to be understaffed. 

Executives, a more savvy sort, lose passion when denied the advancement or consideration they feel they deserve. They’re not obnoxious about this attitude, but they don’t try to cover it up, either. They’re not going to give a real effort to a project or company that acts against their own interests or lowers their own social status. They won’t negotiate against themselves by being “passionate”, either. They want to be seen as supremely competent, but not sacrificial. That’s the difference between them and us. Executives are out for themselves and relatively open about the fact. Programmers, on the other hand, heroize some of the stupidest forms of self-sacrifice: the person who delivers a project (sacrificing weekends) anyway, after it was cancelled; or the person who moves to San Francisco without relocation because he “really believes in” a product that he can’t even describe coherently, and that he’ll end up owning 0.05% of. 

What executives understand, almost intuitively, is reciprocity. They give favors to earn favors, but avoid self-sacrifice. They won’t fall into “love of the craft” delusions when “the craft” doesn’t love them back. They’re not afraid to “get political”, because they realize that work is mostly politics. The only people who can afford to be apolitical or “above the fray”, after all, are the solid political winners. But until one is in that camp, one simply cannot afford to take that delusion on. 

If programmers want to be taken seriously, and we should be taken seriously and we certainly should want this, we’re going to have to take stock of our compromised position and fix it, even if that’s “getting political”. We’re going to have to stop glorifying pointless self-sacrifice for what is ultimately someone else’s business transaction, and start asserting ourselves and our values. 

In defense of defensibility

I won’t say when or where, but at one point in time, a colleague and I were discussing our “red button numbers” for the organization under which we toiled.

What’s that? The concept is this: a genie offers you the option to push a “red button” and, if you do so, your company will go bankrupt and cease to exist. Equity will be worthless, paychecks will bounce, and jobs will end. However, every employee in that company gets the same cash severance. (Let’s say $50,000.) The stipulation that every employee gets paid is important. I’m not interested in what some people might do if they had no moral scruples. Some people would blow up their employers for a $50,000 personal payoff, with everyone else getting nothing, but almost no one would admit this, or do it if it were to become known. If everyone gets paid, it pushing the red button becomes ethically acceptable. At $50,000 for a typical company? Hell yeah. Most employees would see their lives improve. The executives would be miserable, getting a pittance compared to their salaries, but… seriously, fuck ‘em. A majority of working people, if their company were scrapped and they were given a $50,000 check, would be dealt a huge favor by that circumstance.

The “everyone gets paid” red-button scenario is more interesting because it deals with what people will do in the open and consider ethically acceptable. When I get to more concrete matters of decisions people make that repair or dismantle companies, the interesting fact is that most of those decisions happen in the open. Companies are rarely sabotaged in secret, but disempowered and decomposed by their own people in plain view.

The “red button number” is the point at which a person would press the button, end the company, have every employee paid out that amount, and consider that an ethically acceptable thing to do. It’s safe to assume that almost everyone in the private sector has a red button number. For the idealists, and for the wealthy executives, that number might be very high: maybe $10 million. For most, it’s probably quite low. People who are about to be fired, and don’t expect a severance, might push the button at $1. Let’s assume that we could ask people for their red button numbers, and they’d answer honestly, and that this survey could be completed across a whole company. Take the median red button number and multiply it by the number of employees. That’s the company’s defensibility. We can’t actually measure this number directly, but it has a real-world meaning. If there were a vote on whether to dissolve the company and pay out some sum D, divided among all employees equally, the defensibility is the D* for which, if D > D*, the motion will pass and the company will be disbanded, and if D < D*, the company will persist. It’s the valuation the employees assign to the company (which is, often, a very different number from its market capitalization or private valuation).

Of course, such a vote would never actually happen. Companies don’t give employees that kind of power, and there’s an obvious reason why. Most companies, at least according to the stock market or private valuations, assigned values much greater than their defensibility. (This is not unreasonable or surprising, if a bit sad.) I can’t measure this to be sure, and I’d rather not pick on specific companies, so let me give three models and leave it to the reader to judge whether my assessments make sense.

Model Company A: a publicly-traded retail outlet with 100,000 employees, many earning less than $10/hour. I estimate the median “red button” number at $5,000, putting the defensibility at $500 million. A healthy valuation for such a company would be $125,000 per employee, or $12.5 billion. Defensibility is 4 cents on the dollar.

Model Company B: an established technology company with 20,000 employees. Software engineers earn six figures, and engineers-in-test and the like earn high-five-figure salaries. There’s a “cool” factor to working there. I’d estimate the median “red button” number at about 9 months of salary (and, for some of the most enthusiastic employees, it might be as much as five years, but at the median, it’s 6-9 months) or $75,000, putting the defensibility at $1.5 billion. A typical valuation for such a company would be $5 million per head, or $100 billion. Even though this is a company whose employees wouldn’t leave lightly, its defensibility is still only 1.5 cents on the dollar.

Model Company C: a funded startup, with 100 employees and a lot of investor and “tech press” attention. Many “true believers” among the employee ranks. Let’s assume that that, to get a typical employee to push the “red button”, we’d have to guarantee 6 months of pay ($50,000) and 250 percent of the fully-vested equity (0.04%) because so many employees really expect the stock to grow. The valuation of the company is $200 million (or $2 million per employee). We reach a defensibility of $250,000 per employee, or $25 million. That’s a lot, but it’s still only 12.5% of the valuation of the business.

None of these companies are, numerically speaking, very defensible. That is, if the company could be dissolved to anarchy with its value (as assessed by the market) distributed among the employees, they’d prefer it that way. Of course, a company need not be defensible at 100 cents on the dollar for its employees to wish it to remain in existence. If a $10 billion company were dissolved in such a way, there wouldn’t actually be $10 billion worth of cash to dish out. To the extent that companies can be synergistic (i.e. worth more than the sum of the component parts) it’s a reasonable assumption that a company whose defensibility was even 50 percent of its market capitalization would never experience voluntary dissolution, even if it were put to a vote.

In real life, of course, these “red button” scenarios don’t exist. Employees don’t get to vote on whether their companies continue existing, and, in practice, they’re usually the biggest losers in full-out corporate dissolution because they have far less time to prepare than the executives. The “red button number” and defensibility computations are an intellectual exercise, that’s all. Defensibility is what the company is worth to the employees. Given that defensibility numbers seem (under assumptions I consider reasonable) to consistently come in below the valuation of the company, we understand that companies would prefer that their persistence not come down to an employee vote.

That a typical company might have a defensibility of 5 cents on the dollar, to me, underscores the extreme imbalance of power between capital and labor. If the employees value the thing at X, and capital values it at 20*X, that seems to indicate that capital has 20 times as much power as the employees do. It signifies that companies aren’t really partnerships between capital and labor, but exist almost entirely for capital’s benefit.

Does defensibility matter? In this numerical sense, it’s a stretch to say that it does, because such votes can’t be called. Fifty-one percent of a company’s workers realizing that they’d let the damn thing die for six months’ severance has no effect, because they don’t have that power. If defensibility numbers were within a factor of 2, or even 4, of company’s market capitalizations, I’d say that these numbers (educated guesses only) tell us very little. It’s the sheer magnitude of the discrepancy under even the most liberal assumptions that is important.

People in organizations do “vote” on the value of the organization with what they do, day to day, ethically (and unethically) as they trade off between self-interest and the upkeep of the organization. How much immediate gain would a person forgo, in order to keep up the organization? The answer is: very little. I’d have no ethical qualms, with regard to most of the companies that I’ve worked at, in pressing the red button at $100,000. Most employees will be ecstatic, a few executives will be miserable; fair trade.

Thus far, I’ve only addressed public, ethical, fair behavior. Secretive and unethical behaviors also affect a company, obviously. However, I don’t see the latter as being as much of a threat. Organizations can defend themselves against the unethical, the bad actors, if the ethical people care enough to participate in the upkeep. It’s when ethical people (publicly and for just reasons) tune out that the organization is without hope.

The self-healing organization

Organizations usually rot as they age. It’s practically taken as a given, in the U.S. private sector, that companies will change for the worse as time progresses. Most of the startup fetishism of the SillyCon Valley is derived from this understanding: organizations will inexorably degrade invariably with age (a false assumption, and I’ll get to that) and the best way for a person (or for capital) to avoid toxicity is to hop from one upstart company to another, leaving as soon as the current habitat gets old.

It is true that most companies in the private sector degrade, and quite rapidly. Why is it so? What is it about organizations that turns them into ineffective, uninspiring messes over time? Are they innately pathological? Is this just a consequence of corporate entropy? Are people who make organizations better so much rarer than those who make them worse? I don’t think so. I don’t think it’s innate that organizations rot over time. I think that it’s common, but avoidable.

The root entropy-increasing cause of corporate decay is “selfishness”. I have to be careful with this word, because selfishness can be a virtue. I certainly don’t intend to impose any value, good or bad, on that concept here. Nor do I imply secrecy or subterfuge. Shared selfishness can be a group experience. Disaffected employees can slack together and protect each other. Often this happens. One might argue that it becomes “groupishness” or localism. I don’t care to debate that point right now.

Organizations decay because people and groups within them, incrementally, prefer their interests over that of the institution. If offered promotions they don’t deserve, into management roles that will harm morale, people usually take them. If they can get away with slacking– either to take a break, or to put energy into efforts more coherent with their career goals– they will. (The natural hard workers will continue putting forth effort, but only on the projects in line with their own career objectives.) If failures of process advantage them, they usually let those continue. When the organization acts against their interests, they usually make it hurt, causing the company to recoil and develop scar tissue over time. They protect themselves and those who have protected them, regardless of whether their allies possess “merit” as defined by the organization. These behaviors aren’t exactly crimes. Most of this “selfishness” is stuff that morally average people (and, I would argue, even many morally good ones) will do. Most people, if they found a wallet with $1,000 and a driver’s license in it, would take it to the police station for return to its owner. However, if they were promoted based on someone else’s work, and that “someone else” had left the company so there was no harm in keeping the promotion, they’d keep the arrangement as-is. I’m not different from the average person on this; I’ll just admit to it, in the open.

People in the moral mid-range will generally try to do the right thing. On the “red button” issue, most wouldn’t tank their own companies for a personal payout, leaving all their colleagues screwed. Most would press the button in the “every employee gets paid” scenario, because it’s neither ethically indefensible nor socially unacceptable to do so. Such people are in the majority and not inherently corrosive to institutions– they uphold those that are good to them and their colleagues, and bad to those that harm them. However, they hasten the decay of those organizations that clearly don’t deserve concern.

Let’s talk about entropy, or the increasing tendency toward disorder in a closed system. Life can only persist because certain genetic configurations enable an organism, taking in external energy, to preserve local low-entropy conditions. A lifeless human body, left on the beach to be beaten by the waves, will be unrecognizable within a couple of days. That’s entropy. A living human can sit in the same conditions with minimal damage. In truth, what we seem to recognize as life is “that which can preserve its own local order”. Living organisms are constantly undergoing self-repair. Cells are destroyed by the millions every hour, but new ones are created. Dead organisms are those that have lost the ability to self-repair. The resources in them will be recycled by self-interested and usually “lower” (less complex) organisms that feed on them as they decay.

Organizations, I think, can be characterized as “living” or “dead”, to some degree, based on whether their capacity for self-repair exceeds the inevitable “wear and tear” that will be inflicted by the morally acceptable but still entropy-increasing favoritism that its people show for their own interests. The life/death metaphor is strained, a bit, by the difficulty in ascertaining which is which. In biology, it’s usually quite clear whether an organism is alive. We know that when the human heart stops, the death process is likely to begin (and, absent medical intervention, invariably will begin) within 5 minutes and, after 10 minutes, the brain will typically be unrecoverable and that person will no longer exist in the material world. Life versus death isn’t completely binary in biology, but it’s close enough. With organizations, it’s far less clear whether the thing is winning or losing its ongoing fight against entropy. To answer that question involves debate and research that, because the questions asked are socially unacceptable, can rarely be performed properly.

“Red button” scenarios don’t happen, but every day, people make small decisions that influence the life/death direction of the company. Individually, most of these decisions don’t matter for much. A company isn’t going to fail because a disaffected low-level employee spent his whole morning searching for other work. If everyone’s looking for another job, that’s a problem.

In the VC-funded world, self-repair has been written off as a lost cause. It’s not even attempted. The mythology is that everything old (“legacy”) is of low value (and should be sold to some even older, more defective company) and that only the new is worth attention. Don’t try to repair old organizations; just create new shit and make old mistakes. It’s all about new programmers (under 30 only, please) and new languages (often recycling ideas from Lisp and Haskell, implementing them poorly) and new companies. This leads to massive waste as nothing is learned from history. It becomes r-selective in the extreme, with the hope that, despite frequent death of the individual organisms, there can be a long-lived clonal colony for… someone. For whom? To whose benefit is this clonal organism? It’s for the well-connected scumbags who can peddle influence and power no matter which companies beat the odds and thrive for a few days, and which ones die.

In the long run, I don’t think this is going to work. Building indefensible companies in large numbers is not going to create a defensible meta-organism. To do so is to create a con (admittedly, a somewhat brilliant and unprecedented one, a truly postmodern corporate organism) in which enthusiastic young people trade their energy and ardor for equity is mostly-worthless companies, and whose macroscopic portfolio performance is mediocre (as seen in the pathetic returns VC offers for passive investors) but which affords great riches for those with the social connections (and lack of moral compass) necessary to navigate it. It works for a while, and then people figure out what’s going on, and it doesn’t. We call this a “bubble/crash” cycle, but what it really is, at least in this case, is an artifact of limitations on human stupidity. People will only fall for a con for so long. The naive (like most 22-year-olds when they’re just starting the Valley game) get wiser, and the perpetual suckers have short attention spans and will be drawn to something shinier.

Open allocation

What might a defensible company look like? Here I come to one my pet issues: open allocation. The drawback of open allocation, terrifying to the hardened manageosaur, is that it requires the organization be defensible in order to work, because it gives employees a real vote on what the company does. The good news is that open allocation tends naturally toward defensibility. If the organization is fair to its employees and its people are of average or better moral quality, then the majority of them can be trusted to work within the intersection between their career interests and the needs of the company.

Why does open allocation work so well? Processes, projects,  patterns, protections, personality interactions, policies and power relationships (henceforth, “the Ps”) are all subjected to a decentralized immune system, rather than a centralized one that doesn’t have the bandwidth to do the job properly. Organizations of all kinds produce the Ps on a rather constant basis. When one person declines to use the bathroom because his manager just went in, that microdeference is P-generation. The same applies to the microaggression (or, to be flat about it, poorly veiled aggression) of a manager asking for an estimate. Requesting an estimate generates several Ps at once: a power relationship (to be asked for an estimate is to be made someone’s bitch), a trait of a project (it’s expected at a certain time, quality be damned), and a general pattern (managerial aggression, in the superficial interest of timeliness, is acceptable in that organization). Good actions also generate Ps. To empower people generates power relationships of a good kind, and affords protection. Even the most superficial interactions generate Ps, good and bad.

Under open allocation, the bad Ps are just swept away. When one person tries to dominate the other through unreasonable requests, or to socially isolate a person in order to gain power, the other has the ability to say, “fuck that shit” and walk away. The doomed, career-ending projects and the useless processes and the toxic power relationships just disappear. People will try to make such things, even in the best of companies, and sometimes without ill intent. Under open allocation, however, bad Ps just don’t stick around. People have the power to nonexist them. What this means is that, over time, the long-lived Ps are the beneficial ones. You have a self-healing organization. This generates good will, and people begin to develop a genuine civic pride in the organization, and they’ll participate in its upkeep. Open allocation may not be the only requisite ingredient for success on the external market, but it does insure an organization against internal decay.

Then there’s the morale issue, and the plain question of employee incentives. Employees of open allocation companies know that if their firms dissolve tomorrow, they’re going to end up in crappy closed-allocation companies afterward. They actually care– beyond “will I get a severance?”– whether their organizations live or die. In closed-allocation companies, the only way to get a person to care in this way is either (a) to give him a promotion that another organization would not (which risks elevating an incompetent) or (b) to pay him a sizable wage differential over the prevailing market wage– this leads to exponential wage growth, typically at a rate of 20 to 30 percent per year, and can be good for the employee but isn’t ideal for the employer. Because closed-allocation companies are typically also stingy, (a) is preferred, which means that loyalists are promoted regardless of competence. One can guess where that leads: straight to idiocy.

Under closed allocation, bad Ps tend to be longer-lived than good ones. Why? Something I’ve realized in business is that good ideas fly away from their originators and become universal, which means they can be debated on their actual merits and appropriateness to the situation (a good idea is not necessarily right for all situations). Two hundred years from now, if open allocation is the norm, it’s quite likely that no one will remember my name in connection to it. (To be fair, I only named it, I didn’t invent the concept.) Who invented the alphabet? We don’t know, because it was a genuinely good idea. Who invented sex? No one. On the other hand, bad ideas become intensely personal– loyalty tests, even. Stack ranking becomes “a Tom thing” (“Tom” is a made-up name for a PHP CEO) because it’s so toxic that it can only be defended by an appeal to authority or charisma, and yet to publicly oppose it is to question Tom’s leadership (and face immediate reprisal, if not termination). In a closed-allocation company, bad ideas don’t get flushed out of the system. People double down on them. (“You just can’t see that I’m right, but you will.”) They become personal pet projects of executives and “quirky” processes that no one can question. Closed allocation companies simply have no way to rid themselves of bad Ps– at least, not without generating new ones. Even firing the most toxic executive (and flushing his Ps with him) is going to upset some people, and the hand-over of power is going to result in P-generation that is usually in-kind. Most companies, when they fire someone, opt for the cheapest and most humiliating kind of exit– crappy or nonexistent severance, no right to represent oneself as employed during the search, negative (and lawsuit-worthy) things said about the departing employee– and that usually makes the morale situation worse. (If you disparage a fired and disliked executive, you still undermine faith in your judgment; why’d you hire him in the first place?) No matter how toxic the person is, you can’t fire someone in that way without generating more toxicity. People talk, and even the disaffected and rejected have power, when morale is factored in. The end result of all this is that bad Ps can’t really be removed from the system without generating a new set of bad Ps.

I can’t speak outside of technology because to do so is to stretch beyond my expertise. However, a technology company cannot have closed allocation and retain its capacity for self-repair. It will generate bad Ps faster than good ones.

What about… ?

There’s a certain recent, well-publicized HR fuckup that occurred at a well-known company using open allocation. I don’t want to comment at length about this. It wouldn’t have been newsworthy were it not for the high moral standard that company set for itself in its public commitment to open allocation. (If the same had happened at a closed-allocation oil company or bank, no one would have ever heard a word about it.) Yes, it proved that open allocation is not a panacea. It proved that open allocation companies develop political problems. This is not damning of open allocation, because closed allocation creates much worse problems. More damningly, the closed allocation company can’t heal.

Self-healing and defensibility are of key importance. All organizations experience stress, no exceptions. Some heal, and most don’t. The important matter is not whether political errors and HR fuckups happen– because they will– but whether the company is defensible enough that self-repair is possible.

The complexity factor

The increase of entropy in an organization is a hard-to-measure process. We don’t see most of those “Ps” as they are generated, and moral decay is a subjective notion. What seems to be agreed-upon is that, objectively, complexity grows as the organization does, and that this becomes undesirable after a certain point. Some call it “bureaucratic red tape” and note it for its inefficiency. Others complain about the political corruption that emerges from relationship-based complexity. For a variety of reasons, an organization gets into a state where it is too complicated and self-hogtied to function well. It becomes pathological. Not only that, but the complexity that exists becomes so onerous that the only way to navigate it is to create more complexity. There are committees to oversee the committees.

Why does this happen? No one sets out “to generate complexity”. Instead, people in organizations use what power they have (and even low-level grunts can have major effects on morale) to create conditional complexity. They make decisions that favor their interests simple and those that oppose them complicated enough that no one wants to think them through; instead, those branches of the game tree are just pruned. That’s how savvy people play politics. If a seasoned office politicker wants X and not Y, he’s not going to say, “If you vote for Y, I’ll cut you.” He can’t do it that way. In fact, it’s best if no one knows what his true preference is (so he doesn’t owe any favors to X voters). Nor can he obviously make Y unpleasant or bribe people into X. What he can do is create a situation (preferably over a cause seemingly unrelated to the X/Y debate) that makes X simple and obvious, but Y complex and unpredictable. That’s the nature of conditional complexity. It generates ugliness and mess– if the person’s interests are opposed.

In the long run, this goes bad because an organization will inevitably get to a point where it can’t do anything without opposing some interest, and then all of those conditional complexities (which might be “legacy” policies, set in place long ago and whose original stakeholders may have moved on) are triggered. Things become complex and “bureaucratic” and inefficient quickly, and no one really knows why.

The long-term solution to this complexity-burden problem is selective abandonment. If there isn’t a good reason to maintain a certain bit of complexity, that bit is abandoned. For example, it’s illegal, in some towns, to sing in the shower. Are those laws enforced? Never, because there’s no point in doing so. The question of what is the best way to “garbage collect” junk complexity is one I won’t answer in a single essay, but in technology companies, open allocation provides an excellent solution. Projects and power relationships and policies (again, the Ps) that no longer make sense are thrown out entirely.

The best defense is to be defensible

The low defensibility of the typical private-sector organization is, to me, quite alarming. Rational and morally average (or even morally above average) don’t value their employers very much, because most companies don’t deserve to be valued. They’re not defensible, which means they’re not defended, which is another way of saying self-repair doesn’t happen and that organizational decay is inexorable.

Technology’s Loser Problem

I’m angry. The full back story isn’t worth getting into, but there was a company where I applied for a job in the spring of 2013: to build a company’s machine learning infrastructure from scratch. It was a position of technical leadership (Director equivalent, but writing code with no reports) and I would have been able to use Clojure. As it were, I didn’t get it. They were looking for someone more experienced, who’d built those kinds of systems before, and wouldn’t take 6 months to train up to the job. That, itself, is not worth getting angry about. Being turned down happens, especially at high levels.

I found out, just now, that the position was not filled. Not then. Not 6 months later. Not to this day, more than a year later. It has taken them longer to fill the role than it would have taken for me to grow into it.

When they turned me down, it didn’t faze me. I thought they’d found a better candidate. That happens; only thing I can do is make myself better. I found myself, however, a bit irked when I found out that they hadn’t filled the position for longer than it would have taken me to gain the necessary experience. I lost, and so did they.

That’s not what makes me angry. Rationally, I realize that most companies aren’t going to call back a pretty-good candidate they rejected because they had just opened the position and they thought they could do better (if you’re the first 37% of candidates for a job, it makes sense for them not to choose you and, empirically, first and second applicants for a high-level position rarely get it). That’s the sort of potentially beneficial but extremely awkward social process that just won’t happen. What makes me angry is the realization of how common a certain sort of decision is in the technology world. We make a lot of lose-lose decisions that hurt all of us. Extremely specific hiring requirements (that, in bulk, cost the company more in waiting time than training a 90% match up to the role) are just the tip of the iceberg.

You know those people who complain about the lack of decent <gender of sexual interest> but (a) reject people for the shallowest, stupidest reasons, (b) aren’t much of a prize and don’t work to better themselves, and (c) generally refuse to acknowledge that the problem is rooted in their own inflated perception of their market value? That’s how I feel every time I hear some corporate asswipe complain about a “talent shortage” in technology. No, there isn’t one. You’re either too stingy or too picky or completely inept at recruiting, because there’s a ton of underemployed talent out there.

Few of us, as programmers, call the initial shots. We’ve done a poor job of making The Business listen to us. However, when we do have power, we tend to fuck it up. One of the problems is that we over-comply with what The Business tells us it whats. For example, when a nontechnical CEO says, “I only want you to hire absolute rock stars”, what he actually means is, “Don’t hire an idiot just to have a warm body or plug a hole”. However, because they tend to be literal, over-compliant, and suboptimal, programmers will interpret that to mean, “Reject any candidate who isn’t 3 standard deviations above the mean.” The leads to positions not being filled, because The Business is rarely willing to pay what one standard deviation above the mean costs, let alone three.

Both sides now

I’ve been on both sides of the interviewing and hiring process. I’ve seen programmers’ code samples described with the most vicious language over the most trivial mistakes, or even stylistic differences. I’ve seen job candidates rejected for the most god-awful stupid reasons. In one case, the interviewer clearly screwed up (he misstated the problem in a way that made it impossible) but, refusing to risk face by admitting the problem was on his end, he claimed the candidate failed the question. Another was dinged on a back-channel reference (don’t get me started on that sleazy practice, which ought to be illegal) claiming, without any evidence, that “he didn’t do much” on a notable project four years ago. I once saw an intern denied a full-time offer because he lived in an unstylish neighborhood. (The justification was that one had to be “hungry”, mandating Manhattan.) Many of us programmers are so butthurt about not being allowed to sit at the cool kids’ table that, when given the petty power associated with interviewing other programmers, the bitch-claws come out in a major way.

Having been involved in interviewing and recruiting, I’ll concur that there are a significant number of untalented applicants. If it’s 99.5 percent, you’re doing a lot of things wrong, but most resumes do come from people way out of their depth. Moreover, as with dating, there’s an adverse weighting in play. Most people aren’t broken, but broken people go on orders of magnitude more dates than everyone else, which is why most peoples’ dating histories have a disproportionate representation of horror stories, losers, and weirdos. It’s the same with hiring, but phone screening should filter against that. If you’re at all good at it, about half of the people brought in-office will be solid candidates.

Of course, each requirement cuts down the pool. Plenty of companies (in finance, some officially) have a “no job hopper” or “no unemployeds” rule. Many mandate high levels of experience in new technologies (even though learning new technologies is what we’re good at). Then, there are those who are hung up on reference checking in weird and creepy ways. I know of one person who proudly admits that his reference checking protocol is to cold-call a random person (again, back-channel) is the candidate’s network and ask the question, without context, “Who is the best person you’ve ever worked with?” If anyone other than the candidate is named, the candidate is rejected. That’s not being selective. That’s being an invasive, narcissistic idiot. Since each requirement reduces the size of qualified people, it doesn’t take long before the prejudices winnow an applicant pool down to zero.

Programmers? Let’s be real here, we kinda suck…

As programmers, we’re not very well-respected, and when we’re finally paid moderately well, we let useless business executives (who work 10-to-3 and think HashMap is a pot-finding app) claim that “programmer salaries are ridiculous”. (Not so.) Sometimes (to my horror) you’ll hear a programmer even agree that our salaries are “ridiculous”. Fuck that bullshit; it’s factually untrue. The Business is, in general, pretty horrible to us. We suffer under closed allocation, deal with arbitrary deadlines, and if we don’t answer to an idiot, we usually answer to someone else who does. Where does the low status of programmers come from? Why are we treated as cost centers instead of partners in the business? Honestly… much of the problem is us. We’ve failed to manage The Business, and the result is that it takes ownership of us.

Most of the time, when a group of people is disproportionately successful, the cause isn’t any superiority of the average individual, but a trait of the group: they help each other out. People tend to call these formations “<X> Mafia” where X might be an ethnicity, a school, or a company. Y Combinator is an explicit, pre-planned attempt to create a similar network; time will tell if it succeeds. True professions have it. Doctors look out for the profession. With programmers, we don’t see this. There isn’t a collective spirit: just long email flamewars about tabs versus spaces. We don’t look out for each other. We beat each other down. We sell each other out to non-technical management (outsiders) for a shockingly low bounty, or for no reason at all.

In many investment banks, there’s an established status hierarchy in which traders and soft-skills operators (“true bankers”) are at the top, quants are in the middle, and programmers (non-quant programmers are called “IT”) are even lower. I asked a high-ranking quant why it was this way, and he explained it in terms of the “360 degree” performance reviews. Bankers and traders all gave each other top ratings, and wrote glowing feedback for minor favors. They were savvy enough to figure out that it was best for them to give great reviews up, down, and sideways, regardless of their actual opinions. Quants tended to give above-average ratings and occasionally wrote positive feedback. IT gave average ratings for average work and plenty of negative feedback. The programmers were being the most honest, but hurting each other in the process. The bankers and traders were being political, and that’s a good thing. They were savvy enough to know that it didn’t benefit them to sell each other out to HR and upper management. Instead, they arranged it so they all got good ratings and the business had to, at baseline, appreciate and reward all of them. While it might seem that this hurt top performers, it had the opposite effect. If everyone got a 50 percent bonus and 20% raise, management had to give the top people (and, in trading, it’s pretty obvious who those are) even more.

Management loves to turn high performers against the weak, because this enables management to be stingy on both sides. The low performers are fired (they’re never mentored or reassigned) and the high performers can be paid a pittance and still have a huge bonus in relative terms (not being fired vs. being fired). What the bankers were smart enough to realize (and programmers, in general, are not) is that performance is highly context-driven. Put eight people of exactly equal ability on a team to do a task and there will be one leader, two or three contributors, and the rest will be marginal or stragglers. It’s just more efficient to have the key knowledge in a small number of heads. Open source projects work this way. What this means is that, even if you have excellent people and no bad hires, you’ll probably have some who end up with not much to show for their time (which is why open allocation is superior; they can reassign themselves until they end up in a high-impact role). If management can see who is in what role, it can fire the stragglers and under-reward the key players (who, because they’re already high performers, are probably motivated by things other than money… at least, for now). The bankers and traders (and, to a lesser extent, the quants) had the social savvy and sense to realize that it was best that upper management not know exactly who was doing what. They protected each other, and it worked for them. The programmers, on the other hand, did not, and this hurt top performers as well as those on the bottom.

Let’s say that an investment bank tried to impose tech-company stack ranking on its employees, associate level and higher. (Analyst programs are another matter, not to be discussed here.) Realizing the mutual benefit in protecting each other, the bankers would find a way to sabotage the process by giving everyone top ratings, ranking the worst employees highly, or simply refusing to do the paperwork. And good for them! Far from being unethical, this is what they should do: collectively work The Business to get what they’re actually worth. Only a programmer would be clueless enough to go along with that nonsense.

In my more pessimistic moods, I tend to think that we, as programmers, deserve our low status and subordinacy. As much as we love to hate those “business douchebags” there’s one thing I will say for them. They tend to help each other out a lot more than we do. Why is this? Because they’re more political and, again, that might not be a bad thing. Ask a programmer to rate the performance of a completely average colleague and you’ll get an honest answer: he was mediocre, we could have done without him. These are factual statements about average workers, but devastating when put into words. Ask a product manager or an executive about an average colleague and you’ll hear nothing but praise: he was indispensable, a world-class player, best hire in ten years. They realize that it’s politically better for them, individually and as a group, to keep their real opinions to themselves and never say anything that could remotely endanger another’s career. Even if that person’s performance was only average, why make an enemy when one can make a friend?

“Bad code”

Let’s get to another thing that we do, as programmers, that really keeps us down. We bash the shit out of each other’s code and technical decision-making, often over minutiae.

I hate bad code. I really do. I’ve seen plenty of it. (I’ve written some, but I won’t talk about that.) I understand why programmers complain about each other’s code. Everyone seems to have an independent (and poorly documented) in-head culture that informs how he or she writes code, and reading another person’s induces a certain “culture shock”. Even good code can be difficult to read, especially under time pressure. And yes, most large codebases have a lot of code in them that’s truly shitty, sometimes to the point of being nearly impossible to reason about. Businesses have failed because of code quality problems, although (to tell the whole story) it’s rare that one bad programmer can do that much damage. The worst software out there isn’t the result of one inept author, but the result of code having too many authors, often over years. It doesn’t help that most companies assign maintenance work to either to junior programmers, or demoted (and disengaged) senior ones, neither category having the power to do it right.

I’d be the last one to come out and defend bad code. That said, I think we spend too much time complaining about each other’s code– and, worse yet, we tend toward the unforgivable sin of complaining to the wrong people. A technical manager has, at least, the experience and perspective to know that, at some level, every programmer hates other peoples’ code. But if that programmer snitches to a non-technical manager and executive,  well… you’ve just invited a 5-year-old with a gun to the party. Someone might get fired because “tabs versus spaces” went telephone-game into “Tom does shoddy work and is going to destroy the business”. Because executives are politically savvy enough to protect the group, and only sell each other out in extreme circumstances, what started out as a stylistic disagreement sounds (to the executive ear) like Tom (who used his girlfriend’s computer to fix a production problem at 11:45 on a Friday night, the tabs/spaces issue being for want of an .emacs.d) is deliberately destroying the codebase and putting the whole company at risk.

As programmers, we sell each other out all the time. If we want to advance beyond reasonable but merely upper-working class salaries, and be more respected by The Business, we have to be more careful about this kind of shit. I’ve heard a great number of software engineers say things like, “Half of all programmers should just be fired.” Now, I’ll readily agree that there are a lot of badly-trained programmers out there whose lack of skill causes a lot of pain. But I’m old enough to know that people come to a specific point from a multitude of paths and that it’s not useful to personalize this sort of thing. Also, regardless of what we may think as individuals, almost no doctor or banker would ever say, to someone outside his profession, “half of us should be fired”. They’re savvy enough to realize the value of protecting the group, and handling competence and disciplinary matters internally. Whether to fire, censure, mentor or praise is too important a decision to let it happen outside of our walls.

There are two observations about low-quality code, one minor and one major. The minor one is that code has a “all of us is worse than any of us” dynamic. As more hands pass over code, it tends to get worse. People hack the code needing specific features, never tending to the slow growth of complexity, and the program evolves over time into something that nobody understands because too many people were involved in it. Most software systems fall to pieces not because of incompetent individuals, but because of unmanaged growth of complexity. The major point on code-quality is: it’s almost always management’s fault.

Bad code comes from a multitude of causes, only one of which is low skill in programmers. Others include unreasonable deadlines, unwillingness to attack technical debt (a poor metaphor, because the interest rate on technical “debt” is both usurious and unpredictable), bad architecture and tooling choices, and poor matching of programmers to projects. Being stingy, management wants to hire the cheapest people it can find and give them the least time possible in which to do the work. That produces a lot of awful code, even if the individual programmers are capable. Most of the things that would improve code quality (and, in the long term, the health and performance of the business) are things that management won’t let the programmers have: more competitive salaries, more autonomy, longer timeframes, time for refactoring. The only thing that management and the engineers can agree on is firing (or demoting, because their work is often still in use and The Business needs someone who understands it) those who wrote bad code in the past.

One thing I’ve noticed is that technology companies do a horrible job of internal promotion. Why is that? Because launching anything will typically involve compromises with the business on timeframe and headcount, resulting in bad code. Any internal candidate for a promotion has left too many angles for attack. Somewhere out there, someone dislikes a line of code he wrote (or, if he’s a manager, something about a project he oversaw). Unsullied external candidates win, because no one can say anything bad about them. Hence, programming has the culture of mandatory (but, still, somewhat stigmatized) job hopping we know and love.

What’s really at the heart of angry programmers and their raging against all that low-quality code? Dishonest attribution. The programmer can’t do shit about the dickhead executive who set the unreasonable deadlines, or the penny-pinching asswipe managers who wouldn’t allow enough salary to hire anyone good. Nor can he do much about the product managers or “architects” who sit above and make his life hell on a daily basis. But he can attack Tom, his same-rank colleague, over that commit that really should have been split into two. Because they’re socially unskilled and will generally gleefully swallow whatever ration of shit is fed to them by management, most programmers can very easily be made to blame each other for “bad code” before blaming the management that required them to use the bad code in the first place.

Losers

As a group, software engineers are losers. In this usage, I’m not using the MacLeod definition (which is more nuanced) and my usage is halfway pejorative. I generally dislike calling someone a loser, because the pejorative, colloquial meaning of that word conflates unfortunate circumstance (one who loses) with deserved failure. Here, however, it applies. Why do we lose? Because we play against each other, instead of working together to beat the outside world. As a group, we create our own source of loss.

Often, we engage in zero- or negative-sum plays just to beat the other guy. It’s stupid. It’s why we can’t have nice things. We slug each other in the office and wonder why external hires get placed over us. We get into flamewars about minutiae of programming languages, spread FUD, and eventually some snot-nosed dipshit gets the “brilliant” idea to invite nontechnical management to weigh in. The end result is that The Business comes in, mushroom stamps all participants, and says, “Everything has to be Java“.

Part of the problem is that we’re too honest, and we impute honesty in others when it isn’t there. We actually believe in the corporate meritocracy. When executives claim that “low performers” are more of a threat to the company than their astronomical, undeserved salaries and their doomed-from-the-start pet projects, programmers are the only people stupid enough to believe them, and will often gleefully implement those “performance-based” witch hunts that bankers would be smart enough to evade (by looking for better jobs, and arranging for axes to fall on people planning exits anyway). Programmers attempt to be apolitical, but that ends up being very political, because the stance of not getting political means that one accepts the status quo. That’s radically conservative, whether one admits it or not.

Of course, the bankers and traders realize the necessity of appearing to speak from a stance of professional apolitical-ness. Every corporation claims itself to be an apolitical meritocracy, and it’s not socially acceptable to admit otherwise. Only a software engineer would believe in that nonsense. Programmers hear “Tom’s not delivering” or “Andrea’s not a team player” and conceive of it as an objective fact, failing to recognize that, 99% of the time, it means absolutely nothing more or less than “I don’t like that person”.

Because we’re so easily swayed, misled, and divided, The Business can very easily take advantage of us. So, of course, it does. It knows that we’ll sell each other out for even a chance at a seat at the table. I know a software engineer who committed felony perjury against his colleagues just to get a middle-management position and the right to sit in on a couple of investor meetings. Given that this is how little we respect each other, ourselves, and our work, is it any wonder that software engineers have such low status?

Our gender issues

I’m going to talk, just briefly, about our issues with women. Whatever the ultimate cause of our lack of gender diversity– possibly sexism, possibly that the career ain’t so great– it’s a major indictment of us. My best guess? I think sexism is a part of it, but I think that most of it is general hostility. Women often enter programming and find their colleagues hostile, arrogant, and condescending. They attribute that to their gender, and I’m sure that it’s a small factor, but men experience all of that nonsense as well. To call it “professional hazing” would be too kind. There’s often nothing professional about it. I’ve dealt with rotten personalities, fanaticism about technical preference or style, and condescension and, honestly, don’t think there’s a programmer out there who hasn’t. When you get into private-sector technology, one of the first things you learn is that it’s full of assholes, especially at higher levels.

Women who are brave enough to get into this unfriendly industry take a look and, I would argue, most decide that it’s not worth it to put up with the bullshit. Law and medicine offer higher pay and status, more job security, fewer obnoxious colleagues, and enough professional structure in place that the guy who cracks rape jokes at work isn’t retained just because he’s a “rockstar ninja”.

“I thought we were the good guys?”

I’ve often written from a perspective that makes me seem pro-tech. Originally, I approached the satirical MacLeod pyramid with the belief that “Technocrat” should be used to distinguish positive high-performers (apart from Sociopaths). I’ve talked about how we are a colonized people, as technologists. It might seem that I’m making businesspeople out to be “the bad guys” and treating programmers as “the good guys”. Often, I’m biased in that very direction. But I also have to be objective. There are good business people out there, obviously. (They’re just rare in Silicon Valley, and I’ll get to that.) Likewise, software engineers aren’t all great people, either. I don’t think either “tribe” has a monopoly on moral superiority. As in Lost, “we’re the good guys” doesn’t mean much.

We do get the worst (in terms of ethics and competence) of the management/business tribe in the startup world. That’s been discussed at length, in the essay linked above. The people who run Silicon Valley aren’t technologists or “nerds” but machiavellian businessmen who’ve swooped in to the Valley to take advantage of said nerds. The appeal of the Valley, for the venture capitalists and non-technical bro executives who run it, isn’t technology or the creation of value, but the unparalleled opportunity to take advantage of too-smart, earnest hard workers (often foreign) who are so competent technically that they often unintentionally generate value, but don’t know the first thing about how to fight for their own interests.

It’s easy to think ourselves morally superior, just because the specific subset of business people who end up in our game tends to be the worst of that crowd. It’s also a trap. We have a lot to learn form the traders and bankers of the world about how to defend ourselves politically, how to stand a chance of capturing some of the value we create, and how to prevent ourselves from being robbed blind by people who may have lower IQs, but have been hacking humans for longer than we could have possibly been using computers. Besides, we’re not all good. Many of us aren’t much better than our non-technical overlords. Plenty of software engineers would gladly join the bad guys if invited to their table. The Valley is full of turncoat software engineers who don’t give a shit about the greater mission of technology (using knowledge to make peoples’ lives better) and who’d gladly sell their colleagues out to cost-cutting assholes in management.

Then there are the losers. Losers aren’t “the bad guys”. They don’t have the focus or originality that would enable them to pull off anything complicated. Their preferred sin is typically sloth. They’ll fail you when you need them the most, and that ‘s what makes them infuriating. They just want to put their heads down and work, and the problem is that they can’t be trusted to “get political” when that’s exactly what’s needed. The danger of losers is in numbers. The problem is that so many software engineers are clueless, willing losers who’ll gladly let political operators take everything from them.

When you’re young and don’t know any better, one of the appeals of software engineering is that it appears, superficially, to tolerate people of low social ability. To people used to artificial competition against their peers, this seems like an attractive trait of the industry; it’s not full of those “smooth assholes” and “alpha jocks”. After several years observing various industries, I’ve come to the conclusion that this attitude is not merely misguided, but counterproductive. You want socially skilled colleagues. Being the biggest fish in a small pond just means that there are no big fish to protect you when the sharks come in. Most of those “alpha jocks” aren’t assholes or idiots (talk to them, nerds; you’ll be surprised) and, when The Business comes in and is looking for a fight, it’s always best to have strong colleagues who’ve got your back.

Here’s an alternate, and quite possible hypothesis: maybe The Business isn’t actually full of bad guys. One thing that I’ve realized is that people tend to push blame upward. For example, the reputation of venture capitalists has been harmed by founders blaming “the VCs” for their own greed and mismanagement. It gives the grunt workers an external enemy, and the clueless can be tricked into working harder than they should (“they don’t really like us and haven’t given us much, but if we kill it on this project and prove them wrong, maybe they’ll change their minds!”). It actually often seems that most of the awfulness of the software industry doesn’t come directly from The Business, but from turncoat engineers (and ex-engineers) trying to impress The Business. In the same way that young gang members are more prone to violence than elder dons, the most creative forms of evil seem to come from ex-programmers who’ve changed their colors.

The common enemy

So long as software engineers can easily be divided against each other on trivial matters like tabs versus spaces and scrotum versus kanban, we’ll never get the respect (and, more importantly, the compensation) that we’re due. These issues distract us from what we really need to do, which is figure out how to work The Business. Clawing at each other, each trying to become the favored harem queen of the capitalist, is suboptimal compared to the higher goal of getting out of the harem.

I’ve spoken of “The Business” as if it were a faceless, malevolent entity. It might sound like I’m anti-business, and I’m not. Business is just a kind of process. Good people, and bad people, start businesses and some add great value to the world. The enemy isn’t private enterprise itself, but the short-term thinking and harem-queen politics of the established corporation. Business organizations get to a point where they cease having a real reason to exist, and all that’s left is the degenerate social contest for high-ranking positions. We, as programmers, seem to lack the skill to prevent that style of closed-allocation degeneracy from happening. In fact, we seem to unintentionally encourage it.

The evil isn’t that software is a business, but that technical excellence has long since been subordinated entirely to the effectively random emotional ups and downs of non-technical executives who lack the ability to evaluate our work. It’s that our weird ideology of “never get political” is actually intensely political and renders us easy to abuse. Business naturally seems to be at risk of anti-intellectual tendencies and, rather than fight back against this process, we’ve amplified it just to enjoy the illusion of being on the inside, among the “cool kids”, part of The Business. Not only does our lack of will to fight for our own interests leave us at the mercy of more skilled business operators, but it attracts an especially bad kind of them. Most business people, actually, aren’t the sorts of corporate assholes we’re used to seeing run companies. It’s just that our lack of social skill appeals to the worst of that set: people who come in to technology to take advantage of all the clueless, loser nerds who won’t fight for themselves. If we forced ourselves to be more discerning judges of character, and started focusing on ethics and creativity instead of fucking tabs-versus-spaces, we might attract a better sort of business person, and have an industry where stack ranking and bastardized-”Agile” micromanagement aren’t even considered.

If we want to improve our situation, we have to do the “unthinkable” (which is, as I’ve argued, actually quite thinkable). We have to get political.

What’s a mid-career software engineer actually worth? Try $779,000 per year as a lower bound.

Currently, people who either have bad intentions or a lack of knowledge are claiming that software engineer salaries are “ridiculous”. Now, I’ll readily admit that programmers are, relative the general population, quite well paid. I’m not about to complain about the money I make; I’m doing quite well, in a time and society where many people aren’t. The software industry has many problems, but low pay for engineers (at least, for junior and mid-career engineers; senior engineers are underpaid but that’s an issue for another time) doesn’t crack the top 5. Software engineers are underpaid, relative to the massive amount of value (if given proper projects, rather than mismanaged as is typical) they are capable of delivering. In comparison to the rest of the society, they do quite well.

So what should a software engineer be paid? There’s a wide disparity in skill level, so it’s hard to say. I’m going to focus on a competent, mid-career engineer. This is someone between 5 and 10 years of experience, with continual investment in skill, and probably around 1.6 on this software engineering scale. He’s not a hack or the stereotypical “5:01″ programmer who stopped learning new skills at 24, but he’s not a celebrity either. He’s good and persistent and experienced, but probably not an expert. In the late 1990s, that person was just cracking into six-figure territory: $100,000 per year. No one thought that number was “ridiculous”. Adjusted for inflation, that’s $142,300 per year today. That’s probably not far off what an engineer at that level actually makes, at least in New York and the Bay Area.

Software engineers look “ridiculous” to people who haven’t been software engineers in 20 years (or ever) and whose numbers are way out of date. If you’re a Baby Boomer whose last line of code was in 1985, you’re probably still thinking that $60,000 is a princely sum for a programmer to earn. When one factors inflation into the equation, programmer salaries are only “at record high” because inflation is an exponential process. Taking that out, they’re right about where history says they should be.

I would argue, even, that programmer salaries are low when taking a historical perspective. The trend is flat, adjusting for inflation, but the jobs are worse. Thirty years ago, programming was an R&D job. Programmers had a lot of autonomy: the kind of autonomy that it takes if one is going to invent C or Unix or the Internet or a new neural network architecture. Programmers controlled how they worked and what they worked on, and either answered to other programmers or to well-read scientists, rather than anti-intellectual businessmen who regard them as cost centers. Historically, companies sincerely committed to their employees’ careers and training. You didn’t have to change jobs every 2 years just to keep getting good projects and stay employable. The nature of the programming job, over the past couple decades, has become more stressful (open-plan offices) and careers have become shorter (ageism). Job volatility (unexpected layoffs and, even, phony “performance-based” firings in lieu of proper layoffs, in order to skimp on severance because that’s “the startup way”) has increased. With all the negatives associated with a programming job in 2014, that just didn’t exist in the 1970s to ’80s, flat performance on the salary curve is disappointing. Finally, salaries in the Bay Area and New York have kept abreast of general inflation, but the costs of living have skyrocketed in those “star cities”, while the economies of the still-affordable second-tier cities have declined. In the 1980s and ’90s, there were more locations in which a person could have a proper career, and that kept housing prices down. In 2014, that $142,000 doesn’t even enable one to buy a house in a place where there are jobs.

All of those factors are subjective, however, so I’ll discard them. We have sufficient data to know that $142,000 for a mid-career programmer is not ridiculous. It’s a lower bound for the business value of a software engineer (in 1999); we know that employers did pay that; they might have been willing to pay more. This information already gives us victory over the assclowns claiming that software engineer salaries are “ridiculous” right now.

Now, I’ll take it a step further and introduce Yannis’s Law: programmer productivity doubles every 6 years. Is it true? I would say that the answer is a resounding “yes”. For sure, there are plenty of mediocre programmers writing buggy, slow websites and abusing Javascript in truly awful ways. On the other hand, there is more recourse for a good programmer who find quality; rather than commit to commercial software, she can peruse the open-source world. There’s no evidence for a broad-based decline in programmer ability over the years. It’s also easy to claim that the software career “isn’t fun anymore” because so much time is spent gluing existing components together, and accounting for failures of legacy systems. I don’t think these gripes are new, and I think tools are improving, and a 12% per year rate sounds about right. Put another way, one who programs exactly as was done in 1999 is only about 18 percent as productive as one using modern tools. And yet that programmer, only 18% as productive as his counterpart today, was worth $142,000 (2014 dollars) back then!

Does this mean that we should throw old tools away (and older programmers under the bus)? Absolutely not. On the contrary, it’s the ability to stand on the shoulders of giants that makes us able to grow (as a class) at such a rate. Improved tools and accumulated knowledge deliver exponential value, but there’s a lot of knowledge that is rarely learned except over a decades-long career. Most fresh Stanford PhDs wouldn’t be able to implement a performant, scalable support vector machine from scratch, although they could recite the theory behind one. Your gray-haired badasses would be rusty on the theory but, with a quick refresh, stand a much greater chance of building it righjt. Moreover, the best old ideas tend to recur and long-standing familiarity is an advantage. The most exciting new programming language right now is Clojure, a Lisp that runs on the Java Virtual Machine. Lisp, as an idea, is over 50 years old. And Clojure simply couldn’t have been designed by a 25-year-old in Palo Alto. For programmers, the general trend is a 12% increase in productivity; but individuals can reliably do 30 percent or more, and for periods spanning over decades.

If the business value of a mid-level programmer in 1999 was $142,000 in today’s dollars, then one can argue that today, with programmers 5.7 times more productive, the true value is $779,000 per year at minimum. It might be more. For the highly competent and for more senior programmers, it certainly is higher. And here’s another thing: investors and managers and VPs of marketing didn’t create that surplus. We did. We are almost 6 times as productive as we were in the 1990s not because they got better at their jobs (they haven’t) but because we built the tools to make ourselves (and our successors) better at what we do. By rights, it’s ours.

Is it reasonable, or realistic, to argue that mid-career software engineers ought to be earning close to a million dollars per year? Probably not. It seems to be inevitable, and also better for society, that productivity gains are shared. We ought to meet in the middle. That we don’t capture all of the value we create is a good thing. It would be awful, for example, if sending an email cost as much as sending a letter by post or, worse yet, as much as using the 19th-century Pony Express, because the producers of progress had captured all of the value for themselves. So, although that $779,000 figure adequately represents the value of a decent mid-career engineer to the business, I wouldn’t go so far as to claim that we “deserve” to be paid that much. Most of us would ecstatic with real equity (not that 0.05% VC-istan bullshit) and a quarter of that number– and with the autonomy to deliver that kind of value.

What Silicon Valley’s ageism means

Computer programming shouldn’t be ageist. After all, it’s a deep discipline with a lot to learn. Peter Norvig says it takes 10 years, but I’d consider that number a minimum for most people. Ten years of high-quality, dedicated practice to the tune of 5-6 hours per day, 250 days per year, might be enough. For most people, it’s going to take longer, because few people can work only on the interesting problems that constitute dedicated practice. The fundamentals (computer science) alone take a few thousand hours of study, and then there’s the experience of programming itself, which one must do in order to learn how to do it well. Getting code to work is easy. Making it efficient, robust, and legible is hard. Then, there’s a panoply of languages, frameworks, paradigms, to learn and absorb and, for many, to reject. As an obstacle, there’s the day-to-day misery of a typical software day job, in which so much time is wasted on politics and meetings and pointless projects that an average engineer is lucky to have 5 hours per week for learning and growth. Ten years might be the ideal; I’d bet that 20 years is typical for the people who actually become great engineers and, sadly, the vast majority of professional programmers never get anywhere close.

It takes a long time to be actually good in software engineering. The precocious are outliers. More typically, people seem to peak after 40, as in all the other high-skill disciplines. It, then, seems that most of age-related decline in this field is externally enforced. Age discrimination is not an artifact of declining ability but changing perceptions.

It doesn’t make sense, but there it is.

Age discrimination has absolutely no place in technology. Yet it exists. After age 40, engineers find it increasingly difficult to get appropriate jobs. Startups are, in theory, supposed to “trade against” the inefficiencies and moral failures of other companies but, on this issue, the venture capital (VC) funded startups are the biggest source of the problem. Youth and inexperience have become virtues, while older people who push back against dysfunction (and, as well, outright exploitation) are cited as “resistant to change”.

There’s another issue that isn’t derived from explicit ageism, but might as well be. Because our colonizers (mainstream business culture) are superficial, they’ve turned programming into a celebrity economy. A programmer has two jobs. In addition to the work itself, which is highly technical and requires continual investment and learning, there’s a full-time reputation-management workload. If a machine learning engineer works at a startup and spends most of his time in operations, he’s at risk of being branded “an ops guy”, and may struggle to get high-quality projects in his specialty from that point on. He hasn’t actually lost anything– in fact, he’s become far more valuable– but the superficial, nontechnical idiots who evaluate us will view him as “rusty” in his specialty and, at the least, exploit his lack of leverage. All because he spent 2 years doing operations, because it needed to be done!

As we get older and more specialized, the employment minefield becomes only more complicated. We are more highly paid at that point, but not by enough of a margin to offset the increasing professional difficulties. Executives cite the complexity of high-end job searches when demanding high salaries and years-long severances. Programmers who are any good face the same, but get none of those protections. I would, in fact, say that any programmer who is at all good needs a private agent, just as actors do. The reputation management component of this career, which is supposed to be about technology and work and making the world better, but is actually about appeasing the nontechnical, drooling patron class, constitutes a full-time job that requires a specialist. Either we need unions, or we need an agent model like Hollywood, or perhaps we need both. That’s another essay, though.

The hypocrisy of the technology employer

Forty years ago, smart people left finance and the mainstream corporate ladder for technology, to move into the emerging R&D-driven guild culture that computing had at the time. Companies like Hewlett-Packard were legitimately progressive in how they treated their talent, and rewarded for it by their employees’ commitment to making great products. In this time, Silicon Valley represented, for the most technically adept people in the middle class, a genuine middle path. The middle path will require its own essay, but what I’m talking about here is a moderate alternative between the extremes of subordination and revolution. Back then, Silicon Valley was the middle path that, four decades later, it is eagerly closing.

Technology is no longer managed by “geeks” who love the work and what it can do, but by the worst kinds of business people who’ve come in to take advantage of said geeks. Upper management in the software industry is, in most cases, far more unethical and brazen than anywhere else. To them, a concentration of talented people who don’t have the inclination or cultural memory that would lead them to fight for themselves (labor unions, agent models, ruthlessness of their own) is an immense resource. Consequently, some of the most disgusting HR practices (e.g. stack ranking, blatant sexism) can be found in the technology industry.

There’s one really bad and technical trait of software employers that, I think, has damaged the industry immensely. Technology employers demand specialties when vetting people for jobs. General intelligence and proven ability to code isn’t enough; one has to have “production experience” in a wide array of technologies invented in the past five years. For all their faults, the previous regime of long-lasting corporations was not so bigoted when it came to past experience, trusting people to learn on the job, as needed. The new regime has no time for training or long-term investment, because all of these companies have been built to be flipped to a greater fool. In spite of their bigoted insistence on pre-existing specialties in hiring, they refuse to respect specialties once people are hired. Individual programmers who attempt to protect their specialties (and, thus, their careers) by refusing assignment to out-of-band or inferior grunt work are quickly fired. This is fundamentally hypocritical. In hiring, software companies refuse to look twice at someone without a yellow brick road of in-specialty accomplishments of increasing scope; yet, once employees are inside and fairly captive (due to the pernicious stigma against changing jobs quickly, even with good reason) they will gladly disregard that specialty, for any reason or no reason. Usually, this is framed as a business need (“we need you to work on this”) but it’s, more often, political and sometimes personal. Moving talent out of its specialty is a great way for insecure middle managers to neutralize overperformance threats. In a way, employers are like the pervert who chases far-too-young sexual partners (if “partner” is the right word here) for their innocence, simply to experience the thrill of destroying it. They want people who are unspoiled by the mediocrity and negativity of the corporate world, because they want to inflict the spoilage. The virginity of a fresh, not-yet-cynical graduate from a prestigious university is something they want all for themselves.

The depression factor

I’m not going to get personal here, but I’m bipolar so when I use words like “depression” and “hypomania” and “anxiety” I do, in fact, know what the fuck I am talking about.

A side effect of corporate capitalism, that I see, is that it has created a silent epidemic of middle-aged depression. The going assumption in technology that mental ability declines after age 25 is not well supported, and it is in fact contrary to what most cultures believe about intelligence and age. (In truth, various aspects of cognition peak at different ages– from language acquisition at age 5 to writing ability around 65– and there’s also so much individual variation that there’s no clear “peak” age.) For general, holistic intelligence, there’s no evidence of an age-bound peak in healthy people. While this risks sounding like a “No True Scotsman” claim, what I mean to say is that every meaningful age-related decline in cognition can be tracked to a physical health problem and not aging itself. Cardiovascular problems, physical pain and side effects of medication can impair cognition. I’m going to talk about the Big One, though, and that’s depression. Depression can cause cognitive decline. Most of that loss is reversible, but only if the person recovers from it and, in many cases, they never do.

In this case, I’m not talking about severe depression, the kind that would have a person considering electroconvulsive therapy or on suicide watch. I’m talking about mild depression that, depending on time of diagnosis, might be considered subclinical. People experiencing it in middle age are, one presumes, liable to attribute it to getting older rather than a real health problem. Given that middle-aged “invisibility” in youth-obsessed careers is, in truth, unjust and depressing, it seems likely that more than a few people would experience depression and fail to perceive it as a health issue. That’s one danger of depression that those who’ve never experienced it might not realize exists: when you’re depressed, you suffer from the delusion that (a) you’ve always been depressed, and (b) that no other outlook or mood makes sense. Depression is delusional, and it is a genuine illness, but it’s also dangerously self-consistent.

Contrary to the stereotype, people with depression aren’t always unhappy. In fact, people with mild depression can be happy quite often. It’s just easier to make them unhappy. Things that don’t faze normal people, like traffic jams or gloomy weather or long queues at the grocery store, are more likely to bother them. For some, there’s a constant but low-grade gloom and tendency to avoid making decisions. Others might experience 23 hours and 50 minutes per day of normal mood and 10 minutes of intense, debilitating, sadness: the kind that would force them to pull over to the side of the road and cry. There isn’t a template and, just as a variety of disparate diseases (some viral, some bacterial, and some behavioral) were once called “fevers”, I feel like “depression” is a cluster of about 20 different diseases that we just don’t have the tools to separate. Some depressions come without external cause. Others are clearly induced by environmental stresses. Some depressions impair cognition and probably constitute a (temporary) 30-IQ-point loss. Others (more commonly seen in artists than in technology workers) seem to induce no intellectual impairment at all; the person is miserable, but as sharp as ever.

Corporate workers do become less sharp, on average, with age. You don’t see that effect, at least not involuntarily so, in most intellectually intense fields. A 45-year-old artist or author or chess master has his best work ahead of him. True entrepreneurs (not dipshits who raise VC based on connections) also seem to peak in their 50s and, for some, even later. Most leaders hit their prime around 60. However, it’s observable that something happens in Corporate America that makes people more bitter, more passive, and slower to act over time, and that it starts around 40. Perhaps it’s an inverse of survivor bias, with the more talented people escaping the corporate racket (by becoming consultants, or entrepreneurs) before middle age. I don’t think so, though. There are plenty of highly talented people in their forties and fifties who’ve been in private-sector programming for a long time and just seem out of gas. I don’t blame them for this. With better jobs, I think they’d recover their power surprisingly quickly. I think they have a situationally-induced case of mild depression that, while it may not be the life-threatening illness we tend to associate with major depression, takes the edge off their abilities. It doesn’t make them unemployable. It makes them slow and bitter but, unlike aging itself, it’s very easily reversible: change the context.

Most of these slowed-down, middle-aged, private-sector programmers wouldn’t qualify for major depressive disorder. They’re not suicidal, don’t have debilitating panic attacks, and can attribute their losses of ability (however incorrectly) to age. Rather, I think that most of them are mildly but chronically depressed. To an individual, this is more of a deflation than a disability; to society, the costs are enormous, just because such a large number of people are affected, and because it disproportionately affects the most experienced people at a time when, in a healthier economic environment, they’d be in their prime.

The tournament of idiots

No one comes out of university wanting to be a private-sector social climber. There’s no “Office Politics” major. People see themselves as poets, economists, mathematicians, or entrepreneurs. They want to make, build, and do things. To their chagrin, most college graduates find that between them and any real work, there’s at least a decade of political positioning, jockeying for permissions and status, and associated nonsense that’s necessary if one intends to navigate the artificial scarcity of the corporate world.

The truth is that most of the nation’s most prized and powerful institutions (private-sector companies) have lost all purpose for existing. Ideals and missions are for slogans, but the organization’s true purpose is to line the pockets of those ranking high within it. There’s also no role or use for real leadership. Corporate executives are the farthest one gets from true leaders. Most are entrenched rent-seekers. With extreme economic inequality and a culture that worships consumption, it should surprise no one that our “leadership” class is a set of self-dealing parasites at a level that hasn’t been seen in an advanced economy since pre-Revolution France.

Leadership and talent have nothing to do with getting to the top. It’s the same game of backstabbing and political positioning that has been played in kings’ courts for millennia. The difference, in the modern corporation, is that there’s a pretense of meritocracy. People, at least, have to pretend to be working and leading to advance further. The work that is most congruent with social advancement, however, isn’t the creative work that begets innovation. Instead, it’s success in superficial reliability. Before you get permission to be creative, you have to show that you can suffer, and once you’ve won the suffering contest, it’s neither necessary nor worth it to take creative risks. Companies, therefore, pick leaders by loading people with unnecessary busywork that often won’t go anywhere, and putting intense but ultimately counterproductive demands on them. They generate superficial reliability contests. One person will outlast the others, who’ll fall along the way due to unexpected health problems, family emergencies, and other varieties of attrition (for the lucky few, getting better jobs elsewhere). One of the more common failure modes by which people lose this tournament of idiots is mild depression: not enough to have them hospitalized, but enough to pull them out of contention.

The corporate worker’s depression, especially in midlife, isn’t an unexpected side effect of economic growth or displacement or some other agent that might allow Silicon Valley’s leadership to sweep it under the rug of “unintended consequences”. Rather, it’s a primary landscape feature of the senseless competition that organizations create for “leadership” (rent-seeking) positions when they’ve run out of reasons to exist. At that level of decay, there is no meaningful definition of “merit” because the organization itself has turned pointless, and the only sensible way to allocate highly-paid positions is to create a tournament of idiots, in which people psychologically abuse each other (often subtly, in the form of microaggressions) until only a few remain healthy enough to function.

Here we arrive at a word I’ve come to dread: corporate culture. Every corporation has a culture, and 99% of those are utterly abortive. Generally, the more that a company’s true believers talk about “our culture”, the more fucked-up the place actually is. See, culture is often ugly. Foot binding, infantile genital mutilation (“female circumcision”), war, and animal torture are all aspects of human culture. Previous societies used supernatural appeal to defend inhumane practices, but modern corporations use “our culture” itself as a god. “Culture fit” is often cited to justify the otherwise inconsistent and, sometimes, unjustifiable. Why wasn’t the 55-year-old woman, a better coder than anyone else on the team, hired? “It wouldn’t look right.” Can’t say that! “A seasoned coder who isn’t rich would shatter the illusion that everyone good gets rich.” Less illegal, but far too honest. “She didn’t fit with the culture.” Bingo! Culture can always be used in this way, by an organization, because it’s a black box of blame, diffusing moral culpability about the group. Blaming an adverse decision on “the team” or “the culture” avoids individual risk for the blamer, but the culture itself can never be attacked as bad. Most people in most organizations actually know that the “leadership team” (career office politicians, also known as executives) of their firm is toxic and incompetent. When they aren’t around, the executives are attacked. But it’s rare that anyone ever attacks the culture because “the culture” is everyone. To indict it is to insult the people. In this way, “the culture” is like an unassailable god.

Full circle

We’ve traveled through some dark territory. The tournament of idiots that organizations construct to select leadership roles, once they’ve ceased to have a real purpose, causes depression. The ubiquity of such office cultures has created, I argue, a silent epidemic of mild, midlife depression that has led venture capitalists (situated at its periphery, their wealth buying them some exit from the vortex of mediocrity in which they must still work, but do not live) and privileged young psuedo-entrepreneurs (terrified of what awaits them when their family connections cool and they must actually work) to conclude that a general cognitive mediocrity awaits in midlife, even though there is no evidence to support this belief, and plenty of evidence from outside of corporate purgatory to contradict it.

What does all of this say? Personally, I think that, to the extent that large groups of individuals and organizations can collectively “know” things, the contemporary corporate world devalues experience because it knows that the experience it provides is of low value. It refuses to eat its own dogfood, knowing that it’s poisoned.

For example, software is sufficiently technical and complex that great engineers are invariably experienced ones. The reverse isn’t true. Much corporate experience is of negative value, at least if one includes the emotional side effects that can lead to depression. Median-case private-sector technology work isn’t sufficiently valuable to overcome the disadvantages associated with age, which is another way of saying that the labor market considers average-case corporate experience to have negative value. I’m not sure that I disagree. Do I think it’s right to write people off because of their age? Absolutely not. Do I agree with the labor market’s assessment that most corporate work rots the brain? Well, the answer is mostly “yes”. The corporate world turns smart people, over the years, into stupid ones. If I’m right that the cause of this is midlife depression, there’s good news. Much of that “brain rot” is contextual and reversible.

How do we fix this?

Biologists and gerontologists seeking insight into longevity have studied the genetics and diet of long-living groups of people, such as the Sardinians and the people of Okinawa. Luckily for us, midlife cognitive decline isn’t a landscape feature of most technical or creative fields. (In fact, it’s probably not present in ours; it’s just perceived that way.) There are plenty of places to look for higher cognitive longevity, because few industries are as toxic as the contemporary software industry. When there is an R&D flavor to the work, and when people have basic autonomy, people tend to peak around 50, and sometimes later. Of course, there’s a lot of individual variation, and some people voluntarily slow down before that age, in order to attend to health, family, spiritual, or personal concerns. The key word to that is voluntary

Modeling and professional athletics (in which there are physical reasons for decline) aside, a career in which people tend to peak early, or have an early peak forced upon them, is likely to be a toxic one that enervates them. Silicon Valley’s being a young man’s game (and the current incarnation of it, focused on VC-funded light tech, is exactly that) simply indicates that it’s so destructive to the players that only the hard-core psychopaths can survive in it for more than 10 years. It’s not a healthy place to spend a long period of time and develop an expertise. As discussed above, it will happily consume expertise but produces almost none of value (hence, its attraction to those with pre-existing specialties, despite failing to respect specialties once the employee is captive). This means, as we already see, that technical excellence will fall by the wayside, and positive-sum technological ambition will give way to the zero-sum personal ambitions of the major players.

We can’t fix the current system, in which the leading venture capitalists are striving for “exits” (greater fools). That economy has evolved from being a technology industry with some need for marketing, to a marketing industry with some need (and that bit declining) for technology. We can’t bring it back, because its entrenched players are too comfortable with it being the way it is. While the current approach provides mediocre returns on investment, hence the underperformance of the VC  asset class, the king-making and career-altering power that it affords the venture capitalist allows him to capture all kinds of benefits on the side, ranging from financial upside (“2 and 20″) to executive positions for talentless drinking buddies to, during brief bubbly episodes such as this one, “coolness”. They like it the way it is, and it won’t change. Rather than incrementally fix the current VC-funded Valley, it must be replaced outright.

The first step, one might say, is to revive R&D within technology companies. That’s a step in the right direction, but it doesn’t go far enough. Technology should be R&D, full stop. To get there, we need to assert ourselves. Rather than answering to non-technical businessmen, we need to learn how to manage our own affairs. We need to begin valuing the experience on which most R&D progress actually relies, rather than shutting the seasoned and cynical out. And, as a first step in this direction, we need to stop selling each other out to nontechnical management for stupid reasons, including but especially “culture fit”.

Street fighting, HR, and the importance of collective action.

Street fights

Street fighting is a topic on which a large number of men hold strong opinions, although very few have been in a true fight. I haven’t, and I hope never to be in one. Playground fisticuffs are one thing, but for adults, street fighting is far too dangerous to be considered anything but a terrible idea. It’s not likely to happen, but a punch from an adult can kill. The exact mechanics by which punches to the face incapacitate aren’t fully understood, but rotation in the neck (constricting blood vessels) seems to be a culprit, and the forces involved in a proper face punch are enough, with the right conditions, rupture blood vessels in the neck and head and end a life. A person falling and hitting the ground can do it, too.

Anything before age 16 doesn’t count. Martial arts sparring doesn’t count. Boxing doesn’t count. Fight clubs don’t count. Those are a lot safer. In a real fight, you typically don’t know your opponent. If he wins, he might kill you (intentionally or otherwise) once you are on the ground. He may have a weapon and, even if he doesn’t, he can kill you with a few kicks or punches. Losing means you’re likely to end up in the hospital (and possibly dead). Winning means you have to appear in court, possibly for murder. Either way, it’s not a good place to be. If there are bystanders, the loser faces humiliation but probably won’t die. If there are none, it is up to the one who wins the fight, and unintentional (or intentional) deaths from punches and kicks happen all the time.

The moral bikeshedding around fistfighting (and the actual physical risks of it) are discussed in this analysis of the Trayvon Martin case. Is George Zimmerman a huge asshole and a probable racist? Absolutely. Should he have followed and harassed a 17-year-old? No, not at all. Is George Zimmerman a good or admirable person? Clearly not. Was Zimmermann in mortal danger during that fistfight? Yes. He was on the ground, being punched, which alone puts his life in danger. (“Ground and pound” constitutes attempted murder in most jurisdictions.) He had a lethal weapon and, if he lost capacity, he’d also lose control of the weapon. Zimmerman’s wrongdoing was following a stranger in the first place– but not firing a gun in “just a fistfight”. When the opponent is a stranger and there is no one to break up the fight, there is no such thing as “just a fistfight”.

Fistfights (and armed confrontations) aren’t like what people see in movies or on television. Some notable differences are:

  • in most, the loser is incapacitated within a few seconds. One punch can do it, and the element of surprise (that is, being the one to start the fight) confers a huge advantage.
  • fighting is exhausting and most people will be winded (and vulnerable) inside of 15 seconds. Fights become remarkably more dangerous at this point, a common cause of unexpected death being falls after a decisive punch.
  • it’s very difficult to get up when grounded, especially if one has taken several blows to the face or head.
  • an incapacitated opponent is easy to kill, either with continual punching, kicks to the head, or a stomp to the head or neck. Most fights are broken up before that can happen, but unexpected deaths (even from a single punch) occur, sometimes hours after the fight.
  • if there are two or more opponents, fighting skill and strength typically won’t prevent a very bad outcome. Two-on-one is dangerous.
  • most untrained people, when attacked, will panic. Between the short duration of typical fights, the incapacitating nature of blows to the face or head, and the effect of surprise, that initial second or two of shock is enough to cause them to lose the fight.

Knowing all this, I also know to avoid getting into fistfights. I might know more than most people about them (because I’ve done my research) but I won’t pretend to have any idea how I’d perform. It’s not worth finding out.

Enough about fistfighting, though. It’s not that interesting. I just wanted to bring it up, because it’s something that men think they know a lot about, and they tend to underestimate the risks. There are a lot of people who think they know what they’d do in George Zimmerman’s position (panicked, mounted, having sustained several punches). Most of them don’t. They haven’t been in a fight since junior high, and those don’t count.

“I would…” Stop right there. You don’t know.

Let’s talk about organizational life, on which I’ve shared a lot of opinions.

Ask a software engineer in Silicon Valley for his opinion on collective bargaining, and you’re likely to hear a negative response. Unions have been targeted for negative publicity for decades and have a reputation for mediocrity and corruption. “I don’t need no stinkin’ union!” Your typical 25-year-old engineer probably hasn’t been fired from a job he wanted or needed to keep (a summer job doesn’t count). He’s probably never faced financial pain or stress on account of a bad job outcome. If he needs to negotiate a severance (or, a more delicate task, an agreed-upon reference) from an employer unwilling to budge, he probably has no idea what to do.

As with fistfighting, most people have an alarmingly precise image of how they’d get out of a bad situation and, for the most part, they don’t know.

“If I were in that grapple, I’d grab his wrist and bite down, then drive my elbow into his balls, then knee him in the head until he gave.” (No, you wouldn’t. You’d be panicked or confused, having taken several punches, and struggling to stay conscious.)

“If I were in a two-on-one fight like that, I would…” (Sorry, but in a two-on-one fight against capable, adult opponents, you’re fucked.)

“If I were put on a PIP, I would…” Same principle. If you’ve never been in a real fight, you don’t know.

Companies have trained fighters on staff, between their attorneys and HR. They have hard fighters who’ll insult and belittle you, and soft fighters who’ll pretend to take your side, while taking notes that can be used against you. If the company wants you in a two-on-one, you’ll face that: your manager and HR, or your manager and his manager. You basically can’t say anything in a two-on-one meeting. They will plan the meeting beforehand together, decide on “the story” afterward together, and (most likely) you’ll have no credibility in court or internally. If the meeting is adversarial and two-on-one, say as little as you possibly can.

In a police interrogation, you’d have the right to have your attorney present, and the right to remain silent, and you really want to exercise those rights. Watch this incredibly convincing video, backed by decades of experience, if you have doubt on that matter. What about a PIP meeting, which comes unannounced? On Tuesday, your boss was congratulating you on your work. You were an employee in good standing. On Wednesday, you’re told in the middle of a 1-on-1 “not performing at your level”. You were ambushed; there was no sight that this was coming. That corner you cut because your boss told you to do so is now being cited as an example of the low quality of your work. Most people think they would immediately fight intelligently, with the same confidence as applied to fistfights. Not so. Between the initial panic (especially if the person has financial needs) and intense emotions related to betrayal, it’s going to be really hard not to shit the bed when under a PIP ambush. And, as with a fistfight, it’s really easy to lose quickly. Raise your voice? You’ll probably be fired “for cause” because of “threatening behavior”. Attempt to amend the PIP’s factual inaccuracies? (There will always be inaccuracies in a PIP and they’re intentional. Those documents are supposed to piss you off because about 1 in 5 will lose his cool, thereby firing himself on the spot.) That might be construed as insubordination.

HR is probably present in meetings with management, from that point forward. The reason put forward to the PIPee is to protect the employee against managerial retaliation, and that’s complete bullshit. HR is there to corroborate management’s version of the story (even if management lies). Nothing that the employee says can improve his case. As with a fistfight, two-on-one means you’re probably done. Avoid it if you can.

In one way, it’s more perilous to be under PIP ambush than in a police interrogation, though. In the latter, one has the right to be silent and to call in a lawyer. In a PIP meeting, the rules aren’t clearly defined. Is an employee who refuses to answer a manager’s question guilty of “insubordination”, allowing termination on the spot? Can the employee be fired for refusing to sign the PIP “until I talk to a lawyer”? (That will be noticed.) Can the employee turn down an “independent” HR party’s meeting request “to hear your side of the story”? (Don’t fall for this.) Is the employee guilty of insubordination if he shares, with other employees, that he’s under a PIP, and voices his supposition that the decision to place him on it was political? At-will employment is complicated and undefined in many cases, and the answers to these questions are, “no one knows”. The company will terminate (once a PIP is shown, the decision has been made and only a change in management can reverse it) when it feels it can do so with minimal risk and cost. Some companies are aggressive and will use anything that is said in a PIP or HR meeting as cause to fire without severance. Now, at-will employment has a lot of serviceable corner cases (despite what employers claim) but is it worth it to launch a lawsuit against a deep-pocketed adversary? For most people, probably not. And lawsuits (again, like fistfights) are another form of combat in which an untrained person has no idea what to expect, and is at a disadvantage. Say the wrong thing, and it will be used against you. Even if you “should” win because you have the space or superior ability, you can lose in seconds.

With PIP meetings, management has had time to prepare, and the employee is ambushed. That unfairness is intentional. It’s part of the design. Same with the two-on-one intimidation. In a PIP meeting, silence will be presented as a non-option. Even though this is illegal, most employers will claim that not signing the PIP will lead to termination (record this conversation on a cell phone if you can; but if you’re not unusually calm, there’s a low chance that you’ll remember to do this). They can bring anyone they want into the room, but you almost certainly won’t be allowed to have an attorney or representative present. It’s unfair, it’s shitty, and you need to have a plan before it ever gets to that. Your best option is to say, “I need to go to the bathroom”. Take time, calm down, and possibly call a lawyer. If you return to the PIP meeting 20 minutes later, so be it. They can wait.

Macho self-reliance

No amount of self-defense training will make it safe to get into a street fight. The odds may improve, but it’s still an inherently dangerous thing to do. Knives aren’t good protection, being much harder to control in a fight than one might think, and inviting other weapons such as guns. Speaking of guns, most people wouldn’t know what to do with a gun in a situation of actual danger. Many would think that, with a gun against four unarmed assailants, they’d be able to emerge the winner. I wouldn’t count on that.

All that said, there are plenty of people (mostly men) who believe that, because of their martial arts training or because they possess a weapon, they can safely go to places, and engage in behaviors, that make fistfights common. They’re wrong. The issue is that people wish to identify with the winner in any conflict. “I would never end up on the ground like that.” They overestimate their performance in the face of fatigue, panic, confusion, or under time pressure. Until something bad happens to that person, many people assume it never will. “I’d never be put on a PIP, because I’m good at my job”. That’s wrong, too. True low performers are usually eased out through other means. PIPs are generally used to sucker-punch a politically-targeted high performer, and they work. Plenty of people shit the bed the first time they see one.

In the world of work, this macho self-reliance is seen in the resistance of many workers to any sort of protection. You hear this one a lot from software engineers: “I wouldn’t want to work somewhere where it’s hard to fire people.” I’m sorry, but that’s a bit ludicrous. It should be hard enough to fire people that the politically unlucky still end up with a decent severance, and hard enough that people who fit poorly with their first project are given at least one more chance.

Let’s talk about the “10x engineer”. That effect is real. It’s not only about natural ability, so it’s not the same people who’ll be “10x” in every single context. Motivation, project/person fit, political position (those who write code and make decisions will always be more productive than those who read code and are downwind of decisions) and domain-specific expertise all apply. To understand why, consider the Gaussian bell curve, which emerges as random variables compound additively. In most human affairs, though, they compound multiplicatively, producing the “fat-tailed” lognormal distribution. Sometimes, there are knock-on effects that produce an even fatter-tailed power law distribution.  (If this all sounds like jargon, the takeaway is, “outliers are more common than one would think.”) Consider a world in which a new employee’s productivity is a function of the outcome of a 6-sided die, like so:


| Die |  A  |  B  |
-------------------
|  1  |   0 |   4 |
|  2  |   9 |   6 |
|  3  |  10 |  10 |
|  4  |  10 |  20 |
|  5  |  11 |  40 |
|  6  |  11 | 100 |

In scenario A, average employees (10 points) have job security, because even at that point, they’re above the mean productivity (8.5 points) of a new employee. Even the noticeably below average employees are OK. On the other hand, in Scenario B, the mean is 30 points, which means that the mediocre ’3′s and ’4′s (10 and 20 points, respectively) should be fired and will be, even if it’s not their fault. In practice, it doesn’t workout quite that badly. Firing average employees is bad for morale, and the “10x” effect is as much about motivation and purpose as it is about the individual. But we can see, from a theoretical basis and considering the convex nature of software development, why tech companies tend to have such mean-spirited management practices. From their perspective, they’re in the business of finding and exploiting 10x-ers (who won’t be paid anything remotely close to their value, because they’re lucky to still be employed) and anyone who seems not to be a 10x-er should be tossed back into the sea.

Closed allocation tech companies, one notes, are notoriously awful when it comes to internal mobility. This is a severe ethical issue because internal mobility is promised almost inherently by big companies. It’s the one advantage large companies have over small ones. Yet, most technology companies are set up so that median performers are practically immobile (unless their projects are canceled and they must be put somewhere). Headcount limitations and political intricacies (one manager not wanting to upset the other) arrange it so that middling and even average-plus performers can’t really move anywhere. The fact that there are political externalities to deal with, alone, makes them less desirable to the target team’s hiring manager than a fresh recruit from outside the company. Stars generally are worth fighting a broken transfer system to attain, but they generally don’t want to move, because they have a gravy train they’d rather not stop.

Most software engineers think too logically and will take basically reasonable premises (“low performers should be fired”) to unreasonable conclusions (“performance reviews should be part of the transfer packet”). This allows technology companies to abuse them, taking advantage of their just world delusion and rugged individualism. When very real abuses are targeted toward those around them, they fail to perceive that such things could ever happen to them. “I’m a rock star coder. I’ll never be put on a PIP! If I were, I’d just work weekends and show the boss he’s wrong!” This is like claiming that knowing a couple jiujitsu throws makes it safe to get into bar brawls. It’s just not true.

Do we need unions?

Originally, I said no. When I said that, to be frank about it, I didn’t know what the fuck I was talking about. I got professions right, but I had a defective understanding of what labor unions actually do and why they are important.

I’m not ready to come down on either side of that question. We also don’t want to see the seniority-bound nightmare of, say, the airline pilots’ unions. Also, uniformity in compensation begets mediocrity on both sides and we don’t want that; the good news is that writers’ and actors’ unions seem not to destroy the upside in pay for those people. Hollywood’s labor pool, including celebrity actors, is unionized and hasn’t suffered in quality or compensation for it. Professional structure doesn’t necessarily lead to mediocrity.

However, we need collective bargaining. We need the right of a software engineer to have an independent representative, as well as legal counsel, in the room when in hot water with management (to prevent those “shit the bed” PIP ambush scenarios). We need protection against political terminations, such as for revealing salary information (the general prohibition offices have against that isn’t for anyone’s benefit but their own) or committing the overperformance offenses (doing a job too well, and embarrassing others) typical of engineers. We need to be part of a political body that corporate management actually fears. When we act in good faith against the interests of our employers (say, by revealing abuses of power) we shouldn’t have to do it alone. We also need to have the right to have those contractual issues that are embarrassing to negotiate for oneself (severance, health accommodations, privacy) negotiated for us by more experienced parties. We need automatic legal assistance in questionable terminations, and automatic work-stopping shutdowns (strikes) against any employer who resorts to a negative reference. We need sleazy and unreliable “back channel” reference checking to be abolished, with employers who engage in or allow it being sued or struck into oblivion.

Right now, we get none of this, and it’s a total fucking crime.

That delusional programmer self-reliance (“I know how to handle myself in a fight”) allows employers to isolate “troublemakers” and pick them off, one by one. It allows them to negotiate salaries and conditions down, brutally, due to asymmetric information and broken, reputation-driven power structures in which even six months out of traditional employment can ruin a person’s life. It allows stack ranking and closed allocation, which any labor body with self-respect would fight back against.

Because of our nonsensical self-reliance, when we do fight, it is usually an individual acting alone. Employers can easily ruin the reputations of isolated individuals, and few will protect a person thus disparaged. Consequently, those battles are typically ineffective and humiliating to the fighter. The conclusion of the workers is that management simply can’t be fought. Yet those same people believe that, through hard work or some notion of personal “merit”, they’ll never face political adversity, unfair terminations, or humiliating, destabilizing processes such as PIPs. Like armchair street fighters, they don’t know what the fuck they are talking about.

Collective strength

Street and bar fights are depressing, illegal, stupid, and dangerous. Your average person would have no idea how to respond in a sudden, unexpected assault, and be completely at the mercy of the attacker. Others who could assess the situation would have a chance to interpose, but the victim would have lost the fight by the time the initial shock wore off. The good news is that such things are uncommon. Most people haven’t been in a fight since high school, if ever. Why? In public spaces, attacking another person is dangerous and generally a terrible idea. One might get beaten up oneself, especially if others interpose, and jail time is likely. When it comes to physical violence, there’s an understanding that, while rare, bad actors do exist and that it’s worth it to protect a stranger against an attack, if one can. People will often defend each other, which makes the costs and risks of assault sufficiently high that it’s a rarity.

In the workplace, we don’t see this. When a good employee is served with a PIP or fired, nothing really happens. People live in such fear of management that word of the unjust termination is unlikely to travel far, and even more unlikely to have any effect on the company. Those who oppose a company’s management in any way, no matter how trivial and even if it is accidental, are isolated and embarrassed by the organization. If it really cannot afford the image issues associated with a clearly political demotion or termination, it will simply assign him untenable work, interfere with performance, and render it impossible for him to do a good job, making the eventual outcome appear deserved.

The result is that sudden, unpublished assaults are common in the corporate world, sometimes because a person falls into an unknown political trap and winds up in a management-level grudge, sometimes because the organization must protect unrelated interests, and sometimes for no reason at all. They are rarely talked about. The person is isolated and either goes silently, or is humiliated. Opposition to the organization resulting from his treatment will never happen. In light of the severe (and unreasonable) stigma placed on employees who “bad-mouth” ex-employers, he can’t sue or publicly disparage that company without risking far more than he should ever have to put on the line. No one has the individual employee’s back, and that’s a shame.

Software engineers might be the worst in this regard, because not needing support is taken as a badge of pride, as seen in tech’s celebration of reckless firing, punishing work conditions, and psychotic micromanagement in the name of “agile”. “I don’t need no stinkin’ rights. Two fists are all I need.” That just makes no sense. It renders the individual programmer (a person with skills for which demand is so high that, if we acted collectively, we could really improve things for ourselves) easy to isolate, exploit, and humiliate if he even attempts to get a fairer deal.

The enemy, and this is especially a problem for software engineers, is delusional self-confidence. People feel a lot safer than they actually are, and to communicate the truth of one’s lack of safety is taken as admitting weakness, which allows the bad actors to isolate their targets easily. This prevents non-managerial professional groups (such as software engineers) from acting collectively and overcoming their current disadvantage.

Meritocracy is the software engineer’s Prince Charming (and why that’s harmful).

One of the more harmful ideas peddled to women by romance novels and the older Disney movies is the concept of “Prince Charming”, the man who finds a young girl, sweeps her off her feet, and takes care of her for the rest of her days. It’s not a healthy concept, insofar as it encourages passivity as well as perfectionism in mates. But it also encourages women to make excuses for bad (and often abusive) men. Because the archetype is so unreasonable, men who can make themselves seem to fulfill it are the manipulative and sometimes abusive ones, not genuine good (but flawed) men. I’d argue that software engineers have a similar Prince Charming.

It might begin as a search for “a mentor”. Savvy software engineers take knowledge and favor from multiple people, but every Wall Street or Silicon Valley movie showcases a mentor/protege relationship as the path to success. Meet this magical person, and he’ll take care of your career from there on out. That doesn’t exist for most people, either, and most software engineers learn that around age 25. Their counterreaction is to develop a bizarre self-reliance in which they start refusing help, wanting to work alone, and denigrating those who advance their careers based on “politics” or “connections”. Having too much dignity to wait for a magical mentor to rescue them from mediocrity, they insist on their new Prince Charming, an interpersonal force that will recognize and elevate talent: meritocracy.

The problem with meritocracy is that every organization claims to be one, yet almost all organizations are deeply political. Software engineers are not a subtle breed, so I must imagine that they imagine most non-meritocracies perceive themselves as such, and admit so much, and that’s clearly not true. Oil companies, banks, startups and dysfunctional academic bureaucracies all have this in common: they believe in their own meritocracy. Otherwise, they wouldn’t be self-consistent and stable. “We’re a meritocracy” means nothing. And what is “merit”? Organizations make promotion decisions not to recognize some abstract principle of “merit”, but on what is perceived to be in the short-term, narrow interest of the organization. It’s not what software engineers mean when they use the term merit, but one could argue that political acumen is organizational merit. The people who are promoted in and end up dominating organizations are… those most able to convince organizations to promote them, whether through delivering objective value or by trickery and intimidation. It’s a self-referential, Darwinian sense of “merit” akin to “fitness”. Darwinian fitness is neither a matter of good, bad, or anything other than the ability to self-replicate.

Of course, I know what software engineers mean when they say they want to live in a “meritocracy”. They want important decisions that affect their working lives to be made by the right people. The problem is that the ability to make good executive decisions is almost impossible to measure, reliably, especially on a timeframe that businesses would consider acceptable. Political machinations can happen, on the other hand, in split seconds. Saying something stupid in a meeting can end someone’s career, even if that person is, in general, a good decision-maker. It takes too long to select leaders based on the quality of their decisions, so organizations develop political side games that end up consuming more energy, time and attention (especially at high levels) than the actual work or purpose of the organization. Generally, this side game takes on the feeling of a war of attrition. Nonsensical pressures and busywork are added until people embarrass themselves out of contention, or their health fails, or they leave to pursue better options, leaving one person standing. Software isn’t different from that, with the long hours and posturing machismo and general disregard for health.

By believing in meritocracy, software engineers trick themselves into making excuses for awful companies and bad bosses that hurt their careers, destroy their confidence, and unapologetically exploit them. When they enter organizations, they tend (at least, when young) to want to believe in the self-professed “meritocracy”, and it’s hard to let such an idea go even in the face of adverse evidence. When these engineers are betrayed, it’s practically an ambush.

Older, savvier engineers know that few workplaces are meritocracies. In general, the claim of “meritocracy” is nothing more than a referendum on the leadership of the company. For this reason, it’s only in the midst of an open morale crisis (in which firing the obviously unhappy people isn’t viable because almost everyone is obviously unhappy) that one can admit to the organization’s non-meritocracy.

The expensiveness of it all

Software engineers’ belief in meritocracy costs them money and career advancement. By conflating their organizational position (low, usually) with personal merit, their confidence falls to zero. Computer programming, if marketed properly, ought to be “the golden skill” that allows a person unlimited mobility within industry. However, we’ve allowed the businessmen who’ve colonized us to siloize us with terms like DBA, operations, data scientist, etc., and use those to deny opportunities, e.g. “you can’t take on that project, you’re not a real NLP programmer”. As a class, we’ve let these assholes whittle our confidence down to such a low level that our professional aura is one either of clueless youth or depressive resignation. When they beat us down, we tend to blame ourselves.

Our belief in meritocracy hurts us in another way, in that we justify things being unduly hard on us. We hate the idea of political promotion. Perhaps, on first principles, we should. What this means is that engineers are promoted “confirmationally” rather than “aspirationally”. In HR-speak, confirmational promotion means that they’re given formal recognition (and the organizational permission to operate at the level they have been) once they’re already working at the level signified by the title. Aspirational promotion means that people are promoted based on potential, but this opens the door for a host of clearly political promotions. On paper, confirmational promotion is superior, if infuriatingly slow. (It requires people to blow off their assigned duties and to take unrequested risks.) Engineers, of course, prefer confirmational regimes. And what’s wrong with that?

Engineers don’t like to negotiate, they don’t like politics, and they’re against favoritism. Most have a proud self-reliance that would leave them uncomfortable even if personally favored. They’re also, in general, irreverent toward title as long as they believe they’re fairly paid. To them, confirmational promotion is right. The problem? Everyone but engineers is promoted aspirationally. Engineers need long, completed, successful projects to get bumped to the next level. What, pray tell, does it take to become VP of Product or Senior Manager as opposed to Manager, or to rise on just about any of the nontechnical tracks, in most tech companies? Absolutely nothing. There is no fucking magic there. You have to convince someone to “see something” in you. That is, you have to play politics.

To the engineer’s chagrin, playing politics comes easily for most ambitious people. It sure isn’t rocket science. Getting over one’s own moral objections is, for most people, the hardest part. The result of this is that nontechnical tracks, including management tracks that often cross over engineers, are characterized by easy aspirational promotion driven by favoritism and politics. The “meritocratic” engineering track is clearly much more difficult. There are people over 35, with IQs over 140, who haven’t made “senior engineer”, for fuck’s sake. (At a “mere” 125 IQ, you’re smarter than 97% of the nontechnical VPs at most tech companies.) It’s characterized by confirmational promotion, instead. And this is a point of pride for software engineers: it’s really hard to climb the ladder, because one is competing with the smartest people in the organization, and because while favoritism exists, political promotions are much rarer on the engineering track than on non-technical tracks (largely because promotions in general are rarer).

This is something that software engineers don’t really get. What do job titles actually mean in organizations? Companies will say that “Director” means one thing and “VP” means another, with some jargon about “the big picture” and a person’s responsibilities within the organization. The truth is that they mean very little, other than serving as political tokens that prove the person was able to get them. “Director” means, “he was able to negotiate a salary between $X and $Y from HR”. Not more.

Where it leads

If you ask an engineer whether he thinks he’s ready to be VP of Engineering or CTO, you’ll get a half-hearted, self-deprecating answer. “You know, I might be ready to lead a small team, but I’m not sure I’m at the VP/Eng level yet.” Cluelessly, he believes that “the VP/Eng level” exists objectively rather than politically. On the other hand, if you ask a nontech the same question, he’ll take it without hesitation. Even if he’s terrible at the job, he gets a generous severance (he’s a VP) and will fail up into a better job. The relevant concept here is the effort thermocline, or the level in an organization where jobs stop being harder with increasing rank, but become easier (although, more political). It can be politically difficult to get a job above the effort thermocline, but it’s ridiculously easy to keep it. At that point, one has power and credibility within the organization sufficient that one cannot, personally, fail due to a lack of effort.

Nontechs, except for clueless people in their 20s who haven’t figured out what they want to do, go into work with one purpose: to get promoted beyond the effort thermocline. That’s not to say that they’re all unambitious or lazy. They’re just realistic about how the game works. Even if you want to work hard, you don’t want hard work to be expected of you. If you’re an SVP and you show up for work every day and put in an honest effort, you get credit for it. If you’re a worker bee, you get nothing for your 8-or-more hours per day. It’s just what you’re expected to do.

Above the effort thermocline, promotion is political, and people stop pretending otherwise. When you get “into the club”, you’re permitted to speak frankly (and hear frank speech) about how the organization actually works. The issue with the engineer’s mind is that it clings to a belief in right and wrong. It’s moralistic. It struggles to accept what people really are. Engineers don’t want to contend with the basic fact of most organizations, which is that they’re politically corrupt and dysfunctional, because most people are lazy, greedy, and weak. I’d likewise argue that this is connected to the low levels of acquired social skills in people like software engineers. It’s not a neurological disability for most. They never learn to read cues beyond a subconscious and juvenile level, because they hate what they see, which is that humans are mostly defective and that many are horrible.

Engineers don’t like the concept of the effort thermocline, or of political promotion in general. As much as they can, they’d refuse to have it within their ranks. I’d tend to side with the engineers. Who wouldn’t, from first principles, prefer a meritocracy over a political rat’s nest? The business responds by turning off political promotions for most engineers– while the rest of the organization continues to get them. The result is that, while they start off well in terms of pay and occupational dignity, engineers are being surpassed by the nontechs (who gleefully accept political promotions and feel none the worse for it) by age 30 and, by 40, are undervalued and way underpaid relative to their worth to their companies.

Engineering tracks in organizations are notoriously title-deflating, in comparison to the rest of the business world. Most software engineers would be appalled by how little talent and work ethic are required to become a non-technical VP at even the most esteemed tech companies. Many of these people are lazy (11-to-3 with 90-minute lunches) and just plain dumb. And by dumb, I don’t mean programmer dumb (understands the theory behind neural networks, but has never put one in production) but actual shame-to-the-family, village-idiot stupid. You know how towns in the Midwest used to bus their “defectives” to San Francisco in the mid-20th century? Well, so does the corporate world, and they end up as nontechs and upper management in tech companies.

Conclusion?

Meritocracy is the Prince Charming of the software engineer. It doesn’t exist. It never has, and it never will. Some have asked me to comment on recent HR issues occurring at open-allocation technology companies. The only thing I can say is that, yes, open-allocation companies have serious political issues; but closed-allocation companies have those same issues and more. Open allocation is strictly superior, but not a panacea. When there are people, there is politics. The best an organization can do is to be fair and open about what is going on, and hope to achieve eventual consistency.

Every organization defines itself as a meritocracy, and most engineers (at first, until they are disillusioned with a company) will tend to believe it. They aren’t stupid, so they don’t believe their companies to be perfect in that regard, but they (cluelessly) tend to believe that meritocracy is a core value of the leadership. Almost never is that the case. “We’re a meritocracy” is code for, “don’t question promotions around here”.

The Prince Charming delusion of meritocracy is dangerous because it leads people to make excuses for bad actors. Every company has to lay off or fire people, and frequently these choices are made with imperfect information and under time pressure (one large layoff is less damaging to morale than several small, measured, layoffs) so often the wrong people are let go. A self-aware organization understands this and lets them go gracefully: with severance, outplacement assistance, and positive reference. A delusional “meritocracy” has to cook the books, create psychotic policies that impede internal mobility for everyone, and generate useless process in order to build phony performance cases. In practice, just as many people are let go as in established (and less delusional companies) but their reputations have to be demolished first, with bad project assignments and hilariously disingenuous “performance improvement plans“. Personally, I’d rather see the honest, times-are-tough, layoff than the tech company’s dishonest “low performer initiatives”, much less the permanent (and destructive) rolling layoff of stack ranking.

The biggest casualty, however, of the typical engineer’s head-in-sand attitude toward political promotion is that they never stop happening to everyone else. Engineers just make themselves ineligible. Engineers want promotion to be confirmational (that is, resulting from demonstrated merit) rather than aspirational (that is, based on potential and, therefore, subjective, personal, and political). The problem with this is that, after 10 to 20 years, most engineers haven’t been able to demonstrate even 20% of what they’re capable of. They kept getting crappy projects, were never allowed to finish anything, were rushed to produce work that broke under strain, and their lack of finished accomplishment (due to political forces often not their fault) left them ineligible for promotion to more senior roles, but too old to even pretend in the junior roles (hence, the age discrimination problem). After that gauntlet of false starts and misery, they’re still answering to nontechnical people and executives who had the benefit of aspirational, political promotion. By refusing to play politics and believing in the false god of meritocracy, they deprived themselves of the full spectrum of causes for advancement. Politics, however, went on regardless of whether they believed in it.

This false meritocracy is very clever when it comes to reinventing itself. Few expect a large company like, say, Alcoa or Exxon-Mobil to be a meritocracy. Engineers have figured out, as a group, that “big companies” become political. The response? Startups! Venture capital! The actual result of this has been to replace well-oiled and stable (if inefficient) corporate non-meritocracies with the mean-spirited and psychotic non-meritocracy of the VC-funded ecosystem and the feudalistic reputation economy that the leading investors, through collusion, self-dealing, and note-sharing, have created. The cheerleading of intellectually charismatic figures like Paul Graham and Marc Andreessen has managed to create a sense of meritocracy in that world, but belief in those idols also seems to be waning, and I’m proud to say that I contributed to that loss of faith.

If meritocracy is impossible, what should we do? As individuals, we need to learn to fight for ourselves. It’s not undignified or desperate or “pushy” to look out for our own interests. It’s what everyone else is doing, and we should get on board. As a collective, we need to have honest introspection on what we value and how best to achieve it. Perfect meritocracy within any organization is impossible. It is good to strive for that, but bad to believe it has been achieved anywhere. Eventual consistency and technical excellence are achievable, and we should aim for those.

Before we do anything, though, we need to learn how to fight for ourselves. Bringing frank knowledge to the good, in that fight, is what I’ve been striving to do all along.

VC-istan 8: the Damaso Effect

Padre Damaso, one of the villains of the Filipino national novel, Noli me Tangere, is one of the most detestable literary characters, as a symbol of both colonial arrogance and severe theological incompetence. One of the novel’s remarks about colonialism is that it’s worsened by the specific types of people who implement colonial rule: those who failed in their mother country, and are taking part in a dangerous, isolating, and morally questionable project that is their last hope at acquiring authority. Colonizers tend to be people who have no justification for superior social status left but their national identity. One of the great and probably intractable tensions within the colonization process is that it forces the best (along with the rest) of the conquered society to subordinate to the worst of the conquering society. The total incompetence of the corrupt Spanish friars in Noli is just one example of this.

In 2014, the private-sector technology world is in a state of crisis, and it’s easy to see why. For all our purported progressivism and meritocracy, the reality of our industry is that it’s sliding backward into feudalism. Age discrimination, sexism, and classism are returning, undermining our claims of being a merit-based economy. Thanks to the clubby, collusive nature of venture capital, to secure financing for a new technology business requires tapping into a feudal reputation economy that funds people like Lucas Duplan, while almost no one backs anything truly ambitious. Finally, there’s the pernicious resurgence of location (thanks to VCs’ disinterest in funding anything more than 30 miles away from them) as a career-dominating factor, driving housing prices in the few still-viable metropolitan areas into the stratosphere. In so many ways, American society is going back in time, and private-sector technology is a driving force rather than a counterweight. What the fuck, pray tell, is going on? And how does this relate to the Damaso Effect?

Lawyers and doctors did something, purely out of self-interest, to prevent their work from being commoditized as American culture became increasingly commercial in the late 19th century. They professionalized. They invented ethical rules and processes that allowed them work for businessmen (and the public) without subordinating. How this all works is covered in another essay, but it served a few purposes. First, the profession could maintain standards of education so as to keep membership in the profession as a form of credibility that is independent of managerial or client review. Second, by ensuring a basic credibility (and, much more important, employability) for good-faith members, it enabled professionals to meet ethical obligations (i.e. don’t kill patients) that supersede managerial or corporate authority. Third, it ensured some control over wages, although that was not its entire goal. In fact, the difference between unionization and professionalization seems to be as follows. Unions are employed when the labor is a commodity, but ensure that the commoditization happens in a fair way (without collective bargaining, and in the absence of a society-wide basic income, that never occurs). Unions accept that the labor is a commodity, but demand a fair rate of exchange. Professionalization exists when there is some prevailing reason (usually an ethical one, such as in medicine) to prevent full commoditization. If it seems like I’m whitewashing history here, let me point out that the American Medical Association, to name one example, has done some atrocious things in its history. It originally opposed universal healthcare; it has received some karma, insofar as the inventively mean-spirited U.S. health insurance system has not only commoditized medical services, but done so on terms that are unfavorable to physician and patient both. I don’t mean to say that the professions have always been on the right side of history, because that’s clearly not the case; professionalization is a good idea, often poorly realized.

The ideal behind professionalization is to separate two senses of what it means to “work for” someone: (1) to provide services, versus (2) to subordinate fully. Its goal is to allow a set of highly intelligent, skilled people to deliver services on a fair market without having to subordinate inappropriately (such as providing personal services unrelated to the work, because of the power relationship that exists) as is the norm in mainstream business culture.

As a tribe, software professionals failed in this. We did not professionalize, nor did we unionize. In the Silicon Valley of the 1960s and ’70s, it was probably impossible to see the need for doing so: technologists were fully off the radar of the mainstream business culture, mostly lived on cheap land no one cared about, and had the autonomy to manage themselves and answer to their own. Hewlett-Packard, back in its heyday, was run by engineers, and for the benefit of engineers. Over time, that changed in the Valley. Technologists and mainstream, corporate businessmen were forced to come together. It became a colonial relationship quickly; the technologists, by failing to fight for themselves and their independence, became the conquered tribe.

Now it’s 2014, and the common sentiment is that software engineers are overpaid, entitled crybabies. I demolished this perception here. Mostly, that “software engineers are overpaid” whining is propaganda from those who pay software engineers, and who have a vested interest. It has been joined lately by leftist agitators, angry at the harmful effects of technology wealth in the Bay Area, who have failed thus far to grasp that the housing problem has more to do with $3-million-per-year, 11-to-3 product executives (and their trophy spouses who have nothing to do but fight for the NIMBY regulations that keep housing overpriced) than $120,000-per-year software engineers. There are good software jobs out there (I have one, for now) but, if anything, relative to the negatives of the software industry in general (low autonomy relative to intellectual ability, frequent job changes necessitated by low concern of employers for employee career needs, bad management) the vast majority of software engineers are underpaid. Unless they move into management, their incomes plateau at a level far below the cost of a house in the Bay Area. The truth is that almost none of the economic value created in the recent technology bubble has gone to software engineers or lifelong technologists. Almost all has gone to investors, well-connected do-nothings able to win sinecures from reputable investors and “advisors”, and management. This should surprise no one. Technology professionals and software engineers are, in general, a conquered tribe and the great social resource that is their brains is being mined for someone else’s benefit.

Here’s the Damaso Effect. Where do those Silicon Valley elites come from? I nailed this in this Quora answer. They come from the colonizing power, which is the mainstream business culture. This is the society that favors pedigree over (dangerous, subversive) creativity and true intellect, the one whose narcissism brought back age discrimination and makes sexism so hard to kick, even in software which should, by rights, be a meritocracy. That mainstream business world is the one where Work isn’t about building things or adding value to the world, but purely an avenue through which to dominate others. Ok, now I’ll admit that that’s an uncharitable depiction. In fact, corporate capitalism and its massive companies have solved quite a few problems well. And Wall Street, the capital of that world, is morally quite a bit better than its (execrable) reputation might suggest. It may seem very un-me-like to say this, but there are a lot of intelligent, forward-thinking, very good people in the mainstream business culture (“MBA culture”). However, those are not the ones who get sent to Silicon Valley by our colonial masters. The failures are the ones sent into VC firms and TechCrunch-approved startups to manage nerds. Not only are they the ones who failed out of the MBA culture, but they’re bitter as hell about it, too. MBA school told them that they’d be working on $50-billion private-equity deals and buying Manhattan penthouses, and they’re stuck bossing nerds around in Mountain View. They’re pissed.

Let me bring Zed Shaw in on this. His essay on NYC’s startup scene (and the inability thereof to get off the ground) is brilliant and should be read in full (seriously, go read it and come back to me when you’re done) but the basic point is that, compared to the sums of money that real financiers encounter, startups are puny and meaningless. A couple quotes I’ll pull in:

During the course of our meetings I asked him how much his “small” hedge fund was worth.

He told me:

30 BILLION DOLLARS

That’s right. His little hedge fund was worth more money than thousands of Silicon Valley startups combined on a good day. (Emphasis mine.) He wasn’t being modest either. It was “only” worth 30 billion dollars.

Zed has a strong point. The startup scene has the feeling of academic politics: vicious intrigue, because the stakes are so small. The complete lack of ethics seen in current-day technology executives is also a result of this. It’s the False Poverty Effect. When people feel poor, despite objective privilege and power, they’re more inclined to do unethical things because, goddammit, life owes them a break. That startup CEO whose investor buddies allowed him to pay himself $200,000 per year is probably the poorest person in his Harvard Business School class, and feels deeply inferior to the hedge-fund guys and MD-level bankers he drank with in MBA school.

This also gets into why hedge funds get better people (even, in NYC, for pure programming roles) than technology startups. Venture capitalists give you $5 million and manage you; they pay to manage. Hedge fund investors pay you to manage (their money). As long as you’re delivering returns, they stay out of your hair. It seems obvious that this would push the best business people into high finance, not VC-funded technology.

The lack of high-quality businessmen in the VC-funded tech scene hurts all of us. For all my railing against that ecosystem, I’d consider doing a technology startup (as a founder) if I could find a business co-founder who was genuinely at my level. For founders, it’s got to be code (tech co-founder) or contacts (business co-founder) and I bring the code. At my current age and level of development, I’m a Tech 8. A typical graduate from Harvard Business School might be a Biz 5. (I’m a harsh grader, that’s why I gave myself an 8.) Biz 6 means that a person comes with connections to partners at top VC firms and resources (namely, funding) in hand. The Biz 7′s go skiing at Tahoe with the top kingmakers in the Valley, and count a billionaire or two in their social circle. If I were to take a business co-founder (noting that he’d become CEO and my boss) I’d be inclined to hold out for an 8 or 9, but (at least, in New York) I never seemed to meet Biz 8′s or 9′s in VC-funded technology, and I think I’ve got a grasp on why. Business 8′s just aren’t interested in asking some 33-year-old California man-child for a piddling few million bucks (that comes along with nasty strings, like counterproductive upper management). They have better options. To the Business 8+ out there, whatever the VCs are doing in Silicon Valley is a miserable sideshow.

It’s actually weird and jarring to see how bad the “dating scene”, in the startup world, is between technical and business people. Lifelong technologists, who are deeply passionate about building great technology, don’t have many places elsewhere to go. So a lot of the Tech 9s and 10s stick around, while their business counterparts leave and a Biz 7 is the darling at the ball. I’m not a fan of Peter Shih, but I must thank him for giving us the term “49ers” (4′s who act like 9′s). The “soft” side, the business world of investors and well-connected people who think their modest connections deserve to trade at an exorbitant price against your talent, is full of 49ers– because Business 9′s know to go nowhere near the piddling stakes of the VC-funded world. Like a Midwestern town bussing its criminal element to San Francisco (yes, that actually happened) the mainstream business culture sends its worst and its failures into the VC-funded tech. Have an MBA, but not smart enough for statistical arbitrage? Your lack of mathematical intelligence means you must have “soft skills” and be a whiz at evaluating companies; Sand Hill Road is hiring!

The venture-funded startup world, then, has the best of one world (passionate lifelong technologists) answering to the people who failed out of their mother country: mainstream corporate culture.

The question is: what should be done about this? Is there a solution? Since the Tech 8′s and 9′s and 10′s can’t find appropriate matches in the VC-funded world (and, for their part, most Tech 8+ go into hedge funds or large companies– not bad places, but far away from new-business formation– by their mid-30s) where ought they to go? Is there a more natural home for Tech 8+? What might it look like? The answer is surprising, but it’s the mid-risk / mid-growth business that venture capitalists have been decrying for years as “lifestyle businesses”. The natural home of the top-tier technologist is not in the flash-in-the-pan world of VC, but the get-rich-slowly world of steady, 20 to 40 percent per year growth due to technical enhancement (not rapid personnel growth and creepy publicity plays, as the VCs prefer).

Is there a way to reliably institutionalize that mid-risk / mid-growth space, that currently must resort (“bootstrapping”) to personal savings (a scarce resource, given that engineers are systematically underpaid) just as venture capital has done to the high-risk /get-big-or-die region of the risk/growth spectrum? Can it be done with a K-strategic emphasis that forges high-quality businesses in addition to high-value ones? Well, the answer to that one is: I’m not sure. I think so. It’s certainly worth trying out. Doing so would be good for technology, good for the world, and quite possibly very lucrative. The real birth of the future is going to come from a fleet of a few thousand highly autonomous “lifestyle” businesses– and not from VC-managed get-huge-or-die gambits.

VC-istan 7: solving the wrong problem

I’ve written at length about VC-istan, its poor performance and its bigotries. What, however, is VC-istan’s “original sin”? Why is it so dysfunctional? Is there a foundational reason for its pattern of across-the-board moral and financial failure? I think the answer is obviously, “yes”. There’s a simple root cause: it’s solving the wrong problem. This requires two investigations: what problem should venture capital be solving, and what is it actually doing?

What’s the purpose of venture capital?

This is an easy one. The purpose of venture capital is to finance endeavors that require substantial backing in an all-or-nothing transaction. A biotechnology firm that requires $100 million to develop, and put into clinical trial, a new drug or device would be one example of this. With $10 million, it produces nothing salable; with ten times that, it has a chance. Others exist around infrastructure and in more deeply industrial pursuits like clean energy. Venture capitalists do invest in these spaces, and that’s outside of what I’d call “VC-istan”. Not everything that venture capitalists do is ugly, of course, and not all of it is VC-istan.

Venture capital, in a way, was originally intended as the “capital of last resort” for high-risk, capital-intensive businesses that would never qualify for more traditional financing. Why? Because when the proper way to invest is all-or-nothing, that has (unavoidable) negative consequences for all sides. It means that most people won’t get funded, and it’ll be extremely competitive to get capital, and dilution of founder equity will be severe. It’s not ideal, but if all you have is an idea, your product is 3 years away from the market in the best-case scenario, and you’re asking for $125 million to get started, those are the terms you have to take. It is, of course, quite a noisy process. The best ideas might not get funded, because there is literally no one able to assess what the best ideas are.

Venture capital for biotechnology and infrastructure has its own rules and culture. I’m not an expert on that, but it’s not what I consider “VC-istan”. From my perspective, which may be limited, venture capitalists in that space are trying to act in good faith and invest in viable businesses. To be blunt, I don’t think the “cool kids” nonsense (see: TechCrunch) matters so much in those sectors, because the science has to be sound. If you’re trying to turn algae into diesel fuel, Mike Arrington’s half-baked opinion of you matters a billion times less than the chemistry inside your lab.

What problem is VC-istan solving?

VC-istan is a subset of “all venture capital”, and focused on the “hip” stuff that can be likened to “reality TV”. To explain this analogy, ask this question: why are “reality TV” shows so prolific? It’s not about their quality. Cheap production is often cited, but it’s not just amount the numbers in the accounting ledger. The reality show formula is one that admits perfect commoditization. Writers and actors, at high levels of talent, resist commoditization. They won’t work on shows that are bad for their careers, and they have agents whose full-time job is to represent their interests. This makes them non-fungible. At the highest level of talent, labor may be able to push back against commoditization of itself, because there are few enough people at the highest levels to make the market discrete rather than continuous– or, in other words, illiquid. Reality TV does away with those “prima donna” creatives and celebrities: the writing demands are minimal and can be fulfilled with a mediocre staff, and the actors are nonentities. This enables the production studio to iterate quickly with half-baked concepts without needing to concern itself with the career needs of the parties involved.

VC-istan loves social media, and consumer-web marketing experiments, which are like reality television in that they can be produced with mediocre, “commodity-grade” inputs. To launch a biotech firm, one actually needs to have a strong grounding in science. Assessing founders for scientific literacy is hard, and private equity people are rarely up to the task. But any idiot can come up with, and hire someone good enough to implement, a Snapchat or a Clinkle. In the soft space of marketing experiments using technology, as opposed to the much harder sector that is technology proper, commodity founders and engineers suffice, and because the space is a gigantic bike shed, every investor feels entitled to have strong opinions. If genuine technical talent is needed for “scaling” down the road, it can be hired once the company has been covered by TechCrunch and appears legitimate.

Ultimately, the purpose of VC-istan’s “tech” companies is not to innovate or to solve hard problems. It’s to flip teams that have been validated by three to six rounds of venture funding, and possibly by success in the market (but that’s optional). Occasionally there’s an IPO, but those are about as common as big-company spinoffs. More typical is the “acqui-hire”, whose purpose can only be understood in the broader context of corporate dysfunction.

M&A has replaced R&D

A company’s need for top talent tends to be intermittent or subtle, and most often both. An example of the first (intermittent need) is around a short-term crisis that only a small percentage of people will have the insight, creativity, experience, or work ethic that is necessary to surmount it. The second pertains to the long-term existential need for innovation; if the company doesn’t have some engine that produces an occasional positive-impact black swan, it will be torn to shreds by the bad kind: crises that no amount of talent or effort can resolve. While every company pays lip service to its need for top talent, the truth is that most companies don’t need top talent for their day-to-day operations. If they did, that would be irresponsible design: a dependency on something that is somewhere between a highly volatile commodity and an outright non-commodity. The need for top talent tends to be a long-term issue.

Top talent is difficult to truly employ; one merely sponsors it. Old-style corporations understood that and invested in R&D. When the rare crisis that was truly existential would emerge, talent could be borrowed from the R&D pool. Additionally, while R&D could focus on basic research that was of general benefit to the world, and not necessarily in the firm’s immediate, parochial interests, the proximity to that research the corporation enjoyed gave it such an edge in practical innovation to pay for itself several times over.

Unfortunately, basic research was one of the first casualties of the private equity invasion that began in the 1980s. The old R&D labs that built C, Unix, Smalltalk and the internet weren’t scrapped outright, but reduced to a fraction of their former size, and forced to take a next-quarter focus. Conditions weren’t actually made so bad as to flush existing talent out, but positions became scarce enough that new talent couldn’t get in. The executives of those companies weren’t all short-sighted idiots, though. They knew that the high-autonomy, R&D-oriented work was the only thing keeping top talent in place. With corporate R&D near obliteration, that was threatened. So they knew they needed a solution that talent-intake problem. What did private iniquity propose as a solution? More private equity.

Enter venture capital, formerly a subsector of private equity that was generally avoided by those with other career options, due to its infuriatingly intermittent performance. What would it mean, however, if venture capital could be made less “venture”, by filling a need created by the disintegration of another part of the economy? Companies shutting down the blue-sky, high-autonomy R&D work had to get talent somehow. Explicitly paying for it proved to be too expensive, except in investment banking, due to hedonic adaptation– people who are performing at a high level, if their needs for autonomy are not met, require 25-50% per year raises to be content. Tapping high-talent people for managerial ranks proved fruitless as well, because many of these people (while exceptional as individual contributors) had neither the desire nor the ability to manage (and, additionally, middle-management positions were also cut during the private equity invasion). The remaining solution to the talent problem became one that private equity men found extremely attractive, given the premium they collect on deals– companies must buy it.

I don’t intend to insult the low-level employees of the Googles and Yahoos of the world by saying that those companies have “no talent” at the bottom. That’s clearly untrue. Companies don’t acqui-hire (which is far more expensive than internal promotion) because they have no top talent in their ranks. They have plenty, but they acqui-hire because they have lost the ability to discover what they have. It’s a malfunction of the middle-management layer. These companies are like hoarders that buy new coats every winter not for a lack of coats, but because their houses are so out of order that a new purchase is preferable to sorting the old place out.

Moreover, a company cannot, in general, adequately commoditize its highest levels of talent. The best will always seek their own career goals foremost, and perform at their highest only when there is coherency between their long-term personal goals and the work assigned to them. There are also, to put it bluntly, not enough such people to merit any explicit managerial correction to this problem. An executive focused on the career-coherency issues coming out of the most talented 5% is ignoring the day-to-day work completed by the other 95%. Two (problematic) solutions end up emerging. The first is for the company to ignore the high-talent problem and treat its top 5% like everyone else: closed allocation, low autonomy, etc. Then it loses them, plain and simple, and becomes dysfunctional after a few years of brain drain. The second is to leave them alone and effectively let them work on whatever they want. That’s great, in the short term, but it can be politically messy; others (who may belong in the top 5%, but haven’t been recognized) may resent them for their higher level of autonomy, or that set of people may lose sight of their need to continually market themselves and justify their favorable conditions, and then be crushed (not for a lack of value to the organization, but because it fails to market itself) when there is a management or market change.

So what is the “problem” that VC-istan exists to solve? It’s there to commoditize top talent. Although a specific company cannot commoditize its top 5%, the conceit is that an army of dedicated specialists– a mix of investors, corporate biz-dev executives, and “tech press”– can do so. In the consumer web space, venture capitalists have become a sort of high-end headhunter, but one that follows different rules.

For one major difference between the old corporate ladder and the acqui-hire system, employers are not allowed to explicitly discriminate on age, pregnancy status, health issues, race or gender. Investors can. Middle managers are too busy to conduct invasive “back channel” reference checks that, in truth, constitute civil harassment and would admit blacklisting charges if they ever interfered with employment (thus, risk-averse companies prefer not to do so). Investors can do so, and in such a way as to work through people who will keep their secrets (preventing lawsuits). This is a wet dream of the new right wing, an Uberization of executive hiring. The old system, with decades of regulation thrown into it because those rules were actually necessary, has been supplanted by a premium, rule-breaking, and vicious new one. The people who need the regulations imposed by the old system (i.e. women, minorities, people with health problems, people over 40, people with kids) are simply judged unfit to compete.

Here’s a question: how well is VC-istan actually doing, on its own terms? The first question is: what does it mean to “commoditize top talent”? While that sounds like something I might be against, I can’t actually say it’s a bad thing– not even for top-talent people. When something is commoditized, a fair price (that may fluctuate, but is fair relative to published market conditions) is established and it’s very easy to buy or sell it near that price. Currently, the compensation for top (2.0-level) engineering talent swings between about $75,000 and $10+ million per year– there is a clear uncertainty about what it is worth– with a median around $150,000. If that level of talent were adequately and fairly commoditized, that range would be more like $300,000 to $500,000– which would give most of them a hefty pay bump. The truth about the commoditization of labor is that labor generally finds it unobjectionable when the terms are fair. In fact, one effect of labor unions is to explicitly commoditize labor while attempting to ensure fairness (while professions, in general, oppose the commoditization regardless of terms). The murky issue in technology is that “top talent” is very hard to detect, because the people with the requisite skill have better things to do. Those who can, do; those who can’t, evaluate others’ work.

VC-istan, then, is built on the record-company model. Founders and engineers are treated as commodities (and generally, for reasons I won’t get into here, don’t get fair terms) but there is a hope that, thanks to the law of large numbers, top talent will be detected and validated by the outside market.

Where VC-istan went wrong is that it never figured out what top talent might look like, so the resources were thrown behind those who were either best at self-promotion or (increasingly, over time) those who could pull inherited connections. As a mechanism for detecting the rising generation’s top marketing talent, it might not be doing so bad. For picking out the best technical talent, especially as pertains to long-term R&D, it’s worse than abysmal. It’s doubtful that it’s picking up any signal at all. Companies that have a genuine need for R&D talent will be poorly served if they source it through acqui-hires.

VC-istan exists to commoditize top talent, but it has also erected a feudalistic reputation economy in which investors hold the cards. Founders hold few, engineers hold none. The highest levels of technical talent have been rendered, by this new economy, effectively irrelevant, depriving it of any leverage whatsoever. So, the terms are made bad– so bad that top engineering talent is rarely delivered. Whether this will strip VC-istan of credibility in the long run is something that remains to be seen.

The point I’ve made here is that it’s “solving” an ugly problem in a bad way.

What can venture capital do for technology?

Venture capital’s purpose is to build companies that, if successful, will become massive corporate behemoths. On a fundamental level, it’s stuck in the 20th-century mentality where a gigantic organization is the only acceptable prize for winning. Startup life is sold (by founders, and rarely by investors directly) to talented, usually clueless, engineers as an antidote to the ills of “MegaCorp” when, in truth, the explicit purpose of the VC-funded startup is to become exactly that: a new MegaCorp, but usually with crappier health benefits, longer hours, and faster firing.

What the best engineers actually tend to want is high autonomy so they can deliver exceptional work. They’d prefer ownership over it, all else being equal, but as long as they’re fairly compensated, they’re generally happy whether they work for a 20,000-person company or for themselves. When corporate R&D was sold for parts, venture-funded startups were proposed as the solution, the new way forward. Don’t like what happened to your old job? Create a new job for yourself! The lie here is that founding a VC-funded company provides the autonomy associated with true ownership. In truth, venture capitalists become full owners (de facto, if not de jure, due to the power afforded them by VC’s feudal reputation economy) of the company even when they hold a minority stake. Working for VCs is not fundamentally better than working for a boss; in many ways, it’s worse because the social distance is greater. Most bosses don’t consider themselves inherently superior based on favorable birth;

There are several critical misses that have become evident as venture capital has attempted to replace more traditional venues for innovation. One is that it has proven not to be a valid replacement for internal R&D. Nothing that VC-istan has coughed up is anywhere near the order of magnitude of Bell Labs or Microsoft Research. The second is that it has failed to be an engine of small-business generation, which is necessary for economic growth. It hasn’t connected top talent with the autonomy that comes from ownership. Rather, it has abandoned top talent in the pursuit of commodity startups run by commodity founders and commodity engineers. Over time, one might hope top talent to abandon it. That trend seems to be emerging, but I have no idea when or how (or, even at this stage, if) it will mature.

There is a fundamental technical flaw with VC-istan, additionally. That I’ll focus on, because it might lead us in the direction of a solution. If we consider the risk/reward profile of businesses, we see an underserved middle of the spectrum. Low-risk businesses can take bank loans, but those require personal liability, so it’s not wise to use them for anything that might actually fail. High-risk gambits with above-90% chances of failure, but that are capable of returning 20-50x on success, are what VCs love. The mid-risk/mid-growth space– targeting 15 to 50% annual growth, with a low but nonzero chance of business failure– is inappropriate for bank loans (too risky) but unpalatable to venture capitalists (not risky enough). Unfortunately, I don’t see an easy fix for that. Venture capital could become very profitable by funding the 15-50% range, but investment decisions aren’t driven by profits so much as the career needs of the investors. Returning a steady profit (say, 25% per year, with a bit of variance) by investing in a number of solid but moderately-sized businesses is not career-making; having been in on Facebook (even as a minor and late investor) is. The name-dropping world of Sand Hill Road cannot be expected to change, and if it does not, the focus will be less on building quality businesses and more on taking insane risks in the hope of hitting a career-making blockbuster.

This is problematic because the mid-growth/mid-risk space is exactly where true technologists live. They do not become machine learning experts or compiler gurus in an overnight episode of “virality”, and whether Mike Arrington or Paul Graham owes them a favor is irrelevant to whether they can actually code. They get good (and, if possible, rich) slowly. In terms of abstract value-added capacity, 15 to 50% per year seems to be about the natural rate (although most engineers would be thrilled to have salary increases at even half that rate). Technologists are extremely good at delivering these 20 and 40 percent per year improvements. What lies outside their interest (and, usually, their ability) is engineering the social conditions that admit 100x “viral” growth (or, far more often, abysmal failure). It’s just not where they live; they weren’t born in casinos.

The future

VC-istan is not about to die, any more than recording labels have ceased to exist. As a method of shaving 15 years off a rich kid’s corporate-ladder climb via “acqui-hire”, it will persist. As a machine that produces commodity startups run by commodity entrepreneurs, it will persist and probably be profitable for quite some time. As a way of enabling companies to discriminate on age, health, pregnancy status, and other illegal factors at upper levels (filled through acqui-hires, while rendering internal promotion rare) while keeping the discrimination off their books, it will hold that niche quite well. How relevant will VC-istan remain to true top talent? On that one, VC-istan’s lifespan may be limited. In that territory, it’s “ripe for disruption”.

So what shall be built to bring the disruption?

Software engineer salaries aren’t inflated– at least, not for the 99%

It’s autumn 2013, and there’s a lot of discussion around the current bubble (now obviously one) in the VC-funded technology world and how it will end. Business Insider acknowledges that a bubble exists, but gets some crucial details wrong. Let’s talk about one that most of us actually care about. Business Insider claims: “It’s not just tech asset prices that are high. Salaries are high, too.” Them’s fighting words. Is it true? Well, sort-of. Here’s the evidence, from tech recruiter Matt Allen:

Instead, we’re seeing sign-on bonuses for individuals five-years out of school in the $60,000 range. Candidates queuing-up six, eight or more offers and haggling over a few thousand-dollar differences among the offers. Engineers accepting offers and then fifteen minutes before they’re supposed to start on a Monday, emailing (not calling) to explain they found something better elsewhere.

Ok, let’s dissect this. One: a few people (and it’s not clear that they’re engineers) are getting huge signing bonuses. $60,000 isn’t a number to sneeze at, but it’s not that extreme. Management-level hires typically get signing/relocation bonuses that cover the cost of a cross-country move (easily over $20,000, for people with families) and there’s no reason software engineers shouldn’t get the same. Additionally, signing bonuses usually have clawback provisions if the employee leaves (even involuntarily) in the first year, penalizing the job-hopping for which the worst of our generation is known. Given the tax penalty associated with receiving a bonus and risking having to pay it back, I’m not sure I’d want a $60,000 bonus under typical terms. Two: some candidates are queuing up 6 to 8 job offers. I call bullshit on that one, if only because of the scheduling difficulties in a startup ecosystem where 7-day exploding offers are the norm. I’m sure there are people getting 6-8 offers in the course of an entire job search (I’ve had that) and that people are claiming to have portfolios of excellent offers in negotiation, but the logistics of getting 6 active, credible job offers at one time are unfavorable, to say the least. Three: people are being unprofessional dickbags, pulling out of accepted offers on their start date. I’m sure that that is happening, but how is an occasional episode in which a privileged young hotshot acts like a jackass newsworthy, much less the sign of a bubble? It’s not.

Managers and product executives are making a killing in the present-day startup economy, no doubt, and some of those people might be able to pass as programmers due to some PHP scripts they wrote in their teens, but when one actually studies the contemporary startup economy, there are not a lot of software engineers making over $200,000 per year outside of finance– and those who are tend to be either very good or unusually lucky. For a VC-funded startup to offer $200,000 to an engineer would be incredibly rare, even in the Bay Area, and equity allotments after VCs are involved are notoriously stingy.

Twenty years ago, when startups were underdogs almost by definition, the scene had a “Revenge of the Nerds” feel. A bunch of ragtag computer aficionados, typically from middle-class backgrounds and far away from the East Coast’s financial and corporate elite, were showing up the old guard. New, powerful technologies were being developed, and power shifted (temporarily, perhaps) to those few who understood them at a deep level. There was slight subversion of the 1%; they weren’t destroyed or even harmed, but they were visibly outperformed. For a contrast, the hot properties of the current VC-funded world almost entirely come from the 1%. Behind almost every single one of the VC darlings, there’s a series of strings pulled by powerful people repaying favors to the rich daddies of the founders. There’s no meritocracy in it. It’s not a challenge to the established and rich; it’s a sideshow for the supercapitalists. In a surprising reversal, the old-style corporate world (and the enterprise companies existing and being formed to serve their needs) has a much more middle-class culture, because the current-day rich find it boring.

Software engineer salaries in the VC-funded world are not especially low (nor are they high). They’re usually 75 to 95 percent of what more typical employers are offering. Equity distributions, on the other hand, are extremely lopsided. I worked for a company once where the board refused to allow more than 0.04% to go to an engineer. (Why? Because fuck the people doing the work, that’s why.) There’s something that needs to be discussed here, because it applies to the age-old question of why people who do actual work are modestly compensated, while vacuous celebrity types take the lion’s share. It’s the Teacher-Executive Problem.

The Teacher-Executive Problem

As a society, we need teachers, police officers, park rangers, and other such people who are modestly compensated. We don’t need celebrities, business executives, or professional athletes. I’m not going to argue that the latter are overpaid, insofar as it’s difficult and competitive to get to the top ranks in any field. That would be a subjective argument; all I intend to say is that, objectively, the need for the latter class of labor is smaller. If we didn’t have teachers or police, society would fall apart. If we didn’t have corporate executives, companies would find other ways to survive. So why are the more necessary people paid less? Because being necessary means that more workers will be drawn into the field, and that limits individual compensation. We probably pay more, as a society, for teachers and police than we do for corporate executives (as we should) but the individual slices for the larger, more necessary, job categories are smaller.

We have 3 million teachers in the US, and we need that large number of them, because individual attention per student is important. The functioning of society would be greatly impaired if that number dropped to 2 or 1 million. One might argue that competent teachers are “worth” $200,000 (or much more) per year– and I’d say that the best are worth several times that– but can society afford to pay that much for teaching? Three million $200,000 paychecks is a $600-billion annual liability. Taxes would go up substantially– in a time when the base of political power is (unfortunately) divided between a structurally disadvantaged (read: mostly fucked) emerging-adult cohort and retiring Boomers whose children are out of school– and society would likely determine that $200,000 annual paychecks for teachers “can’t be afforded” (especially given the claim that “they get off work at 3:00″). $200,000 isn’t a large amount of money for a single person, but for people who are actually needed in significant numbers, the multiplier of 3 million makes it seem unacceptable. (I am not arguing that teachers don’t deserve $200,000 salaries; only that it would be politically impossible to get them there.)

For a contrast, the social need for corporate executives (excluding entrepreneurs) is pretty minimal, and society recognizes this in a rational way: there aren’t a large number of slots: title inflation aside, there might be ten thousand truly executive roles in powerful companies. However, when the number of people performing a job is low, gigantic salaries (if those people control the distribution of resources) become socially affordable. Three million somewhat high salaries is a problem, ten thousand enormous ones is not. This is paradoxical because the middle-class conceit is that the way to become wealthy is to make oneself valuable (or, better yet, necessary) to society. What the Teacher-Executive Problem shows us is that there’s more potential for outlier compensation in doing things that aren’t necessary, because asking for more compensation doesn’t carry the implicit multiplier based on the size of the labor base. Society “can’t afford” to pay the 3 million teachers such high salaries, but it can afford the huge salaries of corporate executives, and the $850-million acquisition that enriches the top executives of IUsedThisToilet.com.

Why do so few software engineers get a fair shake in the VC-funded world? They’re on the wrong side of the Teacher-Executive Problem. They’re actually necessary. They’re required in order for technology firms to function.

What about 10X?

The generally accepted consensus (even among software engineers) is that average programmers aren’t very valuable. They write all that buggy, hideous legacy code. There’s little that software engineers and business executives agree upon, but the low status of the average programmer is probably not a point of disagreement. I don’t care to speculate on what the “average” software engineer is like, because while I have seen a ton of incompetents (and a smaller number of good engineers) out there in the world, I don’t have a representative sample. I also think that most of the engineering incompetence comes not from a lack of ability, but from an anti-intellectual culture originating in business culture at large, as well as nonexistent mentoring, so it’s not programmers who are mostly at fault. However, I will agree readily that the bulk of software engineers don’t deserve high ($200,000) salaries. They might have the talent, but few have that level of skill.

However, there is the concept of the “10x” software engineer, one who is 10 times as productive as an average engineer. It reflects a truth of software engineering, which is that excellence and peak productivity are tens to hundreds of times more powerful than the average-case output. (In fact, often that ratio is infinite because there are problems that require top talent to solve it.) Moreover, groups of engineers often scale poorly, so a team of 10 isn’t really (most of the time) 10 times as productive, but maybe 2 or 3 times as strong, as an individual. So it’s not surprising that a great engineer would be 10 times as valuable. The degree to which “10x” is real depends on the type of work, the context, project-person fit, and the competence of the engineer. It’s highly context-dependent, it’s not always the same people, and it’s quite unpredictable. The national average salary for a software engineer is about $90,000. The 10x-ers are not earning 10 times that and, to be honest about it, they probably shouldn’t. You can’t know, when hiring someone, whether the context that supports 10x output for that person is going to exist in the role. The bona fide 10x engineers typically earn 1.5 to 2 times that amount ($135,000 to $180,000) in the U.S. I’m not going to argue that they’re underpaid at this level– although, at least in comparison to MBA graduates earning twice that before age 30, I think they clearly are– but they’re far from overpaid at that level.

Why don’t 10x engineers get paid astronomical sums? For a large part, I think it’s because of the context-dependent nature of “10x”. It doesn’t require only a good engineer, but a good engineer connected with the right kind of work. Companies can’t afford (obviously) to pay $900,000 salaries to senior engineers just on the hunch that those (typically highly specialized) talents will find a use. When engineers do find environments in which they can deliver 10x output, they’re happy– and they’re not liable to clamor for huge raises, or to move quickly and risk starting over in a defective environment. This isn’t especially wrong; engineers would rather have interesting work at “merely” 1.5x salaries than risk happiness and growth for a chance at more. It’s just worth pointing out to establish that, in general, software engineers (and especially the most capable ones) are not overpaid. Moreover, the people commanding $500,000+ salaries in technology are typically not real engineers, but managers who might occasionally “drop down” and hack on one of the sexier projects to keep their skills sharp. Finally, the few (very few) software engineers making that kind of money are generally worth it: we’re not talking about top-1% talent at that level, but top-0.05% (and a level almost never achieved before age 40). There are plenty of people drawing undeserved high salaries in the Valley, but they aren’t the ones writing code.

Why must I point this out?

This (bubble) too, shall pass. The era when a well-connected rich kid can raise a $1-million seed round rather than eating his lumps in an investment bank’s analyst program will end. I don’t think that that’s a controversial assumption. Timing the end of a bubble is nearly impossible, and I don’t think anyone has shown reliability in that particular skill; but predicting that it will die off is trivial– they always do. When this happens, there will be a lot of job losses and belt-tightening. There always is. It’ll get ugly, and that’s fine. Most of these businesses getting funded and acquired don’t deserve to exist, and the economy will inevitably purge them. What I don’t want to see is the bubble made into an argument against the middle-class, hard-working software engineers. When the bubble ends, there will be losses to eat and austerity to go around, and it ought to go right to the people who reaped the benefits while the bubble existed. The end of the bubble should not be used to reduce the compensation of software engineers as a whole, whose pay is currently (I would argue) not quite unfair, but on the low side of the fair interval.

For the 99 percent, there is no software engineer salary bubble.