It’s time to stop hating and start learning

There’s a strong resentment that software engineers and technologists have toward “non-technical” people or “business people”. We compare our situations, and our impulse is to find the arrangement unjust. Based on fundamentals, things are a bit off. We technologists work harder, tend to bring more in terms of natural ability, and yet “they” (meaning executives) make three or ten or fifty times as much, for considerably less work, and they have a lot more job and career security. We learn new programming languages and tools (as our jobs, constantly evolving, require) on our own time, and sometimes sacrifice weekends to tackle production issues. They, on the other hand, can work 11-to-4, because their bosses care more about in-group cohesion than what gets done. We get pulled into meetings that we can’t say “no” to. They call those meetings. We live in fear of “Agile” processes being imposed in order to micromanage us. They decide how we are managed, and what we work on. We have little say in how our performance is evaluated. They define their jobs and, unless their assertions are utterly unreasonable, write their own performance reviews, easily justifying raises. We start struggling to get jobs at age 40. They hit their peak in their late 50s. It does seem unfair.

Why are “they” winning? It’s a lesson in collective intelligence. It really matters. We beat them, pound for pound, on individual intelligence– but that doesn’t matter. Often, we use our individual gifts to compete, stupidly, with each other. We need to stop being resentful of “business types”, and start learning from their superior collective intelligence. What are they doing that we aren’t? We’re working harder and we are smarter, individually speaking, but they are working smarter, which is what actually matters.

Look at a typical Harvard or Stanford business school class. Some of the students are brilliant, many are above-average, and some are depressingly mediocre. Perhaps 7 to 10 percent of them have the natural ability to do what a strong programmer can do, and yet almost all of us could do the 6- and 7-figure jobs that they move into. They’re not idiots by any stretch of the imagination, but only a few would be notably sharp by a capable programmer’s standard; some would be our peers, very few would be superior. In spite of this, because they pool resources and connections, they move along to high-status jobs earning $300,000 per year or more, and a number become CEOs earning an order of magnitude more. The ones who become the venture capitalists who run technology are, in fact, the runners-up; they’re the ones who didn’t get into stat-arb funds or private-equity firms or Fortune 500 CEO positions.

I’m not pointing this disparity out for the sake of denigrating anyone. In fact, it should become clear that I intend the opposite: it’s impressive what they’ve achieved as a collective, and we should learn from it. What is it about these people that has them outperforming us, as programmers and technologists, in terms of prestige, economic wealth, and general respect? Can you even imagine a Harvard MBA justifying his own working time in terms of two-week “sprints” or “iterations”? Of course not. It’s inconceivable. So what did they do right that we did wrong?

Rather than getting angry about their superior status and compensation, and our inferior lot, we might do better to observe what these people have done in order to get where they are. It seems to be a lesson in group power and intelligence. Politically speaking, a group of people who protect and assist each other, without question, will always do better than free agents acting alone, no matter how talented the latter group may be.

Like anyone else, I have my bitter moments. With my intelligence as well as the pedigree that would come from, say, Harvard Business School, I’d have an invincible combination. So perhaps I should have done that, instead, during the years that I spent goofing around in startups and tech companies. Had I taken that other route, I’d probably have my own company and be calling shots, instead of spending so much of my career in reaction to others’ suboptimal decisions.

I didn’t, and at 32, it’s probably too late to take that route. (An MBA doesn’t open the same doors when you’re old, and I’m a non-drinker.) When I was the right age for it, I was too idealistic. I spent my mid- and late 20s in a startup, and then Google, and then another startup, because “technology is the future”. That is true, of course. Technology is the future (or, at least, an integral part of it). However, an MBA gives you a future. As for 10 or 20 years in technology… I’m not so sure. With “Agile” and open-plan offices (both designed around the rapid hiring and facile control of cheap, fungible, talentless engineers) becoming the norm, I don’t think that expertise or technical excellence are valued anymore. I’d love to see this change, but it looks like an uphill battle.

To a large degree, though, I’m committed to the technical vision. As lucrative as it could be, were it still an option, I don’t want to be crashing third-world currencies for sport. I want to do something useful and important; that matters more to me than the difference between “comfortable” and “rich”. So, the fact that I’ve “lost” those 7-figure, world-hurting opportunities is hardly something to be upset about. What I don’t want to lose is the battle for the soul of technology.

We’ve let this industry be invaded by the worst kinds of psychopaths. Programming used to be an R&D job; now we have to deal with two-week “sprints” and “user stories” and, worst of all, the sorts of mediocre, mindlessly obedient colleagues who will put up with that nonsense (and for whom such processes are designed). That brings discomfort and humiliation as well as financial mediocrity. Writ large, it also makes us an underperforming industry. Innovation isn’t possible under “Scrum”.

The picture is bleak, but it doesn’t mean that we should abandon technology. It’s fucking important. It’s worth fighting for. What we do, as programmers, isn’t just the production of software. We solve problems. We throw ourselves, day after day, into a kind of work that requires precision of thought (a trait that not many people have developed). That makes us the lifeblood of an advanced economy. The status of our industry, therefore, is worth getting right. We ought to fix it.

The extremity to which we take our tribalism, however, does us no favors. We’ll have to work with people who aren’t like us in order to get anywhere. Don’t get me wrong: the upper ranks of our industry have been invaded by narcissists and psychopaths, and I want to drive every single one of them out. What I don’t like about us is our tendency to downgrade non-technical accomplishments. Our tribal antipathy toward “business people” is one example of this. I’ll admit that I’ve been guilty of this in the past. I’ve said a lot of negative things about “business guys” before but, as I get older, I recognize that there are a lot of intelligent, well-intended people who’ve dedicated their careers to solving business problems instead of programming ones. Pretending that we’re better, just because we can write code, does us no good. It only deprives us of possible allies, because we’ll learn a lot more and get farther if we can work with other people who have the skills we lack. Worse yet are the idiotic sectarian divisions we’ve let form within the programming community: “Agile” versus “enterprise”, tabs versus spaces, Ruby versus Java, women versus “brogrammers”, young versus old, Bay Area versus “the sticks”. We can’t afford this. These divisions make us weak and manipulable. Many of them don’t make any sense. The quarreling turns us into a low-status group of people. Instead, we need to unite around our shared interests.

Why do corporate executives get so much more respect, autonomy, and money than us, despite working significantly less than we do? Because they don’t let people divide them. It’s that simple. They protect each other, and this makes them collectively strong. A graduate of Harvard Business School will place his least favorite classmate in an executive role, for the good of the group; for a contrast, on our side, I still get shit on for stupid things that I said at Google, 5 years hence.

A friend of mine worked at a large bank with “360-degree” performance reviews. Ratings were on a 9-point scale with 5 as the intended average. The lowest-rated group was technology, where the average person got about a 6.5. The salespeople and executives and business analysts all gave each other high ratings, regardless of whether they thought each other “deserved” high scores, and so all of the “soft skills” departments came in with averages above 8.5. The programmers gave each other “honest” ratings, which shows collective stupidity, because it showed a willingness to rat each other out to management. The business people had the collective and political intelligence not to put bad scores, even when they were deserved, on the record. In most of the bank, people wrote reviews for themselves and had them signed by peers. There was a mutually beneficial (i.e. positive-sum) trade of credibility going on. Business people are smart enough to give criticism verbally, off the record, and to give glowing reviews when on the record. They recognize that preserving the credibility of the group is one of the most important things that they can do at work. Not so for programmers.

I can imagine that some programmers will read this and say that it proves that we are the honest ones, and that “business people” are corrupt in some way. I disagree completely with that viewpoint. They played politics effectively, and the programmers didn’t. End of story. The business people (traders, investment bankers, salespeople) saw that the executives were trying to collect data that could be used against them, and so they jammed the signal in order to minimize the probability of harm coming to anyone. The programmers played politics poorly, because they couldn’t keep their mouths shut, and probably got hit harder in the next round of layoffs.

This kind of “honesty”, shown by the programmers, I find to be non-virtuous. While I dislike deception, I also think that many forms of “honesty” are harmful, serving only to cause needless drama and friction. If I find a stranger unattractive, I am going to hide that reaction, because I’m not a jerk. What good would it do, to voice that opinion? None at all. Not all information deserves to be shared and some information should be concealed, even if using half-truths or (on occasion) lies. Likewise, I’m not going to sell out someone I perceive as a low performer unless he’s an existential threat to my career or to the company. If his only crime is to draw a salary of X and furnish work that could be hired at 0.9X, who cares? It’s not “honest” to rat him out to management as a below-average performer; it’s stupid. It would only make a mess.

Programmers are “honest” when it comes to reviewing each others’ performance (which is not to say that they’re correct) and it tends to have three effects. One is that it creates drama. The business people realized that if everyone got top ratings and glowing verbal feedback, the performance-review system could be rendered a formality and they could all get back to work. Problem solved: it becomes a once-a-year annoyance that takes up half an afternoon, rather than something that people fret and grouse about for months beforehand and afterward. The programmers, by being stupid and back-stabby, gave the performance reviews teeth, and created drama that lasted for a long time. The second is that it creates division. Numerical performance scores lead to false objectivity and that leads to permanent rifts within the company, as people withhold information and form alliances in order to game the system. Thirdly, it makes programmers look incompetent. If they’re getting the lowest ratings (because they’re slamming their peers, reports, and bosses) then the organization concludes that must be the worst people in the firm. They’ll never get respect. In the Philippines, this is called the crab mentality: if programmers worked together, they could escape their low professional status (the bucket); however, the constant fighting for slight, typically transient, individual superiorities keeps them all trapped.

Most doctors have peers whom they consider below-average in competence, of course. The same is true with lawyers. It’s also true of business executives, and professors, and everyone else. There’s no professional group in which there aren’t people who consider some of their colleagues to be idiots. Even still, they’ll never say such things about each other, in a context where it could hurt that person’s professional future, except in the case where someone is unethical or dangerous. If you’re seriously ill and have a team of doctors, they might not all like each other, but they’re not going to show it in front of you, the patient. Programmers, unfortunately, don’t have this sense of professional solidarity. They’ll call another programmer a “fucking idiot”, in front of a manager or executive, because they don’t like his indentation style. It’s so easy to turn us against each other, and we all suffer for it.

To continue the example above: on numerical performance scores, it’s obvious to everyone except for the programmers that those don’t exist for the employees’ benefit. (Programmers are too earnest, and actually believe the company when it says that such a management change exists for mutual benefit.) True high performers have no need or use for numerical scores, low performers (or those labelled as low performers) get fired, and the people in the middle get varying degrees of humiliation: being tagged as “good but not promotable” (70th-90th percentile) or “solidly mediocre” (30th-70th percentile) or “probably disposable” (10th-30th percentile). The salespeople and traders and equity analysts at this bank understood, when this numerical peer-review system was rolled out, that a game was being played against them, so they jammed the signal with positive reviews exchanged for other positive reviews and, collectively, won. They created a Lake Wobegon illusion: everyone was above average. The programmers didn’t. They ratted each other out to management– for nothing in return. Consequently, they lost. As a group, this left them with less autonomy and more managerial oversight, which meant more of that thing that programmers claim to hate: politics. We all know that corporate programming is extremely political and hate the fact, but few of us have insight into why it’s that way. It’s because we’re so bad at politics that political problems remain unsolved for years. If we simply learned how to get good at politics, we’d end up spending less time on it, and have more time and emotional energy for the work we enjoy: creating things.

It’s always best, for the workers, to leave management in the dark as to which people are the high and low performers. Obviously, low performers benefit from this. (“But who cares about them?”, asks every reader, “Since that couldn’t possibly be me.”) What’s less obvious is that high performers also benefit from opacity of the performance hierarchy. Why so? If management knows (or thinks it knows) exactly who the high performers are, it only needs to treat about 10 percent of the people being managed with any dignity. The middle 80 percent aren’t perceived to have the credibility that would make it risky to mistreat them. Making things miserable for the middle 80 percent isn’t management’s goal, but it can be an effective way of exerting leverage over the top 10 percent, who might find themselves thrown back into the mid-pack (by, say, being assigned less glamorous work) if they aren’t obedient. Companies want their middling employees to be unhappy with their status. Why? An analogy would be the decline, over the past 20 years, of service and comfort in economy-class airline seats. The airlines don’t do that because they hate economy-class passengers (they don’t care, either way, about them) but to sway those who can afford business- or first-class tickets, but who flew in lower classes in the 1990s when economy-class flight was more tolerable. Employers make it annoying to be a 50th- or 70th-percentile employee not because they hate their average people (again, they don’t care either way) but to put some fear into the top performers. However, if management is somewhat unsure about who is a top performer and who is not, this forces it to treat a larger number of employees with dignity as each might be a key player.

Furthermore, I’ve observed (from many angles) a number of terminations and what I’ve noticed is that it’s neither low performers nor high performers (“tallest nail”) who are most likely to get fired or laid off. Companies can adapt to consistent low performers, and they aren’t stupid enough to throw high performers overboard unless they burn out. Instead, people are most likely to be fired when there’s a change in performance level. Low-to-high is dangerous because shortfalls that were once blamed on inability, and forgotten, can be perceived to have been deliberate. It can open old wounds when someone who was once tolerated as the runt of the litter is now in contention for meaningful projects or promotions. High-to-low is even more dangerous, because it’s seen by management as an immediate risk to morale. If management sees a 90th-percentile worker drop to 60th, she’s still productive, but the fear of contagion will encourage management to isolate her and possibly to fire her. The fact that a 60th-percentile worker adds more business value than her salary costs doesn’t matter; management is more interested in holding up the masquerade of every employee working as hard as he possibly can, which is challenged when a former top performer falls into the mid-pack.

When someone drops from the 90th to the 60th percentile, that person’s literally below expectations. Literal-minded engineers are wont to turn on her, for that reason. This is, in my view, stupid. We all have fluctuations in performance. It’s going to happen to every single one of us: that experience of setting high (and unsustainable) expectations, until reality intrudes, is universal for people who try to do hard things. It’s a part of the human experience. I’d much rather help this person, protect her politically if I can, and have her as an ally. She’s already proven that she’s highly capable and, when she gets her groove back, I’d like to have her on my side. This is how MBAs work with each other, but it’s not how programmers think. This is precisely the collective intelligence that I’m talking about.

It is, I’d argue, often politically beneficial to protect a harmless low performer, even if you’re a high performer. Now, I’m not talking here about negative performers who cause problems, either due to incompetence or malevolence or stupidity. Those have to go. The good news is that the truly, incorrigibly toxic people are rare: maybe 1 out of 50, unless your company is absolutely terrible at hiring. When it comes to the productive but merely-below-average low performers, I have no problem with letting them stay. If they want to better themselves and become high performers, I’ll mentor them. If they don’t, then I’ll ignore them (as they desire). It’s not my company, and they’re not really hurting anyone as far as I can tell. They cost slightly more than one would pay to replace them, but they’re not going to sink the company. I don’t have an ethical issue with the company choosing to fire them, but I’m not going to help management do it; if I did, I would (a) gain nothing, (b) hurt someone, and (c) risk making a career-long enemy. I’d much rather protect that person and increase the odds that that person (who might become a high performer in another context) likes and protects me in the future. If the low performer is a lifelong loser, then it probably doesn’t matter if I rat him out to management. I won’t lose anything, but I won’t gain anything either. If, however, the low performer is a capable person having a bad year, then I make an enemy, for the benefit of a company that would toss me overboard as soon as it thought that I was a “low performer”. No thanks; I’ll protect his secrets.

If technologists protected each other and worked together, in the same way that executives have, then we could change the world not only for our benefit, but for the world’s benefit. Right now, we work so much harder than the caste of private-sector social climbers called “executives”, and get so much less in return. We’ve lost our own industry. If the right people were in charge of our industry, we’d build better technologies and solve more interesting problems and, to boot, make a lot more money.

I recognize that the statement above is a bold one. In fact, it’s hard to believe. The world is increasingly taking stock of Silicon Valley and not liking what it sees. It sees sexism, because VCs fund unqualified, narcissistic young founders based on connections and “gut instinct” instead of talent and capability. That’s not a part of our culture as technologists, but of one that’s been imposed upon us. It sees San Francisco gentrification, because the VCs fund no-remote monoculture shops almost exclusively, forcing people to live in that already-congested city, even though most have no other reason to want to be there. It sees spoiled engineers building products to solve the problems of prematurely affluent goofballs instead of rolling their sleeves up to work on things that matter. That’s what the rest of the U.S. is beginning to think that “we”, as technologists, are. Our reputation is being lost, because our industry is run by people who diverge not only from our values as technologists, but from common decency. When the public sees billionaire venture capitalists who believe women shouldn’t vote or founders’ callous, offensive rants about San Francisco’s working-class population, that’s what the rest of the world thinks we are. We can fix this, but we have to take charge of our industry.

We know who are enemies are. The Silicon Valley elite must be discarded. They have damaged our industry and slowed innovation and, with their antics, are putting our reputation at risk, too. What we need to be better at is making more friends outside of our silly tribal boundaries. For example, we shouldn’t be so quick to reject “business people” as a tribe or group. Rather than resent them for what they have accomplished, as a collective, we ought to learn from it. They’ve been pooling connections and resources while we’ve been drawing lines in the sand by saying “I won’t work with Java programmers” or “I refuse to hire anyone who doesn’t want to do Scrum”. If we had done what they were doing– putting aside differences and working together– then we might actually have a technology industry that reflects our values. More to the point, we’d get more respect and more money, as well, if we could get past petty differences and work together in an intelligent way.

The past ten years have taught me that individual intelligence is very overrated. It shouldn’t have surprised me. Organizational and collective intelligence are far more important, and we as programmers ought to develop them. The quickest way to do this is to reach out to people who already have successfully organized around their own interests. Why do Harvard Business School graduates end up running the business world? They work together, they pool resources and connections, and their first impulse is to cooperate rather than compete. We need to learn this from them, and fast. We’re losing our industry, and it’s time to win it back.

45 thoughts on “It’s time to stop hating and start learning

  1. Michael,

    Long time lurker, first comment. Perhaps the root of the problem here is the low bar to entry into the software development profession. Anyone who can, does. Of course, there is the not insignificant need to have the smarts, perseverance, etc to learn how to program effectively.

    What I mean is there’s no sanctioning body or union or whatever to accredit anyone professing to be a software developer. You just find a job writing code, and then you’re a pro.

    I don’t think it’s even about university credentials or training. Like me, the majority of software developers/engineers are self taught, lack degrees in a relevant field, or both. Consequently, we feel the “imposter syndrome”, lacking even the most basic confidence in our proven on-the-job abilities.

    What’s your feeling about a professional body, or union to “set the bar”? I suppose this would require legislation and licensing to be effective and prevent companies from hiring ” scabs”. If you could make the argument that hiring unlicensed programmers could lead to death or injury, you might get somewhere. Oh, wait, bad work by unlicensed programmers HAS lead to death and injury… you get my drift.

    In most cases though our work is not life or safety critical, so it’s unlikely a governing body would gain support from industry or software developers. Perhaps the cost of supporting such a body would be too high to justify the marginal gain in quality of work.

    I do think mandatory membership in an association, or guild, or union, to unite and support us, and provide a set of ethics for treatment of each other, would be a good counterbalance to the current infighting.

    Hope to read your thoughts on this in a future post!

    All the best,
    Philip

    • I think we should draw inspiration from two sources: (1) the exam-based actuarial professional track, and (2) the actors’ and writers’ guilds in Hollywood.

      I strongly prefer exams over courses for baseline accreditation. Making a degree a requirement is stupid, IMO. I also think that there should be an untimed project-based alternative for each exam, because some people have legitimate learning disabilities and if someone can prove knowledge through code, that’s great. (I might have the manual-review process cost more than a written exam.) Ultimately, I’d like the material to be (a) a full CS curriculum to the MS level, (b) relevant mathematics (including linear algebra, statistics/probability, abstract algebra, real analysis, logic) to the BA level, (c) a couple courses’ worth of EE and hardware design, and (d) a component focused on technical writing, design, and aesthetics. It’d probably be a 12- to 15-year course from zero to Fellow for most people (assuming that most people would be working full-time while taking the exams.)

      I also agree on the mandatory membership in some kind of guild.

      Now, what we’re up against is that some programmers have a visceral negative reaction to centralization. And those who don’t are likely to be the ones who also don’t have a problem with the employer-friendly/corporatist status quo. It’s a hard sell, but if the alternative is long-term Scrum/open-plan culture, then it may be the only option we have.

    • I think it’s better to split the programmers into two tracks.
      1) Professional software engineer who belongs to a professional organization.
      2) Unqualified software “programmers”

      If companies want to hire “only” unqualified programmers, let them do it.
      If they want to hire one or more professional software engineers, they should agree to the terms of the profession. Otherwise, the profession would deny the companies access to its talents.

      This is already powerful enough as long as the profession has credibility in the society, and it enables a profession to enforce its policies internally without resorting to external groups.

      So, for a profession to become powerful, it needs to establish credibility and attract as many intellectually curious people as possible.

      Not every form of centralization or monopoly is bad. Almost no programmer bashes tweeter for achieving natural monopoly by networking effect. I think a profession could obtain natural monopoly on culture by being the “first” professional organization of programmers that gains credibility in society and utilizes networking effect.

      Since many programmers do not like “imposed” centralization(government laws, etc), it could actually work.

      • How do we get programmers to join when, in the early stages, it imposes a professional disadvantage (i.e. being part of the guild means you agree not to work on non-guild projects)?

      • Maybe, it’s better to not impose certain restrictions in the early stages. A profession should probably notify applicants that it’ll introduce certain restrictions when it reaches a certain stage. Any unexpected future changes will upset members.

        Being a member of a profession has natural benefits which might be enough to attract people. A profession could also provide extra benefits rather than impose restrictions in the early stage. I’m not sure if it could be any benefit when the profession is poor, but the profession could fund new startups that strongly prefer profession members to outsiders. I think the profession could give seed funds to small startups for profession members. Looking back at what I just wrote, it starts sounding like a category. I can’t think of other artificial benefits, off the top of my head for the moment.

      • This is just a quick thought. We could also layer the profession so that members who join the higher layer are subject to more restrictions but receive more benefits. Would it be effective? I don’t know.

  2. Interesting post and I basically agree with what you say. I did notice one very slight contradiction though. You make the observation that the “executive class” has less age discrimination and can peak in their late-50’s–whereas the software engineers are notably in decline by age 40. But you also say that you are too old at 32 to do an MBA because an “MBA doesn’t open the same doors when you are old”.

    • “In decline” isn’t the term that I’d use for over-40 programmers. They still have 20+ years of great work ahead of them; it’s just hard to convince employers that it’s worth it (even though it absolutely is!)

      Business culture is ageist but less so than VC-funded startup idiot culture. However, the social life in MBA school centers around being 25-30 and single. I’m a 32-year-old, married non-drinker who’d rather read a paper on elliptic curves and play with my cats than get hammered at a bar (been there, done that).

      Alcohol has a magical property, for young people, in creating powerful bonds that pay dividends in the business world. However, I’m past the age where that makes sense. I’ll drink a really good Belgian beer but, aside from that, I’m just not into that scene anymore.

    • Famous marshmallow/uncooked spaghetti/a few other things building exercise.

      Best results from engineers and 1st graders. Worst from MBAs who spend most of their time trying to game the rules to make half-assed solutions be accepted.

  3. Hey Michael. This makes sense. Now suppose I get a new job and want to educate my fellow programmers, how should I do it?

    Talking it is too risky and ineffective I think.
    I would feel much more comfortable with some kind of inforgraphic or a series of images of some sort. The closer it is to a meme the better. So I can accidentally drop it or share it to them, without revealing my ambitions to anyone.

    Mmm. If I had to pitch this I would do something like “Average Programmer Bob Comics” and do a few strips with the lessons of this blog.

    Maybe like this:

    Panel one: 37173 Programmer Jack wants to push his favorite programming language and will belittle anyone who proposes another language. 37173 Programmer Jack is miserable.

    Panel two: Average Progammer Bob wants to advance the careers of the whole team by picking a language that can do the job and that will open many interesting opportunities in the future. Average Programmer Bob is respected by all.

    That’s it. Repeat the same formula 10 or so times, keep the two panels, the characters and the lasts lines, just change the lessons. We want to make “miserable” and “respected by all” into code words. So that later we can drop it and send a message while mainting plausible denial. They would act as labels to signalize where to put social pressure.

    I’m out of my depth here, but my gut tells me we could create some constructive and effective propaganda in a few hours.

  4. I’m not convinced what you’re saying is true anymore. It used to be more like that, but it doesn’t seem to be like that right now where I’m at. I work in San Francisco. There are lots of acqui-hires where they fire all the non-engineers. I don’t think the managers from failed startups do as well as the engineers.

    I know someone who got an MBA from a top-tier school, and is having trouble finding a job. I was at a company where a new CEO came in and fired most of the management, but kept most of the engineers. I’ve also seen former Google engineers with no business-sense raise ridiculous amounts of money after they go into YCombinator.

  5. I think you’ve captured the essence of the problem as have the other commenters as well. If you consider the products being foisted onto the public, nowadays it shows the mediocrity compared to what was available only a few years ago. Many of the successful web/internet solutions were built by craftsman, but now are deemed old and outdated, only to be replaced by something slapped together by a team of inexperience. It shows, it really shows.

    But to some extent, it is just a numbers game. Look at how few executive positions are available, and how long they stay. Upward mobility is severely limited. For this reason, alone only the most savvy and competitive ascend the ranks, leading to the environment you describe.

  6. Michael, I think you’re subscribing to the apex fallacy to some extent when you talk about salaries.

    Among the non-technical, business-side people, very few ever get to the executive level. For every C-level elite, how many average Joes are there who earn a fraction of what the boss makes?

    These days you hear about programmers being paid six figures right out of college. How many years of service would you need to get that high on the non-programmer track at a company that also hires programmers? Thirty? (I live outside the US, so I welcome more accurate numbers.)

    I don’t begrudge programmers their high salaries — the work is difficult, very productive, and deserving of the high pay. More deserving than regular office workers such as account managers and customer service workers whose jobs can be done with much less intellectual firepower. (I am one such worker, lest anyone reading think I’m looking down on the riff-raff.)

    I’d be interested to hear the median technical versus non-technical salary calculated with all workers at a certain age as the base, not just executives. I wouldn’t be surprised if the technical side is still ahead even at age 50 or 60.

  7. You say you could easily do what an executive manager or Ceo does with your intellect.. Sure I have no doubt you are smarter, but don’t you have some sort of anxiety issue which is why you dislike open plan offices?

    Can you get up in front of 50 people and present to them? Can you convince investors to give you money? Can you communicate to other non technical people you need to convince and get them to like you, in simple language they will understand and agree with?

    • > Can you get up in front of 50 people and present to them?

      Yes. That’s not the same thing as open-plan anxiety. Presenting is a position of strength. Being visible from behind is a position of weakness and low status.

      Spending hours on end divided between actually producing something and the demoralizing side game of appearing productive is something that, over time, utterly drains a person.

      > Can you convince investors to give you money?

      That’s about connections more than anything else, at least in the early stages. In the later stages, it’s more about traction.

      > Can you communicate to other non technical people you need to convince and get them to like you, in simple language they will understand and agree with?

      Not sure. Maybe. I can learn it. If it’s my job, and I don’t know it well enough, then I will learn it.

      • Example of the drain: It’s 5:30 and you are exhausted from 6 hours of continuous work. Want to go home but boss is answering emails and you sit five feet from him so you too have to look busy, since surfing Reddit/Twitter/Facebook will be obvious. So you spend a horrid hour until boss finally leaves.

        • Precisely. And you don’t want to start anything real, not because you wish to be lazy, but because you’ve been surrounded by people in close quarters for a hideously long time, and any work that you do will have to be redone anyway.

          Creative people, who tend almost invariably to be introverts, are becoming an oppressed minority.

  8. I liked that bit about how “innovation isn’t possible under Scrum”. Speaking as someone who job it is to run iteration planning meetings, I can attest to the fact that time to try things out just doesn’t happen when we have our every moment planned out for us. Remember life happens when you are planning. In much the same way, innovation happens when you are reflecting; Scrum doesn’t really leave any time for reflection, and that goes double if you pair all the time.

    I hate our open plan environment. There is no privacy. I am a goldfish in a goldfish bowl.

    • Or maybe realize the ideas tried so far have all failed, thus probably in order to succeed one must do exactly the reverse of what originally seems like a good idea.

      Path to success: it’s time to stop learning and start hating! >:)

    • Scrum has become an excuse for enabling bad behavior in the people on the business side. Welcoming changing requirements is just asking for disaster. Requirements should be few and basic and rarely change.

      Scrum is also sold as a way to make engineering talent horizontally scalable. Throw enough bodies at the issue, and you can solve *any* technical problem. However, that’s both untrue and uneconomical. Commodity engineers (ScrumDrones) become negatively productive as soon as you try to do anything beyond the trivial.

  9. A lot of what you write speaks directly to my own experience, only you word it more eloquently than I ever could. This is why I keep coming back here.

    It’s very hard (at least for me) to understand the trade offs between honest feedback and maintaining group solidarity in performance reviews and in front of other groups. Without some sort of education in the subject and critical thinking it’s hard for literal minded engineers to consider the potential hard of pure honest feedback.

    I think developing a culture of solidarity is necessary for software engineering to evolve out of the high middle income pigeonhole you’ve expected to leave by age 40 that it is now. The question is how to build that culture. Is anyone on here familiar with how other professional or creative guilds formed and whether there are parts of those models we can use?

    Also can you publish your low status of software engineer’s essay? I get anyone I can who’s either in IT or in a relationship with someone in IT to read it.

  10. Interesting point of view, MOC. One aside – while uniting as a profession is the ideal, I think you are right in that technical people distrust centralization too much to make it happen. It’s not that a bunch of evil managers sat down in plush leather chairs, with cigars and brandy, discussing how to divide and conquer the natives – it was that management adapted to technical people’s natural divisiveness and cluelessness.

    My two cents on divide and conquer and the problems you talk about. I run a small outsourcing practice (I live abroad), and one obstacle I’ve encountered is that US clients always try to cut separate deals with team members. How do I prevent this from happening? First, I pay them well above the local rates, even if it means I take home less, so they know that if they work independently with the client they’ll get screwed (I’m a better negotiator). Second, the team’s accomplishments belong to everyone – everything is a “we”, not a “this team member or that team member”. Thirdly, if they do develop a separate relationship with clients, they’re out, and that includes any future clients that come along. Fourthly, team members get paid in equal amounts, no matter how the work falls. It takes a little backbone, but clients get the message and it works out better for all.

    • Where in the world are you based? Do mind expanding on how you run your business? It sounds unusual enough.

  11. Hey Michael, what do you think about organizing software companies as co-operatives?

    I’ll start by mentioning there are some downsides of course, many co-ops don’t actually hire employees so there’s no job security. But that isn’t really much worse than traditional employment in some places like California which can fire people at will.

    The upside is true co-ownership (not 99% worthless stock options), alignment of personal interest with company interest, and less managerial bullshit, since they can just be booted out and don’t have much to contribute anyways. I’m actually wondering why software developers who work together don’t do this more often.

  12. This is was really interesting article. I think part of the problem, is one of the things that makes a good engineer, a desire to understand the world, to know how things work, to be competent, leads to judging harshly those who don’t work tirelessly on those things. In simple black and white terms they are the “incompetent.” Engineers judge business leaders, marketing execs, etc, just as harshly as their fellow programmers but that doesn’t get voiced and doesn’t matter. No CEO asks a programmer what he thinks of the job the CEO is doing. He asks what do you think of the job your fellow programmer is doing, the “idiot” who only put in eight hours and didn’t master the new development platform you learned that day. And so while contempt may abound for all who don’t understand the complicated nature of what you are working on. It is the slightly less talented, less committed engineer who you report which leads to the the entire IT department being singled out for competency issues. Since no other department is reporting these things, damnit this company needs scrum to fix those guys!

  13. @the cooperative guy: that’s probably the only thing that would work but it’s the hardest thing to achieve.

    If possible, it would be like going from unicellular to pluricellular life form. Quiz: please try to get the answers from your head. Even I was surprised at my ignorance and naivity in estimation.

    1) When did pluricellular life first appear on Earth?

    2) When did pluricellular life not just appear but really take off? (OK, ‘now’ is not a bad answer but really).

    3) Was there UNI before PLURI?

    4) For how long?

    … on a side note, I might have just leaked, excreted or just randomly arrived at an idea which just might be called a startup. A pluriversity, a place where I’d wanna study, work, live.

  14. Rounding errors.

    You get them both in trading and wars. They’re bad when you’re on both the receiving side or targeting side once you realize you traded the wrong stuff.

    They’re excellent when I’m not sure how much money I got this month if you ask me to the dime but the overall feeling after buying that X thing just so I compare it with the W thing is that it’s measured in centimeters.

  15. How do you square your claim that MBAs have cornered the $300k jobs with this post from a couple months ago about how programmers at large tech companies routinely make that within 3-4 years? As Mark Dunan above notes, you seem to be incorrectly comparing the less-talented 90% of techies with the few businessmen who scale the top salary rungs.

    Politically speaking, a group of people who protect and assist each other, without question, will always do better than free agents acting alone, no matter how talented the latter group may be.

    Perhaps, sometimes, but productively speaking, they will almost always produce little in the medium-term, which is why large firms like Microsoft inevitably become more about turf wars between these rival gangs, rather than about producing great product.

    Like anyone else, I have my bitter moments. With my intelligence as well as the pedigree that would come from, say, Harvard Business School, I’d have an invincible combination. So perhaps I should have done that, instead, during the years that I spent goofing around in startups and tech companies. Had I taken that other route, I’d probably have my own company and be calling shots, instead of spending so much of my career in reaction to others’ suboptimal decisions.

    What’s stopping you from starting your own venture now? Others have done so with almost no capital and since you’re so intelligent,😉 you should be able to beat those MBA dummies into the ground.

    Worse yet are the idiotic sectarian divisions we’ve let form within the programming community: “Agile” versus “enterprise”, tabs versus spaces, Ruby versus Java, women versus “brogrammers”, young versus old, Bay Area versus “the sticks”. We can’t afford this. These divisions make us weak and manipulable. Many of them don’t make any sense. The quarreling turns us into a low-status group of people. Instead, we need to unite around our shared interests.

    I have never heard of any of these parlor arguments affecting “unity” towards any meaningful purpose and I doubt you have either.

    We all know that corporate programming is extremely political and hate the fact, but few of us have insight into why it’s that way. It’s because we’re so bad at politics that political problems remain unsolved for years. If we simply learned how to get good at politics, we’d end up spending less time on it, and have more time and emotional energy for the work we enjoy: creating things.

    Sounds like the real problem is “corporate programming,” simply giving in and getting “good at politics” doesn’t sound like it will help much. Rather, the real solution is to get rid of the corporations, which the internet is moving us towards. As Gabe Newell noted three years ago,

    “It seems fairly obvious that the Internet does a better job of organizing a bunch of individuals than General Motors or Sears does. Corporations tend to be pre-internet ways of organizing production.”

    The world is increasingly taking stock of Silicon Valley and not liking what it sees… When the public sees billionaire venture capitalists who believe women shouldn’t vote or founders’ callous, offensive rants about San Francisco’s working-class population, that’s what the rest of the world thinks we are.

    Many outside don’t like Silly Valley anymore, but it’s not because of their personal views. It’s because they’re so bad at creating value, what they’re supposed to be actually doing, that their financial performance lags even the dismal S&P 500 over the last decade. That falls on everyone in the bay area, not just the VCs.

    The past ten years have taught me that individual intelligence is very overrated.

    Lol, I draw the exact opposite conclusion.😀 I suspect the difference is that you are calling those who are currently “successful” in the SV circle jerk, which the link above notes don’t actually make money, “intelligent,” whereas their lack of intelligence is precisely why they’re not performing.

    It shouldn’t have surprised me. Organizational and collective intelligence are far more important, and we as programmers ought to develop them. The quickest way to do this is to reach out to people who already have successfully organized around their own interests.

    Organizational and collective intelligence are important, but it’s actually the individual that’s “far more important” these days. If you’re simply saying that engineers should try harder to cooperate with business and design people, rather than dismissing them as you sometimes do, that’s a platitude that nobody’d disagree with. If you’re calling for the type of log-rolling you praise about HBS graduates below, that’s just ridiculous.

    Why do Harvard Business School graduates end up running the business world? They work together, they pool resources and connections, and their first impulse is to cooperate rather than compete. We need to learn this from them, and fast. We’re losing our industry, and it’s time to win it back.

    There’s always a balance between cooperation and competition. The business guys you praise probably cooperate too much, whereas techies may tend to be too competitive sometimes. But platitudes about increasing cooperation and creating unions will accomplish nothing.

    Rather, what’s coming is much more radical and technology trends like open source and decentralized protocols point the way. The future is a decentralized swarm of information workers all collaborating online, with no company to “employ” anyone. In that environment, the politics you’re talking about will play almost no role, which will be precisely one of the strengths of this new approach.

    You should do something to start making that happen faster, as I’m trying to in my own way, rather than constantly whining about the dying corporate present or the bay area idiots.

  16. Great article, you speak the truth and for that you will be hated everywhere by the people who write the checks. Everyone reading this needs to understand that this is probably the one blog on the internet that really tells the truth about this industry and that doing this publicly like this is really a risk.

    I started out with a degree in business (a very mediocre one) and then taught myself to code because it helped me get a huge salary jump (300%+ after five years in the industry). After 4 years of programming full time I observed 100% of the AGILE bullshit you mention – My fellow programmers, like children, were herded and embarrassed into doing work by stupid systematic public shaming tactics disguised as a project management system. I decided coding wasn’t for me because being treated like a child for the rest of my life was not interesting.

    The problem is that manipulating engineers emotionally works. Engineers fundamentally lack the ability to “See through” people’s motivations into the political workings of the organization. Having worked on the business side, I had been trained by some very sharp mentors to be able to guess how and why management will act based on their motivations and imperfections. I came to understand how to decode corporate double-speak and get at the meaning of what is going on in the corporation. Most engineers simply never get there.

    The result of these skills: I was taught to know where to sit in the organization, which teams to avoid, when to jump out from under a manager, when to recognize a manager who will give me good reviews and which parts of the company were likely to undergo growth and which would undergo shrinkage. Hint: Always be in the growth part of the business, never get stuck in the shrinking part of the business.

    Engineers often don’t do that, they tend to just accept the tasks they are given and get stuck in dying parts of the company. They don’t value networking so they never build escape routes. They also don’t have the skills to escape bad situations through self promotion and networking.

    The business side of my brain let me navigate the corporate world and be in the right place in the right time before anyone else knew that was where to be. I see engineers who are significantly smarter and more talented than me getting shitty reviews and being unable to leave their groups as a result.

    My job is easy. Really easy. I get to travel around the world. I get paid more than ever before. I get glowing performance reviews and bonuses. I don’t have to do hard, shitty programming work, no one micro-manages me… in fact, I work roughly 30% as “hard” as I used to before but I get 500% better results. Most of what I do is talking and networking.

    It’s sad but my experience as a software engineer taught me that engineering is truly a low status job.

  17. One reason the “business” people help each other might be precisely because they know that they would be torn apart by engineer savagery if they would get a chance. Most engineers are extremely stupid as defined by “A stupid person is a person who causes losses to another person or to a group of persons while himself deriving no gain and even possibly incurring losses” ( http://harmful.cat-v.org/people/basic-laws-of-human-stupidity/ )

    • In other words, in a reversal of our opinion of ourselves as the pinnacle of civilization and “them” as the barbarians at the gate, we’re the actual barbarians and they work together to protect themselves against us? It’s possible. It’s certainly an interesting idea.

      This form of stupidity that you’re discussing is simultaneously the opposite of civilization and, yet, something that can only be afforded when civilization exists (because it doesn’t survive for long, in the wild).

  18. Question to Michael: have you found someone willing to mentor you into finance? You mention some acquaintances who work or used to work there.

    If you found them, I’d advise you take that route. If not, question is why?

    Finance is hard and ‘talent’ only loosely correlates with success. As Paul Wilmott says: ‘most quants are stupid’.

  19. @MichaelOChurch I think you are still using too many words to express your intent. I thought the new blog would have succinct posts. Just a food for thought for you.

    • I must say that this is a situation where you can’t make everyone happy. I’m all for copious amounts of text (and beer, for that matter :P).

  20. I don’t know where you guys have worked but I have seen many “highly intelligent” programmers promoted to management only to prove the Peter Principle, having risen to the level of their incompetence.

    Management is a totally separate skill set from software engineering. In the half dozen IT and commercial software shops where I have worked in the last 30 years, the most effective projects have been those with effective management. The differences between the programmers and managers (or even *gasp* salesmen) have NOTHING to do with intelligence but rather temperament and certain, specific skills that frankly I rarely encounter in developers, myself included.

    Recently there has been a shift away from management toward “team leads” and “self-organizing teams.” There is nothing uglier than a team that fails to self-organize. I recently saw the founder of a 20 year old company try desperately to separate leadership and mentor-ship functions from “base” management. He recognized that management was crucial but treated it as something menial that he could hire people to do while the real brains “lead” and “mentor”. As his company grows, I fear that poor management will plague the company and chaos will rule rather than unity in direction.

    As you can tell, I strenuously disagree with your assessment of the superiority of the programming class based a quite startling level of self-importance and hubris. I suppose I will be dismissed as one of those lesser developers who just doesn’t get how smart the cool kids are.

    • Well, the cool kids can be very smart. That does not mean they are any good at managing themselves, let alone others.

      I spend many days convinced I am doing it all wrong, only to find out I was doing it right all along. I have been hovering between team lead and manager for many years now. I just cannot give up the command line (just yet) but I feel that day hurtling towards me.

      I have seen chaos. I have seen over-management where engineers weren’t allowed to fix the simplest of issues without permission granted from above. Neither is a great place to be.

      I sit pragmatically, and consider the next best thing to do, that will inch us forward. It isn’t always what the cool kids want, and it often isn’t technologically sexy.

      Self-organising sounds great until you see no one wanting to do it because they have been burnt too often in the past. Even then self-organising is only accepted up to a certain level. Once it needs wider changes, the team gets boxed in by other teams, and cannot achieve the greatness they aspire to. At a certain point, you need organisational clout to move onto the next stage of improvement and it is normally managers that have that clout.

  21. Pingback: Pyramid vs. obelisk – Michael O. Church

  22. Hi Michael,

    Came here after reading your AMA – I’ve read your blog posts before (e.g. when interviewing at Google), but had always imagined you as a grizzled 40+ year old veteran of the tech world, not a contemporary. I’m an MBA from a top 5 programs and would disagree with smaller details (for example – PE and hedge funds are really only an option for MBAs with prior experience – for the rest of us it’s management consulting/IBD/mid-level corporate gigs), but in general, agree that we’re socialized to look after each other.

    It starts in school with the networking activities and so on – and “grade non-disclosure” does in the recruiting process what the MBA network does in the workplace – leverage collective performance to inflate individual performance (and at Wharton, it’s not even an official policy, but a pact that the MBAs have made with each other).

    This has pros and cons – as a guy with no previous business background, but smart and hungry, getting my MBA at 30 enabled me to break into companies which usually require X years of experience in the exact role – even if I learned quickly and became a top performer once in the role. And I can bring immediate value to the company by having a contact in almost any other company around the world to reach out to for partnerships, insights, funding, etc. On the other hand, I’ve also seen entire departments zombified by MBAs and ex-consultants – where they value conformity to how a deck looks and how ideas are communicated more than the content.

  23. Let me explain the problem to you, my man: aspergers. Many tech people were social misfits growing up, and thus their only source of social approval (which we all need) came from an authority figure (teacher) on the condition of them being smart and obedient.
    Business bros bothered to play sports and have friends growing up, so they realize that you and your crew are in this together and thus have to help each other out. It may just come down to autistic tendencies and lack of socialization.

    • This is right on the money.

      Of course, most of the Silicon Valley power-players aren’t “Aspie” at all. They just use faux-aspie chic to justify being assholes.

      I know people with autism spectrum disorders and, while they are not socially adept by any stretch of the imagination, only one of them went out of his way to be an asshole. The rest were really kind people who just can’t read social cues.

  24. Here’s a fun story about cult thinking and self loathing. Worked at a company with the open spaces, crowded tables, and zealous Agilism. The agile coach said many psychotic things. Here’s a few:

    – Sitting in a meeting room, he said: “I’m so proud of you. I think, and many would agree with me, that we are the best Agile team in the city”. This is a relatively big city with a shit load of dev teams.

    – My favorite was this. At a team manifesto review meeting he said: “I think we should set aside one day a week where each developer can create and innovate on his or her own”. Everybody enthusiastically nodded along to this. The gods help our industry.

  25. Great essay. I felt myself nodding and smiling to myself while reading this. I’m curious though, if not Agile / Scrum then which methodology do you prefer?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s