The software career and the Second Design Paradox

I’ll cold-open with a somewhat pointed, but valid question: why are there so few great software engineers? The answer: the software industry makes it extremely difficult to become good at engineering, unless a person is economically autonomous (read: wealthy, or well-connected enough to secure venture funding). Here are some of the obstacles that engineers face if they want to become great.

• Low expectations of individual contributors. Non-managerial programmers are rarely expected, in most work environments, to have multiplier effects that make the team more productive, and therefore are rarely able to have the kinds of infrastructural contributions that would bring them feedback on what works, architecturally speaking, and what doesn’t. Instead, they’re often put to work on parochial business problems of low general interest or application, and often their contributions are to maintain some small piece of a massive system, so they rarely get a sense of large-scale engineering. 
• Rarity of high-quality work. The kind of work that engineers need in order to become great is uncommon and requires substantial concentration. It’s almost impossible to get it without managerial buy-in (because one needs freedom from interruption and low-quality distractions) and, for the tiny amount of such work that companies allow to exist, it is allocated politically, rather than based on any sensible principles.
• Obscene program size. When the program-to-programmer relationship is many-to-one, one gets sensible and small programs and highly productive engineers, but individual efforts are hard to track and measure. Consequently, many managers prefer a world where the program-to-programmer relationship is one-to-many and “headcount” can be tracked at a “per-program” level. These massive single-program teams tend to end up in a state where one person (the “blazer”) is highly productive and the others (the “trailers”) spend the bulk of their time keeping up with the changes, so individual productivity is low. Why is this low productivity tolerable? Because software engineers are massively underpaid compared to the business value they actually provide, so it’s acceptable (from a corporate perspective) to have them running at 25% efficiency.
• Political distractions. When it takes as much time to prove that one is qualified to do interesting work as can be spent actually doing it, it becomes hard to maintain a high level of competence, much less improve upon it. It takes three to four years of campaigning in most companies to be considered credible enough to get the best projects, and by this point, a person’s technical skills have declined.

All of these issues exist, but there’s an obvious question. For high-potential software engineers, why don’t the political problems “solve themselves”? If they’re so good, why aren’t their managers falling over themselves to clear obstacles and let them do what they do best? Businesses would benefit immensely if they could discover and tap the top talent, so why don’t they? The answers are inherent in the concept of “The Design Paradox”, which I’ve split into two related “paradoxes”, one psychological and the other political.

First Design Paradox: people who lack design sense (“taste”) cannot assess it in others.

People with taste but not vision can recognize tasteful visionaries apart from charlatans, but people without taste are hopeless. This is a serious problem in software, because what differentiates a good programmer from a bad one is not how fast code can be written or how many lines are produced per day, but having the taste necessary to design usable, maintainable systems. Are those technical assets actually usable, or will they be tomorrow’s despised legacy horrors? This specifically technological variety of taste needs to be refined with experience, and that experience is hard to come by, so the taste itself is rare. People who don’t interact with software at a code level on a regular basis will have no intuition for what beautiful software is. You’re not likely to find this taste in a person off the street, and if there isn’t a strong reason to believe you have it, you almost certainly don’t.

This requires that technological taste be present at the very top of the company. The CEO will generally not be reading source code, but if the CEO lacks taste, then getting a tasteful CTO or VP/Eng is a crapshoot. If the CTO lacks taste, he won’t be able to tell which of his reports are capable and which are charlatans. Charlatans tend to be more skilled at office politics– they have more experience, due to a lifelong need to compensate for their own mediocrity– so there’s adverse selection involved.

What should a CEO do if he’s self-aware to understand his lack of taste in software architecture? Something very few managers like to do: give back some control. I think one of the reasons why Valve’s open allocation works so well is that it’s an admission of a reality that exists in every complex company: that executives don’t have all the answers. Workers are trusted to vote with their feet and throw their efforts behind the most tasteful projects– and they don’t have to wrangle with a rusty, corrupt permission system in order to do so.

Most discussions of “The Design Paradox” pertain to the first one, but I’m more interested in the second.

Second Design Paradox: people with taste will perform worse in environments with a history of poor taste, giving tasteless people a competitive advantage.

By “perform worse”, I mean that they will be worse at their jobs, and politically less successful. This is actually somewhat counter-intuitive. It’s akin to saying that the best writers are worse than average people at reading poorly-constructed prose– probably not true, for that particular example. In programming, however, it is true. Under a certain definition of “worst” (which I’ll explain, below) the best writers of code are the worst readers of it. That’s right. A critical skill for a corporate software engineer (reading other people’s code) is, under a certain set of circumstances, strongly negatively correlated to the ability to design good systems. What is that set of circumstances? It’s when the systems that must be understood are themselves incoherent or badly designed. Everyone struggles with it, but the best programmers have even more of a problem with bad code. Good designers will learn good technologies quickly and struggle immensely with bad ones.

I know this from painful experience. When I read code, I’m trying to connect what I am reading with how I would solve the problem. Usually, there are discrepancies, and if it’s good code, these are informative. Whenever I encounter a difference between my mental model of the problem and how it is actually solved, it helps me understand it in a deeper and more detailed way. So I enjoy reading good code, just as I enjoy reading a well-written mathematical paper, because there’s a convergence that occurs as I really learn something. The problem with bad code is that there’s such a stark and irreducible difference between how I would solve it and the way it is done, because it is done so tastelessly, and also because the only thing learned from it is parochial garbage pertaining to how ineffective or poorly-trained programmers think. You don’t learn the fundamentals of computer science from an AbstractProxyVisitorSingletonFactory. I can’t find convergence with a pile of FactoryFactory garbage because I would never write FactoryFactory garbage. There is simply no problem or set of circumstances (except a desire to severely punish an employer) that could bring me to write that kind of code, ever.

The Second Design Paradox is an aesthetic cousin of the Peter Principle, which is that “employees tend to rise to their level of incompetence”. There are two ways one can approach this. The first, which is wrong, is the argument that higher-level positions in a company are more difficult to perform, and that people therefore ascend as long as they are competent but reach a point where the work is too difficult for them. The reason this is wrong is that, although it is abstractly more difficult to do an executive job well, the managerial ranks of almost every company are so self-protecting and political that merit and promotion divorce early, and it becomes easier to succeed politically as one ascends. The standards of personal integrity and intelligence are lower once one has “made it”. In most companies, it’s hard to get into the managerial or executive “club”, but relatively easy, once in, to keep rising. In fact, the tribal self-protection of managers is at such an extreme that the obviously incompetent executives get what are known as “exit promotions”, which make their titles more attractive so they can “fail up” into another company and become someone else’s problem.

The second interpretation of the Peter Principle, which is the right one, is that companies tend to tap people for leadership roles based on their success as subordinates, while good leaders make shitty subordinates and good subordinates make ineffective leaders, because the ones who subordinate well are the people who just don’t care about doing things right.

Now I have to reconcile two statements that seem to contradict each other. One is the polite maxim that “Good leaders know when to follow, and do it well”. That’s true. The other is “Good leaders make shitty subordinates”, which is equally true. How can this be? There is a fundamental difference between following and subordinating. One who follows is choosing to do so, because she perceives it to be better for the group and, thus, also herself, to let a more informed or impartial person make calls. That’s the hardest thing about being a leader: knowing when to trust your own judgment, when to defer to someone else’s, and when to step down outright. What I like about Valve’s model, with its absence of entitled, permanent leaders called “managers”, is the recognition of this fact. On the other hand, a person who subordinates is accepting authority outright. Subordination is unconditional following and obedience that comes out of a preternatural acceptance of rank. When you follow, you have a choice. When you subordinate, there is no choice. You are surrendering to rank.

Most corporate managers aren’t leaders, but an internal police force trained to enforce personnel policies that come from a “Theory X” belief that most workers are ethically somewhere between depravity and juvenility and will steal from the company (slacking and prioritization of one’s own career goals both included as “stealing”) if not closely supervised. The problem is that indoctrination into the managerial religion makes it harder to lead, because it embraces such a negative view of the people being managed. True leaders never show contempt for those who are following them. They recognize that they have a transient functional superiority, not an innate moral one, and that if it is better for the group for them to take an inferior (following) role, they should do so.

Design has a natural need for leadership, because of the need for simplicity in the finished product. Many should be encouraged to suggest ideas and avenues for exploration (divergent creativity) but one person must sit in the center with the task of pruning, simplifying and prioritizing (convergent creativity). It’s rare that you get a good design without exactly one person who takes on the “killing” role. If no one takes on that nasty job, conceptual integrity vanishes as requirements and boondoggles are added. If two or more people take it on, all with some share of veto power, then the political machinations that ensue often create a product whose shape reflects the parochial political environment in which it was formed (Conway’s Law) rather than any universal insight about a problem. Decentralized power is good for government because the objectives are global concerns like justice and sustainability that are too large for one party to manage, but for the finished aesthetics of a local entity like a new product, you actually need a single leader who will accept input from many (thereby following, at the right scale, when appropriate) but prune when it is required to preserve the integrity of the whole product.

The question, then, is whether human organizations are effective at bringing design leaders into leadership roles, and I would say, based on experience, that the answer is “no”. Why? Because these organizations tend to promote people based on their ability to subordinate, resulting in a low quality of leadership. They don’t have the design sense, at the top, to recognize emerging design leaders, but it’s obvious to them who are the best subordinates and who are the grumblers.

In software, anyone who takes a maintenance role is expected to follow, by definition. When you maintain code, you can’t just rip it to shreds because you think it’s badly designed, because you’ll break others’ work. You have to modify the software in accord with the contract (often unspecified) that it has set with its users. Even if the CTO of a company takes on a maintenance project (and sometimes this is exactly the right thing for a leader to do) she is taking on a role where she must, to some extent, follow. That’s not a bad thing.

However, in most large companies, maintenance isn’t done by people who choose it for the good of the company, but by people who get stuck on it because they’re junior and don’t have the clout to do anything better. They don’t have the power to do maintenance right. They chug along at 20 lines of code per month, fixing minor bugs and adding low-priority features, working below their frontier of ability, and waiting to be tapped for a role with more substance. They’re not following, because they don’t have a choice. If they voice their desire to work on something better, or assert their creativity in any substantial way, they’ll probably be fired within a year. They’re subordinate. People with design sense tend to be turned off by this sort of thing, so they rarely pay their dues and climb the ranks. The ones who do are those who never bothered to learn design sense, and who think “functional programming” is some academic mumbo-jumbo with no place in the real world.

In the long run, the result of this is that most corporate software is of very low design quality, which aggravates the problem. Left unchecked, it creates an arena where the lack of taste becomes a political advantage, because the people who lack taste are better equipped to adapt to the bad architectural decisions the group becomes stuck with. They rise into positions of authority and then make more bad design decisions, utterly inadvertently because bad designs are the only thing they’ve been exposed to. At this point, the organization isn’t just producing an occasional piece of bad software. It becomes a BadSoftwareFactory.

Programmer autonomy is a $1 trillion issue.

Sometimes, I think it’s useful to focus on trillion-dollar problems. While it’s extremely difficult to capture $1 trillion worth of value, as made evident by the nonexistence of trillion-dollar companies, innovations that create that much are not uncommon, and these trillion-dollar problems often have not one good idea, but a multitude of them to explore. So a thought experiment that I sometimes run is, “What is the easiest way to generate $1 trillion in value?” Where is the lowest-hanging trillion dollars of fruit? What’s the simplest, most obvious thing that can be done to add $1 trillion of value to the economy?

I’ve written at length about the obsolescence of traditional management, due to the convex input-output relationship in modern, creative work, making variance-reductive management counter-productive. I’ve also shared a number of thoughts about Valve’s policy of open allocation, and the need for software engineers to demand the respect accorded to traditional professions: sovereignty, autonomy, and the right to work directly for the profession without risk of the indirection imposed by traditional hierarchical management. So, for this exercise, I’m going to focus on the autonomy of software engineers. How much improvement will it take to add $1 trillion in value to the economy?

This is an open-ended question, so I make it more specific: how many software engineers would we have to give a high degree (open allocation) of autonomy in order to add $1 trillion? Obviously, the answer is going to depend on the assumptions that are made, so I’ll have to answer some basic questions. Because there’s uncertainty in all of these numbers, the conclusion should be treated as an estimate. However, when possible, I’ve attempted to make my assumptions as conservative as possible. Therefore, it’s quite plausible that the required number of engineers is actually less than half the number that I’ll compute here.

Question 1: How many software engineers are there? I’m going to restrict this analysis to the developed world, because it’s hard to enforce cultural changes globally. Data varies, but most surveys indicate that there are about 1.5 million software engineers in the U.S. If we further assume the U.S. to be one-fifth of “the developed world”, we get 7.5 million full-time software engineers in the world. This number could probably be increased by a factor of two by loosening the definition of “software engineer” or “developed world”, but I think it’s a good working number. The total number of software engineers is probably two to three times that.

Question 2: What is the distribution of talent among software engineers? First, here’s the scale that I developed for assessing the skill and technical maturity of a software engineer. Defining specific talent percentiles requires specifying a population, but I think the software engineer community can be decomposed into the following clusters, differing according to the managerial and cultural influences on their ability and incentive to gain competence.

Cluster A: Managed full-time (75%). These are full-time software engineers who, at least nominally, spend their working time either coding or on software-related problems (such as site reliability). If they were asked to define their jobs, they’d call themselves programmers or system administrators: not meteorologists or actuaries who happen to program software. Engineers in this cluster typically work on large corporate codebases and are placed in a typical hierarchical corporate structure. They develop an expertise within their defined job scope, but often learn very little outside of it. Excellence and creativity generally aren’t rewarded in their world, and they tend to evolve into the stereotypical “5:01 developers”. They tend to plateau around 1.2-1.3, because the more talented ones are usually tapped for management before they would reach 1.5. Multiplier-level contributions for engineers tend to be impossible to achieve in their world, due to bureaucracy, limited environments, project requirements developed by non-engineers, and an assumption that anyone decent will become a manager. I’ve assigned Cluster A a mean competence of 1.10 and a standard deviation of 0.25, meaning that 95 percent of them are between 0.6 and 1.6.

Cluster B: Novices and part-timers (15%). These are the non-engineers who write scripts occasionally, software engineers in their first few months, interns and students. They program sometimes, but they generally aren’t defined as programmers. This cluster I’ve given a mean competence of 0.80 and a standard deviation of 0.25. I assign them to a separate cluster because (a) they generally aren’t evaluated or managed as programmers, and (b) they rarely spend enough time with software to become highly competent. They’re good at other things.

Cluster B isn’t especially relevant to my analysis, and it’s also the least well-defined. Like the Earth’s atmosphere, its outer perimeter has no well-defined boundary. Uncertainty about its size is also the main reason why it’s hard to answer questions about the “number of programmers” in the world. The percentage would increase (and so would the number of programmers) if the definition of “programmer” were loosened.

Cluster C: Self-managing engineers (10%). These are the engineers who either work in conditions of unusual autonomy (being successful freelancers, company owners, or employees of open-allocation companies) or who exert unusual efforts to control their careers, education and progress. This cluster has a larger mean competence and variance than the others. I’ve assigned it a mean of 1.5 and a variance of 0.35. Consequently, almost all of the engineers who get above 2.0 are in this cluster, and this is far from surprising: 2.0-level (multiplier) work is very rare, and impossible to get under typical management.

If we mix these three distributions together, we get the following profile for the software engineering world:

Skill Percentile
1.0     38.37
1.2     65.38
1.4     85.24
1.6     94.47
1.8     97.87
2.0     99.23
2.2     99.78
2.4     99.95
2.6     99.99


How accurate is this distribution? Looking at it, I think it probably underestimates the percentage of engineers in the tail. I’m around 1.7-1.8 and I don’t think of myself as a 97th-percentile programmer (probably 94-95th). It also says that only 0.77 percent of engineers are 2.0 or higher (I’d estimate it at 1 to 2 percent). I would be inclined to give Cluster C an exponential tail on the right, rather than the quadratic-exponential decay of a Gaussian, but for the purpose of this analysis, I think the Gaussian model is good enough.

Question 3: How much value does a software engineer produce? Marginal contribution isn’t a good measure of “business value”, because it’s too widely variable based on specifics (such company size) but I wouldn’t say that employment market value is a good estimate either, because there’s a surplus of good people who (a) don’t know what they’re actually worth, and (b) are therefore willing to work for low compensation, especially because a steady salary removes variance from compensation. Companies know that they make more (a lot more) on the most talented engineers than on average ones– if they have high-level work for them. The better engineer might be worth ten times as much, but only cost twice as much, as the average. So I think of it in terms of abstract business value: given appropriate work, how much expected value (ignoring variance) can that person deliver?

This quantity I’ve assumed to be exponential with regard to engineer skill, as described above: an increment of 1.0 is a 10-fold increase in abstract business value (ABV). Is this the right multiplier? It indicates that an increment of 0.3 is a doubling of ABV, or that a 0.1-increment is a 26 percent increase in ABV. In my experience, this is about right. For some skill-intensive projects, such as technology-intensive startups, the gradient might be steeper (a 20-fold difference between 2.0 and 1.0, rather than 10) but I am ignoring that, for simplicity’s sake.

The ABV of a 1.0 software engineer I’ve estimated at $125,000 per year in a typical business environment, meaning that it would be break-even (considering payroll costs, office space, and communication overhead) to hire one at a salary around $75,000. A 95th-percentile engineer (1.63) produces an ABV of $533,000 and a 99th-percentile engineer (1.96) produces an ABV of $1.14 million. These numbers are not unreasonable at all. (In fact, if they’re wrong, I’d bet on them being low.)

This doesn’t, I should say, mean that a 99th-percentile engineer will reliably produce $1.14 million per year, every year. She has to be assigned at-level, appropriate, work. Additionally, that’s an expected return (and the variance is quite high at the upper end). She might produce $4 million in one year under one set of conditions, and zero in another arena. Since I’m interested in the macroeconomic effect of increasing engineer autonomy, I can ignore variance and focus on mean expectancy alone. This sort of variance is meaningful to the individual (it’s better to have a good year than a bad one) and small companies, but the noise cancels itself out on the macroeconomic scale.

Putting it all together: with these estimates of the distribution of engineer competence, and the ABV estimates above, it’s possible to compute an expected value for a randomly-chosen engineer in each cluster:

Cluster A: $185,469 per year.

Cluster B: $89,645 per year.

Cluster C: $546,879 per year.

All software engineers: $207,236 per year.

Software engineers can evolve in all sorts of ways, and improved education or more longevity might change the distributions of the clusters themselves. That I’m not going to model, because there are so many possibilities. Nor am I going to speculate on macroeconomic business changes that would change the ABV figures. I’m going to focus on only one aspect of economic change, although there are others with additional positive effects, and that change is the evolution of an engineer from the micromanaged world of Cluster A to the autonomous, highly-productive one of Cluster C. (Upward movement within clusters is also relevant, and that strengthens my case, but I’m excluding it for now.) I’m going to assume that it takes 5 years of education and training for that evolution to occur, and assume an engineer’s career (with some underestimation here, justified by time-value of money) lasts 20 years, meaning there are 15 years to reap the benefits in full, plus two years to account for partial improvement during that five-year evolution.

What this means is that, for each engineer who evolves in this way, it generates $361,410 per year in value for 17 years, or $6.14 million per engineer. That is the objective benefit that accrues to society in engineer skill growth alone when a software engineer moves out of a typical subordinate context and into one like Valve’s open allocation regime. Generating $1 trillion in this way requires liberating 163,000 engineers, or a mere 2.2% of the total pool. That happens even if (a) the number of software engineers remains the same (instead of increasing due to improvements to the industry) and (b) other forms of technological growth, that would increase the ABV of a good engineer, stop, although it’s extremely unlikely that they will. Also, there are the ripple effects (in terms of advancing the state of the art in software engineering) of a world with more autonomous and, thus, more skilled engineers. All of these are substantial, and they improve my case even further, but I’m removing those concerns in the interest of pursuing a lower bound for value creation. What I can say, with ironclad confidence, is that the movement of 163,000 engineers into an open-allocation regime will, by improving their skills and, over the years, their output, generate $1 trillion at minimum. That it might produce $5 or $20 trillion (or much more, in terms of long-term effects on economic growth) in eventual value, through multiplicative “ripple” effects, is more speculatory. My job here was just to get to $1 trillion.

In simpler terms, the technological economy into which our society needs to graduate is one that requires software (not to mention mathematical and scientific) talent at a level that would currently be considered elite, and the only conditions that allow such levels of talent to exist are those of extremely high autonomy, as observed at Valve and pre-apocalyptic Google. Letting programmers direct their own work, and therefore take responsibility for their own skill growth is, multiplied over the vast number of people in the world who have to work with software, easily a trillion-dollar problem. It’s worth addressing now.

Programmers don’t need a union. We need a profession.

Every so often, I read a blog post or essay about the undesirable state of the software industry, and occasionally someone will suggest that we ought to unionize, in order to put an end to long hours and low pay compared to the value we produce. The argument is that, because software engineers are potentially worth millions per year to their employers, collective bargaining is the best way to capture more of this value. I disagree that a labor union is the way to go, because of the highly variable output of a software engineer, and the need for continuing education in this line of work. What we actually need is a profession.

Unions work best for commodity labor, and I use that term non-pejoratively. Commodity work is easily measurable and people can often be individually evaluated for performance. For example, a fishing boat operator is measured according to the quantity of fish she procures. A lot of very important work is commodity labor, so I don’t intend to disparage anyone by using that term. Commodity work can be unionized because there aren’t large and often intangible discrepancies in quality of output, and collective bargaining is often the best way to ensure that the workers are fairly compensated for the value they produce. Software is not commodity work, however. It’s difficult to measure quality, and the field is so specialized that engineers are not remotely interchangeable. When the work is difficult to measure and large disparities of quality exist, you have a situation in which a certain less-egalitarian (in the sense of allowing top performers to receive high compensation, because it’s essential to encourage people to improve themselves) and more self-regulatory structure is required: a profession.

The term professional is one of the most overused words in the English language, conflated often with white-collar work in general. If you work in an air-conditioned environment, someone will call you a professional. Most white-collar workers are not professionals, even in industries like investment banking, consulting, and management. Professionalism has nothing to do with social status, job category, or behavior (in the sense of an “unprofessional comment”). Rather, it’s all about a certain style of ethical structure, and the structure of a profession is nearly nonexistent in most white-collar industries (and under attack within the true professions).

What is a profession?

A profession is an attempt to impose global structure over a category of specialized, cognitively intensive work where the quality of output has substantial ramifications, but is difficult (if not, in the short term, impossible) to measure, giving ethics and competence primary importance. A profession is needed when it’s clear that not everyone can perform the work well, especially without specialized training. Here are some traits that signify the existence of a profession.

1. Ethical obligations that supersede managerial authority. Professions have global ethical standards that the professional may not break, even under pressure from an employer. “Following orders” is not a defense. Therefore, the professional is both allowed and required to be autonomous in ethical judgment. It is rare for people to be fired, in true professions, because of disagreements with managers or political misfortune, but unethical behaviors are punished severely (in the worst cases, leading not only to termination but expulsion from the profession). This said, ethical demands on a profession may not match the common sense we have of ethics; they must be industry-specific. For example, attorneys’ stringent requirement to keep client information secret (attorney-client privilege) supersedes concerns regarding whether the client’s behavior itself is ethical; it’s not for the attorney to make that call.
2. Weak power relationships. This is directly related to the above. In order to prevent the ethical lapses that are disturbingly common in non-professional work, and the abuses of power that are often behind those, professions deliberately weaken supervisory power relationships, with the intention of making it difficult for a manager to unilaterally fire an employee or damage his reputation. The result of this is that professionals answer to their companies or the profession directly. Professions attempt deliberately to be “bossless” so people will do what’s right rather than what’s politically expedient.
3. Continuing improvement and self-direction as requirements. The professions assume change both in terms of what kinds of work will be valuable, and of what tools will be available to do it, and expect people to dedicate time to adapt to it without managerial permission or direction.
4. Allowance for self-direction. Professionals are expected to place their career objectives at a high priority, since they serve the profession and their employers by becoming better at their work. Traditionally, metered work (that is, work directly relevant to the line of business) expectations in professions have been about 800 to 1200 hours per year, with the expectation that an equal amount will be spent on “off-meter” work such as attending conferences, reading journals, pursuing exploratory projects, mentoring others, and (at senior levels) looking for business or supporting the profession.
5. Except in egregious cases, an agreement between employee and firm to serve each others’ interests, even after employment ends. Professionals are almost never fired, in the cinematic sense of the word. If a firm wants to get rid of someone, that person is encouraged to look for new employment on working time, and therefore allowed to represent himself as employed and retain an income during the job search. When people actually are fired, they’re typically offered a severance. In the rare case that a professional firm must lay people off or close an office for economic reasons, it gives as much notice as it can of the change, and announces the layoff in the public to protect departing employees’ images. That employee will almost always be given a positive reference (not the neutral “name and dates” reference that is often taken as negative) in exchange for his time. Efforts are made to protect the reputations of departing employees, who are expected to return the favor by speaking well of previous companies and colleagues. Additionally, employees who leave the firm are treated as alumni rather than deserters.

If I were to sum it up, I’d say that professionalism is about liberal service, and my use of the word “liberal” has nothing to do with political ideology. Rather, it’s the same etymology as “liberal arts”, where “liberal” means “pertaining to free people”. Professional service is that which is judged to be intellectually and ethically demanding enough that it should only be done by free people, not serfs or “wage slaves” who lack the autonomy for trust to be placed in them. The efforts that professions exert to curtail managerial authority are in place to prevent situations of ethical compromise, as well as political influences that might result in inferior work.

This is not to say that the professions are workplace utopias. I doubt that any workplace is perfectly clean, politically speaking, and the imperatives of a profession may improve behavior, but can’t change human nature wholesale. What I would argue, however, is that the global structure and protection of a profession protects the individual worker, to some degree, from local adversity. A doctor can be fired by his supervisor, but he’s still a doctor. Regular workers lose their credibility when their employment ends. Professionals don’t.

It’s all about the “Bozo Bit”.

The difference between a profession and regular work is the default assumption about an unknown person’s character, intellect, and competence. In typical industrial work, the “Bozo Bit” starts out in the “On” position, meaning that a typical worker is assumed to be stupid, treacherous, and useless. Since barriers to entry are low, the only thing that defines a worker is wanting money, and being willing to do unpleasant tasks in order to get it. There’s no respect for the average, individual member in such an industry; the default assumption is incompetence, ethical depravity, and childlike stupidity. One has to prove non-bozoism, which is usually established with a managerial role or other formal sanction: a “manager-equivalent” job title. (Non-professional companies typically make the work environment bad enough for non-managers that the default assumption is that anyone competent will become a manager.) Proving non-bozoism is often very difficult to do, because workers who attempt to do so are often fired for attempting to make their case instead of doing their assigned work. If you’re assumed to be a bozo, you probably won’t get the autonomy that would enable you to prove otherwise.

In this style of working environment, people without official managerial sanction (such as coveted job titles) have no credibility and therefore can be discarded on a whim. People with such decorations have only minimal power, because they can be deprived of these assets immediately, and often unilaterally. The result is an environment where all the power is held by the people who disburse funds, job titles, performance reviews, and authority, and in which the workers have none. Except at the top, there is no autonomy, because a manager can reduce the worker’s credibility to zero for any reason or no reason at all.

Professions clear the Bozo Bit (i.e. default it to “Off”) by making it difficult for incompetent people to enter, and punishing objectively unethical people severely. This gives people the assurance that a professional is highly likely to be both competent and ethical. In doing so, they create a high-trust zone. A person who completes medical school is a doctor and, unless the license is revoked because of a serious ethical lapse, has valuable expertise. No manager or employer can take that credibility away for political reasons.

Is software engineering a profession?

No. It’s not.

Professionals have the right to disobey managerial authority if they believe they are ethically right in doing so. On the other hand, a software engineer who refuses to cut corners or hide defects under managerial instruction has no recourse and will probably be fired.

Professionals take great pains to avoid disparaging others’ work in the public, preferring to keep feedback private. Many software engineers work in companies where performance reviews are part of an employee’s packet any time she wishes to transfer, and wherein a negative review can leave a person internally black-listed.

Professionals are allowed and expected to dedicate about half their working time to career development and continuing education. Many software engineers will get fired if they get caught in that.

Professionals have an external credibility that is independent of reporting to a specific manager or being employed at a specific firm, which enables them to serve the profession and the abstract ethical principles it values. The rest of workers don’t. They have managers who can unilaterally fire them from their current jobs (or, at least, damage their reputations through the aggressive performance review systems that are now the norm). Therefore, they have no option other than to serve their direct manager’s career goals, even if those diverge from the company’s needs or the employee’s long-term career objectives.

Most software engineers fall into the “rest of workers” category. They don’t have the right to buck managerial authority, and companies rarely allot time for their continuing education and career needs. Some have to fucking use vacation days to attend conferences!

Are the professions still relevant?

In Corporate America, the professions have been losing ground for at least half a century. Attorneys’ metered work demands have grown substantially, so that associates can’t participate in the off-meter work on which their careers depend, resulting in slowed development. It’s now typical that a “biglaw” associate is expected to log 2000 billable hours (metered work) and it’s therefore nearly impossible for her to build the business contacts necessary for transition into partnership. Academia and basic research have lost most of their funding and prominence. Professionalism remains in medicine for doctors, but the aggressively anti-professional insurance companies are working just as hard to deny care as doctors do to deliver it. Journalism is becoming more about entertainment, as objectivity is thrown to the wind by many participants. In sum, it seems that all of the professions have seen an erosion of status and, for lack of a better word, professionalism over the past few decades.

Why is the corporate world killing the professions? There’s an irreducible disparity between the professions and corporations. Professions are republics that exist to serve global, ethical objectives, and professionals happen to make money because the service they provide is valuable to others. Corporations are autocratic, amoral machines that exist to make money regardless of whether they provide any useful service, and the scope of service expected is local: take direct orders, don’t think for yourself. Workers serve their managers, who serve executives, and executives serve the short-term vacillations of the company’s stock price. A worker doesn’t have the right to prioritize work that she considers more beneficial (for the company, or the industry) than what her manager has told her to do. In the non-professional world, she doesn’t get to make that call.

Contemporary corporate leaders consider the professions to be archaic (I consider them to be archaic, but I won’t go there just yet) relics of a pre-capitalist era, unable to compete in a more ruthless business environment. This explains their scorn for government officials, journalists, and most especially professors who are “insulated” from the market. From a business person’s perspective, what the professions value is, if not supported by the market, then not worth preserving. (One exception: attorney-client privilege is a very strong ethical demand for lawyers, and business leaders are very glad for that one!) For this reason, they’re hostile toward the core ideas coming out of professions: the right to resist authority, the expectation of doing what is right rather than what is expedient, the diversion of half one’s working time into continuing growth and development, and the encouragement to seek external visibility that gives a person credibility independent of employment status. Finally, business leaders view professions, which intercede against managerial authority and thereby confer benefits on the professionals, as extortive institutions like guilds, or perhaps even command economies.

Why are the business leaders wrong? They are right about the superiority of market economies over command economies, but I don’t think that point is in contention anymore. They are wrong because they believe the professions can’t thrive in the market. That they can has been proven by a Seattle video game company called Valve, one of the only companies to truly professionalize software development, and one of the most successful software companies on Earth.

I’ve written a lot about Valve and its open allocation policy, under which engineers are trusted to move their desks to join another team as they wish. There’s no transfer process, and teams don’t have “headcount” constraints. Projects that no one wants to do don’t get done, unless they are genuinely important enough for the executives to create an incentive for it. Project supervision is driven by leadership and consensus rather than executive authority. What is open allocation about, philosophically? (For more about Valve’s open allocation, read this excellent blog post by Yanis Varoufakis.) It’s about professionalizing game and software development. Engineers are actually trusted to work for the company, rather than a manager who might use authority to divert their efforts toward the manager’s own goals (including preservation of the power relationship). Valve’s employees have an ethical commitment– make Valve better– they are expected to hold, but they have autonomy in how they do so. They live in the sort of high-trust zone that professions exist to create.

Not only do I believe that the professions can thrive in the market, but I think that professionalized software development will be superior to the industrial-age managed framework. Most of the progress in software comes from creative successes, which can be thought of as black swans. It is impossible to manage them into existence, and subordination will kill them before they are born. The best you can do, if you have managerial authority, is to use it as rarely as possible. At scale, you can be confident that people will come up with great ideas, and that enough of them will be profitable to justify the time and resources committed.

Ethics of a Technologist

Should software become a profession? I would say that the answer is a resounding “yes”. This emerging profession should not require expensive schooling. One of the things that’s great about programming is that no educational credential is required to enter, and I think we should keep it that way. However, I think that the idea of a global ethical structure is a sound one, and here are some thoughts.

I am using the term “technologist” for two reasons. The first is that I don’t want to limit my scope to software engineers, but to include a larger set of people who work on technology, such as designers and software testers and startup founders. The second is that the most important thing for a technologist to do, at a given time, may not be to write code. Sometimes, writing more code is not the answer: cleaning the existing stuff is better. As professionals, we should expect ourselves to do what is right for the problem we are trying to solve, not what allows us to write the most code.

Here are some basic, bedrock ethical principles that I think should be part of the technologist profession.

1. Technologists do not create an inferior product for personal or single-party gain. We do not create “logic bombs” to extort clients or “back doors” that allow us to exploit systems we create. We do not create bugs for the purpose of “finding” them later. While we cannot ensure perfect software quality (it’s mathematically impossible to do so) we will not compromise on quality unless we believe it to be in the interest of all parties involved. We deliver the best product or service we can with the resources given to us, and if the resources given are not enough, we voice that concern.
2. Technologists collaborate. We do not compete by harming another’s performance. We believe the world to be positive-sum and our industry is structurally cooperative. Therefore, to the extent that we compete, it’s in the direction of making ourselves better at our jobs, not making others worse. We also do not make technical decisions for the purpose of reducing another technologist’s ability to perform.
3. Technologists do not disparage another’s performance to a non-technologist. Ever. All people with managerial authority are considered “non-technologists”, in the context of this item. Put simply: we don’t sell each other to outsiders. People who break this policy are fired for life from the profession. We handle our own affairs, period. If we need to remove a technologist from our team for reasons of incompetence or non-performance, we have the right to tell a non-technologist that this person cannot continue as a member of the team, and we are neither obligated nor allowed to give further reasons. We handle matters among ourselves, and do not attempt to use managerial warlords for personal gain. It’s up to us to form and disband teams, and to expel problematic members.
4. Technologists choose their own leaders. It is not for non-technologist meddlers to decide who are the leaders of our groups. We shall not answer to stooges selected by executives. We choose our leaders, typically through democratic processes, and leaders who fail to serve the groups they are supposed to lead shall lose that distinction.
5. Technologists work for the greater good of the company that employs them, the Technologist profession, and the greater good of the world. We serve the world first; our job is to make it better by improving technical processes, solving difficult but important problems, and advancing the state of science and rationality. Secondarily, we serve the Technologist profession and its values. As a tertiary concern, we act in the interests of our employers. We solve their technical problems and work to improve their infrastructure. Technologists are trusted to serve their employers directly, and any management that exists must be purely for providing guidance, not an irresistible authority. As a quaternary concern, technologists are expected and allowed to prioritize their own career growth, making themselves more valuable in the future.
6. Technologists are trusted to work on any project that will have them. It is the right of the project owner to allow (or expel) members. Projects shall not be constrained by “head count” limitations set by non-technologists, who are to be deemed unqualified to make such distinctions.
7. Technologists deliver on commitments they make. If a technologist cannot meet a commitment, he or she explains the cause as soon as possible, and attempts in earnest to address the shortfall. Additionally, it is not for managers to pressure technologists into making commitments that they would otherwise consider unreasonable. This “always deliver” policy only applies to freely made commitments, not any made under managerial duress.
8. Technologists have the right to refuse work, unless it is of existential importance to the firm, or work that they have freely committed to doing. The only time a technologist can be required to complete a project he or she did not choose is when the company is at credible risk of failure or catastrophic loss if the work is not completed. Even then, it is best for the company to try to provide incentives (such as bonuses or promotions) before resorting to authority.
9. Technologists have the right to inquire about other specialties without facing professional repercussions. Technologists are not fired or dismissed as “not a team player” when they voice concerns about a discrepancy between the work expected of them and the direction they want to take in their career, or the specialties into which they want to grow. Rather, technologists are encouraged to be direct and forthright about the specialties that interest them, and companies shall make reasonable efforts to allow them to find appropriate work.
10. Making mistakes is tolerable; honesty about mistakes is required. The employment arena for a technologist must be one where people are not punished for making mistakes, or for discovering and revealing mistakes made by others. However, knowingly hiding a mistake, when it risks harm to others, is a grave ethical fault.
11. Terminated employees recieve proper career support. Companies will sometimes need to fire a technologist, if he is unable to lead (as determined by other technologists) or follow within the context of the firm. When companies terminate, they always grant the right to represent oneself as employed during the job search, and provide a positive reference unless the employee was fired for a grave ethical breach. Companies that would be considered (reasonable person standard) able to afford severance pay it, in a large enough amount to cover the expected time for a job search. This does not apply to pre-funded startups (the risk is well-known) or companies in financial distress (that can’t afford it).
12. Companies and investors do not create “no poach” or “do not hire” lists. Anyone who breaks these rules is censured severely and turned over to law enforcement. Investors are also disallowed from communicating any information about a technologist that might prevent him from getting further investment, unless a formal breach of ethics was involved.
13. 1200 hours of metered work, per year, as a maximal obligation under normal circumstances. The general expectation is that a full-time technologist will deliver 400 to 1200 hours per year of metered work (work that is relevant to the direct line of business, and non-discretionary) per year. Companies have the right to increase metered work expectations in unusual circumstances. It is up to the company to decide what those circumstances are, but if they exist, they should be disclosed before an employment agreement begins (e.g. these are situations we consider abnormal, and these are the expectations we’ll have if they occur). Otherwise, the default assumption is that the company should expect no more than this level of direct dedication. Expecting anything more is to expect the employee to take on unreasonable risk of career stagnation. This 1200-hour standard shall be pro-rated for people who have part-time employment agreements.
14. 800 hours of off-meter work as a minimal expectation of the technologist. ”Full time” (2000 hours per year) technologists are usually expected to dedicate the remaining (800 or more) hours to off-meter work: continuing education, attending conferences, exploratory work and research, career-directed open source work, and pursuit of other specialties. This is not an obligation per se, but technologists should expect to spend this time if they wish to be maximally relevant, and companies shall be expected to allow time for it. This eliminates the excuse that many programmers have currently for stagnation and mediocrity, which is that their bosses won’t allocate time for growth. If we become a technologist profession, we will make allowance for such time an inflexible pillar.

This is just a starting set of principles. I’m sure I will remember more, but my intent here is to indicate what a technologist profession will look like.

One final thought: selectivity

My word count’s getting high, indicating that I should wrap this up, but I’d like to address one other concern: Who do we let in? What’s the barrier to entry? Here are my initial thoughts.

No formal education is required to become a technologist. Technologists are expected to have the breadth of knowledge of an average college graduate from a top-50 university, but how they get it is up to them. Age, socioeconomic status, educational matriculations, and national origin are irrelevant. Anyone with the competence can be a technologist, but we shall set the bar for competence very high. I think the best model for this is actuarial science in the United States, where progress is exam-based. Classes and study guides are available, but not required. Additionally, the profession requires and expects that associates will dedicate a significant fraction of their working time to studying for the exams.

There are a few tweaks I’d make to this system. The first is that, instead of a linear series of eight or ten exams, there shall be a larger number, with some being elective. The courses on technological ethics, basic mathematics, code quality and scientific thought would generally be required, for example. Machine learning, startup economics, and compilers courses would be optional. The second is that some courses would require code, and most courses would have a non-exam option whereby, for example, a high-quality open-source contribution could be substituted for a typical exam. (This is because, while most people who are “not a good test taker” are just lazy, the condition does exist and alternative evaluation is appropriate for making the profession maximally inclusive of talent.)

The purpose of these exams would be to provide an alternative path to success and true independent credibility for technologists, and to deprive parochial managers of the ability to reduce a technologist’s credibility to zero. They would not necessarily be required for a technologist to have employment, but they would be designed to be difficult and relevant enough to increase a professional’s employability dramatically– so dramatically as to give the technologist true independence of managerial authority.

That is all I have to say on this matter tonight, so I yield the floor.

Functional programming is a ghetto

Functional programming is a ghetto.

Before any flamewars can start, let me explain exactly what I mean. I don’t mean “functional programming sucks”. Far from it. The opposite, actually. Not all ghettos are poor, crime-ridden, and miserable. Jewish ghettos existed for centuries in Europe, from the Renaissance to World War II, and many were intellectual centers of the world. Some were quite prosperous. Harlem was, at one time, an upper-middle-class African-American community at the center of some of America’s most important artistic contributions. The same is true of functional programming. It’s the underappreciated intellectual capital of the programming world, in that its ideas (eventually) trickle down into the rest of the industry, but it’s still a ghetto. A ghetto is the urban analog of a geographic enclave: it’s included in the metropolis, but culturally isolated and usually a lot smaller than the surrounding city. It often harbors those who’ve struggled outside of it. Those who become too used to its comforts view the outside world with suspicion, while those on the outside have a similar attitude of distrust toward those within. Ghettos usually imply that there’s something involuntary about being there, but that’s often not the case. Chinatowns are voluntary ghettos, in the non-pejorative sense, as are some religious communities like monasteries. “Functional programming” is, likewise, a voluntary ghetto. We’ve carved out an elite niche in the software industry, and many of us refuse to work outside of it, but we’re all here by choice.

What is functional programming? Oddly enough, what I’m about to talk about is not functional programming in the purist sense, because most “functional programmers” are not averse to using side effects. The cultural issues surrounding functional programming are not about some abstract dislike of computational effects, but rather an understanding of the necessity of managing the complexity they create, and using tools (especially languages) that make sane development possible. Common Lisp, Scala, and Ocaml are not purely functional languages, but they provide native support for the abstractions that make functional programming possible. What real functional programmers do is “multi-paradigm”– mostly functional, but with imperative techniques used when appropriate. What the debate comes down to is the question of what should be the primary, default “building block” of a program. To a functional programmer, it’s a referentially-transparent (i.e. returning the same output every time per input, like a mathematical function) function. In imperative programming, it’s a stateful action. In object-oriented programming, it’s an object, a more general construct that might be a referentially-transparent function, might represent an action in a hand-rolled domain-specific language (DSL) or might be something else entirely. Of course, most “object-oriented programming” becomes a sloppy mix of multiple styles and ad-hoc DSLs, especially as more than one developer comes to work on an object-oriented project. That’s a rant for later.

In general, functional programming is right. The functional approach is not right for every problem, but there is a right answer regarding the default abstractions for building most high-level programs: immutable data, and referentially transparent functions should be the default, except in special cases where something else is clearly more appropriate. Why? A computational action is, without more knowledge, impossible to test or reason about, because one cannot control the environment in which it exists. One needs to be able to know (and usually, to control) the environment in which the action occurs in order to know if it’s being done right. Usually, the tester wants to be able to cover all special cases of this action, in which case knowing the environment isn’t enough; controlling it is also necessary. But at this point, the state of the environment in which the test happens is an implicit parameter to the action, and making it explicit would make it a referentially-transparent function. In many cases, it’s better to do so– when possible. It might not be. For example, the environment (e.g. the state in a computer cluster database) might be too large, complex, or volatile to explicitly thread into a function. Much of real-world functional programming is not about eliminating all state, but about managing what state is intrinsic, necessary or appropriate.

For a concrete example, let’s say I have a blackboard, face down, with a number (state) on it, and I ask someone to read that (call it n) and erase the number, then write n+1 on the blackboard. I’m assuming he won’t lie to me, and that he’s physically capable of lifting the board; I want to determine if he can carry out this operation. If I don’t know n, and only see what is written after he is done, I have no hope of knowing whether the person carried out my order correctly. Of course, I could control the testing enviroment and write 5 on the blackboard before asking him to do this. If he writes 6, then I know he did what I asked him to do. At that point, though, the blackboard isn’t necessary. It’s more lightweight to just ask him, “what is 5 + 1?” I’ve moved from an imperative style of testing to a functional one: I’m determining whether his model of the addition function gives the right answer, rather than putting him through an exercise (action) and checking the state after it is done. The functional alternative is a unit test. I’m not trying to assess whether he knows how to turn over a blackboard, read it, erase it, and write a new number on it, because I only care about whether he can add. If I want to assess all of those as well, then I need to make an integration test of it. Both types of test are necessary in real-world software engineering, but the advantage of unit tests is that they make it easy to determine exactly what went wrong, facilitating faster debugging.

Testing, debugging, and maintenance are a major component of real-world software engineering, and functional programming gives us the tools to tackle these problems in a tractable way. Functions should be referentially transparent and, ideally, small (under 20 lines when reasonable). Large functions should be broken up, hierarchically, into smaller ones, noting that often these small components can be used in other systems. Why is this desirable? Because modularity makes code reuse easier, it makes debugging and refactoring much simpler (fixes only need to be made in one place) and, in the long term, it makes code more comprehensible to those who will have to modify and maintain it. People simply can’t hold a 500-line object method in their heads at one time, so why write these if we can avoid doing so?

The reality, for those of us who call ourselves functional programmers, is that we don’t always write stateless programs, but we aim for referential transparency or for obvious state effects in interfaces that other programmers (including ourselves, months later) will have to use. When we write C programs, for example, we write imperative code because it’s an imperative language, but we aim to make the behavior of that program as predictable and reasonable as we possibly can.

Functional programming, in the real world, doesn’t eschew mutable state outright. It requires mindfulness about it. So why is functional programming, despite its virtues, a ghetto? The answer is that we tend to insist on good design, to such a degree that we avoid taking jobs where we’re at risk of having to deal with bad designs. This isn’t a vice on our part; it’s a learned necessity not to waste one’s time or risk one’s career trying to “fix” hopeless systems or collapsing companies. Generally, we’ve come to know the signs of necrosis. We like the JVM languages Clojure and Scala, and we might use Java-the-language when needed, but we hate “Java shops” (i.e. Java-the-culture) with a passion, because we know that they generate intractable legacy messes in spite of their best efforts. Say “POJO” or “Visitor pattern”, and you’ve lost us. This seems like arrogance, but for a person with a long-term view, it’s necessary. There are a million “new new things” being developed at any given time, and 995,000 of them are reincarnations of failed old things that are going to crash and burn. If I could characterize the mindset of a functional programmer, it’s that we’re conservative. We don’t trust methodologies or “design patterns” or all-purpose frameworks promising to save the world, we don’t believe in “silver bullets” because we know that software is intrinsically difficult, and we generally believe that the shiniest IDE provides us enough to compensate for its shortfalls and false comforts. For example, an IDE is useless for resolving a production crisis occurring on a server 3,000 miles away. We’d rather use vim or emacs, and the command line, because we know they work pretty much everywhere, and because they give us enough power in editing to be productive.

From a functional programmer’s perspective, it’s easy to mistake the rest of the software industry for “the ghetto” (especially considering the pejorative association, which I am trying to disavow, with that word). Our constructions are stable and attractive, and we do such a good job of cleaning up after ourselves that there’s not much horseshit on our streets. Outside our walls are slums with rickety, fifteen-story tenements that are already starting to lean. The city without is sloppy and disease-ridden and everything built in out there will be burned down, to kill the plague rats, in ten years. We don’t like to go there, but sometimes there are advantages of doing so– for one thing, it’s fifty times larger. If we lose awareness of size and scale and what this means, we can forget that we are in the ghetto. That’s not to say we shouldn’t live in one, for it’s a prosperous and intellectually rich ghetto we inhabit, but a ghetto it is.

I think most functional programmers only get a full awareness of this when we’re job searching, and thanks to most of us being in the top 5% of programmers, our job searches tend to be short. Still, I’ve lost count of the number of times over the past five years that I’ve found a job listing that looked interesting, except for its choice of language. “5 years of experience in Java, including knowledge of design-pattern best practices.” Nope. It might be a good company writing bad copy, but its technical choices look exactly the same as those of the bad firms, so how can I be sure? The process quickly becomes depressing. It’s not that Java or C++ are “dirty” languages that I would never use. It’s that any job that involves using these languages full-time is so likely to suck that it’s hardly worth investigating. Occasionally, C++ and Java are the right tools for the job, but no one should try to build a company on these languages. Not in 2012. Java isn’t a language that people choose to use, not for primary development. Not if they’ve used three or four languages in their career. It’s a language that people make other people use. Usually, it’s risk-averse and non-technical managers making that call. A Java Shop is almost always a company in which non-engineers call the shots.

What we call functional programming is somewhat of a shibboleth for good-taste programming. We prefer the best programming languages, like Ocaml and Clojure, but we don’t actually restrict ourselves to writing functional programs. Do we use C when it’s the right tool for the job? Hell yeah. Do we put mutable state into a program when it makes it simpler (as is sometimes the case)? Hell yeah. On the other hand, we trust the aesthetic and architectural decisions made by brilliant, experienced, gray-bearded engineers far more than we trust business fads. We have a conservative faith in simplicity and ease-of-use over the shifting tastes of mainstream managerial types and the superficial attractiveness of silver bullets and “methodologies”. We roll our eyes when some fresh-faced MBA tells us that structuring our calendar around two-week “iterations” will solve every software problem known to humankind. Unfortunately, this insistence (often in the face of managerial authority) on good taste makes us somewhat unusual. It stands out, it can be unpopular, and it’s not always good for one’s career. Few stand with us. Most leave our camp, either to become managers (in which case, even a Java Shop is a plausible employer) or to accept defeat and let bad taste win. It’s hard to live in a ghetto.

Now, I have little faith in the stereotypical average programmer, the one who never thinks a technical thought after 5:01 pm, and who doesn’t mind using Java full-time because the inevitable slop is the problem of some “maintenance guy”. That person probably shouldn’t be programming. On the other hand, we’re about 2 percent of the software industry, if that, right now. We can reach out. We can do better. We’re not so brilliant that the other 98% of programmers have no hope of joining us. Not even close. There are many IDE-using, Java-hacking, semi-bored developers who are just as smart as we are but haven’t seen the light yet. It’s our job to show it to them, and if we fail to convince them that they could become 2 to 10 times more productive than they ever dreamed of being, and that programming can become fun again, then we’re the ones to blame. We must reach out, and we can probably bring 10, 20, maybe even 30 percent of programmers over to the light side, bringing about dramatic changes in the software industry.

I think the integrity of our industry depends on our ability and willingness to figure out how to do this.

Radical Transparency #1: who gets fired, how, and why.

Personal note: I wrote the bulk of this material in February 2011. I recently left a job (from which I was not fired) and admit the timing is less than perfect. There’s no connection between this essay and my personal situation, although its insights are derived from dozens of observations over the past 6 years.

I’m writing this post for a political purpose, not a personal one. Namely, I wish to provoke an epic showdown between the cognitive “1 percent” and the socioeconomic “1 percent”– a pleasant euphemism in the latter case, since it’s really 0.1% of Americans who are in access to make major decisions. I wish to force this conflict as fast as I can I believe technological trends are in our favor and we will win. It is not enough to defeat the socioeconomic “(0.)1 percent” as a class or as individuals. It is far more important that we dismantle the ugly world that they have created. If we destroy them personally but not the execrable processes they’ve set in motion, others will step up to replace each one we remove from power. We need to change the processes, which involves changing the discussion. Radical transparency is the first step toward doing this. 

I’m going to shed some light on the processes by which people are separated from companies. Involuntary termination. Layoffs. Getting fired. Shit-canned. Pink-slipped. There’s obviously a lot of nastiness surrounding this function, despite its absolute necessity to the health of an organization. I don’t intend to say that firing people is mean, or wrong, or unethical. Far from it, it’s something that all organizations will need to do from time to time. Some people are frankly toxic or unethical, and others are just not able to perform well within the organization, and must be let go.

In fact, I’d argue that the results of what happens when incompetent people are rarely fired can be seen in U.S. politics, where we face one of the most disastrously ineffective legislatures in human history because voters are so bad at firing idiots. Political officials are subjected to periodic elections (360-degree performance reviews) which are supposed to be intensely competitive. Yet the incumbent-victory rate is astonishingly high: over 95 percent in American political elections. The job-loss rate of political officials is less than 2 percent per year, despite their poor performance. In finance and technology, even people who are extremely good at their jobs have higher risk of involuntary termination than that. That contrast, I consider illustrative. For reasons like this, I will never say it is wrong to fire people, although it’s important to be decent about it. More on that later.

Two years ago, one of my friends was served with a Performance Improvement Plan (PIP) issued to him from a large company. If there is a TPS Report Museum, there must be an entire wing dedicated to PIPs. I’ll say one thing about these: they should never be used, and they don’t work. The first way in which PIPs fail is that they don’t work at improving performance. A manager who genuinely wishes to improve an employee’s performance will address the matter, one-on-one, with the employee in a verbal meeting (or series of meetings) where the intention is discovering the cause (“blocker”) of low performance, and decide either (a) to resolve this problem, if it can be done, (b) to accept transient low performance if the cause is a temporary one such as a health problem, or (c) to determine that the problem is irresolvable and terminate the employee (preferably in a decent way). On the other hand, written negative communication about performance (formalized most finally in a PIP) is universally interpreted as an aggressive move and will lead to distrust of the manager, if not outright contempt toward him. As soon as a manager is “documenting” negative performance, the relationship between him and his report has failed and he should progress immediately to a decent termination. Never PIP. Fire, but do it right.

What’s a decent termination? It’s one in which the employee is allowed to resign, provided with a positive reference (both in writing, to guarantee it, and verbally when needed) and a severance package equal to between 1 and 1.5 times the average duration of a typical search for jobs of that kind (between 2 and 3 months for junior-level positions, and 6 to 9 months for executives, and with an additional multiplier to 1.5-2 granted to those who are “overexperienced”) to compensate the employee for the unexpected job search. Of course, companies have no legal obligation to any of this, but it’s the right way of doing things. In fact, because financial pressures can result in a suboptimal job search, I’d argue that severance (when the cause issue is fit or performance rather than unethical behavior) is an ethical obligation.

Speaking of this, a lot of extremely unethical things happen in American workplaces, and that is a result not of “bad people”, but the bulk of this behavior comes from morally-average people who are scared. One of the things people fear most at work is a sudden, unjust, badly-structured termination that leads to a long-term career problem. This fear motivates a lot of the bad activities that occur in workplaces that lead to unsafe products and defrauded customers. The best thing a company can do for its culture, and for its macroscopically visible good citizenship, is to establish a policy of managing terminations in a proper way– to say that, yes, we’ll fire people whose poor performance constitutes an unacceptable cost to us, but we’ll always do so in a way that ensures that good-faith low performers (i.e. decent people who are a bad fit for their role) move on to more appropriate jobs.

How does a PIP actually affect performance? First, it destroys the relationship between the manager and the employee, who now feels “sold out”. If claims in the PIP suggest that others contributed to it, it may destroy the working relationship between the employee and his colleagues, causing isolation. PIPs usually carry a biased or even inaccurate summary of the employee’s work as the motivation for the Plan. Second, PIPs often generate a lot of additional work for the employee, making it harder to perform. A PIP usually contains deadlines equivalent to a 40-hour per week work schedule. This seems reasonable, except for the fact that many work demands are unplanned. An employee who faces responsibility for an emergent production crisis during a PIP will be forced to choose between the PIP work or the emergent crisis. A PIP’d employee ends up actually ends up with four conflicting jobs. The first is the work outlined in the PIP, which is already a 40-hour obligation. The second is any emergent work that occurs during the PIP period (which is usually unspecified in the PIP, but claimed to be covered by a vague “catch-all” clause). The third is the legalistic fighting of the PIP– the employee must contest false claims about his performance or the company will represent him as having agreed with them, which damages his position if he ends up in severance negotation. The fourth is the job search process, which must be commenced right away because the PIP usually ends in termination without severance.

This brings us to the real purpose of PIPs, which is to legally justify cold-firing employees. The PIP is the severance package. Defenders of PIPs represent it as cheaper (and it’s not, because of contagious morale effects) to keep a burned-out, miserable employee in the office for a month or two than to fire him and pay a 3-month severance package. Employees who are terminated under a PIP are usually granted no severance, because their termination is “for cause”. There’s a lot of intimidation that usually goes on here. Because the PIP is confidential, employees are usually told they will be breaking the law if they show the PIP to anyone, even an attorney. (Not true.) They’re also told that they can be terminated “for cause” (read: without severance) and that it will damage their ability to get a reference if they don’t sign the PIP, damaging their ability to contest it later. This one is true (employment is at will, and severance is not a legal obligation) but a company that gives a bad reference, especially for retaliatory reasons, will not fare well in court.

I was once privy to managerial statistics at a large company. Over a 12-month window during a time of average job fluidity, 82 out of 100 of PIP’d employees left their jobs within the duration (30-60 days) of the PIP. These might be judged to be “success” cases (they left) but if this was the desired outcome, it could have been suggested in an in-person discussion without involving nasty paperwork. In another 10 cases, the employee fails the PIP and is usually fired. In 6 more cases, the PIP is ruled “inconclusive” because of factual inaccuracies, or because the PIP’d employee is able to garner support from elsewhere in the organization, and transfer to a more sympathetic manager, who kills the PIP. Only 2 actually “pass” the PIP. Also, a PIP’d employee is likely to face another PIP within six months if he stays with the same manager, while most managers don’t want to take a chance on someone who faced a PIP (even successfully) because, statistically speaking, an unblemished new hire is a better bet.

My advice to PIP’d employees is as follows. As I said, you’ve got 4 jobs now: (a) the PIP work, (b) emergent work, (c) legalistic fighting, and (d) finding another job in the event of failing the PIP. Regarding the first, don’t blow off the PIP work entirely, or at least not openly. It may seem like it’s pointless to do this work (and it is, because your future at that company has ended) but one needs to show a good-faith effort in any case. PIPs are not guarantees of employment during the specified duration; they can be “closed” early if it is judged as obvious that the employee will not meet the PIP’s deadlines. However, in the rare case that the PIP is actually fairly structured and you can complete it in time, do not use the “surplus” to move forward on the PIP work. Instead, use this time to learn new technologies (for the job search) and contribute to open-source projects (only on your own resources). Even if you pass the PIP, your days are numbered. Regarding the second, request in writing that the manager place any emergent responsibilities on the PIP calendar. Now is the time to document everything and be a pain in the ass. If he ignores these requests, you can make a case to HR to rule the PIP inconclusive, buying you time until the manager’s next opportunity to be PIP (usually a quarter or two later) you comes along. You should have a new job before that happens. On the third of your four jobs– the legalistic wrangling– document everything, but keep communication terse, objective, and professional. Don’t spend more than an hour per day on it, but if you can waste your manager’s time with HR paperwork in the hope of getting the PIP ruled inconclusive, do it. The fourth of these– the job search– is where one’s highest priority should be focused. Represent interviews (in email; again, document everything) as doctors’ appointments. This is also a great opportunity to disclose health issues, which reduces the probability of being fired without severance.

My advice for companies or managers considering the use of PIPs. Don’t. Sure, you’ve got version control and backups, but do you really want an essentially fired employee in the office for a month? It’s amazing to me that companies have security escort employees out of the building when they are officially terminated (at which point the shock has passed and emotions are likely to be resolved) but keep them in the office for months on PIPs, when they are effectively fired. PIP’d walking dead are far more damaging to morale than terminations.

Finally, my advice for employees who actually get fired is simple. Don’t be a dick. Disparaging firm-wide emails will ruin your reputation, and property damage is illegal and fucked-up and wrong, and therefore not in your interest at all. Depart cordially. On references, there are two kinds: unofficial (which can be a peer you select) and official (manager or HR). Usually official references say very little, but have your reference from that company checked by a professional service and get an attorney involved if anything negative is being said about you. Cultivate positive relationships with colleagues after your exit. Most importantly, move on quickly. When asked about the previous job, stay professional and represent the termination as one that you initiated. If your ex-employer ever contradicts you and says you were fired, then congratulations: you get a huge settlement for a negative reference.

Also, don’t let it hurt your self-esteem. There are two rites of passage in the modern corporate world. The first is getting fired. The second is learning that getting fired is not usually a reflection on the employee (more on this later, on who gets fired). Your self-esteem will benefit if you experience the second before the first.

Ok, so that’s a bit about the process. It’s important for people to know this stuff, so they know how to box if the fight comes to them. Now, I’d like to shed some light on who gets fired. Low performers, right? Well, not always. Companies initiating impersonal layoffs for financial reasons will usually drop low-performing teams and businesses, but not all those people are individually low performers. Personal firings are a different story. These don’t occur because “companies” choose to fire these people; people do. So what are some of the motivations that lead to this? First, let me say that political (non-necessary) firings usually occur in the middle-bottom and middle-top of an organization’s performance spectrum. The bottom 10% are usually so used to lifelong, serial incompetence that they’ve mastered office politics as a survival skill. The top 10% are objective, visible high-performers who tend to have political clout on account of the value they add to the organization. The middle 60 percent rarely stick out enough to get into trouble. This leaves two deciles– the 2nd and the 9th– that are most exposed to personal terminations.

Second-decile firings tend to be “trophy firings”, initiated by managers who wish to impress upon higher-ups that they “take action” on low performers. They aren’t the worst people in the organization, and whether it’s of economic benefit to retain them is not strongly resolved either way, but they’re weak enough that if they leave the organization, they won’t be missed.

Ninth-decile firings are “shot-from-the-bridge” firings, initiated against a high performer by a jealous peer before she develops the credibility and capability to become a threat (i.e. graduate into the objectively and visibly high-performing 10th decile). These people are usually fired because they’re creative, passionate, and opinionated. Now, managers don’t initiate ninth-decile firings, at least not willingly. Managers want more people to be more like their 9th- and 10th-decile reports. Rather, those firings to be initiated by “young wolves”, who are typically same-rank co-workers, but sometimes insecure middle managers, who sabotage the target’s ability to perform. The textbook young wolf is Pete Campbell in the early seasons of Mad Men, the slimy and machiavellian junior account executive who lies and threatens his way to the top. (Disclaimer: I wish the term for such people wasn’t young wolves. I like wolves. I like them better than most people.)

A direct confrontation with a manager about a 9th-decile report is not a politically wise idea, because managers know that 9th-decile employees are highly valuable. So young wolves focus on giving the target some disadvantage in project allocation. Most technical teams have a “people manager” (who is savvy to political machinations) and a technical lead (who is not usually politically savvy, but has authority to allocate projects, make technical decisions that affect the team, and delegate work). Tech leadership is a hard role because (a) it involves a lot of responsibility with minimal glory, and often no additional compensation, and (b) tech leads are responsible as individual contributors in addition to their leadership role. Thus, the tech lead is usually overburdened, and young wolves usually cozy up to him, with the intention of “helping out” on work allocation. Then they start delegating impossible, discouraging, or unsupported work to the target, the 9th-decile employee they want to shoot from the bridge before they become 10th-decile. When the target’s performance drops (transiently) to about the 7th or 8th decile, they begin to speak about how she “does good work, but isn’t as good as we thought she’d be”. As she sees leadership opportunities and the most interesting projects receding from view, her motivation declines further. By the time she’s at the 5th or 6th decile (because she’s now beginning to look for other opportunities, and prioritizing work that benefits her external career) the party line is that she’s “not giving her best”, which is, by this point, entirely true and equally irrelevant to whether it’s economically beneficial to employer to retain her. Social rejection of her is encouraged through gossip about her being “not a team player” and whispered suspicions that she’s looking for employment elsewhere. When she enters the 3rd or 4th decile, the young wolves engage her manager and– if the manager is unaware of what’s happened (a young-wolf situation)– she receives the first official warning signs, such as negative feedback delivered over email instead of in spoken form. This is an aggressive move that confirms this now-declining employee’s suspicions that her future there is over. If she’s still working there (unlikely, in a decent economy) by the time she falls into the 2nd decile, she’s now a trophy-fire.

This process of targeted, intentional demotivation, prosecuted not by managers (whose interests are aligned with the success of the whole team) but by young wolves, can occur over years or over days, but it’s a discernable pattern that I’ve seen play out dozens of times. The problem is that “flat” managerial arrangements don’t work. I don’t like the alternative (official hierarchy) much, so I wish they did work, but they almost always fail. A “flat” organization means “we’re large enough that we need hierarchy but we’re too socially inept to have hard conversations”. In most cases, I actually think that large companies (for post-industrial knowledge work) are inherently broken, because neither the hierarchical model nor the flat model results in good products. Why is “flat” so bad? It means that a manager has a ridiculous number of reports (sometimes over 50) and that it’s impossible for him to be on the lookout for young-wolf behavior, which is immensely damaging to the company because it tends to drive out the most creative individuals. Young wolves thrive in environments of managerial inattention.

How does one combat a young wolf within an organization? As with any progressive, but curable, parasitic infection, early detection is the key. A decent manager who identifies a young wolf will send it to the pound and have it put down, but effective young wolves evade managerial detection for a long time. Defining the term helps, but there’s a danger here. If “young wolf” enters the business lexicon, it will just become a term of disparagement that young wolves use for their targets. The most effective young wolves, instead of declaring their targets “not a team player”, will instead attempt to paint their targets as young wolves. So here, I don’t have the answer, but I have a thought.

I’ve been coding for five years, and I’ve made a serious mistake. At this point, I have no open source contributions. Two and a half years I put into a failed startup that should have allowed me to open-source my work (30,000 lines of Clojure code to build a custom graph-based database, and some of the best technical work of my life) but because the CEO of that failed startup is a vindictive asshole who wishes to punish me for “leaving him”, so that work is gone. For this entire five years, I was a “company man”, pumping 50 to 60 hours per week of software effort into companies with which I no longer have a relationship. I have a “great resume”, but nothing in the open-source world to show that I can actually code. I’ll be just fine, and I’m going to take some time to write some open-source work, but that’s a serious regret I have over how I’ve spent the last half-decade. Why’s open-source software so important? Radical transparency. It provides a socially beneficial mechanisms for programmers to demonstrate, objectively, high levels of skill. It provides sufficient job security to the best programmers that they can fearlessly avoid most office politics and focus on work, which generally makes the quality of that work a lot higher.

I think radical transparency should apply within large workplaces as well. Many middle managers discourage line (i.e. non-managerial) employees from taking initiatives that would make them visible to other departments, for the fear that it would make them a “flight risk”. Managers of undesirable projects are aware that external visibility for their reports will lead to transfers, and managers of average projects are often insecure that their projects might be undesirable. Moreover, many companies discourage unmetered work, such as education, career development and extra-hierarchical collaboration (i.e. helping out other teams) which further isolates line employees. The result of this is that, for many employees, the manager is literally the only person who knows what the employee is doing. This gives the manager free rein to represent his opinion of that employee (which, if the manager has 25 reports, is very probably unreliable) as objective fact. Human Resources (HR) is supposed to ameliorate conflicts that occur when this leverage is abused, but the truth about HR representatives involved in manager-employee disputes (and terminations) is that they work for the company’s lawyers (and that’s why they think PIPs are a good idea) and no one else. At any rate, this pattern of dysfunction and outsized managerial power is sometimes called “manager-as-SPOF”, where SPOF is a technical abbreviation for “single point of failure” (the weak point of a system).

Manager-as-SPOF, isolation, and a lack of transparency about who is doing what allow socially manipulative twerps (young wolves) to succeed. In this kind of environment, 9th- and 8th- and 7th-decile performers get almost no visibility and are defenseless against a political campaign against them. The solution is to make the contributions of these employees visible. It’s to make everyone’s contributions visible and to foster an environment that makes many of the worst manifestations of office politics untenable. It’s to encourage employees to share their contributions with the whole company, and to work on things that benefit all of their colleagues– not just their managers.

A future RT post will focus on this: how to construct a radically transparent workplace, and what it would look like.

REPL or Fail

I wrote a post last week on the idealized trajectory of a software engineer in general professional ability, and I find the 3-point scale (with one decimal point) that I developed to be quite useful. It describes the transition from additive to multiplicative contributions to a team, with 1.0 representing the baseline competence (a net adder, rather than a subtracter) of a professional programmer, 1.2 being about average for the industry, 1.5 being seriously good (“senior” in most contexts) and 2.0 representing a consistent multiplier and technical leader. Sadly, one of the more common roadblocks occurs early (around 1.0 to 1.2) and for most developers, and it’s often associated with-established programming languages like Java, C++, and Visual Basic. What’s going on? Why do so many programmers reach a hard ceiling, with some persisting there for decades, while others pass through this barrier with ease? What is it about certain languages or technologies that holds programmers back?

Programming is a two-class society. We have the mere “coders” who use only one language, hate the command line, and don’t program outside of work. They typically work on bland, “enterprise” projects and solve annoyingly detailed, but not difficult, problems. They generally find programming to be a boring task, but “it pays the bills” and the other smart-person route to management (actuarial science) involves hard exams. If they remain “in programming”, they’re lucky to break six figures when they’re 40, and they are likely to face age discrimination and layoffs in the decade after that. On my scale, they plateau around 1.2 because, in enterprise programming, those who reach 1.3-1.4 are usually brought into management. For a contrast, the other class is comprised of elite “hackers” who prefer languages like Python, Scala and Erlang (although some might have to use Java) and who program outside of work, who are hotly desired by huge companies and startups alike, and who continue growing even into old age. These usually reach 1.3 in their first few years as professional programmers, and reliably break 1.5 by mid-career. What separates the two? What is it about a programming language that makes it highly indicative of a software engineer’s future progress (or lack thereof)?

It’s evident that this problem is outside of languages themselves, because a 1.5 programmer can, after some adjustment, program at a comparable level in Java. So it’s actually not the case (aside from an opportunity cost argument) that Java and C++ make people worse programmers. Rather, something happens in other, more modern, languages that makes people improve faster and helps them smash through that 1.2 barrier. So, what is it? I spent much time trying to figure this out, and when I came upon the answer, it was so simple it shocked me: The Mighty REPL.

Modern programming languages supply an interactive mode, also known as a “read-eval-print loop” or REPL, as part of the programming environment. The REPL allows programmers to try out code and get immediate results, and to explore existing software modules interactively by calling their functions and seeing what they do. Although typically associated with interpreted languages (notably, Lisp) REPLs are supplied for compiled languages such as Scala, Ocaml, and Haskell. (They do not provide the speed of compiled code, but code is not run for speed in the interactive mode.) This tight feedback loop facilitates a style of development and code exploration that is far more engaging and effective than the processes of writing and reading code would be without a REPL, and far superior to anything available from an IDE. (A REPL is, in some sense, a simple but highly effective IDE. Properly built, it makes IDEs unnecessary.)

Programmer productivity is binary, in that a programmer is either in a productive, engaged state of “flow” or in a disorganized, unpleasant, and unproductive state out of flow where 10% as much (if that) is accomplished. (A significant amount of work stress, in my estimation, is caused by a self-inflicted sense of pressure to be productive while one is out of flow.) It takes programmers about 15 to 30 minutes to enter flow, but once in it, they are immensely productive and moreover, quite happy. Developers usually write the most code and their best code when in flow, and are best at reading code when in this state as well. There’s a problem: reading other peoples’ code, if the code presents a lot of accidental complexity obscuring the question the reader is trying to answer, often shatters flow (which is why bad code is hated with a passion that only programmers understand).

Programmers understand flow and its importance from experience, and flow becomes more important as one increases one’s programming skills (to the point that 2.0+ programmers, rather than negotiating for higher salaries, tend to negotiate perks oriented toward flow and engagement, such as a quiet working space and an unconditional right to turn down meetings). An experienced programmer can work at a 1.0 level (cranking out code of nominal additive value) without inspiration, but 1.5+ and especially 2.0+ level contributions require creativity and focus. So experienced, elite programmers know that a REPL-less language is generally a dead-end. In a “green field” environment where the programmer controls the entire context in which is work exists, engaged writing of code in languages like Java is still possible– but engaged reading of code is out of the question. The engaging way to read code is to get a big-picture sense of it through interaction, and then to examine the code for implementation details strictly after this has been achieved.

This, in my mind, singularly explains the ceiling that Java developers hit around 1.2. By some non-satisfactory definition akin to Turing-completeness for programming languages, a 1.2 programmer has the knowledge and resources to “solve any programming problem” (ignoring performance and feasibility concerns). The solution may be inelegant, slow, unmaintainable, and even bug-prone, and it might take a long time for the solution to be delivered, but there aren’t programming problems that a 1.2 (or even 1.0) engineer “can’t solve”. On the other hand, far more interesting is how people solve problems, and some of the things that programmers must do if they want to become elite (1.5+) programmers are (a) figure out which problems are worth solving, and (b) learn how to read solutions that other people have created. To become a great programmer, one must be able to read code (in an engaged state of flow). Moreover, one must read good code, a commodity that is depressingly rare in this industry.

Reading code can be an immense joy that produces “Aha!” moments, or it can be hellishly tedious and unfruitful. Sadly, most real-world code (especially in languages like Java and C++) is closer to the latter extreme– it’s probably over 90 percent. Code rots for a variety of reasons. One is the wine/sewage problem (“a teaspoon of sewage in a barrel of wine makes a barrel of sewage, a teaspoon of wine in a barrel of sewage makes a barrel of sewage”): if a system is corrupted and the nastiness isn’t aggressively refactored, the kludges will beget counterkludges in “maintenance” and destroy the whole system. A related issue is the “broken windows” effect: tolerance of ugliness leads to a sense of abandon, and this is more common than most programmers will admit. Modifying code in a reasonable way (i.e. one that doesn’t, despite solving an immediate bug or adding a specific feature, make the general quality of the code worse) requires understanding it, and that usually involves reading it, and most code is so terrible that programmers who have to use, extend or maintain it just give up on comprehension and “hack” it as far as they can. Programming in this style is akin to the” Jenga” game, where players must remove planks from a tower and place them on top of it, making the structure less sturdy and higher as they go (until it collapses, and the player whose turn it is loses).

There’s no silver bullet for code comprehension, but the REPL is the closest thing. The worst code may remain impenetrable, but few real projects begin their existence as incomprehensible legacy nightmares; they usually start out as average code for a project of that size. REPLs make it possible to explore average-case code and comprehend it without dedicating massive amounts of time to the process, and this makes a huge difference. Aging modules can be refactored in the earliest stages of decay, long before they get anywhere near the “legacy horror” state. By enabling interactive peeking and poking of functions, REPLs allow programmers to explore libraries and get a sense of their interfaces. For example, in Ocaml, it’s possible to get the full type signature of any module:

# module L = List;;
module L :
sig
val length : 'a list -> int
val hd : 'a list -> 'a
val tl : 'a list -> 'a list
val nth : 'a list -> int -> 'a
val rev : 'a list -> 'a list
val append : 'a list -> 'a list -> 'a list
val rev_append : 'a list -> 'a list -> 'a list
val concat : 'a list list -> 'a list
val flatten : 'a list list -> 'a list
val iter : ('a -> unit) -> 'a list -> unit
val map : ('a -> 'b) -> 'a list -> 'b list
[...]
end

From these type signatures, it’s relatively easy to get a sense of what these functions do and to test those intuitions:

# List.length [1; 1; 2; 3; 5; 8];;
- : int = 6
# List.map (fun x -> x*x) [1; 1; 2; 3; 5; 8];;
- : int list = [1; 1; 4; 9; 25; 64]


With Ocaml’s powerful REPL, a person can explore code and get a sense of the big picture before starting to read it. That makes a huge difference: reading code is an order of magnitude easier and more engaging when one understands what one is looking at. Moreover, many Lisps such as Clojure and Common Lisp provide documentation functions at the REPL that allow the user to read a function’s documentation without having to leave the command-line. This provides all of the benefits of an IDE, without the flow-breaking drawbacks.

For an aside, there’s something that elite programmers (1.5+) call “keyboard snobbery”: IDEs are scorned, while the key-combos of emacs and vim are venerated, even with anachronistic names like “meta” for the escape key.  The command line interface is highly valued. This doesn’t apply to all computer use (when web surfing, keyboard snobs use the mouse like anyone else) but it does apply to writing and reading code. Why? Because the mouse is physical, continuous and imprecise, while the keyboard is  cerebral, discrete and exact (and therefore a better tool when programming). When we use web pages, we trust the developer to handle the imprecision (in determining whether a button was clicked, and in interpreting the mouse event). This is fine for this purpose, but when we’re writing code, we want exactitude and total control of our interaction with the machine. We want exactly the result we expect at all times. So that’s why we prefer the keyboard when coding, but there’s something else going on as well. Switching from keyboard to mouse doesn’t only involve a move of the hand. It reframes the interaction between the human and the machine, and that’s a context switch. Seemingly benign context switches inflict major drag on programmer productivity. Switching to the mouse because one’s IDE requires it? That’s 3-5 minutes. Pinging about the filesystem because of some stupid requirement that each class live in its own file? That’s about ten minutes. Managerial interruption? An hour, and half a day if the meeting is unexpected and intense. Programmers hate being nickel-and-dimed by context switches, and they hate being out of flow. This is why seriously good programmers prefer “archaic” tools like the command-line interface, vim and emacs, while considering typical mouse-driven modern IDEs (which are necessary if one is developing in a verbose basketcase of a language like Java, but unnecessary in better languages) to be useless.

The REPL, served at the commandline, allows a person to interact with code as if it were live, and see what the pieces do. At least half of what we must do as programmers is comprehension of assets that other people have created, and the REPL allows us to do this without the painful context switch associated with having to read code cold. It enables “flowful” (that is, engaging) exploration and, later, “lazy reading” of code. (Lazy, in this sense, is a non-pejorative computer science term associated with doing only the work needed to solve a problem.) That’s something REPL-less languages can’t provide, because in them, code is a dead static thing that might be run against some dead static tests, not something a developer can interact with as he works.

Reading code is part of the job description of any programmer, and yet it’s rarely done well because enterprise languages like Java make the process so dismal that most people just give up, falling into abominable development practices. When 90 percent of the code is tedious boilerplate (accidental complexity) that isn’t worth the eye strain, it’s easy to miss crucial details. The accumulation of missed details leads to frank incomprehension quickly, and then development practices akin to “throwing mud at the wall and seeing what sticks” become the norm.

This, I believe, is what holds back most Java developers’ progress. Not only does code in such languages become horrible quickly, but the environment makes it unpleasant to read even “good” (by which, I mean “above average for the language) code. In fact, most IDEs tacitly assume that no one is going to bother to read code after it is first written, and adjust accordingly.

For a contrast, this is something Ocaml got right in a major way. Ocaml is an obscure “niche” language, but it has the highest average quality of programmers that I’ve ever seen (even higher than Haskell and Lisp, although those are close). I don’t believe the reason for this is that only good developers can use Ocaml. Instead, what Ocaml achieves is that it makes it a joy to read average-case code– no small feat. Pattern matching, a core feature of the language, is explicitly designed to make what would otherwise be complex control flows human-readable. Haskell is an excellent language as well, and extremely terse, but in my belief it’s optimized (more than Ocaml) for writers of code (although still far better, from a reader’s perspective, than Java). I would guess that the ML family of languages (which are elegantly simple) are the only languages on earth that go so far to make almost all code readable, even in large systems. Of course, it’s still absolutely possible to write horrible, illegible Ocaml code– the language puts up more of a fight against bad practices than most, but it can be done. The difference, relevant for economic rather than purist discussions, is that average-case Ocaml code is attractive, whereas even very good Java code looks only 20 percent less ugly than typical “bad” Java code.

In a language like Ocaml, there’s so little boilerplate and accidental complexity that one can look at the code and actually see the problem being solved. For larger systems, one can test one’s intuitions at the REPL. No one needs to rifle through 300-page design documents to understand what a well-written Ocaml program does. The consequence of this is that Ocaml has libraries of generally very clean code that people can read as they learn the language. Since it’s not an unpleasant process to read code, they do so, and they grow as programmers at a rate that would be unheard-of in Java or C++: rising from 0.8 to 1.5 in about two to three years is typical. Ocaml isn’t some “hard” language that only 1.5+ programmers have a chance of understanding. It’s a language that turns ordinary programmers into 1.5ers rapidly.

There’s one language that can be cited as a counterexample to the “REPL or Fail” rule, and that’s C. C, invented in the 1970s, doesn’t have a REPL. Why was this acceptable for C? First, the language grew up in a different time, when small programs (that would today be replaced by “scripts” unless performance were an issue) were the norm. Programmers could “grow up” on C in 1985 because the programs they’d be reading were small and had well-defined semantics. Second, to say that “C lacks a REPL” is a bit strict. It doesn’t have a language-native REPL, but the Unix/C environment does have a (rudimentary, but sufficient) REPL: the command-line console. This was the environment in which C programs were run and explored: first you run wc and cat to see what they do, and then you could look at their C code and discover how they do it. C was designed with a “small-program” model of development (because large, megalithic programs were simply untenable in 1975) in mind. If complex behaviors were desired, they could be established by composing independent C programs and having them communicate through pipes and sockets. In this world, one could read “a whole C program” (a small, independent module, usually in one file) in one sitting. One only needed a REPL (command-line console) to understand the bigger environment: Unix.

How’d we end up with these disengaging, REPL-less languages? As I said; speaking superficially and strictly, C has no REPL. This was not a problem for C because large programs were so rarely written in it, and enough small, well-written C programs were distributed in every Linux environment that a programmer could learn the language from those. Where C++ differs is that large, complex, and monolithic programs are written in it, because the language has just enough in the way of high-level support to let people attempt them. The result is that C++ supports beasts of complexity (such as 200-line functions, 1000-line class definitions, and 1-million-line whole programs spanning several directories) that would be unconscionable in C, and yet fails to provide the one tool that might enable a programmer to make sense of such things. Although writing a C++ REPL is possible, it wouldn’t be easy: the language is so deeply imperative and crystalline that overcoming the mismatch between the two models of programming would be a monumental task. Java, as a descendant of C++ in syntax and culture, inherited most of these illnesses from it while becoming the default language for enterprise programming, and was also launched without a REPL. The result is that millions of people are stuck in a REPL-less language and don’t know why, while hacking on monolithic projects of intractable complexity that are doomed to get worse over time.

The REPL isn’t just a tool. It’s an engaging classroom in which one learns how to be a programmer. It’s absolutely necessary for a person assigned a task that involves comprehending a complex piece of code. And unlike the training wheels of an IDE, it doesn’t attempt to hide “difficult” details from the developer; it allows her to explore them to arbitrary depth when she is ready.

For these reasons, the interactive mode can’t be considered a luxury of those who are privileged enough to work in “elite” languages. There’s no reason programming should be that way. If we want to democratize programming (and there’s no reason we can’t have at least ten times as many 1.5+ programmers as are alive now, and 10x is a conservative goal; considering the world population) we need to begin orienting ourselves toward modern languages. And there is one rule that seems more fundamental than any argument about static vs. dynamic typing or imperative vs. functional programming: REPL or fail.

Programming, like writing, gets harder as you get better.

Writing’s hard. I don’t think anyone who has done it and taken it seriously, whether in creative fiction or in precise, technical nonfiction, disagrees with this claim. What makes it either very difficult or endlessly rewarding, depending on one’s perspective, is that it remains challenging as one progresses, because one’s increasing competency is paced equally with, if not outpaced by, escalating standards to which one’s own work is held. Many of the millions of wannabe novelists out there believe that the reason they haven’t written (or even started) “their novel” is because that first novel is just too damn hard. Most writers would say that, although it’s the hardest to sell, the first novel is actually the easiest to write. No reputation is at risk, the first novel is expected to be mediocre, and most importantly, one’s own intense self-criticism hasn’t set in, at least not in full force, yet. From what I’m told by experienced authors, writing never gets easier, even for the immensely talented and skilled. One notable exception to this exists, and it’s those who are writing only to make money and who have consciously decided to make writing purely a matter of economic optimization. As far as I’m concerned, such people don’t qualify as writers, but that’s another topic for another time.

Programming is similar. Performing specific tasks, obviously, becomes easier as a programmer’s competency grows, but good programmers don’t want only to “solve the problem”. They want to solve the problem correctly, which entails writing code that is generally useful, extensible, and of high aesthetic quality. The code should not be needlessly slow, complicated, or brittle, even if those concerns are irrelevant to the immediate use case (e.g. an inefficient algorithm may be acceptable on small data sets, but is intolerable in code that might be expected to scale to larger inputs). “Kludges”– inelegant solutions– and “anti-patterns”, such as busy-waiting to implement an event loop, that may be acceptable to a novice programmer trying to just get a program working, become embarrassments to intermediate programmers and intolerable for experts.

Definitions of good programming often diverge, as well. In the 1960s, self-modifying, clever and fast assembly code might be considered “good”, as it solved hard problems at record speed, although it would be opaque to anyone required to maintain it in the future. The scope of an average program was smaller than it is today, and a large project written in such a style would likely be discarded if major revisions were required by anyone other than the writer of the original code. In the 2010s, such unmaintainable code would hardly be considered good code, even if it were 20% faster than a more maintainable alternative. Then again, such may be perfectly acceptable code if generated by a compiler, as humans rarely read the machine or assembly code their compilers create.

Though there is no strong consensus on what constitutes good code, it’s a matter on which many programmers are immensely opinionated. It has to work, obviously, but that’s setting a low bar. Even the worst programmers can make software “work” according to a minimal specification, given enough time and allowance for inelegance; but the code of a bad or even mediocre programmer is often so unpleasant to read, use, and maintain that it inspires a gnawing and universal desire to throw it all out and start anew.

For my part, I would say that a good programmer must be a good teacher. The code and documentation should be instructive of how the code is to be used, and how each component works. Ideally, programmers would develop in such a manner that the function of every line of code is self-evident, due both to the innate clarity of the language (a virtue of, say, OCaml) and the quality of documentation. In practice, most managers will never budget sufficient time to make this a reality, but it’s what software engineers should aim for when they can.

Here we venture into the thicket of aesthetics, where every rule has exceptions that must be learned through practice, and where “known unknowns”– matters on which one knows of one’s lack of knowledge– are only a fraction of total unknowns. (An example of a “known unknown”, for me, would be the German language. I know that it exists, and a few stray words and grammatical principles, but I can’t read or write it. An unknown unknown would be any of the six thousand extant languages that I’ve never heard of.) And that is what makes writing, and software engineering as well, increase in difficulty as one’s skill increases. As one’s knowledge increases, one’s awareness of the gaps in one’s knowledge increase at a more rapid rate. One’s perceived “knowledge ratio” decreases as one’s actual ignorance wanes. A problem with one known solution is easy to solve; when there are ten, and when one knows there might be a hundred more worthy of study, selecting which is best becomes very difficult.

A genre of essay I sometimes find myself writing is the “problem essay”, the first act of which describes an undesirable or inefficient situation, with the second act managing its logical conclusions and avenues and approach, and the final act proposing solutions. This is a very common pattern in writing. Here would be the point at which I propose a “solution”, but I, frankly, don’t have one. To tell the truth, I don’t know if the counterintuitive tendency of a craft’s difficulty to increase with one’s improving skill and knowledge is a “problem” in the first place.

Actually, as a game-design snob who enjoys a well-structured challenge, I rather like this aspect of disciplines like writing and computer programming. It keeps things interesting.