Before reading this, please read Jeff Vogel’s excellent essay, “How I Deal With Harassment, Abuse, and Crazies In General“. His experiences, as one who has encountered both the positive and negative aspects of publicity, are eerily similar to mine.
“A Reputation Problem”
To put things in context, I’ll recount some recent happenings.
This past February, I left a position as an R&D Director at a “unicorn” startup. Why? A Fortune 1000 company wanted to build an applied logistics lab in downtown Chicago, and had selected me to be its VP of Engineering, in charge of a 7-figure hiring budget.
That turn of events was much of why I removed many of my old blog posts. It’s not that I don’t enjoy writing. I do. When you’re an executive, though, you have to be extremely careful in what you say, because you’re going to have a large number of people watching your every move and fretting about whether it will affect their jobs. Pulling out of the blogging game seemed the prudent thing to do, at least temporarily, for the sake of the people who’d be reporting to me.
This company’s “Labs” division was set up to open on March 28. It didn’t. Why? Out of respect for the privacy of others involved, I’m not going to get into the details. It was a mix of others’ life events and mundane difficulties inside the parent company. I’m not even inclined to complain. Others, who pulled up stakes to move here for the project, were hurt more by the closing (or, more accurately, the never-opening) than I was.
During February and March, that project’s fate became more uncertain. I started talking to recruiters, in order to figure out what I would do if the project were scuttled.
I managed to get myself introduced to someone whom the top hedge funds in Chicago and New York tap when they need CTOs and partner-level hires, and he’s known for speaking candidly. Here’s what he said (emphasis mine):
If it were anyone else with your talents, I could place you pretty much anywhere. 300, 400 grand a year base, direct mentorship from people like [billionaire hedge fund manager] and [another billionaire hedge fund manager]. You’d be one of the best people they’ve seen in years, and unlike the people around them, you’re honest and ethical. The thing that’s holding you back is… a reputation problem. People Google you and think, “This guy’s going to start a union.”
This is going to make it hard to place you. Guys like [name] don’t like people they can’t buy. If I were you, I’d start blogging again and come out against unions. Do that for six months, and I can put you anywhere you want to go.
I’m not going to come out against unions. Collective bargaining is a complicated topic. I could gain, economically, in voicing a simplistic view; but it wouldn’t be intellectually honest to do so.
I might as well take this opportunity to explain what I do and do not believe.
“A reputation problem” is a polite understatement. There’s a misconception about me, which is that I attempted to unionize Google. Nothing of the sort happened. Nonetheless, several of top internet search hits for my name are people attacking me, mostly, for reasons that stem from the events that have been misconstrued in this way.
With all of this misinformation out there, employers might see me as a radical unionist. In reality, I’m an intellectually honest (and harmless) moderate who’d rather just do his job. I like doing things that I’m good at, and I’m extremely good at my job. On the other hand, I’ve never organized a union before and don’t know if I’d be any good at it.
Silicon Valley is aggressively opposed to unions. Companies routinely share lists of suspected unionists, and people who end up on these lists can be blacklisted, assigned to the worst managers if “discovered” once inside the company, harassed in public, and subjected to all sorts of libel.
For example, I’ve had people whom I’ve never met lie about my performance at Google and the circumstances under which I left. I’ve been subjected to two high-profile, bad-faith website bans (Hacker News, Quora). When I started an “Ask Me Anything” thread on Reddit, hoping to keep the discussion confined to my technical interests, I faced a voting ring comprised of at least 45 accounts (that is the number that were detected by Reddit; it could be higher) that drowned out substantive discussion. The top comment asked me why I was a “mega-douche”. It seems that tech thugs, while dedicated to their work, aren’t very creative with their insults. (If these people ever need a competent writer, I happen to “know a guy”.)
I’m not even “a unionist”. I just choose not to reject the idea out of hand. Because of my mathematical training, I don’t reject many ideas out of hand; I wait for proof or refutation before assuming certainty. I’m philosophically agnostic. Yet, for having the intellectual honesty to consider that collective bargaining might be worth consideration, I’ve been attacked, over and over again for years, by some of the worst people in Silicon Valley.
Now, am I going to walk into my next job and start a union? No. I doubt that have the organizational skills for it. We’re talking about programmers. I’d call that a cat-herding task, but cats are prettier and less vicious, in my experience, than most private-sector software people. Do I abstractly believe that software programmers have the right to some form of collective bargaining? Sure, but I’m not planning to build it at my next company. I’d rather build great products and solve hard problems, that being more in line with my core competencies.
So what do I believe?
First of all, I believe that collective bargaining is a right. Corporations are collectives that bargain on behalf of the rich people who own the means of production. The workers ought to have similar backing. It’s only fair. In fact, most respected professional organizations are, in fact, labor unions under a more genteel name. The American Medical Association is a doctor’s union. The Society of Actuaries is a union. Many of these unions are, on the whole, net positive for society. The AMA has done some evil things, but we’re probably better off with it than without.
Second of all, I recognize that there is a diverse array of collective-bargaining arrangements, and that not all are good. “Unions” have a negative reputation in the United States and, sure, there are bad unions out there. There are also good unions, and good union workers. The negative reputation of unions comes more from propaganda and clever redefinition of the term “union” than anything else.
Traditional labor unions emerge (and, often, work very well) when the labor has been deemed by both sides to be a commodity: for example, coal production. If commoditization of the labor is irresistible, then the union tries to ensure that it happens on fair terms.
In fact, the strongest objections against programmer unions come from software engineers who resist allowing their work to be viewed as a commodity. To this, I am sympathetic and in intellectual agreement. It would be a better world if we were artisans. That said, we’ve failed to oppose commoditization of our labor for as long as we’ve been without a collective arrangement; in fact, we’ve been subjected to open-plan offices (surveillance) and “Scrum” practices that reduce our labor to piece-work. If we believe that our work is a commodity, then we should not be resistant to unions. If we truly believe that it’s not, then we ought to consider a professional organization or a guild system that can defeat the commoditization that has happened this far.
Professional organizations are unions that exist to uphold ethical principles (for doctors, don’t kill patients; for journalists, print the truth) that ought to supersede both economic concerns and managerial authority; they exist to dictate terms of partial commoditization. Likewise, high-end unions (as in Hollywood and professional sports) and guilds exist to enable those who view themselves as athletes or artists (and who might, therefore, insist that their work is not a commodity, but must sell their labor to others who see it precisely that way) to coexist with commercial players. Most likely, a structure that will actually work for software engineers will be somewhere between the lightweight unions of Hollywood and the professional societies that doctors and lawyers and actuaries have.
Third, I am (surprisingly?) more skeptical of software engineer unionization than supportive, at least right now. Why is that? For a while, software engineering has been dominated by the venture-backed startup culture. We’ve had the worst people driving out the good for at least 15 years, if not longer. We’ve ended up with an exclusionary macho-subordinate culture. We have sexism, ageism, classism, racism, and widespread harassment in our midst. We can’t blame all of this on management. Tech-company executives certainly use such divisions to keep software engineers from uniting and demanding a larger share of the value that we create, but we as engineers also allowed those divisions to exist in the first place. If we create a union without fixing our culture at the same time, we’ll probably generate one of the bad kinds of unions (the garbage-in, garbage-out principle). We should fix our culture first. That’s the top priority.
Fourth, we can nonetheless learn from existing professional and trade unions, and start figuring out what will and will not work. Most programmers believe that all unions negotiate (and regulate) compensation and that, therefore, top performers will see mediocre wages. That claim is demonstrably untrue: professional athletes have unions, as do screenwriters and actors, and nothing has prevented top performers from earning 7- and 8-figure compensation. Most likely, an effective union for software engineers would not try to set salaries, but would give employees additional rights when it comes to how they are managed. We could kill stack ranking and improve working conditions, and I don’t see a good reason not to do that. We could guarantee employees the right to an independent representative on issues surrounding performance management and (if necessary) separation. Again, that’s a no-brainer. We could force employers to act in a way that avoids harm to the individual’s reputation. Who would be hurt by that? No one.
Fifth, it’s important to note that it is impossible to unionize a workforce that doesn’t want to have a union. I’m well aware that Silicon Valley is terrified of unions. However, any company that declines to hire me based on my reputation or “cultural fit” is making two assumptions: (1) that I want to start a union and (2) that I can. The first is not especially true, and the second is very likely false.
Let’s be honest here. Software engineers are individualistic even when it goes against their own interests. Decades of Silicon Valley propaganda have worked. Programmers have been successfully divided into warring tribes based on gender, geography, age, and even choices of tools. An extremely charismatic person might be able to unite all these tribes, like Mance Rayder in Game of Thrones. As for me? My social skills are pretty average. It’s simultaneously flattering and annoying that some employers have thought that I’m capable of single-handedly unionizing their workers, in addition to everything else that I have to do. In reality, that’s not a credible threat.
Sixth, one should note that unions are often good for the companies that are unionized. The (incorrect) common viewpoint is that, since unions drive up workers’ wages, shareholders are losing money. This isn’t necessarily true. If the union’s effect is to make the internal labor market fairer and more efficient, and to improve workers’ conditions, that might actually result in the company getting more value out of its people. Companies can actually become more profitable after being unionized. The workers, after all, have no more desire to kill the company (that they’ve worked so hard to unionize) than the owners.
Who gets hurt by unions? Executives, the most. Managers lose authority and power, while their jobs become more complicated. They can no longer unilaterally terminate people, which means that extorting the workers into supporting their own career goals is no longer an option. For owners, however, unions often mean that a better product is produced. So, while wages increase, the net effect can be positive or negative for profits.
Managers dislike unions because they’re perceived to be a vote of no confidence from their workers. However, as pertains to Silicon Valley, such a vote is in order and has been for years, because Silicon Valley is so severely mismanaged.
Seventh, a company-specific union won’t work for software engineers. As in Hollywood, we have project-based careers and it’s normal to change companies frequently. The most important function of a technologist’s union will not be to protect jobs but to protect our reputations and our professional autonomy across the course of our careers. Any collective bargaining structure will have to be like the Hollywood actors’ and writers’ guilds in terms of conferring benefits that persist independently of the member’s specific employer.
Eighth, unionbusting is one area in which otherwise competing malefactors will cooperate. This isn’t surprising, but its implications are disturbing.
For example, in Silicon Valley, there are employers who check references on people not provided (“back channel” reference calls). Why is this practice, an instance of ethical degeneracy that facilitates illegal activity, tolerated? It’s because these companies want to share blacklists and bust potential unions. Although companies generally don’t care to share dirt on garden-variety low performers, because the legal risks of doing so are too high, the few people every year (almost invariably high performers) who are deemed to be suspected unionists will have a hard time. Companies will break their own rules, when it comes to those.
These employers know, of course, that most of the people named on these lists are no threat to them. They also don’t care. Paranoia trumps the rights of the innocent.
Why are they so paranoid? It’s hard to say. From where I’m standing, the probability of Silicon Valley technologists unionizing in the next ten years is very low. There is a sense in Silicon Valley’s elite that doomsday is around the corner, and programmer unions are one purported risk. Not only is this unlikely, but if it did happen, it would probably be beneficial to the ecosystem as a whole. A union (or guild or professional society) that killed Scrum and open-plan offices would make programmers more productive and result in better products and higher profit margins. The paranoia around “The U Word” just isn’t justified. For existential threats to the technology industry, I’d worry more about Silicon Valley’s investor class and the industry’s management.
The next 40 miles
As of April 2016, there isn’t much evidence of a desire among software engineers to unionize, professionalize, or organize in any way that would threaten the status quo. This might be the case because the startup culture is powered by its low-level workers’ unrealistic expectations (e.g. the fresh college grad who works 90-hour weeks because he thinks he’s going to be a CEO in three years) and because those who have more experience are usually purged. The VC-supported startup culture of misogyny and age discrimination (see: open-plan offices) serves to exclude people with the diversity of experience that we’d need in order to organize, and I believe that this is the true purpose of that culture.
What should we be doing? The first thing that we need to do is fix software’s culture. With any collective organization, we’re not going to have much success until we do this. We need to stop driving out women, programmers over 30, and people from non-traditional racial or career backgrounds. Diversity is a strength, so let’s not lose it.
Let’s go further and be blunt about a few things. Women, on average, have more organizational and social skills than men. Older people generally have acquired more of these skills than people who just got out of college. If we want to band together and kill employee stack ranking or negotiate for better parental leave policies or change a company’s performance appraisal system, then we can’t afford to tolerate a culture that divides us and that drives such people out, justifying it with juvenile explanations of “cultural fit”, and thereby leaving only the most manipulable. Why does Silicon Valley insist on hiring only young men? Because they don’t have the experience and organizational skill to threaten management’s interests. It’s easy to exploit them. While those fresh college graduates are almost never as good at programming as their more senior counterparts, that often doesn’t matter in a style-over-substance startup culture where companies exist to be sold quickly, rather than being built for the long term.
Next, and here’s where I get truly controversial: I think that it’s important to solve the charlatan problem. We have a labor market flooded with incompetents, and we suffer under micromanagement frameworks like “Scrum” that are sold on the promise of making such people productive (they don’t work, but that doesn’t always matter).
I think that the best way to drive out the non-programmers and the non-serious technologists is to implement an exam track similar to what the actuarial sciences use. We want to keep the programming profession open to everyone who has the ability, drive, and intellectual curiosity; while at the same time preventing management from flooding the labor market with an inferior-but-manipulable substitute: open-plan Scrum drones.
Exams aren’t perfect, and I’d love to refine the process to allow project-based alternatives for people who aren’t great test takers, but they’d bring us a lot closer to meritocracy than what exists right now, in which a programmer’s capability is usually assessed according to outmoded reputation metrics (which I’m good at gaming, but at the cost of my soul) like job titles assigned by employers. The field is polluted with biased and manufactured information. Just knowing who the legitimate programmers are, and how to find and identify them, would be a step in the right direction.
What I like about an exam track is that it’s blind. Your answers are scored by people who have no idea whether you’re male or female, white or black, 17 or 73 years old, Bay Area royalty or Blackfoot Lakota. I’d like a system where everyone who has the talent necessary to be in this field, and the drive to learn the broader field of computer science, because I think that such knowledge is important if one wishes to design systems correctly, can get in. Blind grading delivers a massive improvement over “culture fit”, under which a person can be rejected for not having played beer pong for ten years and not using PUA lingo in regular conversation. I know a lot of women who’ve switched to the actuarial sciences (from software engineering) for exactly that reason; they wanted a career where their progress would be measured objectively, rather than by a bunch of young, arrogant men using the “just like me” metric of human value. Who can blame them, really?
She rides for Zelos
Is what we do, as software engineers, important? Is it ethically meaningful? I would like to say “yes” but, at the same time, most of what I’ve done in my career has been meaningless to the advancement of human society. It hasn’t mattered, and that bugs me. I think that, as a community, we need to start thinking about the ethical ramifications of what we’re doing.
There are the small failures that I see on a day-to-day basis. One trait of true professionals is that people inside the profession never criticize each other to outsiders. That’s not necessarily for the purpose of secrecy, but because the outsiders will lack appropriate context and become dangerous, even if they don’t mean to be. For example, let’s say that two engineers disagree about source code formatting. It’s just one of those stupid topics that programmers get into heated conflict over, because there isn’t one clear right way to do it.
What happens, though, when such a dispute gets escalated to non-technical management? To the manager’s ear, it sounds like one of the engineers is willfully refusing to conform to “correct” source code formatting and therefore singularly responsible for the disagreement. Whom will that manager judge to be in the wrong? Well, since there isn’t an objective right way to format code, the person with more political pull will win, and the other will lose and be labelled as the nonconformist. Non-technical managers exposed to engineering disagreements, but feeling a need to Make A Decision, often take those issues out of context and become TWiGs (Toddlers With Guns). As a general rule, I think that we won’t be respected as a profession until we develop the social and individual skills to handle these matters, whether they’re minuscule disagreements or genuine performance issues, internally. As programmers, we’re far too quick to tattle on each other to management and that gives us the reputation of being spoiled, untrustworthy children.
Even still, there are much bigger ethical failures that are even more worrying than our internal affairs. I don’t consider advertising or statistical arbitrage to be unethical, much less evil. (In fact, Wall Street is, on the whole, far more ethical than its reputation makes it out to be, but that’s another discussion for another time.) However, some industries clearly are actually evil. Employee time-tracking software is a major business. Who writes this code? Someone does. That’s because we don’t collectively have the muscle to deny talent to the purveyors of evil. We ought to be applying our skills to finding cures for cancer, or alternative energy sources to fossil fuels. Most of Silicon Valley, however, is just helping rich people create and profit from unemployment.
The question I’ve had to ask myself, again and again, is: does any of this stuff matter? When I’m reading math or CS papers at 9:30 on a Sunday night, what on earth am I working toward? Is it worth it, given the professional adversity that comes with the fight, to keep giving a damn about the ethics of what we do as software engineers and to fight so hard to be a better person, both technically and morally? Can we actually exert any influence over whether technology becomes a force for good instead of evil? I don’t know. I wish I did.
What gives me hope, oddly, is the adversity that I’ve experienced. I wouldn’t have had to deal with any of the nonsense that I have, if these ideas (even when expressed, as I have, with moderation and justified trepidation) weren’t a threat to people. My experience at Google has had some undesirable publicity. A billionaire venture capitalist extorted Quora into banning my account. Paul Graham blocked me on Twitter, even though we had no significant interaction (we spoke for about two minutes about programming languages in 2007) to that point. The bad guys know who I am. This is indicative of a high likelihood that I’m on to something.
I believe that we, as technologists, can take back our industry. To do so, we need to stop treating our careers as things that other people (managers) and institutions (employers) make happen to us, and step up and make things happen for ourselves. Toward this end, we need to work as a collective and develop a “slapping one is slapping all” mentality. If someone is denied a job based on a back-channel reference call, then it’s morally incumbent on those with knowledge to let that person know, and help that person raise hell. It is the morally right thing to do.
On the same token, we’re in a state that is both degraded and paradoxical. In some ways, we need to make the software industry more inclusive. We need to halt the processes that drive out older programmers, women, minorities, people with disabilities, and others who are culturally vulnerable. We need to make a culture that includes and welcomes all people of talent, and not just people who look like me. At the same time, we need to make the industry more exclusive, in the sense of protecting everything that matters. There are a lot of programmers with no intellectual curiosity, nor any sense of craftsmanship, and those open-plan Scrum brogrammers (who implement the sexual harassment culture and the ageism that needlessly drive out talent) just need to go. This is why I am so adamant in supporting the institution of an exam track. We should let in the people who have the drive and curiosity to become good at programming, while excluding the incompetents who are here because they’ve heard that there’s easy money. (In other words, we need to replace a “get rich quick” easy-money culture, built on lies, with a “get rich slowly” hard-money culture, built on the truth.) The easy-money crowd creates a culture of immaturity, narcissism, centralization of managerial power, and harassment, instead of a culture of technological progress and self-betterment.
Let’s talk about the near future. The VC-backed bubble is going to end, just like the one in the late 1990s did. The toxic unicorns are likely to die, and they might take Scrum and open-plan offices and the harassment culture with them. Let’s hope so. However, my fear is that, due to our lack of organization, legitimate programmers might also see our salaries and conditions decline. How confident should we be that a falling tide will only wreck the boats of the easy-money brogrammers and the sleazy tech executives? I’ve been in this industry for too long to have that kind of confidence. The crash will hurt. The scumbags in charge of Silicon Valley will do everything they can to make sure that we lose our shirts before they lose theirs.
What happens then? When jobs disappear in large numbers, worker leverage declines. Motivation to fix the problem increases (read: people get pissed off) but the ability to do so declines. We can’t assume that “the crash” will hurt the bad guys and not the good. If we don’t plan and organize around our own interests, one way or another, then history suggests the opposite.
The problem faced by those of us who are legitimate technologists is that the labor market has been flooded with an inferior product: the open-plan Scrum brogrammers who aren’t good or even acceptably mediocre programmers, but who look the part (“cultural fit”). They can’t fool real programmers, but they can fool non-technical management and even investors. Why are they beating us? For one thing, they’re winning the battle for cultural influence, just because there are fewer legitimate programmers than charlatans who can be trained and paid to act the part. The open-plan drones have redefined “programmer” to mean “naive semi-privileged idiot” instead of “trusted professional”. They’ve also been able to sell, to management, a perception of flexibility. You can hire twenty-five of them in an afternoon, and they’ll work long hours and tolerate punishing work conditions. This would sound great, except for the fact that their technical abilities are abysmal. They’re so terrible, in fact, that they’re often negatively productive on a project with any technical meat. Worse yet, if they’re given any creative or ethical responsibilities, the results are catastrophic.
Why has the market for programmers been flooded by inferior replacements? The perception is that technical excellence doesn’t matter, and that the market situation proves this. I disagree strongly. As far as I’m concerned, if a technical problem isn’t solved correctly, it hasn’t been solved. A mathematical proof that covers four out of five cases is not a proof. Who would want to use an email system that drops one message out of ten? Would people use a social networking site if their private messages were exposed to the public because of irresponsible engineering? Technical excellence matters a hell of a lot. Getting the right answer is important. Unfortunately, short-term incentives often produce disalignment, and long-term consequences take so long to come to fruition that powerful individuals (such as technology executives) can avoid accountability for their own bad decisions.
Bad software engineering and terrible corporate cultures hurt companies. They hurt society. The problem is that they never hurt the individual managers and executives who create these illnesses, in search of short-term profit. Those people get promoted away from the messes they create, and someone else gets stuck having to clean up.
How do we fix this? I don’t know, but I know who will have to solve the problem. It will have to be us, the competent and responsible technologists. No one else knows just how much society is losing right now. Most of society doesn’t even know that the technology industry is so badly run, much less how much value has never been realized because of that fact. Only we, right now, are remotely aware of that. All of this said, we’ve solved problems that are much harder, and we can hack this one. If we can put a person on the moon, we can organize around technologists’ shared interests (as well as those of society). We just need to wake up to the fact that this time it’s a political problem rather than a technical one that is most important.
Appendix: questions I get asked
Here are some questions I’ve been asked, in my personal and professional life, on the topics above.
“Why did Google think you were a potential unionist?”
I said something on a mailing list, pertaining to a specific product, that had a lot of internal visibility. This was a mistake on my part. The comment was taken out of context and developed a life of its own, leading to a suspicion that I was a union organizer.
It wasn’t, and isn’t, true. At the time, I held no strong opinions about software unionization. After having my reputation damaged (something that unions could protect a person from) by this experience, I am far more sympathetic to that cause.
“How did you find out that you were on a Silicon Valley unionist list?”
I’d heard rumors for years. In March 2014, a source sent me internal documents that confirmed the list’s existence and that I was, at one time, on it. Another source corroborated this, two months later.
Later that year, I applied for a job and was told (by a close friend inside that company) that I had been turned down for an interview only because my name appeared on “Google’s List” of suspected unionists.
“How is blacklisting enforced?”
Since these lists are illegal, they are usually only shared at an executive level.
The targeted person is usually subjected to negative, unsubstantiated rumors pertaining to health, work performance, and (until recently) sexual orientation. For every person in-the-know who engages in deliberate unionbusting, there are several “useful idiots” whose role is just to reliably repeat the gossip they are fed.
“How many people are on suspected unionist lists?”
Because these lists are illegal, it’s hard to know. An educated guess is that this affects at least 200 people per year in the U.S. technology industry. While this represents less than 0.01% of all professional programmers, a surprisingly high number of people on these lists are pillars of our community.
Many people are placed on these lists by mistake and often they have no idea what is going on. If they’re older, I’d presume that they’d attribute their difficulty in finding employment to age.
“Does Google still retain or participate in this sort of unionbusting?”
I don’t know. I’ve been told that Google dismantled the machinery that it uses to track suspected unionists.
However, if the company still uses stack ranking (and I do not know whether it does) then it is overwhelmingly likely that the unionbusting is just hidden in the stack-ranking machinery.
In fact, the point of employee stack ranking is, traditionally, to intimidate employees and prevent them from organizing. Rich companies actually don’t care about harmless low performers who draw salaries but do little or nothing. (I’ve seen people like that hang on for years.) They do care about unions. They’re scared to death of an organized labor force.
“Have you pursued legal action?”
I’ve received settlements due to specific cases of defamation.
I do not believe that I can pursue the larger matter without putting the careers of my sources in jeopardy, so I won’t. Moreover, I live in Chicago and am not especially concerned with whether my name is on a Silicon Valley blacklist.
“I think I’m on a unionist black list. How do I get my name off of it?”
I have no idea. It’s almost impossible to get hold of these lists, because their existence is illegal. My advice would be to leave Silicon Valley.
Wall Street is less paranoid about unions, insofar as it actually pays its people. It’s unlikely that they care about Silicon Valley’s suspected unionist lists. If your professional reputation, in the broader sphere, hasn’t yet been trashed, consider it. If it has been damaged, then you need to hire a competent attorney.
“Someone said you were a poor performer at Google. Were you?”
I have to answer this one directly and truthfully. To start, I’ve been an average or strong (and usually quite strong) performer at every job that I’ve held. However, I was only at Google for five and a half months and did not get much time to distinguish myself. So, I would not place myself in the “strong” bucket, at least not on technical contributions, during that time.
It’s rare, at Google, that someone commits substantial code in the first 6 months, because of the amount of time that it takes to learn the codebase. My experience wasn’t an exception. Since I was there for such a short time, and spent much of that time in a war that I didn’t start, I didn’t get to check in much code. That part is true.
I was considered by my peers to be an average performer.
“Do you expect me to believe that everyone who doesn’t like you is part of an illegal unionbusting conspiracy?”
No, of course not. It’s impossible to have any public presence without being disliked by someone. I hold strong views, and I express them boldly.
Had I not been placed on a suspected unionist list in 2011, I doubt that I would have had nearly the amount of publicity that I’ve had. If I had any notable reputation, in this alternate world, it would be a strongly positive one, as it is among people who actually know me.
“Why do Paul Graham and Y Combinator hate you?”
That is unclear, actually. They started the fight, and I’d be happy to end it. I would guess that it’s a mix of political disagreement and the personal pettiness of specific individuals.
“Why are you banned on Hacker News?”
“Why are you banned on Quora?”
Quora has received funding from, and is at this point largely controlled by, Y Combinator. “An investor” (probably from Y Combinator, but this has not been confirmed) threatened Quora’s management that, unless I was banned in a public way, they would be rendered unable to raise future funding. Vendetta-driven, intentional product failures are more common in Silicon Valley than most people would like to believe, and they come as often from a company’s investors as from its management.
I was a top contributor to Quora. I had more than 8500 followers (and, now, over 9000) and was a Top Writer in each year. My answers were published to many of Quora’s media partners, including Time, Fortune, the BBC, and the Huffington Post.
Because I was a model contributor, my ban caused a substantial loss of faith, among Quora’s community, in its moderation. User engagement, especially among users deemed to be high-value content creators, began to decline. In reaction to this, Quora asked me to rejoin its community in February 2016, and offered a cash settlement. I turned their offer down.
“Could you sign a document agreeing not to support a union, as a condition of employment?”
I don’t think that that’s legal.
In truth, I would find it needless to sign such a thing. Given programmers’ strong anti-union/libertarian tendencies, starting a union among would require monumental organizational ability and charisma. I consider myself very average in those abilities and would not constitute a credible thread.
Besides, I’d rather write code and solve hard technical problems. Organizing labor is admirable and important work, but that’s not my skill set.
“Why do you air ‘dirty laundry’ about ex-employers?”
I don’t. I made that mistake, once, with Google.
There are some who believe that “biting the hand” deserves a professional death penalty. Such people should be counted among the enemies of progress and treated as such. Nonetheless, I still believe that it is unwise, for an entirely different reason, to disparage an ex-employer.
That reason is this: in practice, the workers are more exposed to the corporate reputation than the executives (excluding the CEO). Bad-mouthing an ex-employer hurts the wrong people. That’s why I’ll almost certainly never do it again.
I have exposed unethical behavior by Y Combinator and Quora. I never worked for either of these companies.
“Do you, broadly, support Silicon Valley unionization?”
As above, what’s more important than formal structure is fixing the culture. I would rather commit my time to (a) upholding and promoting a culture of technical excellence while (b) driving out exclusionary behaviors.
Until we have the right culture, formal unionization is probably the wrong way to go: garbage in, garbage out. Developing an exam system (as in the actuarial professions) or some other way of formally verifying talent and credibility is a much smaller step that I would support immediately.
“How do you respond to people who believe that unions in Silicon Valley will kill its ability to innovate?”
Of course, that is possible. Silicon Valley seems to be a nadir in terms of innovation, but the wrong labor structures could make it even worse.
However, a collective arrangement, which probably would look more like a professional organization (Society of Actuaries) or a lightweight union (Screen Actors Guild) than a traditional labor union, could also improve the situation greatly. How so? If people who actually make things get more respect, they’ll be more engaged and the products made will be better. If they make more money, they’ll be able to fund more interesting projects and “bootstrap”.
For the past twenty years, the biggest threat to Silicon Valley innovation hasn’t been some union bogeyman but the technology industry’s management. A common slur against unions is that they can promote a culture of complacent, arrogant mediocrity. That culture already exists. See: Scrum, open-plan offices, business-driven engineering. Unions could make that culture worse, but they could also drive it away.
“Have you ever tried to start a union?”
“What do you think of Google now?”
I harbor no ill will toward Google. In fact, I met a lot of very talented people during my time there, and it’s upsetting that there was so much awkwardness surrounding my own experience, which was atypical for me and for Google.
“What have you been up to recently?”
I mentioned above that I was selected to be the VP/Engineering for a company that, for reasons having nothing to do with me, decided to non-exist. Oops.
Since that opportunity self-vaporized, I’ve been looking around the area for people who need top technical talent: machine learning, functional programming, or distributed programming, at a Principal+ Engineer (as an individual contributor) or Director+ (as a manager) level.
I’ve got 10 years of experience as a software engineer, know more machine learning and statistics than most self-proclaimed “data scientists”, and function well as an individual contributor as well as in leadership. So I don’t expect to be on the market for too long.
To keep myself sane, I’ve been spending a lot of time at the gym and reading a lot of math papers. I’ll probably pick up a book on Erlang/OTP, because that platform seems interesting.
“Have you had significant employment difficulties, because of your vocal support of programmers’ rights?”
Severe ethicality can provoke cowardice in some people, sure, so I haven’t been entirely free of professional adversity; but yes, I’m able to get jobs.
I have had to work very hard to overcome adversity. I’m often reading technical papers at 9:30pm on a Friday night. (“A mind needs books like a sword needs a whetstone.”) I don’t get much rest. I’m always leveling up on something.
“What can I do to help?”
For me, there’s no need to do anything. I’m not a victim. I’m an extremely competent technologist and I am beating this.
What one can do is to prevent the sorts of things that I’ve described from happening to anyone else in the future. Blacklisting or attempted blacklisting are never acceptable, not for any reason.
Certainly, if you’re aware of a unionist blacklist at your company or between companies, share it with the press. If a company conducts a back-channel reference call (except in the context of a federal security clearance, where those are ethically acceptable) then make sure the target knows who was called and what was said. Inform that person of his or her rights under the law.
When people are denied jobs for reasons other than the ability to do the job (e.g. “culture fit”, unsubstantiated rumors, illicit back channeling) it is critical to let them know what happened, and that they should consult an attorney to see if they have legal recourse. This is the only way to actually enforce the laws against unionbusting, blacklisting, and discrimination in employment.
“In the end, what did you fight for?”
I must confess that some of what I am is a product of accident.
At Google, I’d been that the company was open to internal critique and, since it was my first big-company job, I was naive and took that directive literally. Random chance had it land my name on a list of suspected unionists. This has had a positive effect on my work ethic (since I’ve had to work much harder than most of my peers, just to survive) but a negative effect on my view of human nature.
Through that experience, I’ve become committed to pursuing social justice in technology. Not only is it the right thing to do from a moral perspective, but it’s the first step toward something bigger: taking back our industry.
If we can make the technology industry more diverse, and fair, and truly meritocratic, then we can solve more important problems and build better products. We can get ourselves out of the rut that has us implementing the ideas of wealthy, greedy actors who have no love for technology, and we can kill the build-to-flip business culture. In doing so, we can replace our industry’s current form with one that a person can actually believe in.
If the AMA counts as a doctor’s union, then wouldn’t ACM count as a programmer’s union?
ACM has corporate members (called “Preferred Employers). That rules them out.
ACM is also very oriented toward academic computer science.
IEEE does some advocacy for working engineers but it is certainly no AMA or Society Of Actuaries.
Michael: Why are back-channel inquiries all right for security clearances and not all right for other positions of trust? A programmer who steals actual money from the company may be able to do the work all right, but a company definitely doesn’t want to hire him (most especially if it’s a financial/fiduciary institution, but even if it isn’t). Calling the current employer is definitely unethical, but I don’t think other inquiries are. If we had to do business with people solely and only on the basis of what they choose to tell us, we’d be ripped off even more than we are.
If a programmer steals money, then the legal system is the recourse. People who literally steal from their employers should be in jail.
Back-channel references are an extrajudicial institution that has a ridiculously high false positive rate. No one has an “approval rating” higher than 80 percent. That means that if you check 5 back-channel references, you’re going to be rejecting over 2/3 (and possibly a much higher fraction) of good people, just because they were unlucky.
Sure they should. But wouldn’t you want to know before hiring someone? Before working at a bank or brokerage, at least in New York, a check is run to make sure you have no felony convictions. That’s “back-channel”, but is it unethical? I don’t think so.
Companies inform you when they are going to run a background check.
They don’t inform you that the CTO’s cousin was a frat brother of the CEO of that company you worked for last year, you know, the one where you got fired after three weeks and they are going to get the “real info” on you.
Right, and since you don’t know when it happens, you can’t pursue legal action.
Back-channel references are just another way for the well-connected to humiliate the poors, who don’t have the same access to (or control over) information.
From a former defense contractor: Back-channel inquiries aren’t really part of security clearances – you just have to provide a shitload of references, and they go visit all these people personally. Investigators have to stick to a line of questioning, or they expose themselves to legal liability.
The process used to be extremely draconian – Smoked pot in college? ding! Gay? ding! Visited the wrong country? ding! Late on a credit card payment? ding! Know too many foreigners? ding! Didn’t know your neighbors for two months? ding! Like foreign food? ding! And, if you were going for a TS-SCI Full Poly, it often culminated in a polygraph, which basically measured how nervous you were strapped to a chair and being grilled about all the horrible things you probably did. What was worse, investigators got a bonus for everyone they managed to tangle up in an interview, so their incentive to fuck up people’s careers was high. And once you got dinged, there was no chance of getting back in. So yes, your investigator held your career in his hands, and plenty of these people wanted to be Jack Bauer, the One Man Doing his All to protect the Army Logistics Budgetary Research Initiative from the terrorists.
But several years after 911, people realized that this weird system was keeping out too many qualified people, and ways were found to make things a little more reasonable – like not insisting everyone get the highest level of clearance. And now after a 4 year period you can re-apply. If anything, the disturbing thing isn’t that too few people get in, it’s that too many get in, and end up doing very little …
Are you going to create an exam-based professional organization? I’m far away from USA, but I’d like to be involved in the process in some way.
Join the ACM and vote for software practitioner certified licensing next time it’s debated in CACM, which seems to reoccur every 2-3 years. Engineering used to mean something — Civil Engineers in the US are licensed, because if the bridge falls into the river, people die.
I’ve maybe met two software practitioners in 30 years who actually “engineer” software. The title “Software Engineer” is misapplied to 99.5% of jobs. I call myself a “Software Developer”, because I do not practice engineering 100% of the time.
Professionals do not unionize, they license, because experience is not interchangeable and a minimum competence is required — we are not hunched over a factory assembly line. If you feel like a cog in a giant corporate wheel, change jobs.
I don’t know much about ACM. At first glance, ACM looks more like an academic group than a vocal professional group that takes professionalization seriously. If people in ACM were organized for the purpose of professionalization, it might be worth my time. If not, I’d better create a new one or find an organization serious about professionalization.
This is the tag line at http://acm.org:
“Association for Computing Machinery
Advancing Computing as a Science & __Profession__
We see a world where computing helps solve tomorrow’s problems – where we use our knowledge and skills to advance the __profession__ and make a positive impact.”
ACM was founded in 1947. It has > 100,000 members worldwide. Although ACM is a major participant in higher education and pure CS, it is a computing profession organization. Read CACM. Read ACM Queue. Read ACM RISKS (http://www.acm.org/public-policy/acm-committee-on-computers-and-public-policy). Look at the Special Interest Groups (http://www.acm.org/special-interest-groups/sigs-by-knowledge-area). It created the $1,000,000 Turing Price. The 39th Annual ACM International Collegiate Programming Contest (ACM-ICPC). The ACM Digital Library contains almost *every* single CS paper it published since 1947.
I’ve been a member since 1987 and I never finished college.
> Professionals do not unionize
Sure we do! Maybe a lot less in the USA but here in Europe, depending on the country and profession, professionals unionize almost 100 %. Don’t be dumb!
Michael, always enjoy your articles. On guarding our profession against non-programmmers, how effective do you think is the current interview practice? In my company, a candidate has to go through a 2-hour programming test, and do white board coding during on-site. I’ve interviewed at a few other places that do the same thing. Do you think this is an effective way to filter out bad programmers?
Interviews only identify whether you can write code competently or not. They don’t tell you whether the person is productive, writes maintainable code that isn’t expensive to maintain, documents well, etc. That’s why referrals are so important.
So, the truth is that no one wants “interviews” or “referrals” per se. People want information, on both sides. Just as there’s almost no value in “code” (the value is in solving the problem or automating the process) there is no value in interviews, work samples, or references aside from the information contained.
The problem with back-channel references is that they contain unreliable information. If someone passes a 10-ply back-channel reference check, there are two possibilities. One: that person never offended anyone (which means he’s probably not very good, either, but blandly average). Two: that person has intimidated the hell out of people (which means that he’s dangerously unethical).
That’s a very naive view of referrals. In my experience more often a referral helps a candidate. It’s often the case that someone does just “meh” in an interview but the side channel information effectively reduces our risk in hiring that person. It’s helpful to have someone on the inside of the previous employer that can say “yeah, that guy did solid work”.
I’ve only seen it go the other way a couple times. One was interesting, the candidate did fantastic in the interview but in the wrapup the next day several people said they heard misogynistic stories from past coworkers (one of which was the wife of one of our coworkers). That probably saved us firing that guy later or (worst case) a lawsuit. Your reputation matters, there’s no way to get around that.
I think you should have hired a few of the putatively mysogynist guys anyway to test your hypothesis that back-channel references are reliable. Without actually testing some people in the field, you will never know if your hypothesis was actually right.
The thing about bad rejections (or, to use the passive-aggressive term, bad “passes”) is that no one *ever* finds out that they’re wrong, except in the ultra-rare case where that person launches a competitor (in which case, he probably comes from money and didn’t need the job). I tend to think that organizations write off rejections as 100% successful (whereas hires are definitely not that) and therefore develop a bias toward not hiring people. This is probably why people need to be cycled out of interviewing after a couple of years, at least for a while. It’s not just that it gets exhausting and burns people out. People also, I think, converge toward rejecting too often.
I’m sure that back-channelling weeds a number of really bad people. My concern is that it’s going also going to kill a lot of innocents who just got unlucky.
Is it worth catching 1 bad hire if you have 20 bad passes? Certainly not for those 20 people. But also, false negatives actually create false positives. If you pass on good people and still have to fill the role, you increase your chances of getting someone who’s bad.
I think it’s better to recognize that no hiring process is perfect. Occasionally, you’ll make a mistake and have to fire someone, but one hopes it shouldn’t happen too often. Also, a company’s not going to lose a lawsuit because it hires someone who behaves badly (say, sexual harassment). It’s only if management fails to fix the problem, or if the company knows and does nothing, that the company is found liable.
I’m amazed at the tremendous length of this AND the distinct lack of profanity in it. Unless this was written by a ghost writer you hired to clean up your image… money well spent 🙂
It’s a good attempt to clear the air around your reputation, but it still comes off as tin-foil. Regardless of the actual truth, few people will believe that a forum (or whatever you call Quora) was withheld funding because of one of its users. And the continued obsession with Graham (whose relevance is subsiding but is still one of techs little darlings) will similarly smell funny to most. If you want to be a V.P. of engineering or CTO you’ve got to know that people at that level are heavily vetted. Nobody wants to hire someone with a persecution complex. Again, I say this is true regardless of the actual facts; everyone may very well be out to get you, it matters not.
What you say is absolutely right, and it’s part of what pisses me off. The Quora and Hacker News bans are incredibly damaging, because they aren’t seen as regular websites. Quora wants to be a major player in publishing, with the power to make or break reputations. If it gets there, God help us.
That an investor would force Quora to ban a user doesn’t surprise me. Investors, in Silicon Valley, are bosses these days. I’m sure that this has happened to many other people. The difference here is that I have the skills and sheer tenacity to find out about it.
Which brings me also to Paul Graham. From what I’ve heard, he carries a lot of weird vendettas. He used to spend 6 hours per day on Hacker news. I doubt that I would have even made the Top 25 on his enemies list… were it not for the fact that I actually fight back against people like him, which many people don’t.
Not sure you got the point of my post.
He’s not a bot, though; I’ve met him.
Sadly, I believe that in general technical excellence doesn’t matter in the market. And I have worked on quite a few products were it was actively punished.
The problem is that developers are generally subordinated to those in the organization who are incapable of identifying technical excellence – product managers, project managers, program managers, executives, even most Scrum masters. They don’t understand it and therefore don’t value it and thus see any effort put toward it as pure waste and as engineering masturbation. Yes, if a technical problem hasn’t been solved correctly, it hasn’t been solved. But very few are capable of determining if it has been solved correctly, and even only a subset of those care whether it was solved correctly.
How much more in the market will a developer be offered who writes extremely secure code with secure designs and prevent a host of security incidents than a developer who doesn’t? $0. How much more will a developer very skilled and experienced in writing testable code be offered over someone who has never written a single test? $0.
In my experience technical excellence is not something that the market values in general.
I’m afraid that you’re right.
The question is: does the market undervalue technical excellence because it doesn’t know how to spot it, or because no one actually cares? If it’s the former, an exam-based meritocracy can fix the problem. If it’s the latter, then we actually have to organize in a more aggressive sense of the word.
I believe that many engineers care. A minority but still a significant amount. I’ve even seen the interest flourish for short periods of time under managers willing to push back organizational forces until the inevitable and eventual capitulation back to business as usual.
As much as I like the idea, I’m not sure exams are the solution. How do you explain the complete market failure of exams like the IEEE CSDP exam, the IEEE Professional Competency exams, and the Professional Engineer option for Computer Science? I’m not saying that these were necessarily the right exams, but there’s been essentially zero demand for them, leading me to believe that ‘product/market fit’ for such a product is non-existent or (at best) elusive.
My experience is that it depends on the company size and organization. Startups look at you and say “we can only hire 3 engineers right now, can he significantly contribute to our team?” Large companies look at you and say “we need to hire 1000 engineers this year, can we use him for something?” Depending on organization large companies may operate in the other mode.
Maybe in NA. But in Russia and China, I assure you, it is quite the opposite (in the things that MATTER. So yes, real world is shytty everywhere, more or less. But shytiness in NA by far surpass shytiness in ANY other part of the world.)
Good to have you back and posting. Concerning “blind exams” – as a musician (when not building code…) I just wanted to make sure you were aware of the work with “blind auditions” in recent symphony orchestra hiring – it’s a great story, and dramatically changed the gender bias of standard auditions. For example, http://gap.hks.harvard.edu/orchestrating-impartiality-impact-%E2%80%9Cblind%E2%80%9D-auditions-female-musicians Hmm – come to think of it, orchestra musicians are unionized!
This is what I had in mind. I’m well aware of the changes that came in professional music after auditions were made blind, and I think we could achieve similar gains with an exam system.
Thanks for the great post – have some criticism:
> Toward this end, we need to work as a collective and develop a “slapping one is slapping all” mentality.
Starting after this post, right? Or was I mistaken in thinking that anything you wrote here was intended to disparage any current members of your collective?
Also, somewhat related to Meat Donut’s point, you sure seem *fixated* on unions and collective bargaining for someone “skeptical” of them.
> Bad software engineering and terrible corporate cultures hurt companies. They hurt society. The problem is that they never hurt the individual managers and executives who create these illnesses, in search of short-term profit. Those people get promoted away from the messes they create, and someone else gets stuck having to clean up.
This is just as true of political organizations; and unions. Maybe “Union!” isn’t really such a great solution in so far that programmers will now themselves be responsible for maintaining a powerful organization; the same people with such supposedly poor social and political skills. Or do you think that a suitable union could be designed that would run itself?
> Most of society doesn’t even know that the technology industry is so badly run, much less how much value has never been realized because of that fact.
This is true of *every* industry, with the exception of the small number of people that know how badly *their* industry is run, and the even smaller number that realize incompetency is just sadly near-universal. More charitably, *excellence* is just relatively rare. And people *mostly* don’t severely hurt themselves or others.
You might find this parody about software developers unionizing interesting. The author basically makes the case for unions by mocking the view that it is a bad idea.
Game developers must avoid the “pay me for my work” attitude (https://medium.com/@GurthSnisley/game-developers-must-avoid-the-pay-me-for-my-work-attitude-35d354ff3832)
> Why has the market for programmers been flooded by inferior replacements? The perception is that technical excellence doesn’t matter, and that the market situation proves this. I disagree strongly.
Your argument that the market is wrong here is specious at best. Au contraire, the current brand of capitalism which we are stuck with today proves that the vast majority of technical problems when it comes to computer science and software engineering are that technical mediocrity is more than sufficient. it is because Technical Excellence does not sell.
ISV’s sell software not on excellence but on perceived technical excellence greased with executives with a big expense account and a good golf swing. Consultants sell software and services on perceived excellence to clients who are too helpless to tell the difference. Startups sell themselves to larger companies on ideas and headcount. Everybody who stands apart on actual technical excellence are being snubbed out in the market. They are increasingly becoming outliers.
Sales is what makes these companies successful. They are the profit center. Even where software is your bread and butter, technology is actually a COST CENTER and always will be. The great cost center in Capitalism used to be Labor. It was the big factor in the industrial age for capitalists. Now that automation, software, and an increasingly educated populace have commodotized even white collar labor, the labor cost approaches zero so the big cost center that remains to a Capitalist is their Captial itself. When growth in a mature economy is stifled, the only way to survive and continue increasing profits is to commodotize and operate on the margins by volume. They not only do this with product but with labor, and even software and technology.
Tech is becoming commodotized so volume wins the day and the capitalist with the most intellectual capital stands victorious. It doesn’t matter if the young upstart is technically better.
We are a dying relic my friend to a bygone era. Life will only get harder and harder for you and me as we struggle forward.
he, he. Look the stock of ibm. What do you see? ibm went on the shytty path many years ago. So read my lips: ibm (and majority of such companies) are going to – eat their own shyte. These are the companies that prefer sociologists to software engineers any time anywhere, and do it for years on a massive scale. Their rationale is simple: sociologists are much more likeable. Customers will want them on board, which translates to higher earnings (that they produce shyte nobody cares. Except the stock market: once the stock dives to where it should, many shytty executives of those shytty companies who hired those shytty sociologists, english majors, anthropologists, biologists and all other sorts of unimaginable stupidities, will also dive through the sky throwing themselves from the highest floors of their buildings, eager to pay a visit to St. Peter)
A professional organization for software developers/engineers/technologists is probably inevitable; the real question is how far away are we from the time it happens? The AMA and professional engineering organizations exist to self-regulate, which avoids a lot of heavy-handed government regulation in its place. With software becoming more and more crucial to our everyday lives, eventually we’ll have to self-regulate or risk regulation from an outside, largely uninformed committee.
No bureaucracy is perfect, but I think a professional software organization would be a net positive in the long run. It might have been damaging as little as 10 years ago, but software development is reaching a level of maturity that I think we could organize and codify some things without hurting innovation and creativity. While it could be done very poorly, I’m optimistic about what a smart group like us could build. Plus, we’re not all socially/organizationally inept; that’s just a stereotype. Most of us learn how to interact with “normals” by the time we’re 30 (i.e. recognize that we need an outgoing filter, rather than our default incoming filter).
Pingback: Links 4/22/16 | Mike the Mad Biologist
” If it were anyone else with your talents, I could place you pretty much anywhere. 300, 400 grand a year base, direct mentorship from people like [billionaire hedge fund manager] and [another billionaire hedge fund manager]. You’d be one of the best people they’ve seen in years, and unlike the people around them, you’re honest and ethical. The thing that’s holding you back is… a reputation problem. People Google you and think, “This guy’s going to start a union.”
This is going to make it hard to place you. Guys like [name] don’t like people they can’t buy. If I were you, I’d start blogging again and come out against unions. Do that for six months, and I can put you anywhere you want to go.”
I have yet to meet a financial sector recruiter with such excellent English writing skills and the time and disposition to write such a mini essay to a candidate they do not want to work with.
It was a phone call, and so I might be injecting my own writing style.
It definitely wasn’t that he didn’t want to work with me. It was that he thought I could do better if I came out on this issue (which I did, although not in the way that he recommended). Recruiters work on commission, so they get more money if they put you in a better role, and the good ones want to put you in leadership roles, if they can, so they have a contact for the future.
Another bizarre element here is: why would a billionaire hedge fund manager “mentor” a programmer, or even their CTO? You’re not a trader, you’re not a portfolio manager, your’re not a sales man. If anything, they’d expect you to teach them technology.
Many wouldn’t, but the more progressive ones know how much they rely on technology in order to execute. You wouldn’t want to run stat-arb on top of rickety technology.
It depends on the trade frequency. You need high tech for low-latency trading execution, other stuff can be done with off-the-shelf tech (I mean execution, not strategy development).
And you’d be surprised how rickety even the HFT tech can be 😉
That’s probably true. And I’m probably not the go-to guy for ultra-low latency problems (at least, not yet; I could learn that stuff).
One of the things that frustrates me about the tech industry is how much technical excellence has ceased to matter. It really shifts leverage away from people like me (talented, ethical, but not gifted at pretending to actually enjoy a subordinate role) and toward a sort of uninspiring, reliably mediocre person. I had hoped that HFT would provide a different scene (that’s a big part of why I moved to Chicago instead of Seattle; that said, I’m glad that I’m in Chicago) but every signal that I’m getting is that, excluding around That One Guy who’s been doing q/kdb and ultra-low-latency C++ since 1987, even finance doesn’t really have a place where technical excellence matters.
What can I say… I work in HFT (as a quantitative trader, but I interact a lot with developers). Good programmers are still getting a lot of respect in our firm, and enjoy quite a lot of autonomy (you wouldn’t believe how many times we are told to go away with our requests, because the tech team wants to do something else… even though it’s the trading which pays the bills). But the golden days of HFT are over, and technological advantage won’t let you just “print money” anymore, because the market participants have cought up. Now the advantage has shifted to smart strategies, for which the tech teams play a crucial, but supporting role. But we still treat technology seriously, some people from my firm participate in the C++ committee meetings for example. The problem you might be seeing is that – given that the HFT scene is shrinking – the tech people who have been there for years might be reluctant to make space for an amibitious newcomer. And the firms are cutting costs. If the trading desk makes 10-20 millions USD a year, hiring another C++ developer on a $300k base salary is easy. Not so easy if you’re just making 1-2 millions.
The problem non-technical people have is that it’s really hard to for a non-developer to distinguish between a really talented programmer and a bullshit master. And it’s ultimately the fault of the programmer community that they haven’t developed a culture of separating the two.
“The problem non-technical people have is that it’s really hard to for a non-developer to distinguish between a really talented programmer and a bullshit master. And it’s ultimately the fault of the programmer community that they haven’t developed a culture of separating the two.”
This is utterly, and painfully, true. Have you read some of my other writings on the need for, at the minimum, some kind of professional organization? I am sick of this culture of charlatanry and immaturity.
“Have you read some of my other writings on the need for, at the minimum, some kind of professional organization?”
I did. I sympathise with the aim, but I do not think it is achievable now, at least not in the way most people see it – as some sort of legal bar, engineering society or tradesman’s guild.
First of all, medieaval guilds were much more than just certification and standards organisations – they were more like wealthy co-ops. Guilds of Venice were rich enough to build splendid palaces full of art. Hanzeatic Legue guilds maintained armies. This place in capitalist economy has been taken over by corporations and governments.
Second, legal profession has thousands of years of tradition behind it. No comparison with computer programming which is not even 100 years old.
Third, engineering organisations work in disciplines which have a well-defined concept of good practices. Despite everything which online pundits and book authors say, software development DOES NOT have it. People simply cannot agree on what it means that a software is “done” or “correct”. Compare it with e.g. construction industry which has a pretty well defined concept of “this building is finished” or “this bridge is safe to use”. Programmers think of themselves as of almost mathematicians, but in reality 99.9% of them are tinkerers who do not understand fully the machines they are operating, the mathematical/logical theories behind the abstract models they use, etc. Not just that, but also Computer Science researchers do not know what it is a “finished software project”! Engineering disciplines have their own universitites with cadres of professors who collaborate with the industry, pushing both the theoretical and the practical sides of their disciplines forward. In programming, the division between the industry and the academia is huge, amounting to almost hatred.
You think that programmers – by the virtue of knowing something like C++ or Haskell – are automatically entitled to a social status equal to disciplines with long traditions, mature cultures, established standards of quality and systems of accountability for malpractice. No chance. You want respect, you gotta earn it.
computer science and software engineering are two different areas. Like biology and chemistry. Related, but not quite.
I laid my hands on IEEE Transactions on Software Engineering first back in the summer of 1981 and I was immediately attracted to the field. In the next 35 years I maybe met in person or worked with not more than 10 (ten) real sw engineers. Why is there so many computer science departments and only a few sw engineering departments, I do not know. The truth of the matter is that the lack of seasoned sw engineering resources, among other things, led to the so called internet bust, an explosion that resonates even to this day (and will resonate for at least a decade more). Trillions and trillions of losses.
In the absence of sw engineering, industry has shifted to the so called (very expensive) COTS solutions. Obviously, mediocre people immediately took the “ownership” of these COTS solutions, and again they are in the spot light. And real sw engineers? They are hidden inside the core libraries of these COTS solutions and platforms, most of which were originally developed in some garages, and sold to big players for a fraction of the cost.
Also, ask a construction company to build a 100 storey building, for $1000. 10 bucks per storey. The deadline? Yesterday. That is exactly what has been happening in the sw engineering.
Roman, I like your opinions.
> Guilds of Venice were rich enough to build splendid palaces full of art. Hanzeatic Legue guilds maintained armies. This place in capitalist economy has been taken over by corporations and governments.
People say middle age was a dark age, but it had virtues rarely seen in the modern era. Medieval guilds were probably more eager to pool resources than modern professional organizations are. A professional organization of programmers could also learn to pool resources as a co-operative to fund startups(, if not armies).
> Third, engineering organisations work in disciplines which have a well-defined concept of good practices. People simply cannot agree on what it means that a software is “done” or “correct”.
We could try to accelerate establishing well-defined concepts of good practices.
> In programming, the division between the industry and the academia is huge, amounting to almost hatred.
It’s too bad that the industry people avoid the academia. The industry and the academia should reach out to each other more often. A proper professional organization could bridge the two.
“A professional organization of programmers could also learn to pool resources as a co-operative to fund startups(, if not armies).”
What resources? A typical VS has millions of dollars at their disposal. How much could a single programmer contribute out of their savings to this “guild”? Maybe a few thousand USD max. You would need to gather at least tens of thousands of people to achieve anything.
If you want to fund your startup avoiding VCs, it’s more realistic to hope for Kickstarter covering it. But then you’re not accountable to anyone.
Medieval guilds also could punish (and punish harshly) its members for breaking their rules. A voluntary association of developers will not have such powers. Only government could grant it to them (like they granted it to lawyers who have the power to disbar).
“We could try to accelerate establishing well-defined concepts of good practices.”
That’s step 1.
” The industry and the academia should reach out to each other more often. A proper professional organization could bridge the two.”
This is a realistic goal which doesn’t require fighting battles with corporations. A good start for bringing this discpline towards maturity.
you have good ideas, but also a big tinfoil hat.
Even if the whole Graham/quora thing is real, and I believe you, you should understand that you promote it in a way that is quite offputting, and a lot of people really don’t give a crap and it makes you look like a tinfoil hat wearer.
You could get a better response for things that you promote and that matters if you drop the online forum thing entirely and focus on what matters (like exposing bad corporate practices against programmers, promoting professionalizations and way to do it, and other things that are more important than online moderation on silicon valley propaganda channels).
I agree. The dual of “Just because you’re paranoid doesn’t mean no one’s out to get you” is the also-true “Even if people are out to get you, you sound paranoid in discussing it”.
Off topic, but what happened to the rest of your archives? I’m only seeing this year plus June 2015 and July 2013.
Is it really that easy to get another job? Most people want to employ kids they can burn out. The good jobs are given to friends.
I think the hardest part about job searching is striking a certain balance.
On one hand, you have to swallow your pride. For example, it’s humiliating as fuck to send a cold CV, but it’s rare that someone has enough connections to do as well on referrals as through cold-calling. It’s a numbers game and ultimately, you have to do both: some warm intros and networking, some cold calls where there is a 90% chance your CV will be ignored no matter how good you are. But you have to get past your pride and be able to navigate a degenerate stupid system… not just in job searching but in corporate life, because what is corporate system if not the ultimate concoction of the degenerate and stupid?
This is easy advice to give, but hard to take. I’m not good at swallowing my pride. People who are from third-world countries are better at it because they’re used to doing what it takes to survive in a corrupt system. Americans who still harbor vestiges of belief in meritocracy will struggle with it, though.
At the same time, you have to keep enough of that pride to look out for yourself (no one else will) and avoid getting exploited. As you note, most jobs are traps.
Right now, I think that generalizing about “most people” across time is dangerous. Private-sector corruption is at an all-time high. We’re deep into third-world levels of it. It’s so bad that it’s unsustainable. I don’t know what “most people” want. It’s a correct observation that, as of now, most people are just trying to build sham companies that grow quickly, enrich an in-crowd, and farm the marginal in order to exploit them. Will that always be true? I don’t know. I’ve started talking to VCs about this and even they are pretty disgusted by the founder-quality situation, although only a few have the insight to realize why it exists and how it came about.
tl;dr: Write one ticket “Usability study”, take a week on it. Wow, you’ve taken a whole week on one ticket? Make small change, reduce shopping cart abandonment by 10%. Immediate $1 million/month gain. But only one ticket in a whole week, what’s happening to your velocity? Answer, management is measuring the wrong thing.
Michael, re-double your fortitude. Its not your blogs, or a black-list, it’s your non-compliant attitude. Any potential employer can see that you will not “get with the program” because the program is about bilking as much money as possible out of stupid-soon-to-be-failure technological expeditions financed by equally stupid corporations. The corporate model is now obsolete; a zombie that is running out of brains to eat, but for now there’s just enough money to squeeze from this hot-turd called information technology for the developer-turned-project-manager to finish paying off his mortgage by outsourcing to some back water country.
Unions would crush what’s left of the IT industry until USA stops outsourcing and using H1-B programmers to keep wages low and working conditions sweatshopped because, lets face it, Russia, America, (and soon China) produce the best developers in the world; developers educated and experienced in other countries simply produce cheap labor (and buggy code).
“Russia, America, (and soon China) produce the best developers in the world; developers educated and experienced in other countries simply produce cheap labor (and buggy code).”
Like this Finnish dude, what’s his name, Torvalds? He wrote this buggy operating system.
Outsourcing has the Design Paradox (but so does hiring American programmers).
Yes, there are great developers in the Philippines who’ll work for $20/hour. In Russia, you can probably get top-notch minds for $40/hour, which is still substantially cheaper than in the U.S. However, you have to find those people. It isn’t easy. If you’re bad at it, you suffer adverse selection and get shitty programmers.
To get quality, you need to find and trust someone like me who’s actually smart enough to figure out who the good programmers are. But how do you recognize such a person, if you’re not one? The CEO is ultimately taking shots in the dark for his top technical hire.
The Design Paradox still exists in the U.S. The difference is that there’s shared cultural currency, and more room for micromanagement in on-site jobs. So, even though the business guys can’t tell who the good programmers are (unless they have a guy like me, but how do they know if they have a guy like me?) they can at least get a sense (or *think* they have a sense of) work ethic and attitude. If you believe that programming is just a matter of banging symbols together until something works (i.e., commodity labor) then you can convince yourself that you can detect “talent” through purely social means.
“Yes, there are great developers in the Philippines who’ll work for $20/hour. In Russia, you can probably get top-notch minds for $40/hour, which is still substantially cheaper than in the U.S. However, you have to find those people. It isn’t easy. If you’re bad at it, you suffer adverse selection and get shitty programmers. ”
I don’t want to start a discussion about outsourcing – I just objected to The Dude’s assertions that countries with top-notch education systems (e.g. Finland, Germany, France, Great Britain… leaving aside the post-communist countries) somehow fail to produce good programmers.
The solution to your problem is to buy / order finished products from foreign teams. Western video game publishers often hire Polish, Hungarian, Czech, Russian, Ukrainian or even Byelarussian (who created the World of Tanks franchise? Byelarussians!) studios to write games for them. How do they pick the good guys? By looking at their track record, mostly, and discussing the deal in details. But that’s a business-to-business relation. If you treat outsourcing like a B2B problem, it becomes much easier.
That seems to be true but, in that case, you lose a lot of the economic advantages of doing it. Although labor is cheaper in those countries, completed companies tend to converge to value.
I suppose that might make it better to be an employer overseas because, even though it’s harder to control people– you face stronger labor laws in the EU and — there’s more money to be made… and, as you said, there are plenty of talented people everywhere.
What the U.S. excels at is graduate school and academic research. Our PhD programs are super-selective (I probably wouldn’t get in to a Top-5 CS program) and difficult (and poorly paid, but that’s another issue). When it comes to undergraduate education, on the whole, we’re not really above the rest of the developed-world pack.
Its a combination of both education and industry knowledge built up over time in a particular geographical region, that is all. Russia (I guess I really meant old-soviet block) and America have a residual edge because the computer industry in these two countries is the oldest.
P.S. Torvalds started and continued his career in America, and is even a U.S. citizen now.
P.P.S. Belarus built the Opera Browser; awesome browser.
1. He started and was educated in Finland. Anyway, he’s just one of the many great European programmers (Russia is not Europe).
2. Not Belarus, Norway.
“the best way to drive out the non-programmers and the non-serious technologists is to implement an exam track similar to what the actuarial sciences use”
You don’t have to look to actuarial sciences for this. You can occasionally find licensing requirements for engineers as well. For example, in Ontario: http://peo.on.ca/index.php?ci_id=2153&la_id=1
I can all but guarantee that industry trade groups would oppose this for programmers.
Consider how, 25 years ago, lobbyists pushed pushed changes to US labor laws that made entry-level developers a whole lot cheaper to employ, thus undercutting the entire profession:
http://www.nytimes.com/1991/01/28/business/computer-programmers-to-lose-overtime-pay.html Their exact argument, or lack thereof, was that in spite of there being no licensing or certification body for programmers like there is for every other profession, programmers are important. In other words, no one is looking out for their best interests so the industry will happily steal from them. There’s one of the many reasons why the average doctor or lawyer drives around in luxury vehicles while programmers get foosball tables and free snacks. According to what’s written in the FSLA, it’s because our employers think we are very, very important and are compensating us accordingly. But I digress. You would think that if the industry wanted to pass a law that treated programmers as a profession, it would be to establish a professional licensing body. Instead, they passed a law to undercut wages.
I agree with what you’re saying here, but I also think this fact will work against them. Moreover, if we as a group form a (non-profit — that’s important) certification body whose certifications organically become the de-facto standard, that gets us pretty close to becoming a licensed group. Eventually, that would turn into actual licensing.
I see this going a little differently, though. I think we get a long way down this road by more of us becoming contractors/freelancers. There’s already a pretty strong movement in that direction, particularly among high-performing developers. That’s the direction doctors and lawyers came from.
It makes sense in terms of other patterns, too. Everyone should know a little law, but when you need real help, you seek a professional lawyer. Everyone should know basic medicine (aka first aid), but you see the professionals for anything serious. We’re quickly reaching a point where everyone should know how to write a simple program, but you’ll hire an expert when you need critical or complicated software that isn’t an off-the-shelf product. It makes sense that we will eventually become licensed to prove we’re authentic, and likewise charge professional-level prices.
As I mentioned in another comment, I think professionalization of software “engineering” is inevitable. The real question is how long until that happens.
> no one is looking out for their best interests so the industry will happily steal from them
This is the modus op. of capitalism, and the reason unions are absolutely vital to an industry as commoditized as ours.
That’s not the M.O. of capitalism at all, just a false claim anti-capitalists make. Capitalism is successful when exchanges are beneficial to both sides. Making one-sided deals in your favor hurts you in the long run, and good businessmen know that. Yes, abuses can and do happen, and unions are one counter-measure workers have against that. But likewise, unions making deals that benefit them more than the employer(s) hurt them in the long run, such as 50% of GM workers being laid off because the union refused to take a much smaller, likely temporary, across-the-board pay cut despite the fact GM was filing bankruptcy and warned them in advance that would happen. Capitalism is complicated, as are most economic systems. So far, despite our somewhat poor implementations of it, capitalism has shown itself to be the best way to encourage rapid development and improve the standard of living for everyone in the system.
Software development is far from commoditized. If that were true, there would be no reason to post job requirements or do interviews beyond meeting the person to make sure they exist and are sane. You’d hire the cheapest programmer, end of criteria. There would be no salary progression, and we’d keep getting cheaper, not more expensive. Mass software development outsourcing is dying out, which is another sign it’s not commoditized. I can’t think of any skilled labor that’s “commodity.” Mechanics and hair stylists might be close to commoditized these days, but even that’s an oversimplification of reality.
I believe a professional guild structure would benefit both programmers and systems engineers. Not primarily in the bargaining area, but the establishment and maintenance of professional standards, including codes of conduct. It would become a point of pride and excellence.
It would also eliminate the “elite degree” as the measure of purported competence. While having a degree would get you farther, faster on the apprentice track, real experience is what would be used past that. Yes, I’m talking code reviews, take-home tests, and things like that.
When you hired a guild engineer, you would know what you were getting. Yes, they would command more salary, and not be willing to be open-plan scrum slaves. But they would be able to design and build more robust applications in less time. To me this seems like a better formula for commercial success, which appears different than fund-raising and other startup hijinks.
… the fact that Michael Church was the only person in my six years at Google to get so hated by literally everybody that ready any of the (many, many) mailing lists he posted lunacy to suggests that maybe this isn’t an “industry-wide conspiracy” and maybe just a bunch of people who, from experience, know that they don’t want to work with Michael Church. Ever.
Really? You worked for 6 years at a company that currently has nearly 60K employees and $75B per year in revenue, and you only know of 1 person that was widely disliked? That sounds really unlikely.
I have no doubt somebody like Michael rubs a lot of people the wrong way. I’m sure some of his cynicism for the status quo in corporations and throughout the software industry overflows into his professional interactions. That’s bound to make him a few less “friends” in business. But frankly, I think more of us should show open contempt for bad business practices, policies, and processes when they are so often abusive to the average worker.
Also, there’s a big difference in “don’t want to work with [person]” and “try to stop [person] from working for anyone in the industry because we hate them.” Namely, the latter is illegal, not to mention childish and unprofessional. Not that it has ever stopped union-busters from doing it — or are we supposed to believe that doesn’t still happen, either? Industry-wide conspiracy? Probably not. A bunch of Silicon Valley venture capitalists colluding against someone, as is already well-known (and documented) to happen? Quite likely.
The truth is usually somewhere in the middle of the two sides of the story. In this case, I’m willing to believe it’s slightly more toward Michael’s version of things.
This is an adult response to an unscrupulous comment.
“Industry-wide conspiracy” are your words, not mine. I said that there are a number of people who’ve tried to damage by reputation for reasons that, when fully explored, leave said people (and their claims) lacking credibility.
What a wonderfully thoughtful discussion of the Bay Area madness this is. Ten years ago I had to ninja-back my own NDA (suitable-for-framing) to write freely about a Google-related boondoggle, still in progress. I guess security is better now. Bravo to anyone who speaks up about the crisis, never mind the push-back.
VC stealth and hype only hides pervasive incompetence from investors; there is no secret-sauce. I first groked the cultural sickness in Austin, in the mid-80’s, at the lavish private parties of young noveau-riche digeratti. The suppressed angst of hundreds of equally brilliant 12-dollar-an-hour coders was a silent-scream.
A friend and extreme-sport athlete died at his desk one mid-morning working on a GoogleX “moonshot” (none of which have hit any moon). The Coroner found no cause of death, but it sure looked like Karoshi to me, the over-work syndrome that the Japanese first legally recognized. Apparently you have to look very closely at the heart, which somehow tries to turn inside-out suddenly. More than two years at “start-up pace” with +60hr weeks is enough to win a Karoshi case in Japan. This kid had gone four. In the Bay Area you get a temporary memorial web page.
Part of the sickness is group-pretend that some folks are simply insufferable. Will Rodgers never met anyone he did not like, and he would have liked Michael just fine.
I have found in my imagination that the SECOND you here that “we need to change the” industry, country, society, etc, it is ALREADY at least 20 years too late, it takes 10 to even start to notice there is a problem and only the most imaginative start to “detect” it, but it takes 10 more to finally say so.
Only world war will change this industry but in the same trajectory as its current evolution, BUT FOR THE WORSE! LOL
Pingback: Farisa’s Courage publication update – Michael O. Church
Pingback: Crossing the Equator 1: From Tech Blogging To Fiction – Michael O. Church
Pingback: Silicon Valley's Gilded Cage | naked capitalism | Me Stock Broker
Pingback: Silicon Valley’s Gilded Cage | naked capitalism – Courtier en Bourse
Pingback: The Purpose of this Blog – J. Savage
Pingback: Big Brother is Already Here, He Employs You – J. Savage