Farisa’s Courage Table of Contents As Of May 1, 2017.

Of course, chapter titles may change, and the page numbers are unlikely to match those of the printed book. I’m just sharing the current TOC because I think it’s interesting data about the book.

Currently, Farisa’s Courage is 470 pages and 124,758 words. I suspect that, by time of publication, it will be somewhere between 115,000 and 125,000 words.

Table of contents:
01: enter farisa          ................   1
02: house 139             ................   7
03: reverie '94           ................  23
04: the city of exmore    ................  39
05: knowledge and virtue  ................  58
06: train's out           ................  79
07: aftermath             ................  92
08: the global company    ................ 105
09: knight's move         ................ 136
10: high summer           ................ 160
11: playing for keeps     ................ 186
12: the high road         ................ 208
13: royal jelly           ................ 227
14: switch cave           ................ 237
15: joyful september      ................ 251
16: the ivory ashes       ................ 266
17: blank spot on a map   ................ 296
18: brown shoulders       ................ 314
19: what we have lost     ................ 327
20: the magic forest      ................ 361
21: there's always another way ........... 397
22: p.s.                  ................ 427
23: crossing the equator  ................ 444
END                       ................ 470

Crossing the Equator 1: From Tech Blogging To Fiction

I had originally intended to quit blogging. In fact, I started shutting down social media accounts. For example, I went from 2,500 Twitter followers to zero. I don’t miss it. That addiction to internet micro-approvals wasn’t healthy. “Likes” feel like genuine social connection and retweets feel like cultural influence, but let’s not kid ourselves. Social media is crack for anyone insecure enough to seek external proof of value– and all of us become guilty of that, at some point in our lives.

In the 2010s I learned, the hard way, that technology companies can’t be trusted. It began in 2013 when Hacker News began representing my comments as having performed more poorly than they actually had– a moderation practice called “rank-ban”– largely because I criticized the prevailing business model of Bay Area venture capital. I was able to prove the existence of this “rank ban” using accessible historical data. In 2015, I was banned from Hacker News and then Quora (also owned by Y Combinator) under false pretenses of a more serious nature. Quora, in addition to blaming a Y Combinator partner by name, threatened to expose my viewing history to the public.

You know what? To some extent, it’s on me. I trusted my reputation to privately-owned technology platforms. That was naive, and I got burned. I spoke out about the tech industry’s social injustices and moral failures and was attacked by everyone from anonymous trolls to billionaire venture capitalists.

So, I quit tech blogging. On that topic, I’m still quit. Fuck tech, and fuck tech blogging, and fuck everything the private-sector software industry stands for… which is perhaps a vacuous fucking (like wind fucking other wind) because it stands for nothing. I refuse to waste any energy, or to put my livelihood or reputation further at risk, in a vain effort to save it. Given the risks, if I ever publish essays on those topics, I’ll be charging for it– either self-publishing e-books or through print magazines. Because of what I’ve written, I’ve had to take expensive security measures to protect myself and others close to me. That cost has gone unpaid.

Still, I need to write. In late 2016 and early 2017, I found myself in a negative mood. I don’t wish to expose precise political opinions or economic interests, but… this is not a time to sit down. Too often, I hear people express a wish to sleep through unfavorable political events or adverse economic circumstances. Here’s the scary secret: it’s very easy for most people to “sleep through” bad times. That’s how the villains win. It galled me in 2016 (especially, leading up to Nov. 8) when people talked about how “this election” had been so exhausting and dreadful (thus, creating a false equivalency between a classical campaign, with as many negative elements as any other, and a terrifying one). Yeah, our right to vote is such a bummer; fuck off with that spoiled whining.

In February, I pulled together the hundred pages or so that I’d written for Farisa’s Courage, which I originally figured I’d finish in my late 30s– I’d attempted it a couple of times, but it never quite “took”. Of that hundred pages, a decent proportion was still good. About sixty would make it into the final novel.

In late March, I figured out what was wrong with the original story, and how to make it better. The truth is that writing (like programming) is either a lot of fun or an absolute slog and, when it’s the latter, it’s a sign of something wrong. In fiction, one might not be ready for the project. That’s not a big deal. I had a very successful blog here, even six years ago, but I wasn’t ready to write Farisa until this year. It takes time. The tech industry is quick to crown people as “experts” after they give one presentation or write a popular blog post– I was once crowned an expert, by a reputable source that shall go unnamed, in a technology that I’d picked up three months before– but writing projects can span ten years between conception and completion, even for the most advanced writers.

I realized the one best way to tell the story of the first 21 years of Farisa’s life. (Farisa’s Courage is the first in a series, called The Antipodes.) I’m speaking vaguely here, because I want to avoid spoilers. Once I figured it out, I got to it. I wrote. Soon, I had a 134,000-word novel on my hands. (After revision, it’s 124,000.) The story, as far as I can tell, works. It’s compelling, it has a couple of great characters, and there’s the right mix of sad, scary, funny, and sweet moments. I’ll probably do one or two more passes’ worth of editing before I start querying agents, but it’s good.

The last chapter of Farisa’s Courage is entitled “Crossing the Equator”, and so is this series of essays.

What do I wish to write about? Well, let me talk about what I’ve learned over the past few months.

First of all, deleting social media accounts leads to happiness. Try it. The danger is that it can be hard to get them back. As someone who’s going to be publishing a book soon, that 2500-follower Twitter account had value. Oops! Social media can be useful, but it has become, for most people, a waste of time that compromises real-life interactions and that drives out more meaningful channels of information– like books. It’s really nice to be reading books again.

Second, I learned that I need to write. I guess that that was obvious. I’m glad that I quit tech blogging. I hope to be out of private-sector software entirely before I’m judged to be too old for it. Always better to leave on your own terms, right?So I didn’t write for a while. Still, that need to use the written word would come out, one way or another. Even after I quit, I wrote several unpublished (never-to-be-published) blog posts and I spent too much time, some anonymous, on Internet forums. I got to a point where I said, “Fuck it, might as well write something that matters.” So, I got to work on Farisa.

Third, I’ve improved as a writer. I was a 99th-percentile tech blogger for some time, but the standards of publishable fiction are higher than that of business writing– especially, internet business writing. Like I said, it wasn’t until my 30s that I had what it takes to write a novel. It’s not that hard to write 50,000 words of grammatically correct prose. It is much harder to come up with a complex story that anyone would want to read, and to tell it in a compelling way. I’ve already learned a lot in the (I hope, successful) attempt. Authors don’t get second chances. If someone gets bored at page 17, that reader’s typically gone forever. There are too many other things competing for the reader’s time. I know, because I worked in the tech industry. I helped build some of them.

Fourth, I’ve gained a sense of what I care about and what I don’t. In 2012, I would have said that the most important thing would be for every technology company to ditch Agile and implement open allocation. I still feel that this would be an improvement, but my mind fixates itself on bigger issues now. In 2012, I saw global techno-capitalism as inefficient. Those irritating product managers and under-capable executives were getting in the way of programming savants like me. (How dare they?) In 2017, I recognize global techno-capitalism as dangerous. Look at the political situation, anywhere in the world… and then keep looking for the next ten years because, where it isn’t bad, it will be soon enough. This is a lot bigger than open allocation versus Agile or whether Google’s cafeteria is better than Facebook’s.

Fifth, writing helps give purpose to suffering. The past 10 years have been a mix of good and bad, but there’s been a lot of of bad. I did meet my wife and get married. That has been good. I have two cats. Again, good. I wrote a novel of better-than-publishable quality. Good. Yet, for the challenges, in no particular order: I lived through the global financial meltdown; in 2012, I was fired for refusing to commit felony perjury; there were several deaths in my family; I was flagged in Silicon Valley as a “union risk”; I received death threats from highly-positioned people in the Bay Area;  conditions within my chosen industry (private-sector software) deteriorated; that same industry became an adversary of the world… and now I can only watch as technology-driven unemployment sets off a global wave of dangerous, reactionary populism. I’ve had many unpleasant experiences in the past few years… but I wouldn’t have been able to write Farisa’s Courage without them. That puts the weight on me to make it good.

On that note, back to work.

Farisa’s Courage (novel) is Revision Complete

“Revision Complete” means that I won’t be adding characters, changing scenes, or altering storyline in any major way. (ETA 5/2/17: I lied. I’ve added a couple scenes.) “Edit Complete” (i.e. fine-tooth comb, zero typo tolerance) will be a couple months from now. I’d like to have something finished and ready for the world by Oct. 1, 2017. (ETA 5/2/17: I doubt that I’ll make this date. The book will be ready, but publishing is slow.) We’ll see what I can do.

I’m sending out a finite (fixed but undisclosed) number of copies, even before I shop this out to publishers. It’s an intermediate draft (obviously) and So, to people who’ve read my writing, if you’re interested, please let me know.

Here’s the summary / blurb / trailer for Farisa’s Courage, intended as the first in a series (“The Antipodes”).

The Antipodes

The planet is hot. Civilization thrives close to the poles, but the tropics are uninhabitable. Sea temperatures exceed 50°C (122°F), so violent storms make the equator impassible by ship. Deserts broil. Jungles are full of strange creatures like skrums, squibbani, and ghouls. Thus, two hemispheres have been out of contact for thousands of years. There are rumors of a high-altitude path between the two worlds: the Mountain Road. The known path passes through dangerous cities, cursed caves, and a desert reaching 80°C (176°F). No one has made the trek and returned.

State of the World

Humans have won. Dragons, orcs, and elves still exist, but the human world stands at a population over 1 billion. Technological marvels like steamships and machine guns dominate the world. Trains achieve a blistering pace of 25 miles per hour. Plank turnpikes supporting carriages connect the cities. Yet, all is not well. The industrial economy is in decline. Age-old ethnic hatreds are broiling. Cryptic graffiti on city walls suggests danger. Economic inequality and climate change are roiling continents.

The Global Company

The lynchpin of the modern world is the Global Company. The Global Company in the business of… everything, from alcohol to fuel oil to railroads to murder. It began as a detective agency specializing in witch hunts, strike breaking, and bounty hunting. Now controlling 70 percent of the world’s economy, it hasn’t lost its taste for mayhem. It funds pogroms, rigs elections, and topples nations.

Hampus Bell, the largest stakeholder in the Global Company, is its Patriarch. (In addition, he holds political offices, but those don’t matter because the Company won.) Even Bell isn’t safe from his own firm. He faces internal intrigue, bureaucratic incompetence, and the mysterious syr Konklava. Bell seems to be losing his grip on his firm and his own mind. An executive includes a grisly murder in a corporate presentation– and his career thrives. Mysterious suicides by high-ranking officials mount. There are rumors of atrocities by (and within) the Company that even Bell does not know about.

The Blue Marquessa

Magic is real, but people with the gift, or “mages”, suffer from a terrible disease. Known as “The Blue Marquessa”, it can cause infertility, amnesia, insanity– even death. Every spell has a cost. This is a world where everything has consequences. A mage must be careful, for her practice is one that has led many to insanity or early death.

The Heroine

Farisa La’ewind is a smart, good-looking 20-year-old girl “from everywhere and nowhere”. She’s a brown-skinned girl in a snow-white land, a bookish erudite in a dumb war, and a lover in a world where hate thrives. Worst of all, she’s a known person in a society where invisibility is the greatest asset.

She’s also one of the most powerful mages in the known world. She stands accused of two crimes. One, she could not have committed. The other, against Hampus Bell’s only son, she did. This has made her a witch hunter’s prize. With the bounty on her head, she’s literally worth her weight in gold.

26 April ’94

It’s two in the morning. Something awful happened. Farisa can’t remember what. Barefoot and in ill-fitting clothing, she runs into the declining industrial city, Exmore. She believes that she can find safety in “House 139”, but she’s never been there. In the dilapidated outskirts, she encounters cryptic, threatening graffiti. She learns something dangerous. Someone is watching her– and seems to know precisely where she is.

The two most powerful people in the world are drawn into a conflict that neither of them wants to fight. Farisa must avoid Hampus (and his spies) to survive. Hampus must find Farisa or face danger within his own company. The only safe place left for Farisa is… the Antipodes. Yet the Global Company, running out of world to conquer, wants to head South as well. The stakes get higher and higher with every mile, and soon it’s not only Farisa’s fate that hangs in the balance. If the worlds are joined, much more is at risk.

Farisa meets a gun-toting steam-era knight in a leather jacket. She meets a beautiful resistance fighter with a secret past. She purchases Jakhob’s Gun, a trash novel believed to hold coded messages. She fights orcs and ghouls and dragons and even other mages. Her skills develop. She finds love and friendship. As she fights to regain her memory, she learns not only who she is but who she was– and that may be what threatens her the most.

End of blog

“When we reach the top,” Farisa said, “it’ll be worth it.”

Raqel was out of breath. Both girls had been trudging through snow since noon and it was only getting deeper as they climbed.

“You’ve never been up there?”

“Not in winter, Farisa. I’m a city girl.”

The peak was barely a hundred feet above, but each way up looked as treacherous as any other: snow and rock, mostly snow.

“Follow me,” Farisa said. “I know the path.”

“My hands are freezing.”

Reaching the summit exposed them to a fierce northerly wind. In its brief spells of rest, Raqel could hear farm dogs, baying in the shaded valley.

“It’s so cold! The lakes are frozen, the roads covered, the trees all bare–”

“Ay,” Farisa said, “and winter means you see farther. Farther than anyone. Farther than you would have ever known.”

The Disruption Algorithm

I’ve observed that business processes tend to follow three phases.

  • innovation: an opportunity is found or a problem is solved.
  • refinement: improvements are made. The process becomes cheaper, more reliable, and generally better.
  • externalization: there are few identifiable improvements to be made. Value capture can possibly be increased, and opportunities to externalize costs are exploited.

In the refinement stage, there are still plenty of opportunities to reduce costs and improve value capture (or yield) without anyone getting hurt. A process that took six months, when performed the first time, might be reduced to two. Genuine waste is getting cut. It’s not zero-sum. However, returns diminish in the refinement stage as the process improves, and the available gains are made.

At this point, talented and creative people look for new opportunities to innovate. Average or risk-averse people look for other processes to refine, and that’s fine: we need both kinds. Vicious and malignant people, however, start making false refinements that externalize costs and risks. Pollution increases, companies become more brittle, and ethics decline. The world becomes sicker, because in the cops-and-robbers game that exists between cost externalizers and regulators, there are just far more of the former. That’s how one gets corporate processes like employee stack ranking and, in software, the bizarre cult that has grown up around two-week “sprints” (as an excuse that allows management to demand rapid but low-quality work). These appear to yield short-term gains, but externalize costs within the company, diminishing the quality of the work.

Most of the sleazy activities for which business corporations are (justifiably) despised are performed in the externalization phase, when the company starts running out of viable refinements and declines to innovate. It may not be (and probably is not) the case that true improvements cease to be possible, at this point. What seems to happen is that there is a shift in political power between those who seek genuine improvements and those who mercilessly externalize costs. Once this happens, the latter quickly and often intentionally drive out the former. They are often able to do so because genuine improvements to processes usually take time to prove themselves, while cost externalizations can be devised that show profits immediately.

It is, at this point, no longer novel to point out that venture-funded startups have ceased to back genuine technical innovation and have, instead, become a continuation of the process (starting in the 1980s) by which the staid corporations and bilateral loyalty of yesteryear have been replaced by quick gambits, degraded working conditions, and a lack of concern by these new companies for their effects on society. This is what “disruption” often looks like.

The realization that I’ve come to is that Silicon Valley, by which I mean the aggressive and immediate financialization of what purports to be technological innovation, is now deep into the third phase. It still “changes the world”, but rarely in a desirable way, and most often by externalizing costs into society in way that regulators haven’t yet figured out how to handle.

While Silicon Valley is marketed, especially to “the talent”, as an opportunity for people to break free of old rules and take on established interests, it’s actually better thought-of as a massive and opaque, but precisely tuned, genetic algorithm. The population is the space of business processes, up to the scale of whole companies. The mutation element occurs organically, due to inexperience and deflected responsibility– when rules are broken or scandals occur, a 23-year-old “founder” has plausible deniability through ignorance that a 45-year-old financier or executive wouldn’t have. The crossover component is far more destructive: corporate mergers and acquisitions. In this way, new business processes are generated in a way that is deliberately stochastic and, when seemingly cheaper processes are discovered, they’re typically snapped into existing businesses, with losses of jobs and position to occur later.

In the three-phase analysis above, Silicon Valley had its innovation phase in the 1970s and ’80s, when new ways of funding businesses, and looser attitudes toward risk, began to take hold. Business failure in good faith was destigmatized. No doubt, that was a good thing. The refinement phase began in the late 1980s, brought with it the golden age of the technology industry in the 1990s, and ended around 2005. Silicon Valley is now in the externalization phase, exemplified most strongly by Y Combinator, an incubator that openly champions the monetization of reputation, self-indulgent navel-gazing, disregard for experience (a polite way of saying “ageism”), and a loose attitude toward executive ethics. The genetic algorithm of Silicon Valley still operates but, in the Y Combinator era, it no longer produces more efficient business processes but, rather, those that are cheaper and sloppier.

The innovation and refinement phases of Silicon Valley are long gone. Cost externalization– the replacement of trusted corporations with good benefits by a “gig economy” culture of itinerancy, the pushy testing of regulations by founding billion-dollar companies that flout them, and the regression into a 21st-century Gilded Age– has replaced the old Silicon Valley and driven it out. This makes it the responsibility of the next generation’s innovators to come up with something entirely new. It is clear that Paul Graham and Y Combinator, as well as those who’ve subscribed to their culture of flash over substance, will have no place in it.

Card counters

In the world of casino gambling, card counters are legendary. Most “systems” promising to make it possible to beat casinos are ludicrous. However, blackjack, if played very well, can be winnable by the player. For every successful counter, there are probably hundreds who fail at it, because one mistake per hour can annihilate even an optimal player’s edge. Casinos love the legend of the card counter, because it encourages so many people to do it ineptly, and because there’s a lot of money to be made in selling books on the subject. They don’t love actual card counters, though. Those get “burned out”. Casinos share lists of known counters, so it’s pretty typical that a too-skillful player will be banned from all casinos at approximately the same time, and therefore lose this source of income.

There’s danger involved, as is documented in Ben Mezrich’s Bringing Down the House. In Vegas, they’ll just ban you for counting. Shadier outfits in other jurisdictions will do a lot worse. It’s important to note that card counters aren’t, in any sense of the word, cheating. They’re skilled players who’ve mastered the rules of the game and disciplined their minds well enough to keep track of what is happening; nothing less, and nothing more. Even still, they face physical intimidation.

Lousy players are money-makers, and good players are seen as costs. How damaging are good players to a casino’s economic interests? Not very, I would imagine. Card counting is legitimately hard. Don’t believe me? Try it, in a noisy environment where cards are dealt and discarded rapidly. Of course, most people know they will lose money, because gambling is a form of entertainment for them. Casinos will always make money, but it’s not enough to have 98 percent of the players be lousy ones. It’s better to select lousy players exclusively and toss the skilled ones out. “You’re too good for us. Don’t come back.”

In other words, the possibility of earning an edge through skillful play is used as a lure. Most people will never acquire such skill, and casinos can hardly be faulted for that.

Play too well, however, and you won’t have a spot at the table. Lousy players only. Sure, you can say that you beat the system. It might make for interesting discussion at a party, but your playing days are over. You’ve won, now go away.

SetBang 3: Fast(er) math in SetBang

SetBang, even when limited to the finite world, is inherently “inefficient”. We might like to live in a world where straightforward set operations can be performed quickly. Unfortunately, that’s not the case. Short programs with no obvious infinite loops can still take a long time.

Let’s restrict ourselves to hereditarily definite sets. A definite set in SetBang is that is derived from other finite sets, and hereditarily definite means that it’s a definite set whose elements are all (hereditarily) definite sets.

There are many finite and probably-finite sets that aren’t definite. For example, if we admit this macro:

:macro magic #003<[003<[~#1?(_\_\_2>'2>,__2>_12>0)]_2>(4<~5<2>/4>'4>_,4<'4>_)]_;~22>?(__0,_2003<[003<[~#1?(_\_\_2>'2>,__2>_12>0)]_2>(4<~5<2>/4>'4>_,4<'4>_)]_;[~~03>[\3<~3<[\_2>{'"}2>]_4<[~3<~4<2>4<.3>&{'"}]_3>2>]__3<~4>~3<~4<2>4<=(;;,_2>~3<~4<2>4<-3>2>-(~{}3<~4>{~3>2>~4>?(_",__0)}\2>[\3<&2>]_;(~"4<2>-3>2>-1[~3<~4<2>4<.3>&{'"}]_[~3<~4<2>4<.3>&{'"}]_,___0),_)(_1))(_~3<~4<2>4<(03>~{}3<~{}3<-#'[\_#3<~3<~4>[\_2>{'"}2>]_~5<~6>~3<~4<2>4<=(;;,_2>~3<~4<2>4<-3>2>-(~{}3<~4>{~3>2>~4>?(_",__0)}\2>[\3<&2>]_;(~"4<2>-3>2>-1[~3<~4<2>4<.3>&{'"}]_[~3<~4<2>4<.3>&{'"}]_,___0),_)(_1))(_3<~4>6<2>/5>4<2>~3<~4<2>4<-3>2>-(~{}3<~4>{~3>2>~4>?(_",__0)}\2>[\3<&2>]_;(~"4<2>-3>2>-1[~3<~4<2>4<.3>&{'"}]_[~3<~4<2>4<.3>&{'"}]_,___0),_)3<,__3>)]_;,2>)(__1[~3<~4<2>4<.3>&{'"}]_,____00),___10)]_)

we can create these sets:

:macro mersenne ${^\_#~:magic:(_",__0)}

:macro fermat ${^^'~:magic:(_",__0)}

:macro maybebot :fermat:`5_`1; 

What does the magic macro do? Its behavior is:

... N -> ... (1 if N is prime, 0 if N is composite).

As a result, mersenne produces the set of Mersenne primes, which is possibly infinite although that remains unknown, and fermat produces that of Fermat primes, of which there are probably exactly 5 (although there may be infinitely many). The set maybebot, derived from fermat, is nonterminating empty (“bottom”) if there are only 5 Fermat primes, and a one-element set if there are 6 or more. The last of these is a set that is trivially, provably, finite; but its derivation (ultimately from $) makes it a risk of non-termination. It’s finite, but not definite.

If we remove $ from SetBang, we can make every set not only definite but hereditarily definite (meaning that the elements are definite sets as well). We still have the potential for non-terminating computations due to [] loops, but otherwise, even if we use {}, everything will terminate.

Is SetBang still “inefficient”? Yes. So let’s talk about what an efficient SetBang would look like, and assume that we’re only working hereditarily definite sets (i.e. there is no $). An efficient SetBang would:

  • canonicalize sets so that set equality checks (=) can be performed in bounded time as a function of the size of the set.
  • perform membership checks (?) in bounded time.
  • represent structured large sets (e.g. 8^#^, the set of all subsets of {0, …, 255}) in an efficient way that doesn’t require storing the entire set.
    • The example above has 2^256 elements. We can’t store it in its entirety but, because it’s a highly structured large set, we can easily check for membership in it.
  • perform primitive operations (namely, \and /) in constant or logarithmic time.
  • “execute” {} comprehensions in a way (strict or eager) that doesn’t take too much time or space upfront, but that allows the resulting sets to be used efficiently, assuming that the code inside the comprehensions runs efficiently.

Is it possible, on some physical device possibly unlike an existing computer, to make an efficient SetBang, even excluding non-definite sets by removing $? It’s extremely unlikely, as we’ll see below.

Before we do that, we note that ordinals are an extremely inefficient way to represent numbers, so let’s use binary encoding to represent them as sets of ordinals instead. (An implementation, of course, can handle small finite ordinals as efficiently as if they were constants, urelements, or near equivalently, machine integers.) So we represent the number 37 as {0, 2, 5} (where the numerals on the right-hand side are still ordinals) as opposed to {0, …, 36}.

We could go further than binary and use von Neumann indices (or “hereditary binary” representations) defined like so:

I({}) = 0

I({a, b, …}) = 2^a + 2^b + …

This creates a bijection between the natural numbers and the hereditarily finite sets. Letting J be I’s inverse, we’d represent 37 as:

J(37) = {J(0), J(2), J(5)} = {{}, {J(1)}, {J(0), J(2)}} = {{}, {{J(0)}}, {{}, {J(1)}}} = {{}, {{{}}}, {{},{{{}}}}}

Why don’t we do this? In practice, I don’t consider it worth it. The multiple recursion of the index encoding and decoding complicate code considerably. Besides, an implementation can hard-code the first 256 ordinal constants and get the same efficiency (for numbers below 2^256) that we would have as if we treated them as urelements. While we gain efficient storage of certain “sparse” numbers (e.g. 2^(2^(2^(2^7 + 1))) + 23) by using hereditary binary, we complicate our algorithms (by, among other things, replacing single recursion with multiple recursion, requiring unpredictable amounts of stack space) and we still can’t do a lot with them. It does not enable us to make tractable operations out of those that would otherwise be intractable (e.g. precise arithmetic on such massive numbers).

With the binary representation, we have a compact addition function:

S∈tBang> :macro addbinary [~3<~4<2>4<.3>&{'"}]_
Stack: {0, 1, 6, 3, 5} {7, 4, 5, 8}
S∈tBang> :addbinary:
Stack: {0, 1, 4, 3, 9}

We see it correctly adding 107 + 432 = 539. The way it works is by repeatedly executing the behavior:

... X Y -> ... (X xor Y) (X and Y) -> ... (X xor Y) {z + 1 : z ∈ X and Y}

until the latter set (TOS) is zero/empty. This propagates the carries in the addition function, and it terminates when there are no more carries. It may not be the most efficient job, but it does the job in polynomial time (of the size of the sets, or the bit-sizes of the numbers).

With this, we can now illustrate concretely that our dream of efficient (as defined above) SetBang is probably not possible. Consider the following code.


with sumbinary (using a loop, but guaranteed to terminate in polynomial time) defined like so:

:macro sumbinary 02>[\3<:addbinary:2>]_

and the former’s behavior is

S X -> 1 if some subset Y of X has Sum(Y) = S, 0 if else.

This is the subset-sum problem, known to be NP-complete. Ergo, if P is not NP, an efficient SetBang implementation is not possible. Even by eschewing $, and restricting ourselves to the most trivial measurement of a set (“Is it empty?”) it will always be easy to write terminating but intractable programs.

In short, one could use laziness to make the {} operator “efficient” and simply not do gnarly computations until needed, and there’s room for cleverness around equality and membership checks (which often don’t require computation of the entire set) but even an empty-check or a \ creates a risk, even in the definite world, of exponential-or-worse time cost.

These results shouldn’t be surprising. Power sets are structured and one can therefore check quickly that, for example, {{{{5}}}} ∈ P(P(P(P(N)))), but they’re still very big. It’s possible, in principle, that SetBang could be implemented in such a way that a large number of complex queries happen efficiently. It’s just not possible, even with $ and [] taken out, to make SetBang fast in all cases. The language is too powerful.

Although there is much FUD around programming languages and speed, SetBang is an example of a language that is literally “not fast”.

Fast(er) math

On Github, I’ve included a library of faster math functions in resources/fastmath.sbg. These operate on numbers that have been converted to binary, like so:

Stack: 37
S∈tBang> :tobinary:
Stack: {0, 2, 5}

If you actually want to convert such numbers back into the more familiar ordinals, you can use frombinary. We’ve seen addbinary, and there isn’t much more to the other arithmetic operators: they’re standard “school book” implementations of subtraction, multiplication, and division-with-remainder on binary numbers. They improve the performance of our arithmetic considerably.

Recall that we previously had an implementation that would take 21 million years to determine that 65,537 was prime. We can now do it in about 30 seconds.

S∈tBang> 00/9'''''''/
Stack: {0, 16}
S∈tBang> :primebinary:
Stack: 1

That’s still a few hundreds of thousands (if not millions) of times slower than, say, trial division in C, but it’s an improvement over what it was.

Here’s a program that (quasi-)efficiently puts the set of all primes on the stack:

S∈tBang> ${~:tobinary::primebinary:(_",__0)}
Stack: {2,3,5,7,11,13,17,19,23,29,31,37,41,43,47,53, ...}

In fact, full SetBang allows one to put nearly any describable mathematical object on the stack. You can’t always do much with it, of course. Depending on what it is, you might not be able to do anything with it, as it could be:

{n where n is a Gödel number for a proof of theorem T}

which might equal {} when T is provably false (assuming a much smarter implementation of SetBang) but is “bottom” when T is unprovable.