I want to write software that helps kill people.

softwarehelpskill.md

I want to write software that helps kill people.

Please, before you call the police and get my github account put on lockdown, allow me a moment to explain. What I really want to do is work on projects that advance the human condition and improve people's lives. I've been in a mad dash to learn how to program for the past four or five years exactly because I realized how much good I could do for the world with a computer.

The open source software movement has produced my tools, provided my teachers and mentors, and, recently, has become a major focus of my attention and time. I can whole heartedly say that, at the tender age of 22, open source has already directly had a massive and positive benefit on my life. I'm reasonably certain I would be orders of magnitude less well off and less happy if open source didn't exist as a concept.

Which is why a realization I had a few weeks ago has caused me so much personal strife lately: open source software is used to help kill people. Specifically, Palantir Technologies uses open source to identify who should be killed. In a world where the American military probably has enough power to kill everyone currently alive, the hard questions have shifted from how to effectively kill the most people possible to who should be killed. Palantir is in the business of providing the 33 bits of information needed to identify the people who pose enough threat to America's national security that they need to be eliminated.

Consider the following quote, taken from a profile of Palantir in Business Week:

In Afghanistan, U.S. Special Operations Forces use Palantir to plan assaults. They type a village’s name into the system and a map of the village appears, detailing the locations of all reported shooting skirmishes and IED, or improvised explosive device, incidents. Using the timeline function, the soldiers can see where the most recent attacks originated and plot their takeover of the village accordingly. The Marines have spent years gathering fingerprint and DNA evidence from IEDs and tried to match that against a database of similar information collected from villagers. By the time the analysis results came back, the bombers would be long gone. Now field operatives are uploading the samples from villagers into Palantir and turning up matches from past attacks on the spot, says Samuel Reading, a former Marine who works in Afghanistan for NEK Advanced Securities Group, a U.S. military contractor. “It’s the combination of every analytical tool you could ever dream of,” Reading says. “You will know every single bad guy in your area.”

The last sentence is what got me. Palantir uses open source software to tell soldiers when they should be pulling the trigger. The statement "the Python programming language murdered a 15 year terrorist in Iraq" is rickety logic at best, but it does contain a discomforting modicum of truth.

Here is a list of open source projects that Palantir probably uses or plans to use, as indicated by their jobs posting, blog posts, and various tech talks scattered across the web. While Palantir does much, much more than identify "bad guys", I'm lumping all of the listed projects, tools, and languages into the same conceptual bucket simply for lack of detailed information about what happens behind closed doors. I don't know that these projects are directly involved in informing soldiers, but there is a higher than normal chance that they are directly involved compared to a random project.

Languages:

• Bash
• Coffeescript
• C++
• Groovy
• Java
• Javascript
• Perl
• PHP
• Python
• Ruby
• Scala

Front end[0]:

• Android
• backbone.js
• Java Swing
• less
• rrd4j

Back end:

• Cassandra
• Chef
• Hadoop
• HyperSQL
• JAXB
• jMock
• JMX
• Linux/UNIX
• Log4J
• Lucene
• Nagios
• Postgres
• Puppet
• Rails
• Spring
• Zenoss

The vast majority of the above projects are large and successful. Tens of thousands of developers have been involved with helping build them. In Linux's case alone, there are about 10,000 people who have been directly involved over the past few decades. There's probably an order of magnitude more people who have submitted bug reports (think automated Ubuntu crash reports) and at least another two orders of magnitude who have helped harden Linux via sustained and heavy use.

The questions I've been struggling with is the relationship of the developer and how these libraries are used. The above libraries are general purposes tools that can be used for most anything. They do a tremendous amount of good for humanity as a whole and the world is better off for having them. The question I've been asking myself is "How does the use of a tool reflect back on those who developed the tool?" The purposes of these libraries are not to kill, but they have been used by Palantir to help kill someone[2].

To start with, I've publicly written before about how to use backbone.js. If the front end folks at Palantir saw that[1] and figured out a better way to display lists of "bad guy" names or something trivial, I wouldn't feel so bad about that. That information is out there regardless and the Palantir folks are smart enough that they would've figured it out within a few minutes anyway. Frequenters of stackoverflow needn't worry that their answers have made the difference between life and death for someone.

The same goes for people who have submitted even the most detailed bug reports. These people have pointed out how to harden the library and thus made it more reliable, but that is mostly incremental progress. Just because Postgres happens to not print out a comma when you use a certain SQL statement won't stop Palantir from identifying who should die. Even reports of dire, world ending bugs don't bother me much. I haven't written any of the code that is executed when Palantir executes some query looking for enemy combatants.

Now, let's take the obvious next step up. What if I submit a pull request to Cassandra that fixes a bug? Every time a soldier tells the Palantir suite to go talk to Cassandra, asking for updates on the current status of a battle and find out where the person trying to kill him is hiding, code that I wrote could be executed. In that manner, software that I've written would both hurt (and protect) someone. While I've yet to submit a patch to any of the projects above (at least as far as I can remember), it's not unreasonable to assume that it could happen in the near future.

And now we reach the divide between the trivial and the nontrivial pull requests. Trivial pull requests could be written by anyone. Adding a new command line option that was previously undocumented, fixing a small bug, anything a reasonably competent developer could do in half an afternoon, these tasks are what I call trivial. The trivial pull requests would happen regardless of whether or not I was involved. It would hurt me some to know that a library I had helped improve was used by Palantir in their defense work, but I could get over it in a weekend or so with a bottle of vodka and a good friend. The pull request would have happened anyway by some other developer.

But what if I wrote code that nobody else was likely to write? After a certain point in a project, the number of people who can submit a nontrivial pull request for a useful feature goes down pretty fast. To get a nontrivial pull request into a project like Rails, you have to be pretty damn good and work even harder. And even then, it's not guaranteed at all to go through. At the same moment, you can generalize this idea out to creating the initial seeds of what would become Rails. For the purpose of this post, I don't see much of a difference between making some feature of Postgres 10x faster and making Postgres itself. In a sense, a pull request that makes something 10x better is a big enough change that the request is creating a new project.

At this level, where I'm committing code that nobody else would likely commit, the game changes. I'm now enabling Palantir as they help kill people. And most likely, I would be doing it unintentionally. That's what shocked me. If I wrote a really good open source library that Palantir liked, they would be free to use it to design systems to help kill (or even just straight up kill) people and I would never know. If somebody at Palantir came back to me and said that they've had a great success using a library I had written though, I would probably curl up into a ball and cry for a month. Killing people is the opposite of why I got into software in the first place.

And If I vigorously tried to stop Palantir from using my open source library, I would probably be quickly arrested. Palantir is good at what they do and I think governments around the world will only come to rely on their software even more as time goes on. In some sense, I would be threatening national security by actively trying to prevent Palantir from using software that it needed to use to ensure the continued peace and prosperity of the United States. Although a lack of direct control is the case with open source software in general, in this case it could be illegal and seditious for me to even attempt to prevent Palantir from using my code. I'm not even sure how I could attempt to stop them short of gaining access to their servers and ritually deleting all the code I created (which would be hard, if not impossible, to do).

I've been raised as developer on open source. In my heart of hearts, I've matured as a programmer thinking that the best possible thing I could do to give back is create a powerful and useful open source library for other people to use how they see fit. I still have a decade of learning ahead of me before I think I would be good enough to do so, but I'd like to create a project that could be considered a peer of some of the projects listed above. But Palantir is, and will probably still be, using some of the best software ever written to help kill people. If I were successful and wrote great software, Palantir would probably want to use it. And so, I want to write software that helps kill people.

And that hurts.

---

Note that this was not written as a slam against Palantir in any way. I respect their work and recognize that what they are doing has to be done. They provide a much needed service and I'm happy that somebody else has to deal with deciding who should die. The core of it is that I'm distressed that work that I want to do to help improve people's lives could be used to end someone's life. I'm not comfortable yet with the idea that I could help end someone's life, even if they wanted to destroy mine. It's been a shock realizing that a fair portion of the top repositories on Github are potentially being used by Palantir to help kill people.

---

Note to all of those who comment: Thank you for reading this! I'm flattered that you've commented on this, no matter what your reaction was. It's a powerful feeling knowing that what I say affected people all over the globe. Please note though that I don't plan on making public comments on github, hacker news, or, god forbid, reddit. This is a touchy subject and one which requires a fair amount of thought to consider properly. As people have already noted here and in other forums, this strikes at a philosophical problem that depends very much on your system of values and what you think is inherently "good" or "evil". If you really want me to respond, write out your response, find my email, and wait a week or two before sending it. If you aren't trolling, I'll happily respond within a few weeks.

---

Footnotes:

[0] I'm going to take a wild swing and guess that the Palantir front end folks are currently evaluating or using d3.js on the front end right now. The library is just too damn good not to use for what they are doing. Nobody from the company has mentioned it online as far as I can tell, but I'd be shocked if they aren't using it or haven't built an equivalent library internally.

[1] That's assuming what I have written is worth reading, which is a big assumption, but bear with me here. I try to write things worth reading. I ended up taking down that article because I didn't want to maintain it any longer and I've since stopped using backbone.

[2] I've been unable to find any supporting evidence online that Palantir's tech stack has been used to provide the information needed to identify a specific individual to be killed, i.e. a report that a specific terrorist was identified and neutralized thanks to Palantir's tech. I believe that it is reasonable to assume that the above has happened though. I think that it is also reasonable that a public report detailing the above would be against Palantir's interests and they would work to prevent or suppress it.

Ernest Hemingway once wrote: "The world is a fine place and worth coding for." You can agree with the second part.

Kurt Vonnegut's Cat's Cradle deals with a similar subject, namely the invention of the atom bomb and a substance called ice-nine by an oblivious scientist. I suggest giving it a read.

I think it's important to keep in mind that tools like Cassandra are purpose-agnostic. They can be used for a variety of things. A tool like a machine gun, however, does have a specific purpose. They can only be used for a small number of things.

I want to make sure that folks reading this article that there are two different companies named "Palantir" who are active in the open source community:

The company that zmaril is writing about is Palantir Technologies, a Silicon Valley startup founded in 2004 that works with clients in the defense industry, among others.

I am the founder and CEO of Palantir.net, also known as "Palantir", a Web strategy, design, and development company founded in 1996. We are very active participants and contributors to several open source projects, most notably the Drupal project. We are unaffiliated with Palantir Technologies in any way, and we are not, and never have been involved with any of the kinds of activities described by zmaril in his post. We believe in using open source technologies for good and in making the world a better place for everyone.

We occasionally get confused for Palantir Technologies, and they occasionally get confused for us, but it's important for everyone to understand that while we have similar names, we are two very different companies. Any efforts that can be made to help others understand that difference as well would be greatly appreciated. Thanks!

Breaking radio silence to confirm that this post is in no way related to Palantir.net. I've updated the post to reflect that, via using the full name for Palantir Technologies the first time Palantir is mentioned. \cc @gdemet

This isn't restricted to software. You do as well 'aid' in killing if you develop (or sell them) better office furniture, better food, better anything they happen to use. For me, general purpose open source software is well this side of the line - flight controllers or image regocnition starts to get shady.

Palantir Technologies does not kill people. US Government agencies (and other governments/non-governments of the world) kill people.

Given your "tender age," and that you have only been professionally employed less than a year I am guessing you haven't paid a substantial amount of federal income taxes. As you progress in your career, you will discover that federal income taxes:

Note from Zack: the above is factually incorrect. Starting with my work at Wolfram, I've been employed professionally as a developer intern or consultant on and off for at least two years now. I've been paying taxes for longer because of various summer jobs and internships before I entered the software industry.

A. Substantially exceed local and state taxes (which might go to schools, roads and other things visible in your community)
B. Appropriate about $614B for FY2013 (20% of total) toward defense.

Let's consider what B is. This is money the government takes directly from you to spend on building jets and tanks and, yes, software. This is indeed, your direct involvement in contributing to building things that actually do kill people. And though there are "10,000" people historically contributing to Linux, there are millions of tax payers, contributing hundreds of billions of dollars to government defense contractors and our Nobel Peace Prize-winning president's drone program.

Until the American people elect representatives who uphold a non-interventionist foreign policy perspective, many more dollars (and lines of code) will be dedicated toward defense and warfare.

So, where to go from here? The power of open source, open data, and the internet allows for many possibilities, some of which we value, some of which we do not. Technology does too empower the individual, not just governments and corporations. Therefore, if I may offer some advice, I recommend you:

1. Vote.
2. Contribute your time/money/code toward projects and institutions that share your values.
3. Inspire others to do (1) and (2).

If you feel passionately about a cause or movement, then pursue it. This is what you can offer.

Best regards.

The important question, from an ethical standpoint as well as a legal standpoint in case law, is the intrinsic nature of the tool. People are stabbed to death with steak knives all the time, yet you can buy them at the local corner store and no one at a cutlery firm loses sleep over it. A hammer is a lethal weapon, but I can buy them by the dozen at Home Depot. They all have legitimate mundane uses.

Certain poisons, however, are heavily regulated because their only real use is to kill. Or, to touch the 3rd rail, guns exist only to kill or threaten to kill. (Self-defense being still killing or threatening to kill.) Certain locksmithing tools exist solely to break into locks, and therefore require a license to own (because the use itself is always breaking-and-entering, but sometimes that is legitimate for an owner to hire someone to do for his own house).

With the Cassandra database, its raison d'etre is to store and retrieve data. The data is abstract; Storing data is not an intrinsically morally questionable act the way that violence is (even if sometimes justified) or breaking and entering is (even if it's sometimes your own house). So no, I wouldn't feel guilty about working on the Cassandra database.

Working on a guidance system for a rocket? It may use Cassandra internally, but then you're working on the rocket. Rockets kill people. Then you have to struggle with whether improving its targeting is bad (easier to kill people) or good (better accuracy means fewer collateral deaths by missing, you hope). That's... a tricky problem.

Similarly, storing data is fine. Storing stock data for doing micro-trading of the sort that is rotting our economy? Now you're at the level where you can and should be questioning your actions. (Maybe deciding one way, maybe another, but it's a legit question to ask.)

There's still plenty of wiggle room in that approach, of course. How close do you need to get to the "evil use" (for some definition of evil) before you're culpable. It's not always an easy question, but it is the question to be asking. Building hammers and steak knives shouldn't keep you up at night.

I work with @gdemet at Palantir.net (the one that is NOT a defense contractor), and we have a policy that we only take clients that we feel good about; that is, if they're successful in what they want to do with the OSS software we build for them, can we sleep at night? If the answer is no, we don't take the project. That means we work with lots of institutional non-profits, but do a fair bit of corporate work as well. But we do make sure it's not a morally objectionable (by our standards) task before we engage a client. At the same time, might someone do something objectionable with the Drupal modules we release on Drupal.org? Maybe. But if we refrained from making steak knives because someone might stab someone with them, then I'd never be able to have a proper steak dinner. (And I happen to like a good steak dinner.)

See also: http://willy.boerland.com/myblog/the_gpl_is_the_gpl_for_good_and_bad_and_even_ugly