May 2003(This essay is derived from a guest lecture at Harvard, which incorporatedan earlier talk at Northeastern.)When I finished grad school in computer science I wentto art school to study painting. A lot of people seemed surprisedthat someone interested in computers would also be interested in painting.They seemed to think thathacking and painting were very different kinds of work-- thathacking was cold, precise, and methodical, and thatpainting was the frenzied expression of some primal urge.Both of these images are wrong. Hacking and painting have alot in common. In fact, of all the different types of people I'veknown, hackers and painters are among the most alike.What hackers and painters have in common is that they'reboth makers. Along with composers, architects, and writers,what hackers and painters are trying to do is make good things.They're not doing research per se, though if in the course oftrying to make good things they discover some new technique,so much the better.I've never liked the term "computer science." The mainreason I don't like it is that there's no such thing.Computer science is agrab bag of tenuously related areas thrown togetherby an accident of history, like Yugoslavia.At one end you have people who are really mathematicians,but call what they're doing computer science so they can get DARPA grants.In the middle you have people working onsomething like the natural history of computers-- studying thebehavior of algorithms for routing data throughnetworks, for example. And then at the other extreme youhave the hackers, who are trying towrite interesting software, and for whom computers are just amedium of expression, as concrete is for architects orpaint for painters. It's as ifmathematicians, physicists, and architects all had to be inthe same department.Sometimes what the hackers do is called "software engineering,"but this term is just as misleading.Good software designers are no more engineers than architects are.The border between architecture and engineering is not sharplydefined, but it's there.It falls between what and how: architects decide what to do,and engineers figure out how to do it.What and how should not be kept too separate. You'reasking for trouble if you try to decide what to do withoutunderstanding how to do it.But hacking can certainly be more than just deciding how toimplement some spec. At its best, it's creating the spec-- thoughit turns out the best way to do that is to implement it.Perhaps one day"computer science" will, like Yugoslavia, get broken up into itscomponent parts. That might be a good thing. Especially if itmeant independence for my native land, hacking.Bundling all these different types of work together in onedepartment may be convenient administratively, but it's confusingintellectually. That's the other reason I don't like the name"computer science." Arguably the people in the middle are doingsomething like an experimental science. But the people at eitherend, the hackers and the mathematicians, are not actually doing science.The mathematicians don't seem bothered by this. They happilyset to work proving theorems like the other mathematiciansover in the math department, and probably soon stop noticingthat the building they work in says computer science'' on theoutside. But for the hackers this label is a problem.If what they're doing is called science, it makes them feel theyought to be acting scientific.So instead of doing what they really want to do, which is to design beautiful software, hackers in universities andresearch labs feel they ought to be writing research papers.In the best case, the papers are just a formality. Hackers writecool software, and then write a paper about it, and the paperbecomes a proxy for the achievement represented by the software.But often this mismatch causes problems. It's easy todrift away from building beautiful things toward building uglythings that make more suitable subjects for research papers.Unfortunately, beautiful things don't always make thebest subjects for papers.Number one, research must be original-- andas anyone who has written a PhD dissertation knows, the way tobe sure that you're exploring virgin territory is to stakeout a piece of ground that no one wants. Number two, research must besubstantial-- and awkward systems yield meatier papers,because you can write about the obstacles you have to overcomein order to get things done. Nothing yields meaty problems likestarting with the wrong assumptions. Most of AI is an exampleof this rule; if you assume that knowledge can be representedas a list of predicate logic expressions whose arguments representabstract concepts, you'll have a lot ofpapers to write about how to make this work. As Ricky Ricardoused to say, "Lucy, you got a lot of explaining to do."The way to create something beautiful is often to make subtletweaks to something that already exists, or to combine existingideas in a slightly new way. This kind of work is hard toconvey in a research paper.So why do universities and research labs continue to judgehackers by publications?For the same reason that "scholastic aptitude"gets measured by simple-minded standardized tests, orthe productivity of programmers gets measured in lines of code.These testsare easy to apply, and there is nothing so tempting as an easy testthat kind of works.Measuring what hackers are actually trying to do, designingbeautiful software, would be much more difficult. You needa good sense of design to judge good design. Andthere is no correlation, except possiblya negative one, between people's ability to recognize gooddesign and their confidence that they can.The only external test is time. Over time, beautifulthings tend to thrive, and uglythings tend to get discarded. Unfortunately, the amounts of timeinvolved can be longer than human lifetimes. Samuel Johnsonsaid it took a hundred years for a writer's reputation toconverge. You have to wait for the writer'sinfluential friends to die, and then for all their followersto die.I think hackers just have to resign themselves to having a large randomcomponent in their reputations. In this they are no differentfrom other makers. In fact, they're lucky by comparison. The influence of fashion is not nearly so great in hacking as itis in painting.There are worse things than having people misunderstand yourwork. A worse danger is that youwill yourself misunderstand your work. Related fields arewhere you go looking for ideas. If you find yourself in the computer sciencedepartment, there is a natural temptation to believe, for example,that hacking is the applied version of what theoretical computerscience is the theory of. Allthe time I was in graduate school I had an uncomfortable feelingin the back of my mind that I ought to know more theory,and that it was very remiss of me to have forgotten all thatstuff within three weeks of the final exam.Now I realize I wasmistaken. Hackers need to understand the theory of computationabout as much as painters need to understand paint chemistry.You need to know how to calculate time andspace complexity and aboutTuring completeness. You might also want to remember atleast the concept of a state machine, in case you have to writea parser or a regular expression library. Painters in fact have to remember a good deal more about paint chemistry than that.I've found that the best sources of ideasare not the other fields that have the word "computer" intheir names, but the other fields inhabited by makers.Painting has been a much richer source of ideas than thetheory of computation.For example, I was taught in collegethat one ought to figure out a programcompletely on paperbefore even going near a computer. I found that I did notprogram this way. I found that I liked to programsitting in front of a computer, not a piece of paper. Worsestill, instead of patiently writing out a complete programand assuring myself it was correct, I tended to just spewout code that was hopelessly broken, and gradually beat it intoshape. Debugging, I was taught, was a kind of final pass whereyou caught typos and oversights. The way I worked, itseemed like programming consisted of debugging.For a long time I felt bad about this, just as I oncefelt bad that I didn't hold my pencil the way they taught meto in elementary school.If I had only looked over atthe other makers, the painters or the architects, I wouldhave realized that there was a name for what I was doing:sketching. As far as I can tell, theway they taught me to program in college was all wrong.You should figure out programs as you're writing them,just as writers and painters and architects do.Realizing this has real implications for software design.It means that a programming language should, above all, bemalleable. A programming language is for thinking ofprograms, not for expressing programs you've already thoughtof. It should be a pencil, not a pen. Static typing wouldbe a fine idea if people actually did write programs the waythey taught me to in college. But that's not how any of the hackers I know write programs. We need a language that lets usscribble and smudge and smear, not a language where you haveto sit with a teacup of types balanced on your knee and makepolite conversation with a strict old aunt of a compiler.While we're on the subject of static typing, identifying withthe makers will save us from another problem that afflictsthe sciences: math envy. Everyone in the sciencessecretly believes that mathematicians are smarter than they are. I think mathematicians also believe this. At any rate,the result is that scientists tend to make theirwork look as mathematical as possible. In a field likephysics this probably doesn't do much harm, but the further youget from the natural sciences, the more of a problem itbecomes.A page of formulas just looks so impressive.(Tip: for extra impressiveness, use Greek variables.) Andso there is a great temptation to work on problems youcan treat formally, rather than problems that are, say,important.If hackers identified with other makers, like writers andpainters, they wouldn't feel tempted to do this. Writers and painters don't suffer from math envy.They feel as if they're doing something completely unrelated.So are hackers, I think.If universities and research labs keep hackers from doingthe kind of work they want to do,perhaps the place for them is in companies.Unfortunately, most companies won't let hackers do what theywant either. Universities and research labs force hackersto be scientists, and companies force them to be engineers.I only discovered this myself quite recently. When Yahoo boughtViaweb, they asked me what I wanted to do. I had neverliked the business side very much, and said that I just wanted tohack. When I got to Yahoo, I found that what hacking meantto them was implementing software, not designing it. Programmerswere seen as technicians who translated the visions (ifthat is the word) of product managers into code.This seems to be thedefault plan in big companies. They do it becauseit decreases the standard deviation of the outcome.Only a small percentage of hackers can actually design software,and it's hard for thepeople running a company to pick these out. So instead ofentrusting the future of the software toone brilliant hacker, most companies set things up so that it isdesigned by committee, and the hackers merelyimplement the design.If you want to make money at some point, remember this,because this is one of the reasons startups win. Big companies wantto decrease the standard deviation of design outcomes because theywant to avoid disasters. But when you damp oscillations, youlose the high points as well as the low. This is not a problem forbig companies, because they don't win by making greatproducts. Big companies win by sucking less than other big companies.So if you can figure out a way to get in adesign war with a company big enough that its software is designed by product managers, they'll never be able to keep upwith you. These opportunities are not easy to find, though.It's hard to engage a big company in a design war,just as it's hard to engage an opponent inside a castle in handto hand combat. It would be pretty easy to write a betterword processor than Microsoft Word, for example, but Microsoft,within the castle of their operating system monopoly,probably wouldn't even notice if you did.The place to fight design wars is in new markets, where no onehas yet managed to establish any fortifications. That's whereyou can win big by taking the bold approach to design, andhaving the same people both design and implement the product. Microsoft themselves did this at the start. So did Apple.And Hewlett-Packard. I suspect almost every successful startuphas.So one way to build great software is to start your ownstartup. There are two problems with this, though. One isthat in a startup you have to do so much besides write software.At Viaweb I considered myself lucky if Igot to hack a quarter of the time. And the things I had to do the other three quarters of the time ranged from tediousto terrifying. I have a benchmark for this, because Ionce had to leave a board meeting to havesome cavities filled. I remember sitting back in thedentist's chair, waiting for the drill, and feeling likeI was on vacation.The other problem with startups is that there is not muchoverlap between the kind of software that makes money and thekind that's interesting to write. Programming languagesare interesting to write, and Microsoft's first product wasone, in fact, but no one will pay for programming languagesnow. If you want to make money, you tend to be forced to workon problems that are too nasty for anyone to solve for free.All makers face this problem. Prices aredetermined by supply and demand, and there is just not as muchdemand for things that are fun to work on as there is forthings that solve the mundane problems of individual customers.Acting in off-Broadway plays just doesn't pay as well aswearing a gorilla suit in someone's booth at atrade show. Writing novels doesn't pay as well as writingad copy for garbage disposals.And hacking programming languages doesn't pay as wellas figuring out how to connect some company'slegacy database to their Web server.I think the answer to this problem, in the case of software,is a concept known to nearly all makers: the day job.This phrase began with musicians, whoperform at night. More generally, it means that you have onekind of work you do for money, and another for love.Nearly all makers have day jobs early in their careers.Painters and writers notoriously do. If you're luckyyou can get a day job that's closelyrelated to your real work. Musicians oftenseem to work in record stores. A hacker working on someprogramming language or operating system might likewise be able toget a day job using it. [1]When I say that the answer is for hackers to have day jobs, and work on beautiful software on the side, I'm not proposingthis as a new idea. This is what open-source hacking is all about. What I'm saying is that open-source is probably the rightmodel, because it has been independently confirmed by all the other makers.It seems surprising to me that any employer would be reluctantto let hackers work on open-source projects.At Viaweb, we would have been reluctant to hire anyonewho didn't. When we interviewedprogrammers, the mainthing we cared about was what kind of software theywrote in their spare time.You can't do anything really well unlessyou love it, and if you love to hack you'll inevitablybe working on projects of your own. [2]Because hackers are makers rather than scientists,the right place to look for metaphors is not in thesciences, but among other kinds of makers. What else can paintingteach us about hacking?One thing we can learn, or at least confirm, from theexample of painting is how to learn to hack. You learn topaint mostly by doing it.Ditto for hacking. Most hackers don't learn to hack bytaking college courses in programming. They learn to hackby writing programs of their own at age thirteen. Even in college classes, you learn to hack mostly by hacking. [3]Because painters leave a trail of work behind them, youcan watch them learn by doing. If you look at the workof a painter in chronological order, you'll find that each painting builds on things that have been learned in previousones. When there's something ina painting that works very well, you can usually find version 1 of it in a smaller form in some earlier painting.I think most makers work this way. Writers and architects seemto as well. Maybe it would be good for hackersto act more like painters, and regularly start over from scratch,instead of continuing to work for years on one project, andtrying to incorporate all their later ideas as revisions.The fact that hackers learn to hack by doing it is anothersign of how different hacking is from the sciences. Scientistsdon't learn science by doing it, but by doing labs and problem sets.Scientists start out doing work that's perfect, in the sensethat they're just trying to reproduce work someone else has already done for them.Eventually, they getto the point where they can do original work.Whereas hackers, from the start, are doing original work; it'sjust very bad. So hackers start original, and get good, andscientists start good, and get original.The other way makers learn is from examples.For a painter, a museum is a reference library of techniques.For hundreds of years it has been part of the traditionaleducation of painters to copy the works of the great masters,because copying forces you to look closelyat the way a painting is made.Writers do this too.Benjamin Franklin learned to write by summarizing the points in the essays of Addison and Steele and then trying toreproduce them. Raymond Chandler did the same thingwith detective stories.Hackers, likewise, can learn to program by looking at good programs-- not just at what they do, but the sourcecode too. One of the less publicized benefitsof the open-source movement is that it has made it easierto learn to program. When I learned to program, we had to relymostly on examples in books. The one big chunk ofcode available then was Unix, but even this was not open source. Most of the people who read the sourceread it in illicit photocopies of John Lions' book, whichthough written in 1977 was not allowed to be publisheduntil 1996.Another example we can take from painting is the way thatpaintings are created by gradual refinement. Paintings usuallybegin with a sketch.Gradually the details get filled in.But it is not merely a process of filling in. Sometimes the original plans turn out to be mistaken.Countless paintings,when you look at them in xrays, turn out to have limbs thathave been moved or facial features that have been readjusted.Here's a case where we can learn from painting. I think hackingshould work this way too. It's unrealisticto expect that the specifications for a program will beperfect. You'rebetter off if you admit this up front, and write programs ina way that allows specifications to change on the fly.(The structure of large companies makes this hard for themto do, so here is another place where startups have an advantage.)Everyone by now presumably knows about the danger of prematureoptimization. I think we should be just as worried aboutpremature design-- deciding too early whata program should do.The right tools can help us avoidthis danger.A good programming language should, like oil paint, make iteasy to change your mind. Dynamic typing is a win here becauseyou don't have tocommit to specific data representations up front.But the key to flexibility, I think, is to make the languagevery abstract.The easiest program to change is one that's very short.This sounds like a paradox, but a great paintinghas to be better than it has to be.For example, when Leonardopainted the portrait of Ginevra de Benciin the National Gallery, he put a juniper bush behind her head.In it he carefullypainted each individual leaf. Many painters might have thought,this is just something to put in the background to frameher head. No one will look that closely at it.Not Leonardo. How hard he worked on part of a painting didn'tdepend at all on how closely he expected anyone to look at it.He was like Michael Jordan. Relentless.Relentlessness wins because, in the aggregate, unseen detailsbecome visible.When people walk by the portrait of Ginevra de Benci,their attention is often immediately arrested by it,even before they look at the label and notice that it saysLeonardo da Vinci. All those unseen details combine to producesomething that's just stunning, like a thousand barely audiblevoices all singing in tune.Great software, likewise, requires a fanatical devotion tobeauty. If you look inside good software, you find thatparts no one is ever supposed to see are beautiful too.I'm not claiming I write great software, but Iknow that when it comes to code I behave in a way that wouldmake me eligible for prescription drugs if I approached everydaylife the same way.It drives me crazy to see code that's badly indented,or that uses ugly variable names.If a hacker were a mere implementor, turning a spec into code, thenhe could just work his way through it from one end to the other likesomeone digging a ditch. But if the hacker is a creator, we haveto take inspiration into account.In hacking, like painting,work comes in cycles. Sometimes you get excited about somenew project and you want to work sixteen hours a day on it. Other times nothing seems interesting.To do good work you have to take these cycles intoaccount, because they're affected by how you react to them.When you're driving acar with a manual transmission on a hill, you have to back offthe clutch sometimes to avoid stalling. Backingoff can likewise prevent ambition from stalling.In both painting and hacking there are sometasks that are terrifyingly ambitious, and others that arecomfortingly routine. It's a good idea to save some easytasks for moments when you would otherwise stall.In hacking, this can literally mean saving up bugs.I like debugging: it's theone time that hacking is as straightforward as people think it is. You have atotally constrained problem, and all you have to do is solveit. Your program is supposed to do x. Instead it does y.Where does it go wrong? You know you're going to winin the end. It's as relaxing as painting a wall.The example of painting can teach us not only how to manage ourown work, but how to work together. A lot of thegreat art of the past is the work of multiple hands, thoughthere may only be one name on the wall next to it in themuseum. Leonardo was an apprentice in the workshop ofVerrocchio and painted one of the angels in his Baptism ofChrist. This sort of thing was the rule, not the exception.Michelangelo was considered especially dedicated for insistingon painting all the figures on the ceiling of the SistineChapel himself.As far as I know, when painters worked together on a painting,they never worked on the same parts. It was commonfor the master to paint the principal figures and for assistantsto paint the others and the background. But you never hadone guy painting over the work of another.I think this is the right model for collaboration in softwaretoo. Don't push it too far. When a piece of code isbeing hacked by three or four different people, no one of whomreally owns it, it will end up being like a common-room. It willtend to feel bleak and abandoned, and accumulate cruft.The rightway to collaborate, I think, is to divide projects into sharplydefined modules, each with a definite owner, and with interfacesbetween them that are as carefully designed and, if possible,as articulated as programming languages.Like painting, most software is intended fora human audience. And so hackers, like painters, must haveempathy to do really great work. You have to be able to seethings from the user's point of view.When I was a kid I was always being told to look at things fromsomeone else's point of view. What this always meant inpractice was to do what someone else wanted, instead of whatI wanted. This of course gave empathy a bad name, and I made apoint of not cultivating it.Boy, was I wrong. It turns out that looking at things from other people's point of view is practically the secret ofsuccess. It doesn't necessarily mean being self-sacrificing.Far from it. Understanding how someone else sees thingsdoesn't imply that you'll act in his interest; in somesituations-- in war, for example-- you want to do exactlythe opposite. [4]Most makers make things for a human audience.And to engage an audience you have to understand what they need.Nearly all the greatest paintings are paintings of people,for example, because people are what people are interested in.Empathy is probably the single most important differencebetween a good hacker and a great one. Some hackersare quite smart, but when it comes to empathy arepractically solipsists. It's hard for such people to design great software [5], because they can'tsee things from the user's point of view.One way to tell how good people are at empathy is to watchthem explain a technical question to someone without a technicalbackground. We probably all know people who, though otherwise smart,are just comically bad at this. If someone asks them ata dinner party what a programming language is, they'llsay something likeOh, a high-level language is whatthe compiler uses as input to generate object code.''High-level language? Compiler? Object code? Someone who doesn't know what a programming language is obviously doesn'tknow what these things are, either.Part of what software has to do is explain itself. So to write good software you have to understand how little users understand.They're going to walk up to the software with no preparation, andit had better do what they guess it will, because they'renot going to read the manual. The best system I've ever seen in this respect was the original Macintosh, in 1985.It did what software almost never does: it just worked. [6]Source code, too, should explain itself. If I could get people toremember just one quote about programming, it would be theone at the beginning of Structure and Interpretation of ComputerPrograms. Programs should be written for people to read, andonly incidentally for machines to execute.You need to have empathy not just for your users, but for your readers. It's in your interest, because you'll be one of them.Many a hacker has written a program only tofind on returning to it six months later that he has no idea how it works. I know several people who've sworn off Perl aftersuch experiences. [7]Lack of empathy is associated with intelligence, to the pointthat there is even something of a fashion for it in some places.But I don't think there's any correlation.You can do well in math andthe natural sciences without having to learn empathy, and people in thesefields tend to be smart, so the two qualities have come to beassociated. But there are plenty of dumb people who are bad atempathy too. Just listen to the people who call in with questions ontalk shows. They ask whatever it is they're asking insuch a roundabout waythat the hosts often have to rephrase the question for them.So, if hacking works like painting and writing, is it as cool?After all, you only get one life.You might as well spend it working on something great.Unfortunately, the question is hard to answer. There is alwaysa big time lag in prestige. It's like light from a distant star.Painting has prestige now because of great work people did five hundredyears ago. At the time, no one thoughtthese paintings were as important as we do today. It would haveseemed very odd to people at the time that Federico da Montefeltro,the Duke of Urbino, would one day be known mostly as the guywith the strange nose in a painting by Piero della Francesca.So while I admit that hacking doesn't seem as cool as painting now,we should remember that painting itself didn't seem as cool inits glory days as it does now.What we can say with some confidence is that these are the glorydays of hacking. In most fields the great work is done early on.The paintings made between 1430 and 1500 are still unsurpassed.Shakespeare appeared just as professional theater was being born,and pushed the mediumso far that every playwright since has had to live in his shadow.Albrecht Durer did the same thing with engraving, and Jane Austenwith the novel.Over and over we see the same pattern. A new medium appears, andpeople are so excited about it that they explore most of itspossibilities in the first couple generations. Hacking seemsto be in this phase now.Painting was not, in Leonardo's time, as cool as his workhelped make it.How cool hacking turns out to be will depend on what we cando with this new medium. Notes[1] The greatest damage that photography has doneto painting may be the fact that it killed the best day job.Most of the great painters in history supportedthemselves by painting portraits. [2] I've been told that Microsoft discouragesemployees from contributing to open-source projects, even intheir spare time.But so many of the best hackers work on open-sourceprojects now that the main effect of this policy may beto ensure that they won't be able to hire any first-rateprogrammers.[3] What you learn about programming in college is much likewhat you learn about books or clothes or dating: what bad taste youhad in high school.[4] Here's an example of applied empathy.At Viaweb, if we couldn't decide between two alternatives, we'dask, what would our competitors hate most? At one point acompetitor added a feature to their software that was basicallyuseless, but since it was one of few they had that we didn't, theymade much of it in the trade press.We could have tried to explain that the feature was useless,but we decided it would annoy our competitor more if wejust implemented it ourselves, so we hacked together our ownversion that afternoon.[5] Except text editors and compilers. Hackers don't need empathy todesign these, because they are themselves typical users.[6] Well, almost. They overshot the available RAM somewhat,causing much inconvenient disk swapping, but this could be fixedwithin a few months by buying an additional disk drive.[7] The way to make programs easy to read is not tostuff them with comments. I would take Abelson and Sussman'squote a step further. Programming languages should be designedto express algorithms, and only incidentally to tell computershow to execute them. A good programming languageought to be better for explaining software than English.You should onlyneed comments when there is some kind of kludge you need to warnreaders about, just as on a road there are onlyarrows on parts with unexpectedly sharp curves.Thanks to Trevor Blackwell, Robert Morris, Dan Giffin, and LisaRandall for reading drafts of this, and to Henry Leitnerand Larry Finkelstein for inviting me to speak.
