May 2001(This article was written as a kind of business plan for anew language.So it is missing (because it takes for granted) the most importantfeature of a good programming language: very powerful abstractions.)A friend of mine once told an eminent operating systemsexpert that he wanted to design a really goodprogramming language. The expert told him that it would be awaste of time, that programming languages don't become popularor unpopular based on their merits, and so no matter howgood his language was, no one would use it. At least, thatwas what had happened to the language he had designed.What does make a language popular? Do popularlanguages deserve their popularity? Is it worth trying todefine a good programming language? How would you do it?I think the answers to these questions can be found by looking at hackers, and learning what they want. Programminglanguages are for hackers, and a programming languageis good as a programming language (rather than, say, anexercise in denotational semantics or compiler design)if and only if hackers like it.1 The Mechanics of PopularityIt's true, certainly, that most people don't choose programminglanguages simply based on their merits. Most programmers are toldwhat language to use by someone else. And yet I think the effectof such external factors on the popularity of programming languagesis not as great as it's sometimes thought to be. I think a biggerproblem is that a hacker's idea of a good programming language isnot the same as most language designers'.Between the two, the hacker's opinion is the one that matters.Programming languages are not theorems. They're tools, designedfor people, and they have to be designed to suit human strengthsand weaknesses as much as shoes have to be designed for human feet.If a shoe pinches when you put it on, it's a bad shoe, howeverelegant it may be as a piece of sculpture.It may be that the majority of programmers can't tell a good languagefrom a bad one. But that's no different with any other tool. Itdoesn't mean that it's a waste of time to try designing a goodlanguage. Expert hackers can tell a good language when they seeone, and they'll use it. Expert hackers are a tiny minority,admittedly, but that tiny minority write all the good software,and their influence is such that the rest of the programmers willtend to use whatever language they use. Often, indeed, it is notmerely influence but command: often the expert hackers are the verypeople who, as their bosses or faculty advisors, tell the otherprogrammers what language to use.The opinion of expert hackers is not the only force that determinesthe relative popularity of programming languages — legacy software(Cobol) and hype (Ada, Java) also play a role — but I think it isthe most powerful force over the long term. Given an initial criticalmass and enough time, a programming language probably becomes aboutas popular as it deserves to be. And popularity further separatesgood languages from bad ones, because feedback from real live usersalways leads to improvements. Look at how much any popular languagehas changed during its life. Perl and Fortran are extreme cases,but even Lisp has changed a lot. Lisp 1.5 didn't have macros, forexample; these evolved later, after hackers at MIT had spent acouple years using Lisp to write real programs. [1]So whether or not a language has to be good to be popular, I thinka language has to be popular to be good. And it has to stay popularto stay good. The state of the art in programming languages doesn'tstand still. And yet the Lisps we have today are still pretty muchwhat they had at MIT in the mid-1980s, because that's the last timeLisp had a sufficiently large and demanding user base.Of course, hackers have to know about a language before they canuse it. How are they to hear? From other hackers. But there has tobe some initial group of hackers using the language for others evento hear about it. I wonder how large this group has to be; how manyusers make a critical mass? Off the top of my head, I'd say twenty.If a language had twenty separate users, meaning twenty users whodecided on their own to use it, I'd consider it to be real.Getting there can't be easy. I would not be surprised if it isharder to get from zero to twenty than from twenty to a thousand.The best way to get those initial twenty users is probably to usea trojan horse: to give people an application they want, whichhappens to be written in the new language.2 External FactorsLet's start by acknowledging one external factor that does affectthe popularity of a programming language. To become popular, aprogramming language has to be the scripting language of a popularsystem. Fortran and Cobol were the scripting languages of earlyIBM mainframes. C was the scripting language of Unix, and so, later,was Perl. Tcl is the scripting language of Tk. Java and Javascriptare intended to be the scripting languages of web browsers.Lisp is not a massively popular language because it is not thescripting language of a massively popular system. What popularityit retains dates back to the 1960s and 1970s, when it was thescripting language of MIT. A lot of the great programmers of theday were associated with MIT at some point. And in the early 1970s,before C, MIT's dialect of Lisp, called MacLisp, was one of theonly programming languages a serious hacker would want to use.Today Lisp is the scripting language of two moderately popularsystems, Emacs and Autocad, and for that reason I suspect that mostof the Lisp programming done today is done in Emacs Lisp or AutoLisp.Programming languages don't exist in isolation. To hack is atransitive verb — hackers are usually hacking something — and inpractice languages are judged relative to whatever they're used tohack. So if you want to design a popular language, you either haveto supply more than a language, or you have to design your languageto replace the scripting language of some existing system.Common Lisp is unpopular partly because it's an orphan. It didoriginally come with a system to hack: the Lisp Machine. But LispMachines (along with parallel computers) were steamrollered by theincreasing power of general purpose processors in the 1980s. CommonLisp might have remained popular if it had been a good scriptinglanguage for Unix. It is, alas, an atrociously bad one.One way to describe this situation is to say that a language isn'tjudged on its own merits. Another view is that a programming languagereally isn't a programming language unless it's also the scriptinglanguage of something. This only seems unfair if it comes as asurprise. I think it's no more unfair than expecting a programminglanguage to have, say, an implementation. It's just part of whata programming language is.A programming language does need a good implementation, of course,and this must be free. Companies will pay for software, but individualhackers won't, and it's the hackers you need to attract.A language also needs to have a book about it. The book should bethin, well-written, and full of good examples. K&R is the idealhere. At the moment I'd almost say that a language has to have abook published by O'Reilly. That's becoming the test of matteringto hackers.There should be online documentation as well. In fact, the bookcan start as online documentation. But I don't think that physicalbooks are outmoded yet. Their format is convenient, and the defacto censorship imposed by publishers is a useful if imperfectfilter. Bookstores are one of the most important places for learningabout new languages.3 BrevityGiven that you can supply the three things any language needs — afree implementation, a book, and something to hack — how do youmake a language that hackers will like?One thing hackers like is brevity. Hackers are lazy, in the sameway that mathematicians and modernist architects are lazy: theyhate anything extraneous. It would not be far from the truth tosay that a hacker about to write a program decides what languageto use, at least subconsciously, based on the total number ofcharacters he'll have to type. If this isn't precisely how hackersthink, a language designer would do well to act as if it were.It is a mistake to try to baby the user with long-winded expressionsthat are meant to resemble English. Cobol is notorious for thisflaw. A hacker would consider being asked to writeadd x to y giving zinstead ofz = x+yas something between an insult to his intelligence and a sin againstGod.It has sometimes been said that Lisp should use first and restinstead of car and cdr, because it would make programs easier toread. Maybe for the first couple hours. But a hacker can learnquickly enough that car means the first element of a list and cdrmeans the rest. Using first and rest means 50% more typing. Andthey are also different lengths, meaning that the arguments won'tline up when they're called, as car and cdr often are, in successivelines. I've found that it matters a lot how code lines up on thepage. I can barely read Lisp code when it is set in a variable-widthfont, and friends say this is true for other languages too.Brevity is one place where strongly typed languages lose. All otherthings being equal, no one wants to begin a program with a bunchof declarations. Anything that can be implicit, should be.The individual tokens should be short as well. Perl and Common Lispoccupy opposite poles on this question. Perl programs can be almostcryptically dense, while the names of built-in Common Lisp operatorsare comically long. The designers of Common Lisp probably expectedusers to have text editors that would type these long names forthem. But the cost of a long name is not just the cost of typingit. There is also the cost of reading it, and the cost of the spaceit takes up on your screen.4 HackabilityThere is one thing more important than brevity to a hacker: beingable to do what you want. In the history of programming languagesa surprising amount of effort has gone into preventing programmersfrom doing things considered to be improper. This is a dangerouslypresumptuous plan. How can the language designer know what theprogrammer is going to need to do? I think language designers woulddo better to consider their target user to be a genius who willneed to do things they never anticipated, rather than a bumblerwho needs to be protected from himself. The bumbler will shoothimself in the foot anyway. You may save him from referring tovariables in another package, but you can't save him from writinga badly designed program to solve the wrong problem, and takingforever to do it.Good programmers often want to do dangerous and unsavory things.By unsavory I mean things that go behind whatever semantic facadethe language is trying to present: getting hold of the internalrepresentation of some high-level abstraction, for example. Hackerslike to hack, and hacking means getting inside things and secondguessing the original designer.Let yourself be second guessed. When you make any tool, people useit in ways you didn't intend, and this is especially true of ahighly articulated tool like a programming language. Many a hackerwill want to tweak your semantic model in a way that you neverimagined. I say, let them; give the programmer access to as muchinternal stuff as you can without endangering runtime systems likethe garbage collector.In Common Lisp I have often wanted to iterate through the fieldsof a struct — to comb out references to a deleted object, for example,or find fields that are uninitialized. I know the structs are justvectors underneath. And yet I can't write a general purpose functionthat I can call on any struct. I can only access the fields byname, because that's what a struct is supposed to mean.A hacker may only want to subvert the intended model of things onceor twice in a big program. But what a difference it makes to beable to. And it may be more than a question of just solving aproblem. There is a kind of pleasure here too. Hackers share thesurgeon's secret pleasure in poking about in gross innards, theteenager's secret pleasure in popping zits. [2] For boys, at least,certain kinds of horrors are fascinating. Maxim magazine publishesan annual volume of photographs, containing a mix of pin-ups andgrisly accidents. They know their audience.Historically, Lisp has been good at letting hackers have their way.The political correctness of Common Lisp is an aberration. EarlyLisps let you get your hands on everything. A good deal of thatspirit is, fortunately, preserved in macros. What a wonderful thing,to be able to make arbitrary transformations on the source code.Classic macros are a real hacker's tool — simple, powerful, anddangerous. It's so easy to understand what they do: you call afunction on the macro's arguments, and whatever it returns getsinserted in place of the macro call. Hygienic macros embody theopposite principle. They try to protect you from understanding whatthey're doing. I have never heard hygienic macros explained in onesentence. And they are a classic example of the dangers of decidingwhat programmers are allowed to want. Hygienic macros are intendedto protect me from variable capture, among other things, but variablecapture is exactly what I want in some macros.A really good language should be both clean and dirty: cleanlydesigned, with a small core of well understood and highly orthogonaloperators, but dirty in the sense that it lets hackers have theirway with it. C is like this. So were the early Lisps. A real hacker'slanguage will always have a slightly raffish character.A good programming language should have features that make the kindof people who use the phrase "software engineering" shake theirheads disapprovingly. At the other end of the continuum are languageslike Ada and Pascal, models of propriety that are good for teachingand not much else.5 Throwaway ProgramsTo be attractive to hackers, a language must be good for writingthe kinds of programs they want to write. And that means, perhapssurprisingly, that it has to be good for writing throwaway programs.A throwaway program is a program you write quickly for some limitedtask: a program to automate some system administration task, orgenerate test data for a simulation, or convert data from one formatto another. The surprising thing about throwaway programs is that,like the "temporary" buildings built at so many American universitiesduring World War II, they often don't get thrown away. Many evolveinto real programs, with real features and real users.I have a hunch that the best big programs begin life this way,rather than being designed big from the start, like the Hoover Dam.It's terrifying to build something big from scratch. When peopletake on a project that's too big, they become overwhelmed. Theproject either gets bogged down, or the result is sterile andwooden: a shopping mall rather than a real downtown, Brasilia ratherthan Rome, Ada rather than C.Another way to get a big program is to start with a throwawayprogram and keep improving it. This approach is less daunting, andthe design of the program benefits from evolution. I think, if onelooked, that this would turn out to be the way most big programswere developed. And those that did evolve this way are probablystill written in whatever language they were first written in,because it's rare for a program to be ported, except for politicalreasons. And so, paradoxically, if you want to make a language thatis used for big systems, you have to make it good for writingthrowaway programs, because that's where big systems come from.Perl is a striking example of this idea. It was not only designedfor writing throwaway programs, but was pretty much a throwawayprogram itself. Perl began life as a collection of utilities forgenerating reports, and only evolved into a programming languageas the throwaway programs people wrote in it grew larger. It wasnot until Perl 5 (if then) that the language was suitable forwriting serious programs, and yet it was already massively popular.What makes a language good for throwaway programs? To start with,it must be readily available. A throwaway program is something thatyou expect to write in an hour. So the language probably mustalready be installed on the computer you're using. It can't besomething you have to install before you use it. It has to be there.C was there because it came with the operating system. Perl wasthere because it was originally a tool for system administrators,and yours had already installed it.Being available means more than being installed, though. Aninteractive language, with a command-line interface, is moreavailable than one that you have to compile and run separately. Apopular programming language should be interactive, and start upfast.Another thing you want in a throwaway program is brevity. Brevityis always attractive to hackers, and never more so than in a programthey expect to turn out in an hour.6 LibrariesOf course the ultimate in brevity is to have the program alreadywritten for you, and merely to call it. And this brings us to whatI think will be an increasingly important feature of programminglanguages: library functions. Perl wins because it has largelibraries for manipulating strings. This class of library functionsare especially important for throwaway programs, which are oftenoriginally written for converting or extracting data. Many Perlprograms probably begin as just a couple library calls stucktogether.I think a lot of the advances that happen in programming languagesin the next fifty years will have to do with library functions. Ithink future programming languages will have libraries that are ascarefully designed as the core language. Programming language designwill not be about whether to make your language strongly or weaklytyped, or object oriented, or functional, or whatever, but abouthow to design great libraries. The kind of language designers wholike to think about how to design type systems may shudder at this.It's almost like writing applications! Too bad. Languages are forprogrammers, and libraries are what programmers need.It's hard to design good libraries. It's not simply a matter ofwriting a lot of code. Once the libraries get too big, it cansometimes take longer to find the function you need than to writethe code yourself. Libraries need to be designed using a small setof orthogonal operators, just like the core language. It ought tobe possible for the programmer to guess what library call will dowhat he needs.Libraries are one place Common Lisp falls short. There are onlyrudimentary libraries for manipulating strings, and almost nonefor talking to the operating system. For historical reasons, CommonLisp tries to pretend that the OS doesn't exist. And because youcan't talk to the OS, you're unlikely to be able to write a seriousprogram using only the built-in operators in Common Lisp. You haveto use some implementation-specific hacks as well, and in practicethese tend not to give you everything you want. Hackers would thinka lot more highly of Lisp if Common Lisp had powerful stringlibraries and good OS support.7 SyntaxCould a language with Lisp's syntax, or more precisely, lack ofsyntax, ever become popular? I don't know the answer to thisquestion. I do think that syntax is not the main reason Lisp isn'tcurrently popular. Common Lisp has worse problems than unfamiliarsyntax. I know several programmers who are comfortable with prefixsyntax and yet use Perl by default, because it has powerful stringlibraries and can talk to the os.There are two possible problems with prefix notation: that it isunfamiliar to programmers, and that it is not dense enough. Theconventional wisdom in the Lisp world is that the first problem isthe real one. I'm not so sure. Yes, prefix notation makes ordinaryprogrammers panic. But I don't think ordinary programmers' opinionsmatter. Languages become popular or unpopular based on what experthackers think of them, and I think expert hackers might be able todeal with prefix notation. Perl syntax can be pretty incomprehensible,but that has not stood in the way of Perl's popularity. If anythingit may have helped foster a Perl cult.A more serious problem is the diffuseness of prefix notation. Forexpert hackers, that really is a problem. No one wants to write(aref a x y) when they could write a[x,y].In this particular case there is a way to finesse our way out ofthe problem. If we treat data structures as if they were functionson indexes, we could write (a x y) instead, which is even shorterthan the Perl form. Similar tricks may shorten other types ofexpressions.We can get rid of (or make optional) a lot of parentheses by makingindentation significant. That's how programmers read code anyway:when indentation says one thing and delimiters say another, we goby the indentation. Treating indentation as significant wouldeliminate this common source of bugs as well as making programsshorter.Sometimes infix syntax is easier to read. This is especially truefor math expressions. I've used Lisp my whole programming life andI still don't find prefix math expressions natural. And yet it isconvenient, especially when you're generating code, to have operatorsthat take any number of arguments. So if we do have infix syntax,it should probably be implemented as some kind of read-macro.I don't think we should be religiously opposed to introducing syntaxinto Lisp, as long as it translates in a well-understood way intounderlying s-expressions. There is already a good deal of syntaxin Lisp. It's not necessarily bad to introduce more, as long as noone is forced to use it. In Common Lisp, some delimiters are reservedfor the language, suggesting that at least some of the designersintended to have more syntax in the future.One of the most egregiously unlispy pieces of syntax in Common Lispoccurs in format strings; format is a language in its own right,and that language is not Lisp. If there were a plan for introducingmore syntax into Lisp, format specifiers might be able to be includedin it. It would be a good thing if macros could generate formatspecifiers the way they generate any other kind of code.An eminent Lisp hacker told me that his copy of CLTL falls open tothe section format. Mine too. This probably indicates room forimprovement. It may also mean that programs do a lot of I/O.8 EfficiencyA good language, as everyone knows, should generate fast code. Butin practice I don't think fast code comes primarily from thingsyou do in the design of the language. As Knuth pointed out longago, speed only matters in certain critical bottlenecks. And asmany programmers have observed since, one is very often mistakenabout where these bottlenecks are.So, in practice, the way to get fast code is to have a very goodprofiler, rather than by, say, making the language strongly typed.You don't need to know the type of every argument in every call inthe program. You do need to be able to declare the types of argumentsin the bottlenecks. And even more, you need to be able to find outwhere the bottlenecks are.One complaint people have had with Lisp is that it's hard to tellwhat's expensive. This might be true. It might also be inevitable,if you want to have a very abstract language. And in any case Ithink good profiling would go a long way toward fixing the problem:you'd soon learn what was expensive.Part of the problem here is social. Language designers like towrite fast compilers. That's how they measure their skill. Theythink of the profiler as an add-on, at best. But in practice a goodprofiler may do more to improve the speed of actual programs writtenin the language than a compiler that generates fast code. Here,again, language designers are somewhat out of touch with theirusers. They do a really good job of solving slightly the wrongproblem.It might be a good idea to have an active profiler — to pushperformance data to the programmer instead of waiting for him tocome asking for it. For example, the editor could display bottlenecksin red when the programmer edits the source code. Another approachwould be to somehow represent what's happening in running programs.This would be an especially big win in server-based applications,where you have lots of running programs to look at. An activeprofiler could show graphically what's happening in memory as aprogram's running, or even make sounds that tell what's happening.Sound is a good cue to problems. In one place I worked, we had abig board of dials showing what was happening to our web servers.The hands were moved by little servomotors that made a slight noisewhen they turned. I couldn't see the board from my desk, but Ifound that I could tell immediately, by the sound, when there wasa problem with a server.It might even be possible to write a profiler that would automaticallydetect inefficient algorithms. I would not be surprised if certainpatterns of memory access turned out to be sure signs of badalgorithms. If there were a little guy running around inside thecomputer executing our programs, he would probably have as longand plaintive a tale to tell about his job as a federal governmentemployee. I often have a feeling that I'm sending the processor ona lot of wild goose chases, but I've never had a good way to lookat what it's doing.A number of Lisps now compile into byte code, which is then executedby an interpreter. This is usually done to make the implementationeasier to port, but it could be a useful language feature. It mightbe a good idea to make the byte code an official part of thelanguage, and to allow programmers to use inline byte code inbottlenecks. Then such optimizations would be portable too.The nature of speed, as perceived by the end-user, may be changing.With the rise of server-based applications, more and more programsmay turn out to be i/o-bound. It will be worth making i/o fast.The language can help with straightforward measures like simple,fast, formatted output functions, and also with deep structuralchanges like caching and persistent objects.Users are interested in response time. But another kind of efficiencywill be increasingly important: the number of simultaneous usersyou can support per processor. Many of the interesting applicationswritten in the near future will be server-based, and the number ofusers per server is the critical question for anyone hosting suchapplications. In the capital cost of a business offering a server-basedapplication, this is the divisor.For years, efficiency hasn't mattered much in most end-userapplications. Developers have been able to assume that each userwould have an increasingly powerful processor sitting on theirdesk. And by Parkinson's Law, software has expanded to use theresources available. That will change with server-based applications.In that world, the hardware and software will be supplied together.For companies that offer server-based applications, it will makea very big difference to the bottom line how many users they cansupport per server.In some applications, the processor will be the limiting factor,and execution speed will be the most important thing to optimize.But often memory will be the limit; the number of simultaneoususers will be determined by the amount of memory you need for eachuser's data. The language can help here too. Good support forthreads will enable all the users to share a single heap. It mayalso help to have persistent objects and/or language level supportfor lazy loading.9 TimeThe last ingredient a popular language needs is time. No one wantsto write programs in a language that might go away, as so manyprogramming languages do. So most hackers will tend to wait untila language has been around for a couple years before even consideringusing it.Inventors of wonderful new things are often surprised to discoverthis, but you need time to get any message through to people. Afriend of mine rarely does anything the first time someone askshim. He knows that people sometimes ask for things that they turnout not to want. To avoid wasting his time, he waits till the thirdor fourth time he's asked to do something; by then, whoever's askinghim may be fairly annoyed, but at least they probably really dowant whatever they're asking for.Most people have learned to do a similar sort of filtering on newthings they hear about. They don't even start paying attentionuntil they've heard about something ten times. They're perfectlyjustified: the majority of hot new whatevers do turn out to be awaste of time, and eventually go away. By delaying learning VRML,I avoided having to learn it at all.So anyone who invents something new has to expect to keep repeatingtheir message for years before people will start to get it. Wewrote what was, as far as I know, the first web-server basedapplication, and it took us years to get it through to people thatit didn't have to be downloaded. It wasn't that they were stupid.They just had us tuned out.The good news is, simple repetition solves the problem. All youhave to do is keep telling your story, and eventually people willstart to hear. It's not when people notice you're there that theypay attention; it's when they notice you're still there.It's just as well that it usually takes a while to gain momentum.Most technologies evolve a good deal even after they're firstlaunched — programming languages especially. Nothing could be better,for a new techology, than a few years of being used only by a smallnumber of early adopters. Early adopters are sophisticated anddemanding, and quickly flush out whatever flaws remain in yourtechnology. When you only have a few users you can be in closecontact with all of them. And early adopters are forgiving whenyou improve your system, even if this causes some breakage.There are two ways new technology gets introduced: the organicgrowth method, and the big bang method. The organic growth methodis exemplified by the classic seat-of-the-pants underfunded garagestartup. A couple guys, working in obscurity, develop some newtechnology. They launch it with no marketing and initially haveonly a few (fanatically devoted) users. They continue to improvethe technology, and meanwhile their user base grows by word ofmouth. Before they know it, they're big.The other approach, the big bang method, is exemplified by theVC-backed, heavily marketed startup. They rush to develop a product,launch it with great publicity, and immediately (they hope) havea large user base.Generally, the garage guys envy the big bang guys. The big bangguys are smooth and confident and respected by the VCs. They canafford the best of everything, and the PR campaign surrounding thelaunch has the side effect of making them celebrities. The organicgrowth guys, sitting in their garage, feel poor and unloved. Andyet I think they are often mistaken to feel sorry for themselves.Organic growth seems to yield better technology and richer foundersthan the big bang method. If you look at the dominant technologiestoday, you'll find that most of them grew organically.This pattern doesn't only apply to companies. You see it in sponsoredresearch too. Multics and Common Lisp were big-bang projects, andUnix and MacLisp were organic growth projects.10 Redesign"The best writing is rewriting," wrote E. B. White. Every goodwriter knows this, and it's true for software too. The most importantpart of design is redesign. Programming languages, especially,don't get redesigned enough.To write good software you must simultaneously keep two opposingideas in your head. You need the young hacker's naive faith inhis abilities, and at the same time the veteran's skepticism. Youhave to be able to think how hard can it be? with one half ofyour brain while thinking it will never work with the other.The trick is to realize that there's no real contradiction here.You want to be optimistic and skeptical about two different things.You have to be optimistic about the possibility of solving theproblem, but skeptical about the value of whatever solution you'vegot so far.People who do good work often think that whatever they're workingon is no good. Others see what they've done and are full of wonder,but the creator is full of worry. This pattern is no coincidence:it is the worry that made the work good.If you can keep hope and worry balanced, they will drive a projectforward the same way your two legs drive a bicycle forward. In thefirst phase of the two-cycle innovation engine, you work furiouslyon some problem, inspired by your confidence that you'll be ableto solve it. In the second phase, you look at what you've done inthe cold light of morning, and see all its flaws very clearly. Butas long as your critical spirit doesn't outweigh your hope, you'llbe able to look at your admittedly incomplete system, and think,how hard can it be to get the rest of the way?, thereby continuingthe cycle.It's tricky to keep the two forces balanced. In young hackers,optimism predominates. They produce something, are convinced it'sgreat, and never improve it. In old hackers, skepticism predominates,and they won't even dare to take on ambitious projects.Anything you can do to keep the redesign cycle going is good. Prosecan be rewritten over and over until you're happy with it. Butsoftware, as a rule, doesn't get redesigned enough. Prose hasreaders, but software has users. If a writer rewrites an essay,people who read the old version are unlikely to complain that theirthoughts have been broken by some newly introduced incompatibility.Users are a double-edged sword. They can help you improve yourlanguage, but they can also deter you from improving it. So chooseyour users carefully, and be slow to grow their number. Havingusers is like optimization: the wise course is to delay it. Also,as a general rule, you can at any given time get away with changingmore than you think. Introducing change is like pulling off abandage: the pain is a memory almost as soon as you feel it.Everyone knows that it's not a good idea to have a language designedby a committee. Committees yield bad design. But I think the worstdanger of committees is that they interfere with redesign. It isso much work to introduce changes that no one wants to bother.Whatever a committee decides tends to stay that way, even if mostof the members don't like it.Even a committee of two gets in the way of redesign. This happensparticularly in the interfaces between pieces of software writtenby two different people. To change the interface both have to agreeto change it at once. And so interfaces tend not to change at all,which is a problem because they tend to be one of the most ad hocparts of any system.One solution here might be to design systems so that interfacesare horizontal instead of vertical — so that modules are alwaysvertically stacked strata of abstraction. Then the interface willtend to be owned by one of them. The lower of two levels will eitherbe a language in which the upper is written, in which case thelower level will own the interface, or it will be a slave, in whichcase the interface can be dictated by the upper level.11 LispWhat all this implies is that there is hope for a new Lisp. Thereis hope for any language that gives hackers what they want, includingLisp. I think we may have made a mistake in thinking that hackersare turned off by Lisp's strangeness. This comforting illusion mayhave prevented us from seeing the real problem with Lisp, or atleast Common Lisp, which is that it sucks for doing what hackerswant to do. A hacker's language needs powerful libraries andsomething to hack. Common Lisp has neither. A hacker's language isterse and hackable. Common Lisp is not.The good news is, it's not Lisp that sucks, but Common Lisp. If wecan develop a new Lisp that is a real hacker's language, I thinkhackers will use it. They will use whatever language does the job.All we have to do is make sure this new Lisp does some importantjob better than other languages.History offers some encouragement. Over time, successive newprogramming languages have taken more and more features from Lisp.There is no longer much left to copy before the language you'vemade is Lisp. The latest hot language, Python, is a watered-downLisp with infix syntax and no macros. A new Lisp would be a naturalstep in this progression.I sometimes think that it would be a good marketing trick to callit an improved version of Python. That sounds hipper than Lisp. Tomany people, Lisp is a slow AI language with a lot of parentheses.Fritz Kunze's official biography carefully avoids mentioning theL-word. But my guess is that we shouldn't be afraid to call thenew Lisp Lisp. Lisp still has a lot of latent respect among thevery best hackers — the ones who took 6.001 and understood it, forexample. And those are the users you need to win.In "How to Become a Hacker," Eric Raymond describes Lisp as somethinglike Latin or Greek — a language you should learn as an intellectualexercise, even though you won't actually use it: Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot.If I didn't know Lisp, reading this would set me asking questions.A language that would make me a better programmer, if it meansanything at all, means a language that would be better for programming.And that is in fact the implication of what Eric is saying.As long as that idea is still floating around, I think hackers willbe receptive enough to a new Lisp, even if it is called Lisp. Butthis Lisp must be a hacker's language, like the classic Lisps ofthe 1970s. It must be terse, simple, and hackable. And it must havepowerful libraries for doing what hackers want to do now.In the matter of libraries I think there is room to beat languageslike Perl and Python at their own game. A lot of the new applicationsthat will need to be written in the coming years will be server-basedapplications. There's no reason a new Lisp shouldn't have stringlibraries as good as Perl, and if this new Lisp also had powerfullibraries for server-based applications, it could be very popular.Real hackers won't turn up their noses at a new tool that will letthem solve hard problems with a few library calls. Remember, hackersare lazy.It could be an even bigger win to have core language support forserver-based applications. For example, explicit support for programswith multiple users, or data ownership at the level of type tags.Server-based applications also give us the answer to the questionof what this new Lisp will be used to hack. It would not hurt tomake Lisp better as a scripting language for Unix. (It would behard to make it worse.) But I think there are areas where existinglanguages would be easier to beat. I think it might be better tofollow the model of Tcl, and supply the Lisp together with a completesystem for supporting server-based applications. Lisp is a naturalfit for server-based applications. Lexical closures provide a wayto get the effect of subroutines when the ui is just a series ofweb pages. S-expressions map nicely onto html, and macros are goodat generating it. There need to be better tools for writingserver-based applications, and there needs to be a new Lisp, andthe two would work very well together.12 The Dream LanguageBy way of summary, let's try describing the hacker's dream language.The dream language is beautiful, clean, and terse. It has aninteractive toplevel that starts up fast. You can write programsto solve common problems with very little code. Nearly all thecode in any program you write is code that's specific to yourapplication. Everything else has been done for you.The syntax of the language is brief to a fault. You never have totype an unnecessary character, or even to use the shift key much.Using big abstractions you can write the first version of a programvery quickly. Later, when you want to optimize, there's a reallygood profiler that tells you where to focus your attention. Youcan make inner loops blindingly fast, even writing inline byte codeif you need to.There are lots of good examples to learn from, and the language isintuitive enough that you can learn how to use it from examples ina couple minutes. You don't need to look in the manual much. Themanual is thin, and has few warnings and qualifications.The language has a small core, and powerful, highly orthogonallibraries that are as carefully designed as the core language. Thelibraries all work well together; everything in the language fitstogether like the parts in a fine camera. Nothing is deprecated,or retained for compatibility. The source code of all the librariesis readily available. It's easy to talk to the operating systemand to applications written in other languages.The language is built in layers. The higher-level abstractions arebuilt in a very transparent way out of lower-level abstractions,which you can get hold of if you want.Nothing is hidden from you that doesn't absolutely have to be. Thelanguage offers abstractions only as a way of saving you work,rather than as a way of telling you what to do. In fact, the languageencourages you to be an equal participant in its design. You canchange everything about it, including even its syntax, and anythingyou write has, as much as possible, the same status as what comespredefined.Notes[1] Macros very close to the modern idea were proposed by TimothyHart in 1964, two years after Lisp 1.5 was released. What wasmissing, initially, were ways to avoid variable capture and multipleevaluation; Hart's examples are subject to both.[2] In When the Air Hits Your Brain, neurosurgeon Frank Vertosickrecounts a conversation in which his chief resident, Gary, talksabout the difference between surgeons and internists ("fleas"): Gary and I ordered a large pizza and found an open booth. The chief lit a cigarette. "Look at those goddamn fleas, jabbering about some disease they'll see once in their lifetimes. That's the trouble with fleas, they only like the bizarre stuff. They hate their bread and butter cases. That's the difference between us and the fucking fleas. See, we love big juicy lumbar disc herniations, but they hate hypertension...."It's hard to think of a lumbar disc herniation as juicy (exceptliterally). And yet I think I know what they mean. I've often hada juicy bug to track down. Someone who's not a programmer wouldfind it hard to imagine that there could be pleasure in a bug.Surely it's better if everything just works. In one way, it is.And yet there is undeniably a grim satisfaction in hunting downcertain sorts of bugs.
