Richard Stallman: The Coder

Richard Stallman launched the GNU project in 1984 and created the now famous GNU General Public License. The Free Software Foundation is a tax-exempt charity that raises funds for work on the GNU Project.

Linuxcare: One of my favorite documents on the GNU web site is a talk you gave at Sweden's Royal Institute of Technology in 1986. The way that talk was structured was 1/3 historical, 1/3 technical, and 1/3 social. I was wondering if we could spend a bit of time to discuss the technical part now.

Richard: Okay.

Linuxcare: I understand it's not your focus right now. One of the initial questions I have is since the portability of ITS [Incompatible Timesharing System] failed...

Richard: It had no portability.

Linuxcare: Okay, then in the absence of portability, what really made you think that UNIX was a good idea?

Richard: It was a portable operating system and it had real users on different kinds of machines. So it wasn't just portable in theory, but the portability had been tested. In addition, UNIX had some good ideas by the standards of the day. Nowadays we take them for granted perhaps, but compared with other operating systems that I had worked on they were interesting and powerful; things like pipes and redirection, shell programming, the convenient fork() and exec() system calls; these were an advance. So I thought UNIX was a pretty good idea. Since I had never used it, I didn't know about some of the things that were actually badly designed. There were some ways in which UNIX was a step backwards from the other operating systems I had worked on. For example, the lack of atomic supersede. These days we've pretty much forgotten what that is. It means that when a program starts to write a file in UNIX, it starts by truncating the file to zero length, then it rewrites the contents. This means if somebody tries to read it at that time they will get a half written file. On the operating systems I used in the 1970s, there was no danger of that because the new version would appear instantaneously when it was closed. Until a certain point, anybody reading the file would get the old version, and then instantly it would be replaced by the new version. So anybody reading the file would always get a complete version.

Linuxcare: For example, sometimes it's nice to be able to "tail" a file as it's being created.

Richard: Yes it is. There are some uses in that, but it seems to me that having a special system call you can use to get at a file being written is better than lacking atomic supersede. At least there was some way you could get to look at the file while it was being written.

Linuxcare: Doesn't file locking take care of a lot of that?

Richard: To an extent it works. EMACS has the ability to write it under another name and rename it. Of course, there's no other name you can be sure won't cause a problem. There is some ugliness in the design of UNIX, but fortunately most of it is pretty good.

Linuxcare: One of the ugly points you mentioned in the 1986 talk was that of a security model. You explained that rather than make it easy for people to do the right thing, it simply forbids them from doing the wrong thing.

Richard: You're mixing several issues. One is that I don't like the idea of having security within a shared computer system at all. I also don't like the idea of saying no to everyone by default. Now, in fact, I do that. Our computers have security now, and because I am so bitter about what I have done, I'm so ashamed of what I have done, that I treat myself like one of those outsiders. I cannot log in remotely to the new machines. I know a way that I could, but I won't do it because the same maltreatment I am giving to everybody I must give to myself as well out of shame.

Linuxcare: One thing I noted in talking to Richard Stevens was the fact that you were pretty responsible for coming up with the name POSIX.

Richard: Yes.

Linuxcare: Originally it was going to be IEEEIX.

Richard: I was part of the committee, and that's why I knew what it was going to be called. When I realized people were all going to be calling it "UNIX-like," standards for "UNIX-like" operating systems, because they didn't want to say IEEEIX, I decided I had better get them a euphonious name on the double. This was after the standard had been finalized and they were about to officially publish it.

Linuxcare: It might be said that the new system could almost be a reference to the product.

Richard: It could, except that we don't religiously try to follow it. You have the POSIXLY_CORRECT environment variable.

Linuxcare: There are specific examples where POSIX is specifically seen as backwards compatibility.

Richard: Basically, my attitude towards standards is that they are useful. They help users figure out how to support a variety of systems, and then they help system implementors figure out how to give the users what the users will expect. But you shouldn't treat standards as though they were gods. There's no need to. We support standards in the ways that are useful to users, and we depart from them when that becomes more useful to users.

Linuxcare: Can you think of any major coups or major features that were brought to the POSIX table?

Richard: I can't remember any more. That was in the 1980s. However, I do remember one where we didn't succeed in persuading them. This was in the POSIX.2 specification for the utilities. They made a decision that they would follow System V by default, and in System V when you did "df" or "du," file sizes were measured in disk block units of bytes.

Linuxcare: Right; instead of kilobytes.

Richard: Well, it occurred to me that this was not good for anybody, so I asked them if they could please change it for the sake of the users. They said, "No, our rules say we follow System V." So I took a poll and asked users which they preferred, and it was 20:1 in favor of kilobytes. I sent them the poll results and they said they didn't care. Unfortunately, I didn't catch this at a time when I could officially object to it. It was already approved and we had already had a chance to object, so all I could do was make a comment. So I couldn't block approval of the standard because I was a voting member of the committee. I couldn't block approval of the standard on those grounds, so instead... speaking of coups, this was about the time of the coup that eliminated the Soviet Union. So, I posted a notice about the coup in which the evil repressive forces of POSIX were being thrown off, and as we speak, teams of new developers are taking control of the major new utilities because they were making the changes to support K by default. To have an excuse to say that we still support the spec, if you define the environment variable, POSIX_ME_HARDER was the original way. Then a slightly prudish board member convinced me to change it to POSIXLY_CORRECT which I now think was a mistake. I should have left it as POSIX_ME_HARDER.

Linuxcare: I'll ask a stupid question. Have you put any Easter eggs into anything? Just some wacky key code combination or something?

Richard: I did once. It was a joke about the C specification which said #pragma was supposed to do something about implementation design. So I decided that a particular #pragma should do some absurd thing, just to point out that it's ridiculous to use pragma for anything, because you never can tell what it's going to do in some other compiler.

Linuxcare: Which pragma was it?

Richard: I don't remember, but we took it out. There was some pragma that other people were using and we decided to have compatibility to support it. We took that out and made it something that was somewhat useful. The other problem with pragma is you can't use it in macros. This means that for almost all purposes it's a bad solution. To have a mechanism that's guaranteed to be wrong is inappropriate for almost all cases.

Linuxcare: Right.

Richard: I think they finally got the message and started designing other mechanisms that could be used inside a macro extension. We did eventually find one situation where a pragma could actually be used to be appropriate because it was something that never was useful to do in a macro. I can't remember what it is, but I think you might be able to find it in the GCC manual.

Linuxcare: Speaking of GCC, we've got a couple of questions submitted here by somebody.

Richard: Okay.

Linuxcare: To what extent does the basic architecture of GCC need changes to support the language front-ends, such as the new GCC Java compiler?

Richard: I don't know anything about what's happening with the Java compiler. I stopped dealing with GCC around 1992 and I don't know what changes, if any, were needed. Some changes were needed mainly in the tree data structure, which is what programs get parsed into. Occasionally, when you handle a language that has the kind of construct that hasn't been handled before, if there's no way to represent it with the existing tree data structure, you need to add something so you can represent it. Things like methods and core methods needed a way to be represented, so we added them.

Linuxcare: Does GCC still use a LISP-like intermediary language?

Richard: Printed syntax is what's LISP-like. It's not really LISP-like in the sense that it consists of tuples that point to a number of other tuples, which is not by any means a new format for priority use. It's printed out in a LISP-like syntax because that makes it easy to work with in EMACS.

Linuxcare: How did you come to choose LISP as the EMACS engine?

Richard: LISP is the most powerful programming language, and if you want an interpreter, LISP is the best. None of the other languages come anywhere near LISP in their power. The most exciting things about LISP are read, eval, and print. If you look at other languages, they have no equivalent for any of those.

Linuxcare: Why are there so many parameters?

Richard: The minimal set of parameters are small, but on the other hand, there are many standard functions available to do lots of interesting things with lists. With other languages, you need to define exactly the data types you want and define all of the basic things for working on them. Well, they're starting to wise up about template libraries where you can define various kinds of applications, specific list data types, and then have various functions that you can substantiate for any one of those. However, you're still limiting each of those data types you define to using one particular data type for the elements, which is not very convenient. With lists, you have one data type, and you can put anything in any list. You can mix together different kinds of types of elements. It's so flexible.

Linuxcare: One technical point is you don't necessarily limit yourself to a specific set, but the beautiful thing about LISP is that everything is a list.

Richard: Yeah, everything is generic.

Linuxcare: You can build functions using a very small set of primitives. Those functions become useable over lists themselves, or lists of lists and so on.

Richard: Any list of anything in your program and you can have all these built in things that you can do. Not only that, programs are representable as data, which gives you all sorts of convenient, powerful things.

Linuxcare: That's not unique to LISP though.

Richard: Well, it is mostly unique to LISP. Yes, you can write a parser for some other language, and you can invent some data structure to parse them into, but you will have been the one who invented that data structure. There will be nothing you can do on it except the things you provide ways to do.

Linuxcare: That's not true. You can do that in Perl for example.

Richard: Well, with Perl what is their format? They're strings.

Linuxcare: Yeah, you can create strings which are programmed.

Richard: Yes, with string-based interpreters you can represent programs as strings, and you can interpret them as strings, but you can't parse their syntax very easily as strings. In LISP, programs are represented as data in a way that expresses their structure, and allows you to easily do things to the programs that are based on understanding them.

Linuxcare: Right, but you have to count the strings.

Richard: No, you don't have to count strings. Strings are only in a textual representation. When it's data, you have lists that are nested in exactly the same way that the programs are.

Linuxcare: This data structure versus pure list argument is one which the XEMACS people often make.

Richard: They've apparently been indoctrinated by some rather rigid kind of object oriented design philosophy, and they feel they should make as many layers as possible in the program, and at each layer make a whole bunch of primitives to do everything you could possibly want to do. This is a recipe for making a program very big and hairy. Sometimes it's a good thing to do. When the thing has enough hair anyway, then it's sometimes good to make such a data structure. In the 1980s, there were people trying to run it in terms where you could only get an address space of 1 meg. Nowadays, that's an irrelevant design goal, and EMACS is much bigger than it was and that's okay. If it were just technical issues separating us and the EX people, I would be willing to make compromises on those things, but unfortunately it's not just technical issues.

Linuxcare: How much of the current development are you still doing?

Richard: I'm doing a substantial part of it, but not most of it anymore. I found a very good person who I think is about to be hired as an EMACS retainer, who just in the past year and a half rewrote the read/write completely to support variable response and images in the documents, and all sorts of other nice things which seem to work. You'll be able to put postscript into your document and have it display and all sorts of other neat things. We're headed towards word processor.

Linuxcare: Any release dates?

Richard: No. It will be done sooner if you help.

Linuxcare: So what are you doing more of now?

Richard: I have been compelled to involuntarily promote myself into management.

Linuxcare: Which means no more coding?

Richard: Not much coding; sad to say. It's much less fun, but it has to be done.

Linuxcare: Also, you seem to be doing a lot of going out and dealing with issues such as these.

Richard: Exactly. It's the leadership of the movement that I'm doing. Sometimes it involves managing people within new projects, sometimes it involves recruiting, negotiating with other projects, or persuading people to change their licenses. For instance, I worked for two years to persuade Berkeley to change its license to get rid of the obnoxious advertising clause. It took a long time and now it's done, but now we have lots of other people to persuade.

Linuxcare: The BSD people seem to use a new code that they wrote.

Richard: Exactly. I persuaded them a year and a half or more before I persuaded Berkeley. They took to the idea very quickly, so they don't put that into their code, but now they can remove it from the Berkeley code too.

Linuxcare: Getting back to EMACS, I've heard that there are some rather peculiar methods by which it's compiled.

Richard: Well, it has to dump itself out. The reason is to avoid taking the time to load up the standard LISP code every time you start it, so it actually unexecs itself.

Linuxcare: In other words, you compile the base LISP interpreter, then you load up the LISP codes.

Richard: Exactly.

Linuxcare: Do you share libraries at all?

Richard: No. Well, it may if you build it on a system where LISP libraries are being shared. In that case it's using shared libraries, but other than that, no. The C shared library mechanism is not designed to be used to link any LISP codes.

Linuxcare: Is that how LISP programs are normally compiled? To say it core dumps itself sounds kind of frightening.

Richard: No. LISP is interpreted. This is a different issue. This is the issue of: "How can I go out and make something start running quickly?" So much of EMACS's interpreted LISP code is the idea that, rather than starting with something small, having it bootstrap itself, and then start running interpreted LISP, you actually see something pop up on the display. There are about 30 files of LISP codes that are standardly included in EMACS. If EMACS had to load them each time, it would take probably a minute or so to start up. That would be inconvenient.

Linuxcare: It's basically just a snapshot? Okay, it makes an executable file that represents the state of what is in memory at that time.

Richard: Exactly, and that way it starts up like that and all of the loading and preloading... Well, it still takes much longer to start up than we wish, but it's much faster than it would be if it didn't do this. In the old days, it used to take 10 minutes to do that pre-loading. That was really a pain.

Linuxcare: What inspired you to write EMACS?

Richard: It sort of happened almost by accident. Let me give you the whole story. In the early 1970s, what we used for editing was TECO. Except, our version of TECO had a nice display screen feature where at the end of each TECO command string... because the way you used TECO was, you typed a command string which was a long or short sequence of commands, and at the end you would type escape/escape, and that executed all of these commands. In most versions of TECO you never saw what was in your text unless you used the command to type it out. But we had some display consoles, so we put in an automatic redisplay thing where, at the end of the command string, it would display the current area of the buffer on the screen automatically. It would keep track of what was on the screen already, and it would not redisplay the things that were changed. So that made redisplay faster.

Anyway, somebody else decided to implement a real time editing feature where you would type single character commands and they would update the screen immediately. He decided to do this with its own completely separate redisplay mechanism though, which turned out to be utterly unusable and painfully inefficient. It didn't work and nobody used it, so I decided to reimplement that completely and make it display compatibly with the usual redisplay mechanism. So, you'd go into the real time editing mode and you wouldn't see any of your text on the screen change, but then as you typed characters which changed the text, you would see the redisplay happen immediately. I optimized this and people actually started to use it. At first there were only a very limited set of things you could do to your text in this real time editing mode, so you had to edit it to do anything else like delete or save a file.

So, somebody said to me, "How about giving us a couple of characters in the real time editing mode that we can redefine and make them run TECO macros?" A TECO macro was just a command string that you had written in advance to serve as a subroutine that you could call up when you wanted to. So I looked at the idea, and I saw it would be just as easy to let you redefine any and all of the characters.

Linuxcare: Right.

Richard: So I did that, and other people really went to town redefining these realtime editing characters, until eventually just a very few of them were left with their standard definitions. People had written several different packages, essentially editors written in TECO. So TECO had changed from being the editor, to a system for writing editors. Then Guy Steele had the idea of looking at these various TECO-based editors that people had written, collecting the best ideas of them all, and synthesizing them into one TECO-based editor that would end all TECO-based editors. He did a lot of work designing a new command set that would be clean and symmetrical, and then we got together to implement it. Then after the first night it happened, he dropped out, so I did most of the work. This became EMACS, and it basically replaced the other TECO-based editors. When it caught on, other people wrote implementations of the same basic idea, but not using TECO typically. Bernard Greenberg, I believe, was the first one to use LISP as the language to implement the editor in. Then, people working for a system I can't recall, tried the approach of writing their own LISP interpreter and implementing most of the editor in that, and it worked well. So I used that for doing my second implementation of EMACS. Then, to gain portability, I wrote the LISP interpreter, the parts of the editor that had to run really fast, and some editing primitives, all in C.

Linuxcare: Like which ones?

Richard: Well, like insert some text in the buffer, delete some text in the buffer, communication with subprocesses, and so forth.

Linuxcare: Wasn't the LISP interpreter written using lex and yacc?

Richard: No, absolutely not. There's no reason to use lex and yacc to write a LISP interpreter. The syntax is so simple you don't need it. You'd just be making things harder.

Linuxcare: I understand that you had a version on your LISP machine too.

Richard: Yeah, there was an EMACS-like editor written in LISP machine LISP.

Linuxcare: Did you work on the development of that?

Richard: I did some, but not very much except in the later stages in the time of the war with Symbolics. There were some neat features I added, like the selective undo feature where you could select a region and undo the last change in this region.

Linuxcare: Oh, that's cool.

Richard: Now the new EMACS has that feature too.

Linuxcare: What's the key for that?

Richard: You use it with CTRL-U as an argument that requests selective undo. Also, a transient mark mode. When you have a region active then undo the last action.

Linuxcare: So C-u and then what?

Richard: Either C-Xu or C-_.

Linuxcare: Are there any plans to include Guile bindings in EMACS?

Richard: Well, I want to switch over to replace EMACS LISP with Guile.

Linuxcare: Oh, that's interesting.

Richard: Because Guile, which is the new project scheme interpreter, is our standard for extensibility. It's a library. It's designed that you build it into your program, and then you make the lower level parts of your program into added scheme packages. Oh, here's a new feature you can see. You can see here that one of the words in this buffer is highlighted. EMACS 21 supports faces on TTY's.

Linuxcare: Cool.

Richard: It has lots of other neat features too.

Linuxcare: So you're running EMACS 21 here?

Richard: Well the development sources are not finished. We're still putting new features into it.

Linuxcare: It's been a rumor that TECO had such a first impact.

Richard: It was designed to be something you edit with, not designed to be a programming language.

Linuxcare: Right.

Richard: It was really lousy as a programming language. So the lesson of EMACS was, pick a good programming language to write your editor in, and your sensibility language really ought to be designed to be a good programming language. You shouldn't think you should throw away what you know about programming language design. Well, obviously the best programming is LISP.

Linuxcare: Any real words can be considered a TECO macro?

Richard: Yeah, just about anything.

Linuxcare: So you can figure out what your names did?

Richard: I never did that. We didn't have that game. However, I do remember that there was an editor, not TECO, in which the command EDIT {carriage return} would wipe out the text that you were editing because it means "everything delete, insert T." Now if you only did everything DELETE {carriage return} you could undo the change. There was just a one level undo buffer.

Linuxcare: But not with a standard text editor.

Richard: Not on that system. I never edited on a UNIX machine until I had EMACS to edit with. I was at MIT, and there were other things that I could edit with. I could save the files over the net, so I did my editing on a LISP machine.

Linuxcare: Every time, I've heard people say that the reason they're still working on the HURD is because it was a good design and it's almost done.

Richard: Yes. Unfortunately, progress right now is very slow. We don't have anybody working on it full time, and I wish we did.

Linuxcare: What are some of the major big wins?

Richard: Well, the big win is that it's a collection of individual servers that communicate with each other, and each user can replace the ones he or she doesn't like.

Linuxcare: Such as the scheduling, or what?

Richard: Well, the scheduling is actually done inside the micro kernel, but you can write your own memory managers, you can write your own file systems, and you can write your own terminal driver. So, you can replace your own terminal driver. You can't replace somebody else's, and as an ordinary user you can set up additional file systems that you've written, different file names, and then people can open them.

Linuxcare: What advantages would this have over other micro kernels?

Richard: I don't know very much about a lot of micro kernels, but I know that the OSF multiservers were designed specifically to mimic the features of UNIX. That would trap the system call handler, which would send a message to a server, and the server would cooperate to get the job done. As far as the user program was concerned, it would have only the features of UNIX. The HURD was designed to completely expose to the user program the various servers. So something that would be a system call in Unix would just be a function in the library. It would send various messages to get the job done, and you could send messages to those same servers any time you wanted. So, all of the underlying implementation was made available, and you could take advantage of whatever you wanted. In addition, the multiserver derived some benefit of modularity due to the fact that it was divided into these several servers, but you couldn't replace the servers you were using. Only the operator could make an alternate kind of server run. That would be good for debugging a new version of the servers, but it didn't give the user any additional power.

Linuxcare: Can you make reference to the new Mach?

Richard: There is a new version of Mach that we maintain, but we didn't write Mach, and we did that basically once all the work on Mach came to an end.

Linuxcare: Could you expand a little bit on the PDP-10?

Richard: The PDP-10 had several nice things. One is that it was a very symmetrical design. The instructions formed very symmetrical classes, including a number of instructions that were NOOP's. If you hit the class of instructions that were complete, there were a few things that were trivial and didn't do anything useful. In many cases, there were several different instructions that would do the same job just because trivial cases of two different classes of instructions were the same. Another very nice thing about the PDP-10, was that each word of 36 bits was big enough to hold two addresses. In effect, it was designed for LISP, you see. It was the original LISP machine. So, because it was a word addressed machine, you could address a megabyte using just 18 bits, and this megabyte consisted of 256K of 36 bit words, each capable of holding two pointers to other words.

Linuxcare: Right.

Richard: Another nice thing about the PDP-10 was, if you looked at a word, you could pretty much tell from its contents what kind of data it contained, whether it contained text, or instruction, or a pair of addresses, or just one address, or just some number. Typical values were different enough that you could generally tell, and this is a big help for debugging.

Linuxcare: Other than the PDP-10, what would be your favorite architecture?

Richard: As you point out, the PDP-10 was great for its day, but eventually its address space just became too small. There was no way to extend it cleanly to a larger address space, so it really was obsolete. Nowadays, I basically don't pay attention. It doesn't matter so much anymore, and now that you write in something portable rather than assembler language, you have no reason to prefer one architecture to another. Back when everybody was writing in assembler language, it really was much more of a joy to program for the PDP-10.

Linuxcare: Any final thoughts?

Richard: I'm not very good at making up final thoughts.

Linuxcare: Anything?

Richard: If you're trying to debug a program, get into the symbolic debugger as soon as possible. Assuming that you've got a reproducible test case, don't bother wasting any time looking at the cases that don't fail. Just use the debugger to find out what happens in a case that does fail. Use breakpoints, and then stepping, to localize the problem until you see where it's happening. Other approaches that might seem like shortcuts usually end up taking longer. This one is sure and steady, and it will generally find you the problem faster than any other method. But when there is no reproducible test case, then that's a different kind of fish, and you may have to put debugging buffers and save information into your program, so you can get enough information to figure out what happened.

Linuxcare: There was also a program change with the debugger. [Heisenbug]

Richard: Do you know about the bug that depends upon the phase of the moon?

Linuxcare: I've heard about this.

Richard: We always liked to talk about the bugs that depended on the phase of the moon. So, when Guy Steele wrote the Rabbit compiler, which is a scheme compiler, he made it print out a comment at the beginning which showed the time it was compiled and so on, but it also put in the phase of the moon. So, you could always look. If you had a bug that depended on the phase of the moon, you could look at the thing and see at what phase of the moon it was compiled, and that might help you figure out what went wrong. Eventually, he got a bug report about a certain program that had been compiled once, and worked, and when it was compiled at another time it didn't work. So, he looked and he discovered that when the initial comments were printed out, the LISP feature that would automatically put in a line break if a line got too long was activated on one occasion, because the phase of the moon took too many characters to print out. So, it triggered that feature, and the last part of the phase of the moon was on another line, and therefore it wasn't marked by comments. So it was just sitting there in a file, whereas at another time the phase of the moon didn't take up so many characters, and the whole thing was properly commented. So, this was a bug that actually depended on the phase of the moon. You can take that as a final thought.