An Interview with Harry Nelson
HN = Harry Nelson
GAM = George Michael
GAM: It's July 13, 1993. Harry, why don't you begin by telling us when you started at the
HN: George, I started in the summer of 1960, and my first assignment was to go
to the cooler and wait till I got my clearance.
GAM: Well, what was the machine that they got you introduced to?
HN: I basically became interested in the Lab through a representative from
Remington Rand who had just installed LARC computer at Livermore. And I
expected that I would be working on a LARC in some aspect or other. As it
turned out, by the time I got here, the LARC was already pretty well under
control. The Stretch computer was coming in soon, and I got assigned to work
on the Stretch.
GAM: Was this from the point of view of developing what we call software, or
were you doing an applications program, or what?
HN: I was assigned to work on an applications program called ABLE , one of
the Lab's major and mainstream programs that had been around for years.
GAM: Did you generate any reports that I could go get out of the library or
TID, or something like that?
HN: I don't know�it's possible that there are still some things around, but I
doubt if there's anything with my particular name on it relating to that
GAM: Okay. Well, who was your supervisor then?
HN: It seems to me that Clarence Badger was my first supervisor. I worked with
Bill Schultz, though, of A Division, directly. He was the main contact, and
the author of the ABLE program. Barbara Snow and I were assigned to reduce
Bill's algorithms and instructions to a computer code that would run on the
GAM: What was the language you were using then?
HN: Well, at that time we were just using the hardware machine
language�assembly language, basically. Also, the way we split up the work was
that Barbara did the hydrodynamics section, which is the main physics of the
code. And I did everything else�the sort of surrounding operating system, and
all of the peripheral I/O code and some of the physics, also. But, at that
time, there wasn't even an operating system for the Stretch; it was under
construction by IBM, and hadn't been delivered yet. We basically were able to
produce our own operating system as part of the code package. So, when we
wanted to run, we would dead-start the machine, and then we'd go on and
actually load everything.
GAM: Do you remember STIPIC?
HN: STIPIC? No.
GAM: It was the thing that Norman Hardy wrote to do all the I/O and things like
HN: I never used it. We did everything ourselves, using essentially the
machine language and our own I/O. So we produced an operating system. Later
on, IBM's operating system and Livermore extensions that Clarence Badger and
Garret Boer basically put together became available. Of course, we were
totally incompatible with that operating system.
HN: So, basically, we shared the machine with the official operating system.
Whenever they wanted to run the regular operating system they would have to
kill our program and re-dead-start the machine, and then the other users would
operate under that system. We had our own system. Since the ABLE code was
occupying more than half of the clock hours of that machine anyway, it didn't
matter very much.
GAM: I guess that's true. Do you remember how big your codes were?
HN: Oh, let's see�the Stretch had, as I recall, 80,000 words of memory.
GAM: I think it was 96K�there were three 32K banks.
HN: In any case, roughly that order of
number of words of memory. And we would use it all, whatever was available.
We managed to fit our code into whatever space was available and not leave any.
And then most of the data for a typical physics problem would have to reside
off of the memory, on a disk storage system or even on tape. In fact, the
problems would be generated on a different machine and brought over on tape.
Then we would pull in the data on tape and put it over on the disk, and then do
the operations. It was a spooling proposition where you read in the data for
one row, then you worked on it, then you wrote it out, and then you read in the
data for the next row, because you couldn't keep them all in memory.
GAM: Yes. Well, it's clear you didn't stay on the Stretch forever, so what
HN: Well, I worked on the Stretch until the code seemed to be finished and in
real production, and it was handled by operators then. We didn't have to
contribute any further to that part of it. And, let's see, by the time that
was all down and seemed to be working well, we had actually two projects. The
first was a 7090�an IBM 7090 version. We had an IBM 709 for a lot of years,
and then they upgraded it to 7090, which made it fast enough that it was
reasonable to try to run ABLE on that machine. So at that time, we
basically started a project to convert the code we had to a 7090
version�again, using assembly language.
GAM: Well, I'm sort of re-interested in languages. What did you do about the
language situation there?
HN: We just basically translated by hand, one-to-one, each Stretch assembly
language statement over to some equivalent�
GAM: So you stayed away from FORTRAN?
HN: We didn't have anything to do with the compilers at that time. Anyhow,
that project actually never was completed; after some months working on it, we
decided that it would be a better idea to work on the 6600, which was also
coming about at the same time. Basically, the 6600 was Seymour Cray's first
big machine that the Lab got. They had a 1604 before that�maybe a 3600, also,
which was a Control Data product.
Anyway, the Control Data 6600 came along, and William Bennett, who was working
here at that time, had begun to write the ABLE code for the 6600.
Basically, he had worked only on the physics part. Once again, at the time the
machine came in, there wasn't an operating system. The operating system didn't
work. It was there, but it was poor, and it had problems. I went over and
sort of built a shell around Bennett's program, to do the disk reads and writes
and the tape functions and so on. And that was quite interesting. I'd worked
on a new machine, and I got to travel back to Wisconsin and meet Seymour Cray
and to be involved in that project. But then that took some little time�I
don't remember, a year or two probably�before that was completely functional.
There were interesting things�for example, when the first 6600 came in, it
had problems; like when you would ask for the machine to read a word from
memory, you almost always got the right word, but occasionally you got zero, or
you got something else, and there was no error correction, and very little
checking, other than some simple parity checks. So almost every code that was
being tried to run on the 6600 was having big problems, because they couldn't
stand this occasionally getting a zero instead of the real answer.
ABLE, on the other hand, had no problems whatsoever. It was written in such a way
that if it happened to get a
zero instead the real data item, it just could go merrily ahead.
Of course, the answer didn't agree with the previous run exactly, but to 99.9%
accuracy you got the same answers. So, actually, the software was better than
the hardware in some sense.
GAM: That's great.
HN: Along in the process we devised better ways to use the disks, and better
ways to use the tapes and so on, and to get our problem run faster than ever
before. In fact, my recollection is that it ran about three times as fast on
the 6600 as it had been running on the Stretch, for the same test problem. But
the 6600 was really just an interim machine before the 7600, which was the
basic goal that Seymour had to start with. He just needed to develop this
other machine first to try out some of the ideas. The 7600 had a lot of
improvements; not only was the cycle time cut by more than a factor of 2, but
it had a much larger memory with the so-called LCM. Well, they had fast memory
and slow memory. In any case, they were able to build, at that time, much
bigger memory than we'd ever had before�approximately, in my recollection,
400,000 words of memory.
GAM: It was 500,000, I think.
HN: Well, it was supposed to be 500,000, but in fact, you couldn't really get
at all of those. So, basically, the programmer was restricted to using
400,000. In any case, this meant that some problems that had to use disk
before were now able to run completely inside the memory. And for those
problems, we got an additional speed up of six times over the 6600.
HN: Despite the fact that it was only two times internal�but with these other
savings, it was very important. Basically, then, we were running approximately
twenty times as fast on the 7600 as we had been ever before with this code.
Finally, at that time we were able to actually catch up with the demand for
ABLE. That was fast enough so we were able to run 20 problems in an hour,
where it used to take a day. For really the first time in the Laboratory's
history we'd actually caught up with the demand.
GAM: Yes. Well, in a way, that speedup gave you the opportunity to run ABLE
as a kind of test problem over and over again.
HN: Yes, and people began to use it in different ways, and did parameter
studies, and in fact, generated a lot more use of it than ever before. That's
the typical thing.
As a matter of fact, it was in that era that Wilson J. Frank devised the "Jim Frank
Law." Jim Frank was a physicist and a leader of A Division at that time. He
had "Jim Frank's Law," which was that for any code that you ran on any machine,
if it took less than 5 minutes to complete, then it wasn't interesting. If it
took more than 30 hours to complete, it wasn't interesting either. So,
basically, you'd have a code which in a certain era, on a certain machine, was
interesting to people because it was sufficiently complicated to do something
reasonable, but took, say, 10 hours to complete. And then when a machine came
along that was a hundred times as fast, and only took 6 minutes to complete,
they lost interest in it�the same code. What they wanted then was more
physics in it, more complex stuff. And it wasn't interesting to run that old
The opposite was also true. In the original�in the Stretch�days, a ABLE
problem might run 200 hours, or 300 hours. If they seemed to be important
enough to the physicist, he would kind of fight the problem, and it would run
for several hours, maybe 50 hours. Then it would have difficulties and would
have to be rezoned and continued. Then it would run another 50 hours. So, to
actually get a 300-hour run might take a month of human time, real time.
When we got that problem down to about 15 hours, with no necessity for human
intervention, then they were really happy. That's what they had wanted all
along. Now, today, that same problem would probably take 10 minutes on a Cray
C90, and so they really wouldn't run it anymore. They wouldn't care. They
would want something that did more for them, something that did automatic
rezoning or some other complexity. They'd want to throw in some more to get it
up to the point where it seemed like it would be reasonable to do.
GAM: Do you remember anything about difficulties with round off error and
HN: No. As I mentioned before, ABLE was a very accommodating code. It
could stand an arbitrary zero thrown into the middle of it, for example, at
most any time, and round off was never a big problem at all. It had been a
problem in the 709 days. The 709 had basically 27 bits of fraction. It turned
out that it was sort of a borderline amount that you could stand.
Now, when we got the CDC Star computer, which came after the 7600, it offered
two options. There were single precision words with 24 bits of fraction, or
double precision words with 48 bits of fraction. The 48-bit fraction was fine,
and everything worked great. The 24-bit fraction was not quite good enough.
The problems actually did have a lot of round off errors, and after a long
amount of work on them, mostly done by Chris Hendrickson, we actually
determined that 24-bit precision was not adequate to get the answers that we
hoped to get out of the codes. However, in the earlier days 27-bit precision
was adequate, although 27 bits might not have been adequate at this time,
because they were running more complex problems at that time than they were ten
GAM: When you mentioned Chris and so forth, by that time the thing had been
stuck into FORTRAN, hadn't it?
HN: Yes, the first FORTRAN version was for the CDC Star. Actually, I'm not
certain about that. There may have been a FORTRAN version started earlier for
the 7600, but it was never used, because my machine language version was so
much better and faster. The FORTRAN version was just a development tool. But,
when we came to the Star computer, FORTRAN was far more developed, generally
speaking. A lot more people knew about it, and in fact, even I learned FORTRAN
at that time. This was 1970, so I had worked here ten years before I ever even
attempted to use FORTRAN.
GAM: That's great.
HN: I learned FORTRAN, and I also had some kind of feeling for the actual
machine language that FORTRAN was generating because I had worked with machine
language all my ten years.
So, we signed a contract to get the Star computer and began working on it. It
was supposed to be delivered in 1968. ABLE was one of the first codes that
was going to be run on it. In any case, in 1968, we first were able to
actually get our hands on the machine back in Minneapolis at the CDC plant, and
we quickly discovered that the machine didn't work, basically. The hardware
did not perform according to the specifications. Many different kinds of
instructions simply gave the wrong answer, and the ones that worked didn't work
all the time.
The new thing about the Star was that it had vector instructions. The vector
instruction would use a single hardware instruction to do a whole loop, a whole
DO loop equivalent. It would turn out that you could write a program in which,
if the DO loop length was like 500, it would work, but if it was 515 it would
fail. Somewhere along the line something would go wrong.
So at that time, instead of working on ABLE, I started working on the Star.
I started writing diagnostic programs to try to discover exactly what was
wrong, and to show to the engineers exactly how we were seeing these failures.
Then I got to working very close with the CDC people to actually write real
diagnostics that not only would say that something is bad, but would tell them
a lot about what was bad and how it could be repaired. The Star wasn't
actually delivered in what I consider a working condition until 1972, so that
was four more years after the initial time I got my hands on it. Anyway, over
that period, I devised a quite complex diagnostic program for the Star, which
supposedly tested every possible combination of every possible instruction.
One of the interesting things that I remember about the diagnostic program was
that it was very important for a diagnostic program to have a random element.
You could not write a diagnostic program that essentially ran from beginning to
end with exactly the same subset of machine instructions. If you did that, the
engineers would tune the machine up to make it work. Instead, I had to write a
program that would randomly choose the vector lengths, for example, but would
still have self-diagnosing check sums, or various ways of checks�or you'd run
an instruction, then you'd rerun the instruction to make sure you got the same
answer both times. But if you just picked, say, length 600, they would make it
work. But if I said let's choose a random number between 500 and 5000, and now
run it twice in a row, then it was a much more difficult problem, because the
machine basically had tuning problems that allowed it to work well under only
I have an anecdote from that time. I worked at Livermore, and the machine was
being built in Minneapolis. Periodically, CDC would give me a call and say
"Well, we've solved all the problems that we know about. Come back and check
it out." And they had my test program, and they'd run it. What normally
happened was that as soon as my airplane set down on the runway in Minneapolis,
the machine would begin to fail.
GAM: It knew you were coming.
HN: It knew I was coming, or so they told me. In any case, by the time I got
there, it definitely was failing. So once or twice, I just got on a plane and
came right back, and they worked on it a little more. But the Star project was
probably as complicated a computer project as was ever attempted. Maybe the
WHIRLWIND was worse�I'm sure it was worse. This was very complicated, and the
reason was because they didn't have integrated circuits. I mean, when you
wanted to do something more, you simply added on more and more hardware to take
care of this thing. And vector instructions are already extremely complicated
anyway, and, you know, they could be length 64,000 or something.
GAM: Yes. I've heard it said that if you read the first two chapters of
Iverson's book you got to know the Star programming language.
HN: Yes, called A Programming Language, or APL. The Star instruction set was
modeled after that language. But I'll tell you another interesting anecdote,
which is that the APL language itself was implemented on the Star, and it was
never successfully implemented. The Star had an APL compiler that simply never
worked in the life history of the machine as far as I know.
GAM: But it had the instructions.
HN: Exactly right, except it was only 99% APL. That other 1% was crucial. For
example, they didn't have the reshape instruction. They had sort of a reshape
instruction in hardware, but it wasn't quite exactly the same thing. It didn't
have all the variations. In order to do a reshape you had to employ many
hardware instructions, a whole complicated routine. To my knowledge, APL was
nearly always a failure, and when it did run, it was so slow and poor that
nobody could possibly want to use it, despite the fact that supposedly the
hardware basically imitated the instructions.
With FORTRAN, on the other hand, we've found you have the ability to get
probably about 50% of the maximum that you could get out of a hardware by any
kind of careful assembly language programming. With the advanced vectorizing
FORTRAN programs you could get up to 95% of the theoretical performance. So,
FORTRAN was a satisfactory language for that machine, because it gave you the
performance that you could have gotten no matter what you chose, and it had the
advantage of some compatibility. You could move it back and forth to other
machines, and you could check it on other machines.
GAM: Well, when the Star first got here, it didn't have an operating system on
HN: Yes, it did have an operating system. Let's see, I could tell some of
those stories. I actually have a book here that has my notes about the
acceptance test for the Star, because I was involved with the Star for about
five years. Anyway, it tells all, and it has some of my notes about the
attempt. Here it is. Here we go.
"The first Star arrives and the acceptance test begins on 20 October 1974.
Okay. First thing, I got my hands on the machine at 11:00. My tape could not
find the load point on Unit 3, the first note, so we had to call the engineer.
He said, 'Use Unit 4. Unit 3 isn't very good.' So we moved my tape to Unit 4.
All right. At 11:05 we're now on Unit 4. 11:06: I did a program called 'Read
Files,' which, as you can imagine, was fairly simple. That worked okay.
11:08: Illegal instruction indicator on VH test." I don't know, 'VH test' was
probably von Holdt test, I suspect.
GAM: Sounds like it.
HN: Von Holdt gave me some program to check. In any case, I have kept notes
over the years on all the machines that I've been responsible for the testing,
beginning with this machine in 1974, for at least the next ten or fifteen
years. For all the new machines that came to the Laboratory, I had the job of
giving them the "Harry test," so to speak, and to make sure that they were
GAM: Well, in all that experience, did you develop a set of tastes�which one you
liked the best, which language you liked the best, or if they're all the
HN: No, actually, as I mentioned before, the fact that you have a random
element in the test is very important. And another thing is that probably the
worst person to test a new computer is one of the people who built it. The
machine does everything the engineers wanted it to do, and it does everything
the programmers wanted it to do, or they wouldn't let you have it. But,
actually, a room full of monkeys with typewriters would probably be an
excellent testing device, if you just had some way to cross-check to see if
they were always getting the right answer. It's important, in any kind of a
complex project, that you have testers who are totally unfamiliar with
everything, because they will simply do things that nobody else had ever tried
GAM: Yes, of course.
HN: So I kind of made it a project to know as little as possible about all this
equipment when it came in, so that I wouldn't be biased to not try everything.
Then I would learn about it as I worked on it, and get more and more
sophisticated. I would still use some of the old programs and revise them as
necessary. I tried to not have any sort of predetermined way of going about
the test. I remember in the Star days, our contract with Control Data was that
they would ship the machine when they believed it was ready. Then we would
test it, and if, in any 30-day period from the beginning of testing, the
machine had remained in the condition that we called "up," for 90% of the hours
in that 30-day window of time, and the window kept moving, the machine would be
GAM: Yes, of course.
HN: For example, if it was down all the first two days, you'd basically start
over with a new 30-day window. So they had this moving target. And if
anywhere in the first 90 days of its actually being there, there was a 30-day
window when it was up for 90% of the time, then they would have considered it
to have passed the test and would be able to get paid for the project. What
constituted "up" was kind of mutually agreed on between us and the
manufacturer. Some periods were allowed to not count because of maintenance
and other factors.
So, in the case of the Star, they actually delivered two machines within one
month of each other. We got one machine in October, and the other came in
about November. So we were simultaneously doing a test on two machines. The
test began on October 20, which meant it would be sometime into January before
the 90-day window would have passed. This meant that if they didn't ever have
a period of 30 days in there when it was up 90% of the time, then they would
have been deemed to have failed the acceptance test, and then some other
provision of the contract would take place�probably penalty clauses or
something, I don't know.
In any case, the first Star came in and just had unbelievably many
difficulties, problems, errors, and memory failures, and parts that were bad
and had to be replaced. The machine was basically down 50% of the time for the
first several weeks or so. It wasn't until about mid-November that it began
being "up" for a reasonable amount of time during a given day. It still had
intermittent problems, but along about the first of December the earlier bad
time had all slipped out of the 30-day window, so they were beginning to get
sort of up into the 80% range above ground. It hovered there for a long time
(the last week in November had been a particularly bad time), but it kept
getting better, and more and more up, and fewer and fewer failures. More and
more repairs had been made, and so on, so that by the Christmas holidays, it
meant that the next week, if they could have good time, then they would be able
to ignore the part before the first of December, because that would go away,
and we'd be in a December window.
Well, it was very nip-and-tuck. One day they'd be at 88%, and then they would
drop to 87%, and then they'd go to 89%. In my personal opinion, any computer
that only has a 90% up time is a piece of junk, and you ought to throw it away
anyhow. But at that time, that was all we could ask for; that was in our
contract, and we had to accept it when it was 90% up.
Another thing was, there weren't very many people leaning on the machine when
it was brand new, first thing in the door. There were a lot of people trying
to develop a code and make it work. But only my diagnostic program, which had
been around already for four years, and under development, was really able to
fully run and occupy the machine to its ultimate. So basically, I was the only
user in some sense who was running real work.
I tried very hard to keep it running, but over Christmas I took some
vacation. I didn't come in, and I didn't run it. Well, all the time I wasn't
there, the machine was considered up, and was working beautifully. So, by the
time I got back from Christmas holidays, which was January 5, the machine was
dead, for my program. I mean, the first time I went in and ran on it, there
were things that weren't working, that were wrong, that had probably been that
way for several days. But we don't know�I wasn't there to check it.
I have a remark here in my notes, which says "System 102", which was the
number of the first Star, "was considered to have passed the test barely, as of
the morning of January 5th, which was Monday, and the first day I had come in
after the holidays." That was it. When I came in, I said, "The machine is
dead. It's bad, it's gone, it's down." And it took them five days to repair
that problem. It would have failed�at that point it fell back well under the
90% up time again�but it had already passed the test.
Meanwhile, the second machine was also under test, and it worked very well.
It did not have the problems of the first. It had some problems, but it was
probably passed with 95% up time in the first 30 days. So what had happened
was that the first one had all these problems, and was under development for
eight years or something, and they finally started another one.
GAM: Based on what they'd already learned.
HN: Yes. They'd learned a lot, and they didn't make a lot of mistakes the
second time. The other thing was that since they had gone in and fixed
everything on the first one, they messed it up. The engineers were in there
with their fingers on the chips, and it's no wonder it never worked right.
Well, those were interesting times.
GAM: Yes, I think they would have been.
HN: I have a little comment here that one morning we opened up the back console
and we found a fried mouse that had gotten into the circuitry somehow and had
GAM: That was like that Grace Hopper finding a bug inside the Mark 1�a
real bug, a beetle.
HN: We had a mouse�we didn't mess around with no beetles.
GAM: Going back to the Stretch, did you ever have much to do with any of the
designers from IBM? Did they come out and talk to you?
HN: No, in the case of the Stretch, I didn't. I worked with Norm Hardy, who
did have a lot to do with them.
GAM: He'd gone back to New York to work on it, I remember.
HN: Yes, but other than Norm, no, none of the engineers. The first time I
really got involved in it was the hardware side of things with the Star. But
then, of course, I continued being involved with it with the Cray 1 and the
other Cray machines that followed.
GAM: Well, computers have sort of come a long way.
HN: Well, nobody ever gets one fast enough. I mean, the "Jim Frank Law" always
applies. When your problems get running fast enough that they only take ten
minutes, then you put more physics in them. They're not interesting anymore,
so you make them harder. I doubt that this will ever be overcome.
GAM: Right now we're entering an era of parallelism of sorts. What's your
attitude about that?
HN: Well, all these machines from at least the Stretch days had a lot of
parallelism in them. It was a very low level parallelism; that is, you had
instruction decoding going on, you had arithmetic going on, you had I/O going
on, and so on, all at the same time. Simultaneously, you had checking
circuits, and these were all working in parallel. In the case of the 6600, it
was the first time that the instruction itself had been broken down into
parallel segments so that you could have several hardware instructions
proceeding in parallel, each in different stages. It may have been done
earlier by other people, but it's the first machine I worked on that had that
As computers have progressed, they've gotten more and more parallel. The
thing that is different today is that the parallelism is at a higher level.
You have entire computers running in parallel with each other and communicating
and so forth. In the first place, you have 64-bit parallelism in a word. All
the bits move simultaneously, and then the whole arithmetic is done
simultaneously. All these things have made it possible for the machines to run
faster. I mean, the speed of light hasn't changed since the 1960s, and in
fact, the speed of electricity moving over the wires hasn't changed very much
either. It actually has improved slightly in the last thirty years, but not
But the things are closer together. The time it takes to get from one place
to another is the same per inch, but there aren't as many inches. So a modern
circuit would do the same thing in a quarter of an inch that the Stretch used
to need 72 feet to do. In fact, the Stretch itself was de-rated from its design
goal of 200 nanoseconds per clock tick to 300 nanoseconds per clock tick,
because it got so big that the signal simply couldn't progress down to one end
of the machine and back in 200 nanoseconds. It took 300, or whatever the
amount of time was involved.
GAM: It was a big machine, but the Star was bigger.
HN: The Star was even bigger, right. And that same problem did happen for the
Star, also. It had some similar problems�it simply got too big. Not that
Livermore ever learned anything, of course�they built their own Star later,
called the S1.
GAM: That was terrible.
HN: Well, it's what you get when you try to put everything into a machine, and
then you add stuff besides. When you realize at the last minute that you don't
have everything yet, and you've got to put more into it, then you lose
GAM: That was contrary to RISC (reduced instruction set computer).
HN: Right. And meanwhile, other people were doing RISC-type projects where
they were really trying to see what hardware they could take out, and leave to
the compiler, and leave to the operating system. Get it out of hardware!
The Stars were trying to pile the whole language of APL into the hardware. It
didn't work very well. But the Stars eventually became reasonably reliable,
and were a very important addition to the Laboratory's equipment. They were
used for many years to run very important physics problems with great success.
GAM: What do you mean by many years?
HN: Until at least 1978, so I'd say five years. It at least had a five-year
life in which they were the fastest machines available for most of the work.
GAM: In your opinion, did these people who vectorized so thoroughly produce a
fairly fragile code, or not?
HN: No, I don't think so in particular. On the Star, in order to get any kind
of performance, you did have to have a highly vectorized code. What that
simply meant was that you tried to break down a piece of physics into loops
which did very little in each loop. In other words, instead of saying, "take
something and add, multiply, look up a random number, generate, this, that, and
the other thing, all in one loop," the loop became, "do a hundred thousand
adds, do a hundred thousand multiplies, do a hundred thousand random number
lookups, do this," and so on. It was just sort of a matter of the amount of
scale that you put into one loop. But the code was just as readable, I think,
generally speaking. It wasn't exactly the same as the physicists conceived it,
but it was fairly straightforward to do that.
Now, later on, the compilers got good enough that they could do that
themselves. You could write the code in the other order, and the compiler
would, in fact, break it down into those small loops by itself. That made it a
little more readable to the authors. To the programmers it didn't make very
GAM: I was thinking that when you get the thing very tightly vectorized, and
then you suddenly decide you've got to put in this additional piece of stuff on
a given physics loop, then all of a sudden you're all out of sync.
HN: Well, you are if you have to now have a new data structure and a new way of
thinking about it. Sometimes it's more complicated, sometimes it isn't.
GAM: That didn't happen too much.
HN: It didn't happen too often, particularly for the important applications
which hadn't changed a lot over the years. We were still running ABLE in
1978, and it was written in 1960, or in the 1950s. In fact, it was only
beginning to get extremely heavily used in those days.
GAM: Thinking back about the experiences, did you ever develop a taste for
saying, "My God, I wish I had this instruction on this machine," or something
like that�but you didn't quite have?
HN: Well, yes, actually some of that happened. In particular, I remember a
case with Seymour Cray machines. The Cray 1�well, I can look on my notes to
see when the first one was delivered, since I helped do the acceptance test.
GAM: Please do.
HN: Let's see. Here we go�Cray 1, Serial 10. That was the first one that we
were doing acceptance test on. It says "January 2, 1979," Okay, so that's when
that came in. And here's the first comment, "19:28." So that would have been
7:30 at night. "All Appendix A tests okay. Started Appendix B memory tests."
At 20:20 (20 after 10:00), "six memory parity errors occurred." But these were
correctable�on this machine we now had memory correction, so the fact that six
errors happened meant nothing. They were simply corrected and continued. And,
then finally, at 20:37, "High-temperature alarm bell rang due to overheated
power supply." Then I was able to go home for that night while they fixed
that, and I started over the next day. But the Cray 1 passed its acceptance
test on the 30th day, the earliest possible time, and was working very well.
But one thing that the Cray 1 did not have was an instruction called
"gather/scatter," We'd had this instruction on the Star, and it proved very
valuable for certain kinds of physics applications where it was necessary to
sort of reorder your data from time to time in order to get efficient codes.
And in one particular code that I knew about, they basically spent half of
their actual compute time simply reordering the data, which, had there been a
special machine instruction to do that, would have happened a lot quicker. So
we did notice that, and we were able to make the case for the fact that we
needed this instruction which would be very useful.
We went back to Chippewa, Wisconsin, and talked to Seymour Cray about it, and
explained to him what we needed, and what we thought the improvements would be,
and so forth. And in the next Cray, the XMP Cray, that instruction was added
to meet our request.
GAM: But that's not Seymour's machine really�
HN: Well, at that time he was still helping design the second model. In fact,
now I recall�I did have the facts slightly wrong. Seymour was working on the
Cray 2 project at that time, and we got him to add gather/scatter to the Cray
2. This meant that in order to compete, the XMP designer, Steve Chen, had to
put it in his machine. So we got it in the XMP, since the LC (Livermore
Computing Center) never did get a Cray 2. I'd forgotten that it was the initiator. But we made our argument correctly, it seemed.
GAM: Did you do the acceptance checking out for NERSC (National Energy Research
HN: No, but I did help. I did contribute some programs to NERSC, and I did run
on Cray 2 in early times, but I wasn't a member of NERSC and I didn't
officially have anything to do with their acceptance. I did help check out the
first XMP that we got on February 8, 1984. And among other things, it did have
GAM: Now we've passed the history period at that point. Well, I'm trying to
flesh out the stuff from the first thirty years. All the names, and the
notebooks like the one you've got there�those are just very important.
HN: All right. You know, throughout my career, I was always interested in
mathematics as such. I have a degree in mathematics. I have a lot of fun with
it, particularly recreational mathematics�various things like prime numbers,
largest prime number, and so forth. As I mentioned, in order to do diagnostic
programs on computers, the very best thing is to do new, different things that
no one ever thought of before, particularly the engineers who built it. So, it
made it possible to use the latest and best hardware at a time when it wasn't
very busy, to do interesting problems like find the world's largest prime
number or something of that nature.
I've gotten involved in a couple of those projects over the years. We put
together a program that applied the appropriate algorithms for testing very
large numbers to see if they were primes, and we surrounded that with careful
diagnostic routines that would do cross checking, self checking, and so on.
Those routines were actually quite useful to the project at hand, which was
getting the machine in good condition so that we could accept it. It meant
that we were able to do fun, interesting stuff at the same time we were
carrying out our mission, and that was quite interesting.
GAM: Well, I'm remembering that beautiful poster that you had made of the 27th
HN: Right. In the case of the Cray 1, Dave Slowinski and I were actually able
to find what was the largest prime that was known to anyone in the world at
GAM: At that time.
HN: It's been surpassed many times since, but that was the largest then. The
time for that calculation was available because the machine was in its
pre-acceptance phase, and no one had codes ready to run on it. This code was
ready to run, and simply ran in the background when nothing else was going on.
And it used probably a thousand hours of compute time. But all that time, the
Cray 1 would have been idle otherwise. And, as I say, it did have
self-diagnosing features to it.
GAM: Well, just as a matter of interest, was it instrumental at any time at
showing a bug?
HN: Yes, oh yes, absolutely.
HN: Several times we had stops�the program would halt, and say, "something's
wrong here�call an engineer!"
GAM: I love it. And then there's your chess stuff, you know?
HN: Ah, that came about 1980. Yes, well, that's in the same era, right? Do
you want to go to 1984, roughly?
HN: Okay. Another interesting project that I worked on involved the Cray 1
also. A group lead by Robert Hyatt, who was at that time a student in his
senior year at the University of Southern Mississippi, had written a
chess-playing program in FORTRAN. Actually, most or all previous chess-playing
programs were written in machine language, because it's very important for
chess programs to get the greatest speed they can out of the hardware. In any
case, Hyatt had this FORTRAN program running on various machines at the
university there, and he was looking around for a faster machine. So he had
the idea of contacting Cray Research to see if he could get time on the new
Cray 1, just coming out, so it would run a lot faster. So, he did, and they
The marketing department said, hey, this would be great�let's get a chess
program on here, and so on. Because it was written in FORTRAN, you just had to
make a few changes to get it ready for Cray FORTRAN to do the little details.
So he put it on, and he was using it, and running on a Cray machine, in the
computer chess tournaments, which are held annually�at that time in connection
with the ACM meetings.
Well, I was very much involved with the acceptance of Crays, and checking
them. I'd heard about this program and was interested in it. I got a copy of
the program to run at Livermore. That was part of the deal that Hyatt's group
had with Cray�they could use the Crays to develop their program and run it,
but they had to make it available to any customers who would like to have it.
HN: That was sort of the payment. So I got a copy, and I checked it out. I'd
already had a chess program that worked on the 7600 that was written many years
earlier. I got this new program, and I discovered that it was slower on the
Cray 1 than the old 7600 program had been on the 7600. Well, the earlier 7600
program was written in careful machine language, whereas this was FORTRAN, and
it was sloppily done. It wasn't really sloppily done, but it didn't utilize
the hardware features of the Cray.
So, I called the guy up and volunteered. I said, "Hey, I know something about
this. I can help you, I can make some changes, and here's an example." So I
rewrote one of the programs, one of the subroutines that I knew was taking a
lot of time, and sent it to him. And I said, "Why don't you just substitute
this for what you've got?" And lo and behold, the program as a whole now is
suddenly running 25% faster than it was before. And this was about a ten-line
rewrite of one subroutine. So they immediately called back and said, "Hey,
this is great! Let's do some more!" So I developed, really, a very
interesting relationship with these people. And this was about 1980.
Meanwhile, their program always had done poorly in all the chess tournaments
prior to that time, because they didn't have fast hardware. And because it was
in FORTRAN, it probably wasn't taking really complete advantage of any of the
hardware they were running on. Well, in any case, now we had a definite
goal�to make this program work well on this specific machine.
So, what I did was to run it a little bit, do some timing studies, find out
where it was actually spending its time, and suggest revisions and ways to
improve it�put a table in where they were doing a computation, or revise the
size of this array, etc. And the Cray 1 had millions of words, you know, as
opposed to whatever they were using before. Well, the first one had a million
words as opposed to a hundred thousand or whatever. We could expand certain
things and so on.
Well, the upshot of it was, after basically a three-year effort, we had gotten
the code to run 25 times faster than the original version that I'd seen. In
1983, we also were going to use this version, which has been greatly improved,
in the so-called world computer chess championship.
We also had the XMP II at that time, which is the first supercomputer with two
processors. So we were able to do some parallel things. We did a quite simple
thing�just basically, if there were two possible moves you could make, one
processor tried one of them and one tried the other. And they didn't
communicate very well, either. It was crude, but it was effective. And in
1983, then, the Cray BLITZ program, actually, won the world championship
against BELLE, which had been the champion.
GAM: Yes, that was Ken Thompson of Bell Labs.
HN: Ken Thompson of Bell, yes. It was the first time BELLE had ever lost in a
computer chess event of that nature.
HN: We continued to make improvements and we got faster machines,
four-processor machines. We learned how to use their parallel nature and stuff
much better, so that by 1986 the program was again sped up by another factor of
5 or so. And we repeated as world champion in 1986 in Koln, Germany, where the
tournament was held. Since then, we've been soundly beaten by many people with
new projects and new hardware and so on, but a lot of interest was generated in
those days by our program.
GAM: That's great.
HN: In any case, I would run, basically from my office in Livermore, on a
machine in Minneapolis. That's where we did our development work that Cray
Research Marketing Division provided. It was another interesting project.
GAM: Yes, I think it's very interesting, and it had a certain kind of
HN: Visibility, yes. And even today, computer chess is still quite
interesting. Computer chess programs are much better�one of them is rated
even at Grand Master level, but is still not as good as the human world
GAM: I understand.
HN: But it's a goal.
Well, I guess the other thing I didn't mention, George, was that I worked on a
Stretch back in the '60s, till maybe '65 or so. And by that time I'd
gotten�foolishly�onto the management track.
HN: I started helping the management of codes like ABLE on various
processors, in particular. And I kept doing this until about 1970, for about
five years, and finally decided that it wasn't my direction.
HN: That's when I got involved in the diagnostic programming of the Star. But
somebody's got to do the management. It didn't have to be me, of course.
GAM: Right. Well, I'm not sure someone has to do it�not in a research lab.
HN: Well, I disagree. I think somebody has to at least keep their fingers on
GAM: Maybe so.
HN: I'm not sure how, though.
GAM: So, presently you're sort of still trying to explore the limits of the
capabilities of these parallel, larger machines.
HN: Right. It seems that now it's becoming more and more difficult to get more
parallelism into these machines at the lowest level.
HN: Now, that's pretty much been entirely exploited. We can always have faster
circuits, and we'd be glad to incorporate those as long as they allow this
parallelism. But we haven't exploited the higher level parallelism to its
GAM: Well, it's probably a language problem.
HN: It's a language problem. It's also perhaps a hardware problem to some
extent. I like to look at it this way: There's this triangle, that you made
famous at the Salishan Conference, of hardware, software, and application.
And, sort of, the parallel programming problem, parallel computer problem, can
be looked at from those three aspects.
Look at it this way: Suppose I'm a computer manufacturer. What I say is
this: "You give me a lot of money, and I'll develop the perfect hardware to do
parallel programming." Or I'm a company like, say, NASA/Ames. I say, "Hey,
I'm an applications guy. I've already got the applications that are going to
run on this machine. You give me a lot of money, and I'll run these
applications on the best machine I can buy." And then you take a place like
Lawrence Livermore Lab, which has software capabilities. What they say is,
"Well, we know there's certain problems with parallel machines, difficulties
and so on, but you give us a lot of money, and we'll write operating systems
and languages that make it very easy to run all the applications you'd ever
want on any old piece of hardware that these guys provide." If you didn't
notice, there's one common theme among these three groups.
HN: You give me a lot of money, and I'll take care of the problem.
GAM: Very good. Oh, that's great! Okay, I want to hear your favorite story
HN: Okay. Well, you know I'm over 60. I've been around a while. I'm one of
the few guys, today, whose grandchildren�when they went to school�were able
to say, "My grandfather was a programmer!" But, in any case, you know, people
come to me and ask for advice, and I'm glad to give advice. I mean, they
probably don't listen anyway. I have a favorite story about giving advice that
I think is important.
When I was young and leaving home for the first time, my father came to me and
he said, "Before you leave, I want to give you four pieces of advice." He
said, "And here they are. The first piece of advice is: 'Never get married.'"
"Now," he said, "The second piece of advice is: 'After you get married, don't
have any children.'" And he said, "Okay, that's number two. Now, here's the
third piece of advice I have." He said, "'After the first child is born, don't
have any more.'" And so I'm beginning to look askance here, and finally he
said, "And now I'm going to give you the fourth piece of advice, which is
really the most important: 'After the second child is born, for heaven's
sakes, quit taking my advice!'" That was it�quit taking my advice!
GAM: Great! Where did you grow up?
HN: Oh, I spent the first 18 years of my life in Topeka, Kansas, or the nearby
environment. I went to high school there. Then I applied and was accepted at
most of the Ivy League schools, and I went to Harvard. I took an A.B. in
mathematics, which, I thought, was interesting.
GAM: Oh, I didn't know that.
HN: Then I was drafted right after that. And that was in 1953. I went to the
Army for a couple of years. By that time I was married, and then I used the GI
Bill to go back to graduate school at the University of Kansas, in Lawrence,
Kansas. I got a master's degree in mathematics, and spent a couple of more
years teaching there and working on a Ph.D. But I never completed it, and I
still don't have it. 
GAM: I don't think it matters anymore.
HN: Not anymore, no.
GAM: What was the name of the little company down in the Los Angeles area where
you worked prior to LLNL?
HN: Oh, yes. In 1959 I finally decided (having a couple of kids by that time
also) that it was time to go out and start earning some real money instead of
living off of the GI Bill, which had run out anyhow�and stop this TA, partial
teaching, stuff. I looked around. And at that time the work for
mathematicians�other than teaching, for which you had to have the doctorate
pretty much to get anything reasonable�was in the burgeoning, new computer
industry. And I had several offers of work from various people. One of them
was for a company called Autonetics, in California.
GAM: Autonetics�yes, I remember them.
HN: Autonetics was an offshoot of North American Aviation, which was making
computers for their aircraft. And they wanted to spin off a civilian division,
which they called Autonetics, which was to adapt their airborne computers for
commercial use. Anyway, I went there.
Editor's note, added June 9, 2006
One of the regular columns in the Tentacle (see page four for some samples of Tentacle
issues) was a set of puzzles supplied by Harry Nelson, who it turns out is one of the
world's most prolific puzzle creators. Annually, he invents and sells a charming
collection of puzzles often based on geometry, or algebra or logic, and it seems that
many of the readers of these interviews would enjoy examining a small selection of his
A word of caution: the three selected for inclusion here may seem easy, but don't be
fooled. When you think you have an answer, you can look on Page 2 for the correct answers.
A Sample of Nelson puzzles:
How should 36 be partitioned into positive summands so that their product is
maximized? [Example: 36 = 24 + 12; 24 * 12 = 288.]
- 100, which equals ten to the power two, can be factored into two
factors, 4 and 25, neither of which has any zeroes. What is the SMALLEST power of
ten that CANNOT be factored into (exactly) two factors, neither of which has any
- What is the LARGEST power of ten that CAN be factored into (exactly) two
factors, neither of which has any zeroes?
- By referring to any standard dictionary, one finds that any given letter,
with the exceptions of j,k and z appears in the "spelled out in English" form
of one or more of the positive integers. (Zero is not a positive integer.) What
is the smallest positive integer containing all 23 possible letters?
 The name "ABLE", which was never used for a computer
code, will be used instead of the actual code name.
 Harry's wife, Claire, who also reviewed this manuscript, added
that Harry wanted to do his Ph.D. thesis on computers but that the Math
Department did not approve of that topic.
For information about this page, contact us at: email@example.com