source: https://www-cs-faculty.stanford.edu/~knuth/email.html

Not that email or any of the million other apps don't produce constant interruptions - but in terms of what I want my role to be and what I find pleasure in.

A world of high stimulation and constant social interaction? A million things shouting at me and I have to sort through them and grab the most important one and hop from thing to thing fast? That's what I like.

Focusing on one topic for hours, days, even weeks to really understanding every facet of it? I think it'd kill me. I know people theoretically need these big blocks of uninterrupted time to get work done, but I'd much rather have a bunch of meetings where we can figure out the path forward.

I have mad respect for him... but I never could have been an academic.

I like to take in all those digested kernels of knowledge, I like to digest as many of them as I can; I know I lose the details, but integrating them all into something bigger has always been my dream; it's just how I live my life.

The problem I find with surface level knowledge is you make a lot of mistakes. When I have a surface level knowledge of a piece of tech and make a decision based upon that, then talk to someone with deep knowledge it's pretty common for them to say "why didn't you just do XYZ, it's easier/more robust".

Ultimately you can produce OK or even sometimes good solutions to problems but not great ones, unless you have direct access to people that know more than you.

This becomes even more pronounced when different pieces of tech can be combined with eachother.

I'm ADHD (diagnosed, etc), and medication did help, but didn't solve it. For much of my life, I fought against this natural tendency. But for most part I essentially kept telling myself "pay more attention"! It didn't work.

Compensation is important. Detecting the mistakes I make, and avoiding situations where the cost of them is very high. One part of that is accepting things I can never do, like be an academic :).

In the engineering world there's lots of means of compensating. One you mentioned; knowing people who know more than you. Another one would be automated testing; at almost every workplace I've had, I have put a lot of effort into expanding and increasing it, and generally detecting errors wherever they might occur. I'm always asking "what if there's a typo somewhere, will we know, will that go to production?" Might I create a PR I think I've tested but I haven't? The odds I've done this are higher than my peers, so I need to protect myself from that.

If I'm at a workplace that values accuracy above else - that views being able to focus on the code and get the best (not "good enough") solution to every problem as the main trait that I'm evaluated on, I will not succeed. I'm never going to be the guru of anything. This kind of sucks, because a lot of places view being really good at those things as necessary before you can move to another area (IE, management) where you don't use those skills.

I'm a "good not great" or "move fast and break things" kind of person. My usual working style is to produce prototypes very fast; then whoever I'm making it for can see that I didn't quite read the design thoroughly and miss something before it's a huge disaster.

I need the feedback of fast release cycles. I can detect errors fast, take action fast, fix them fast. But "take your time and make sure it's perfect before you throw the switch"? That's just not my personality. I'm high volume (when not artificially constrained) but error prone.

I'm smart enough to know it, smart enough to insist that my code gets reviewed, smart enough to say we really have to test for these cases, things like that. But it's absolutely something I have to compensate a lot for.

A solution that’s worked well for me is to only study deeply when I have a concrete need.

For example, when I felt my work suffering because I didn’t deeply understand z-index, it was easy to sell my brain on reading the complete z-index spec.

One of my hobbies is music, and I’m just starting to feel the limitations imposed by not understanding music theory deeply. That means it’s a good time for me to study music theory.

Some people feel compelled to learn things. I feel compelled to do things, and if I can’t do something as well as I want to for lack of knowledge, then I know I’m ready to learn things.

Confuses the hell out of employers, too. "You did a great job on X, you really improved things, but when we put you on Y, you totally halfassed it" - because X was my idea, and Y was forced on me. I respect the (social) contract that you are my employer and I should do what I'm told; I know at least mentally I should try to excel on any task given me, but it's just so hard to invest in those kind of things.

Anyone can get demotivated; it's just that the gap for me is bigger than people are used to. I'm not blaming my employers here, to be clear - thinking that way does me no good, anyway(+). While the vast majority do a terrible job of recognizing the motivations of employees - and anyone can get demotivated - I'm going to be way worse at this than the average employee and it's up to me to compensate.

Another strategy: I learned early on ANY social context is motivating for me. So I try to schedule regular checkins, even if the org doesn't mandate them. Even a few minutes of a manager explaining the next project will yield much better work than getting a ticket thrown over the wall. Constantly engaging with others and feeling like they are personally depending on me is one of the most motivating things out there - so I have to keep reminding my bosses that YES, I do want to be in all those meetings.

I will say on the subject of learning - I mostly agree with you. I learn all the time in a huge variety of fields, and when I need to learn new tech to accomplish a task I relish it. But if I try to make myself learn something because it would push my career ahead? Force myself to devote some number of hours to a project in a new language, something like that? It's like pulling teeth. I can't get into it.

(+: In general, if the world discriminates against you, is difficult for you, etc - whether or not it's truly worse for you or a perception you have, it doesn't matter to you - what matters is how you react to it. Sitting at home crying how about how it's not fair will never benefit you.)

> I need the feedback of fast release cycles. I can detect errors fast, take action fast, fix them fast. But "take your time and make sure it's perfect before you throw the switch"? That's just not my personality. I'm high volume (when not artificially constrained) but error prone.

So true, and it's made past jobs very hard work, most especially before I was diagnosed at age 31 and had the benefit of medication and being able to formulate better coping strategies :)

Reminds me of a "hermit-like wizard in his tower in solitary uninterrupted study of their craft."

Can I ask how has it benefited your thinking?

I'm curious to know not only how it benefits an understanding of software development, programming languages and functional programming, but also has it helped you understand the world at large in new and novel ways?

I should warn you that there is a social bias towards saying that you found a lot of insight in studying something, and a bias against saying that studying it was a waste of time, irrespective of whether or not the thing is actually helpful. Aristotle could stare at a rock and end up with some insight, so anyone who stares at a rock and admits it hasn't helped them has to publicly admit they are not as smart as Aristotle. They say this is how geology got started. ;)

He rejected atomic theory, believed the Earth was the center of the universe and the four fundamental elements were air, fire, water, and earth, and thought heavy objects fell faster and moving objects would tend to come to a stop if nothing interacted with them.

These were observations that Aristotle and his audience accepted as "obvious". It took more careful and patient observers to discern the less obvious truth.

My goal is to become a better thinker in general. Specifically with regards to seeing how one type of thing relates to another type of thing.

I'm actually more interested in better understanding and reasoning about the ecology of software development than becoming a better programmer per se but only because on some level I think that's actually necessary in order to achieve the writing of better software.

To put that into English... we don't write software in a vacuum. We write it in a team, in a company, in a world full of other programmers and there is a rich set of relationships among all of those things that is exerting pressure on the final shape the software takes.

Category Theory seems like fun and useful tangent that might help me understand the world better and be mildly useful in an abstract way.

Part of good software development is knowing where and when to create abstractions. Category theory has a collection of interesting and very general abstractions - if you can spot these patterns in your software development then they will be good candidates for abstraction/generalisation

It did and does that too (not much, since I am still a learner): It helps me to write APIs that behave well; usually, that means being an API which outward facing parts compose well. (The opposite would be an API where you configure it's internal state until it does what you want - the more complex the API, the harder it gets to mantain the suiting mental model). To be honest, it will help me getting the theoretic motivation behind many FP-concepts like monads, but I am not yet deep enough, I would not say I reaped a practical benefit in my haskell-writing. Reading (and implementing) "Category for programmers", which is a great blog, should take me there tho.

> but also has it helped you understand the world at large in new and novel ways?

On a local scale, it was fascinating to see how you get logic and algebras by putting a little bit of additional order onto some abstract concept. These formal logic rules I leard? Turns out they are not only funny ways to play with symbols, but also funny ways to play with arbitrary finite sets. It was also nice to see how in category theory, you can "fix" nice properties by creating a new structures where these nice properties are objects. It is all pretty handweavy, since I am mostly going by my intuition unless I am pressed to prove something formally. This thread might help you: https://www.reddit.com/r/haskell/comments/16i322/whats_the_i.... Globally, I am a christian, so I see everything and everyone in relation to God anyways. Category Theory might at best helped me to identify God as the intitial and terminal object in the Category `Human` ;)

To give a short answer: I think CT is another tool in the box to gain a more connected view of the world, the main ingredients being age and observation of processes.

Interestingly, the Chinese students in that class did fairly well, even though they could not follow the actual lectures (given in French) and were studying exclusively from the lecture notes (given in English).

When I was at Stanford with the AI project [in the late 1960s] one of the things we used to do every Thanksgiving is have a computer programming contest with people on research projects in the Bay area. The prize I think was a turkey.

[John] McCarthy used to make up the problems. The one year that Knuth entered this, he won both the fastest time getting the program running and he also won the fastest execution of the algorithm. He did it on the worst system with remote batch called the Wilbur system. And he basically beat the shit out of everyone.

And they asked him, "How could you possibly do this?" And he answered, "When I learned to program, you were lucky if you got five minutes with the machine a day. If you wanted to get the program going, it just had to be written right. So people just learned to program like it was carving stone. You sort of have to sidle up to it. That's how I learned to program."

[0] http://www.softpanorama.org/People/Knuth/index.shtml

In the 70's I had a co-worker, perhaps the best programmer in the department, that had gone to school with Knuth. He told me that one day while in college Knuth was using one of the available key-punch machines to punch his program on cards. My friend was ready to punch his program so he stood nearby to wait for Knuth to finish. Knuth, working on a big program, offered to Keypunch my friends program before finishing his own because my friend's program was shorter and Knuth could keypunch quite fast.

While watching over Knuth's shoulder, my friend noticed Knuth speeding up and slowing down at irregular intervals. Later he asked him about that and Knuth replied that he was fixing the bugs in my friend's Fortran as he punched it out.

I think you mean WYLBUR.

I had the "opportunity" to work with WYLBUR once in 1993, and I remember it to this day. "the worst system with remote batch" dramatically understates how bad it was. Hearing this raises Knuth even higher in my estimation.

https://en.wikipedia.org/wiki/ORVYL_and_WYLBUR

How funny, because Wylbur was a large improvement on the default at the time, MVS's TSO (Time-Sharing Option). Wylbur was miles easier and faster than TSO.

So, not actually 'the worst system with remote batch' because that trophy would go to TSO.

http://www.leancrew.com/all-this/2011/12/more-shell-less-egg...

The tl;dr is that Knuth wrote an elaborate implementation of a program to solve a particular problem, and Doug McIlroy replaced it entirely with a six step shell pipeline. (Knuth's program was written using his literate programming tools, could be typeset in TeX, and involved some precise work with data structures and algorithms.)

I love this story as an example both of Knuth's genius and perspective, but also as a way to show what his level of dedication can achieve. It's an amazing intellectual accomplishment.

I also love this story as a demonstration what those of us without that skill and dedication can achieve using the advancements built on the work of Knuth and others.

- Bentley, the author of the column, invited Knuth to demonstrate literate programming using a program of his choice.

- Knuth insisted that to be fair, Bentley ought to specify the program to be written; else someone might object that Knuth chose a program that would be good for literate programming.

- Bentley chose (what we'd now call) the term frequency problem (list the top k most frequent words in a text file), and accordingly Knuth wrote a system program for solving just this one particular task. (Did everything from opening the input file to formatting the output, etc.)

- Doug McIlroy was asked to “review” this program, the way works of literature are reviewed. He happens to be the inventor of Unix pipes. Towards the end of his review, along with many other points (e.g. Knuth didn't include diagrams in cases where most of us would appreciate them, something I still struggle with when reading TeX and other programs), he used the opportunity to demonstrate his own invention, a shell pipeline using now-standard Unix tools (tr, sort, uniq, sed).

There are a few things wrong with this criticism:

- The main thing is that DEK wrote the program he was asked to write, so pointing out that he shouldn't have written that program is a criticism of the one who chose the program (Bentley mentioned this when printing McIlroy's review).

- At the time, Unix wasn't even widely available outside Bell Labs and a few places; it definitely wasn't available to Knuth or most of the column's readers.

- Knuth's program, fine-tuned for the task, is more efficient than the shell pipeline.

- Even if you use a shell pipeline, someone has to write the “standard” programs that go into it (the "tr", "sort", "uniq" and "sed" above), and literate programming can be used there. In fact, Knuth did exactly that a few years later, rewriting Unix's “wc” (IIRC) and compared the resulting program with that of Sun Unix's wc, and his LP version had, among things, better error handling. (He's explained it by saying that in conventional programming, if you have a small function and 90% of it is error-checking, it looks like the function is “about” error-checking, so there's a psychological resistance to doing too much of that, while with LP you move the error-handling to a separate section entirely about error-checking and then you tend to do a better job. BTW, TeX's error handling is phenomenally good IMO; the opposite of the situation with LaTeX.)

All that said, there is some valid criticism that Knuth prefers to write monolithic programs, but that works for him. He seems not to consider it a problem that to change something you have to understand more-or-less the entire program; he seems to prefer doing that anyway (he reads other people's code a lot: in Peter Seibel's Coders at Work, he was the only person interviewed who read others' programs regularly).

At that time, 1986, Unix was widely available - there were more than 100k Unix installations around the world by 1984. AT&T Unix Sys V and UCB 4.3bsd were available. Knuth was friendly with McIlroy, who was head of the Bell Labs computer science research group that begat Unix. Sun Microsystems was formed in 1982 - their boxes ran Unix, and Sun was a startup spun off from Stanford.

Right. Even their name was derived from Stanford University Network:

https://en.wikipedia.org/wiki/Sun_Microsystems#History

Even today, if you would get the exactly same task, with the goal to make the most efficient solution when you have to care about the limitations of hardware available to you and to produce the self contained program (e.g. because your algorithm should run with hundreds of billions of words of input) you'd still at the end probably produce something closer to what Knuth did then what McIlroy did.

Which doesn't mean that it's not brilliant. But it's also not obvious, i.e. not something a "normal user" would "know":

- even if you knew that "tr command translates the characters to characters" did you know that you could (and must) write

   tr -cs A-Za-z'
   '

- did you know what the fifth line was supposed to do "sort -rn" Would you know that you're to sort "numerically" (-n) and that it would "work"?

- "sed ${1}q" how many people even today would know that one?

And after all that, the first of two sorts needs to sort the file that is as big as the original input! If you have hundreds of gigabytes of input, you'd have to have at least that much more just to sort it. McIlroy's approach is a good one for one-off program or not too big input processing, and if you knew that you can use these commands as he used them. But it's still not "a program" in the same sense a Knuth's program is.

Knuth's algorithm would, unsurprisingly, handle the huge inputs orders of magnitude more efficiently. And that is what McIlroy was aware of and intentionally hand-waved it in his "critique." Read the original text:

https://www.cs.tufts.edu/~nr/cs257/archive/don-knuth/pearls-...

But the major point is still: Knuth's task was not "use the existing programs" (or libraries) but "write a program" that does what's said to be done: The fair comparison would then include the source of all the sort, uniq, tr etc. programs which McIlroy used.

And once that is being done, McIlroy's code would still be both less readable, less efficient and worse overall.

Which on the other side also doesn't mean that for some purposes "worse" isn't "better": https://yosefk.com/blog/what-worse-is-better-vs-the-right-th... But for some purposes, on "better" works and "worse" simply doesn't, e.g. when the scale of the problem is big enough. And Knuth teaches us how to solve such, harder problems. And presents the complete solution v.s. doing tricks (just call that library/executable which I'm going to avoid to explain you how it is implemented and what its limitations really are).

And giving the misunderstanding in the difference between showing how something is implemented (most efficiently) and that "just use pre-written tool X" approach, I understand even more why Knuth uses assembly in his "The Art of Computer Programming" books.

> - even if you knew that "tr command translates... "sed ${1}q" how many people even today would know that one?

Are you suggesting it's ever been more likely for people to understand how to manage a trie structure in Pascal than use Unix command line tools? Or look flags up in the manpages?

Personally speaking, I'm comfortable doing both, but can't imagine many scenarios where I'd rather have ten pages of a custom datastructure than six lines of shell. (And they all involve either high volumes of data or situations where I can't easily get to the process-level tools.)

> The fair comparison would then include the source of all the sort, uniq, tr etc. programs which McIlroy used.

If you're including the code that provides the surface abstractions, where do you draw that line? If the code for sort, uniq, etc. is fair game, why not the code for the shell that provides the pipelining, the underlying OS, the file system, the firmware on the disk? After all, who's to say that the programs in the pipeline don't just run one after another with temporary files written to disk, rather than in parallel? (Which I've seen happen in reality.)

The same is true for the other side, of course. The 'fair comparison' could easily require Knuth's solution to include the source for weave/tangle, TeX/Metafont/CMR, the OS, etc.

> And once that is being done, McIlroy's code would still be both less readable, less efficient and worse overall.

What definition of 'worse' are you using?

* I expect sort/uniq/tr/sed to be more well tested and understood than a bespoke program.

* If there are issues with the program, it'll be easier to find skills/time to maintain a shell pipeline than custom trie-balancing code written in Pascal. (Sitting aside a prose description of the same.)

* The shell pipeline runs on a machine that can be found in a retail store today, rather than requiring an elaborate download/build process.

* It's possible that the custom solution runs faster, but not obvious without testing. (None of which is worthwhile outside of demonstrated need.)

Point being: it's very easy to find a definition of 'worse' that applies more to the custom solution than to the pipeline.

That argument points to the fact that your “view” of the whole topic changes the assumed definition of the problem that was given to Knuth to solve. Read once again the original text: he was supposed to illustrate how the “Literate programming” could be ised while writing a program which solves a given program. It was definitely not “write an example of calling the existing pre-written programs”.

And, of course, it was all in 1986, definitely not “to target the machine which can be found in the retail store in 2018.”

McIlroy already behaved as the goal had been different than it was.

How would you feel about McIlroy's solution if it was semantically exactly the same, but written in a literate approach? (Essentially a version of 'weave/tangle', but for shell scripts.)

In the context of that requirement, the it's the use of literate programming that's more of a concern than the specific implementation. (Which is why I asked about a literate version of the shell pipeline.)

Earlier in the thread, you also mention this concern around data volumes:

> If you have hundreds of gigabytes of input, you'd have to have at least that much more just to sort it. McIlroy's approach is a good one for one-off program or not too big input processing,

There, your concern is not justified by the stated requirements of the problem: "I did impose an efficiency constraint: a user should be able to find the 100 most frequent words in a twenty-page technical paper"

I do think McIlroy failed to solve the problem of demonstrating the value of literate programming, but I'm not sympathetic to arguments that he should've used more complex algorithms or relied on less library code. This is particularly the case when the additional complexity is only relevant in cases that don't fall into the given requirements.

(A literate program that uses SQL server or Excel might be an interesting read....)

And McIlroy's "solution" is provably not the "best possible solution" if you are interested in the algorithms, algorithmic complexity, the resources used, you know, all the topics studied by people doing computer science. All these topics are still relevant today.

That is the domain that was of interest to both Bentley and Knuth, and McIlroy "sabotaged" the whole by presenting effectively only a list of calls to the stand-alone programs which he hasn't developed himself, which he avoided to present, and even without looking at them, just by analyzing what are the best possible implementations of these, every student of computer science can prove that McIlroy's solution is worse.

If you carefully read the original text (and if you understand the topics of computer science), you can recognize that McIlroy was aware of the algorithmic superiority of Knuth's solution.

Knuth definitely and provably won.

Sure, it wasn't a competition. The fact remains: McIlroy criticized Knuth's presentation of complete algorithms which effectively solved specified problem, by presenting just a sequence of calls which provably have to call the implementations that must implement provably worse algorithms. In the column in ACM whose topic were... algorithms, edited by Jon Bentley.

So if you consider that two sides presented their arguments regarding the algorithms related to the specific problem, we can still say that Knuth "won" that "dispute."

You've never really established why this matters given that the goal of the challenge was to present the value of literate programming.

The goal wasn't an optimal algorithm or minimal resource consumption - the goal was to demonstrate the value of literate programming on a small data processing problem.

This is a very different problem than writing an optimal or highly scalable algorithm.

No. You obviously still haven't read the column and the article that explained what the goal actually was. The actual goal was to demonstrate Knuth's WEB system, which was explicitly made only for Pascal in 1986. That was what Bentley asked Knuth to do (quoting from the article "Programming pearls: literate programming", Communications of the ACM, Volume 29 Issue 5, May 1986 Pages 384-369):

"for the first time, somebody was proud enough of a substantial piece of code to publish it for public viewing, in a way that is inviting to read. I was so fascinated that I wrote Knuth a letter, asking whether he had any spare programs handy that I might publish as a “Programming Pearl.” But that was too easy for Knuth. He responded, “Why should you let me choose the program? My claim is that programming is an artistic endeavor and that the WEB system gives me the best way to write beautiful programs. Therefore I should be able to meet a stiffer test: I should be able to write a superliterate program that will be noticeably better than an ordinary one, whatever the topic. So how about this: You tell me what sort of program you want me to write, Crin d I’ll try to prove the merits of literate programming by finding the best possible solution to whatever problem you pose’--at least the best by current standards.”"

So, in the context of demonstrating Knuth's WEB on the substantial piece of code, the modification of the request was only that Knuth wasn't allowed to use the program he already wrote, but that he had to write a new one! (So the starting goal was in effectively all the premises exactly the opposite of what McIlroy then showed!)

So the goal was to write and present a wholly new program in Knuth's WEB, which, under the standards of evaluation of the quality of the solution as widely accepted by computer scientists, would be "the best solution." Which is exactly about the optimality of the algorithms, resources use etc.

If you still don't fully appreciate the context of the Knuth's program, do search for all the other columns and computer science books written by both Bentley and Knuth -- the topics of both were never "how to use existing programs" but how to develop/use the best algorithms.

> You obviously still haven't read the column and the article that explained what the goal actually was. ... > do search for all the other columns and computer science books written by both Bentley and Knuth

Since this is public, I'll conclude by noting here that I had indeed read both of the articles, and a bunch of other text by both Bentley and Knuth besides. (In fact, the Programming Pearls books are particularly high on my list of recommendations...)

I think it's possible and desirable to simultaneously respect both approaches for their respective merits. Maybe ironically for this profession, the choice isn't either/or.

Just an interesting counterpoint.

It all serves as a good reminder of how Knuth's work should be used & viewed: It shouldn't be taken as insight into how to manage and develop for software projects. It's more useful as teaching tools for how to think about solving problems. In my mind it's the difference between "pure" science and practical engineering, though I'm not sure that's a perfect analogy.

To try another analogy, it's like Knuth was asked to make a custom set of clothes. And McIlroy's critique seems like saying "Not everyone can afford custom made clothes, and the GAP produces perfectly serviceable clothes that are far more practical in most situations" It's correct, sure, but rather besides the point in the context of the original request.

Agreed... if that was the goal, then McIlroy's example was terrible (in that it didn't use literate programming at all).

The real lesson is to program using the most powerful tools at your disposal.

It'd be interesting to see how true this is in reality. I'm not at all convinced that most scripting languages can do this quite as concisely as bash and the Unix userland. (But, being honest, this problem seems very well aligned to that tooling's strenghts.)

Here is a Perl solution for comparison.

    use strict;
    use warnings;

    my $limit = shift @ARGV;

    my %count;
    for my $line (<>) {
        while ($line =~ /(\w+)/g) {
            $count{lc $1}++;
        }
    }

    @words = sort {$count{$b} <=> $count{$a} or $a cmp $b} keys %count;
    $limit = @words if @words < $limit;
    print "$_\t$count{$_}\n" for @words[0..($limit-1)];

    sub MAIN($limit = Inf) {
        my %bag is Bag = words.map(&lc);
        say "{.key}\t{.value}" for %bag.sort( {
          $^b.value cmp $^a.value || $^a.key cmp $^b.key
        } )[^$limit]
    }

    %bag.sort({ -.value, .key })

  import re
  import sys
  import collections
  
  words = re.findall('[a-z]+', sys.stdin.read().lower())
  c = collections.Counter(words)
  for word, count in c.most_common(int(sys.argv[1])):
      print(f'{count:>7} {word}')

  import re
  import sys
  import collections
  
  c = collections.Counter()
  for line in sys.stdin:
      words = re.findall('[a-z]+', line.lower())
      c.update(words)
  for word, count in c.most_common(int(sys.argv[1])):
      print(f'{count:>7} {word}')

  puts STDIN.read
    .split(/[^A-Za-z]+/)
    .group_by(&:downcase).transform_values(&:size)
    .sort_by(&:last).reverse
    .take(ARGV[0].to_i)
    .map{ |l| l.join("\t") }.join("\n")

  tr -cs A-Za-z '\n' |
  tr A-Z a-z |
  sort |
  uniq -c |
  sort -rn |
  sed ${1}q

What provides the pipe functionality?

Nothing prevents you from unit testing in scripting languages. Multi-process, sure, but most people aren't looking for that.

Ok, there's a single shell substitution, if it was fixed would you still call it a script? Technically the result of that is itself a sed script "3q", but if you count either of those then there isn't a lot of wiggle room between script and command, the arguments to tr are by far the most complex "script" involved.

> Nothing prevents you from unit testing in scripting languages.

That is a world away from what I'm talking about. Each line of that command can be executed on the CLI in isolation, you'd be replicating a lot more in nearly any scripting language, except maybe perl and awk.

> Multi-process, sure, but most people aren't looking for that.

Neither am I generally, still quite nice when you get it for free though.

Curiously, I independently came up with this metaphor in my review of The MMIX Supplement to The Art of Computer Programming [1]:

"You can marvel at its intricacies like one marvels at a Fabergé egg."

[1] https://www.amazon.com/gp/customer-reviews/R25NHN9UP7PLJI/

I had seen it a while ago and blogged about it here, with solutions (approx.) in Python and shell:

The Bentley-Knuth problem and solutions:

https://jugad2.blogspot.com/2012/07/the-bentley-knuth-proble...

Also, previous HN thread about it here:

Revisiting Knuth and McIlroy's word count programs (2011) (franklinchen.com)

https://news.ycombinator.com/item?id=15264997

https://www.amazon.com/Exercises-Programming-Style-Cristina-...

The main reason I like the story is that it's good fodder for discussion about priorities in software development. After all, counting words isn't exactly an interesting problem in 2018, but deciding the effective use of libraries and programmer time is still very interesting.

And about the story you shared, Knuth has elaborated in an interview (with typical humility, he plays it down): http://www.informit.com/articles/article.aspx?p=1193856

> As to your real question, the idea of immediate compilation and "unit tests" appeals to me only rarely, when I’m feeling my way in a totally unknown environment and need feedback about what works and what doesn’t. Otherwise, lots of time is wasted on activities that I simply never need to perform or even think about. Nothing needs to be "mocked up."

It was not an auspicious start to my career in computer science.

When it came time to try my code, I spent a while correcting some basic misconceptions that I had developed but that hadn't been corrected by an actual compiler.

I'd argue the opposite!

Learning how to "run" code in your head, cover edge cases and invariants, is a great skill to practice. When I was at a hospital after a difficult operation, I used to write down pages and pages of C64 programs in my notebook, too :-)

Even if I never even typed all them out later, I like to think this made me a better programmer. It trains your brain in a way that REPL doesn't.

Worked out really well for me - I'm employed well as a programmer, and I got to come to Shakespeare basically fresh when I was old enough to actually appreciate the work.

https://youtu.be/74BfHoE66rc?t=42m17s

https://geekz.co.uk/shop/store/show/knuth-tshirt.html

And here is a shot of the recursive shirt in action: https://laughingsquid.com/jacob-appelbaum-donald-knuth-demon...

"Mayday! ... We're sinking! We're sinking!"

"What are you sinking about?"

While on pronounciation topics, how does one pronounce "Knuth"? Is the k silent like in "know"?

Edit: According to Wikipedia it is kə-NOOTH.

Simon: Are you Alliance?

Early: Am I a lion?

Simon: What?

Early: I don't think of myself as a lion. You might as well though, I have a mighty roar.

Simon: I said "Alliance."

Early: Oh, I thought...

Simon: No, I was...

Early: That's weird.

One of the first times I was ever asked about the title of my books was in 1966, during the last previous ACM national meeting held in Southern California. This was before any of the books were published, and I recall having lunch with a friend at the convention hotel. He knew how conceited I was, already at that time, so he asked if I was going to call my books "An Introduction to Don Knuth." I replied that, on the contrary, I was naming the books after him. His name: Art Evans. (The Art of Computer Programming, in person.)

Some of his writings on the subject:

https://www-cs-faculty.stanford.edu/~knuth/things.html

https://www-cs-faculty.stanford.edu/~knuth/316.html

"It's a pleasure to meet you, Professor Knuth," Steve said. "I've read all of your books."

"You're full of shit," Knuth responded.

http://www.folklore.org/StoryView.py?project=Macintosh&story...

Whereas I recall Gates claimed to have read them, or at least some of the volumes, and found them mind-expanding. Gates wrote a fair amount of software himself early on and later would still challenge people on very technical points.

That comment was then misinterpreted as sarcasm, which is what the folklore author remembered. Later having forgotten the words, and only remembering the mistaken interpretation's overall feel, it was paraphrased, poorly.

cheers!

I thought that the article was pretty decent, considering it is aimed at an audience of people who aren't computer scientists and who have never heard of Donald Knuth.

I'd say she did something positive with the opportunity.

Instead of just signing it, he opened to his article and corrected a typo in the Potrzebie article--I think it amounted to two numbers being transposed somewhere around the millionths place (My copy is packed away from a recent move so it's not too convenient to verify).

What was funny was he had a poem he used to remember the order of the numbers after the decimal. He couldn't quite remember it, so he ssh'd into his home machine (or University, dunno), fired up Emacs, and opened a file containing the poem. Then he corrected the typo.

And that's how Donald Knuth "signed" my copy.

FTR, the answer to how I got it: I made an off-hand comment to my wife about the magazine and she tracked it down for the next gift-giving occasion.

3:16 Bible Texts Illuminated

https://smile.amazon.com/3-16-Bible-Texts-Illuminated/dp/089...

https://www.amazon.com/Things-Computer-Scientist-Rarely-Lect...

That's remarkable. I just wish he'd finish the book.

He wrote TeX82 (modern TeX) in WEB, a literate programming language he invented that could be tangled into the Pascal machine code or weaved into literate documentation in ... TeX itself.

In fact he wrote WEB itself in WEB and by hand compiled the output of the tangle program, which he ran on the tangle source to get the same output.

He also wrote the program METAFONT to produce the typefaces, which he used to design the Computer Modern font.

And then he used that to write instruction books for TeX and METAFONT as well as new editions of The Art of Computer Science and others.

TeX is still in regular use in the academic community (ported via Web2C) over 35 years later, despite the changes in technology in that time (with hacks to deal with modern fonts, outputting PDFs instead of DVI, including images and the like).

He taught the yak to shave itself, and it's still clean shaven 35 years later.

My son used to refer to Knuth as "that amazing harpsichord player" and didn't care about the computer science "stuff" I tried to tell him was really important. And he had plenty of time to sit down with a kid and go through the score of Fantasia Apocalyptica and laugh together at the musical jokes in the score.

OTOH I either have to consciously decide not to stress about all the things I want to get done or else...stress and push hard. Yet the things I feel driven to accomplish are quite minor compared to Knuth's to do list!

His 4th volume of TAOCP is so big that its beginning for him to take decades to write sections. Backtracking search + 3-SAT solvers is really interesting (and very closely related to Constraint Programming), but I'm deeply curious to know what his take on Constraint Programming will be.

Its really odd to be "waiting" for the next chapter of TAOCP. Its a serious body of work that has spanned decades of research and writing... I really hope he finishes the next section sooner.

Maybe I'll go and buy the fascicle for 3SAT and read it while waiting.

I know the feeling of waiting for new chapters of TAOCP; I'm waiting for Chapter 8 on recursion because his idea of it is so different from everyone else's.

This seems to tie in with his attitude towards programming in general: while in the zeitgeist the idea is that a good program builds better abstractions until the structure of the program reflects human thought, Knuth seems to think that a good programmer is one who starts with a human-level description of the solution and carefully refines it until it translates into a good set of instructions (in machine code). (There's no contradiction, but the point of view seems different from the usual.)

He briefly mentions something related in his Structured Programming with Go To Statements (1974) https://pic.plover.com/knuth-GOTO.pdf#page=21

> I have always felt that the transformation from recursion to iteration is one of the most fundamental concepts of computer science, and that a student should learn it at about the time he is studying data structures. This topic is the subject of Chapter 8 in my multivolume work; but it's only by accident that recursion wasn't Chapter 3, since it conceptually belongs very early in the table of contents.

Even today, for many "systems level" programs, you have to take extreme care around recursion, since you are likely to blow the stack limits. (That is, you can't just use recursion freely, but instead have to prove the maximum depth that you will trigger it.)

That is to say, I don't see what makes this different than how most others view recursion. If you can afford to do it, then you should do so. If you can't, then you should be ready for a lot of his other tricks. (Threaded trees being my personal favorite, at the moment. DFS with no stacks? Crazy talk! :) )

(BTW I'm not sure that Knuth would wholeheartedly agree with “If you can afford recursion, then you should use recursion”. Perhaps he'd consider the program half-written. That's in some sense the crux of what I think is different.)

Let me put it another way: I thought I knew how to generate all permutations of a set of numbers. When I saw questions about it on Stack Overflow or Quora, I answered. When I opened Volume 4A, it disposed of what I knew in the first half-page, and proceeded to discuss the problem for over 50 more pages. I thought I knew about backtracking, having written backtracking programs hundreds of times (always implemented using recursion). Knuth's section on backtracking obliterated everything I knew (and with no recursion in sight). So I'm looking forward to when he gets around to writing about recursion, to see what I can (un)learn about something so fundamental.

Your point on him being one of the main programmers from his time still programming is fair.

Here is what he says in the (very early) draft of fasc16a (only 2.5 pages written):

> We've already seen hundreds of examples of recursion in previous chapters. Early on, we encountered sequences of numbers that were defined by relating each member to earlier members after getting started; these “recurrence relations” were sometimes simple formulas like […], sometimes more involved like […], and sometimes considerably more elaborate and complex. We also found that basic data structures for trees are inherently recursive, because all but the simplest trees are composed of subtrees. We discussed numerous instances where problems of sorting or searching or optimization could be solved effectively by a “divide and conquer” approach, with which a large problem is reduced to smaller problems of the same kind.

> Yet we didn't dwell on the recursive nature of those concepts. We transformed them into equivalent notions that could be dealt with directly and nonrecursively. Because “at bottom” a computer operates by means of the physical properties of electrons, which are not recursive.

> For example, in Algorithm 2.3.1T we traversed the nodes of a binary tree by using an auxiliary stack A, not by using the recursive definition of symmetric order to define that algorithm in terms of itself.

Don't want to quote the whole thing (though there isn't much), but from the sample I feel I'm going to learn from a new perspective (new to me, anyway).

I know he has program in one of his books where he uses recursion to print the contents of a tree. Maybe I was just confusing that. (Don't have my books handy.)

That said, this is definitely going to make me study those programs more. Hilarious how off I was in my memory.

(The use of goto statements like he does in the main routine cracks me up, btw. Curious to know if these made a measured difference. With Knuth, I would assume so.)

Depth and coverage.

Case in point: Knuth's coverage of Quicksort doesn't calculate the Big-O. It calculates the precise average running time. He literally counts the assembly-language statements. You might say "But arrays might be randomly sorted", and yeah, Knuth averages the runtime across all possible ways that an array can be sorted. Its a beautiful work of math.

There's almost no room for questions once Knuth finishes a subject. Its so exhaustively covered that I can't even imagine questions I have on any subject after he's done writing about it.

The few questions I do have are the "Difficulty 40+" questions that Knuth puts at the end of each section, which would serve as excellent questions for Masters Thesis or a PH.D Thesis. IE: There's no answer to those questions yet.

Anyway, while most textbooks hand-wave the idea of "Quicksort is faster than Mergesort, even both are O(n*log(n))". Knuth is pretty simple, he says: Look at these two precise numbers I've calculated (and proved). Quicksort is faster (or at least, the Assembly-language version of Quicksort he wrote on his imaginary computer is faster)

Basically, Knuth got "to the bottom of it", there's no hand-waving involved in the explanation. He just tackles the hard questions head on and solves it more precisely than everyone else.

----------

Some of the older chapters are going out of date, which is natural for a work so complete. There is so much material out there that Knuth has to make a decision on whether to revise older stuff (which he does occasionally) or to write about subjects which don't have a major unifying work yet (ie: Constraint Programming has some textbooks here and there, but most of its ideas are only in academic journals, and haven't really been unified into a singular work yet IMO. Even the textbooks on the subjects are mostly introductions + references to papers)

And sometimes, Knuth gets lucky. Knuth's Book3 "Searching and Sorting" includes a HUGE table of sorting methods applied to tape drives.

But today's RAM is now faster sequentially rather than random-access. So Knuth's calculations on Tape-Drive sorts actually applies to modern DDR4 RAM better than any other treatment on the subject. No one else calculated the cost of "internal movements" (aka: cache on modern systems, but it was RAM on older computers) + "external movements" (which was Tape Drives in Knuth TAOCP / Book 3, but is really DDR4 RAM on today's machines).

So yeah, Book 3 is out of date and focuses a huge amount of time on external sorting methods. But... it just so happens that if you know a bit about the sequential nature of modern DDR4 RAM, it is actually super-relevant to today's computers.

This doesn't feel different from everyone else's idea, though. Unless you just mean in the general complexity analysis, where most of us are basically taught to stop at the highest order terms. It is interesting, because Knuth seems credited with pushing Big-O analysis, but he never stopped at the highest term when analyzing an algorithm. Far as I can tell, he never has. He has done this to compare some algorithms, but even then he has always been empirical for actually comparing things. Treating the Big-O as a hypothesis to test, if anything.

We attended the conference as students but still got to see the conference, listen to the music and be at the birthday dinner free of charge. When I talked to Knuth I was amazed by his humility. Every presentation was about how grand he was, but he still was humble and friendly.

They say you shouldn't meet your heroes, but they're obviously not talking about Knuth!

I love that quote.

https://youtu.be/dhh8Ao4yweQ

He was ready for Moneyball and sports data science decades before everyone else.

I'm interested in this low level systems opimization. What type of jobs hire for this work? What areas in graduate school should one focus in? Is this type of skill set still employable?

As with all things, the way to get experience and knowledge is to try things. Take a look at some software that you use that does something slow, and try to make it faster; it's easier if you have the source available. Lots of applications these days are slow to start, and that's easy to time, and probably easy to bucket things into things that don't need to be done or things that can be done later. If you interact with any long data processing loops, those are often good choices for optimization as well.

This skill set is most employable for companies that have scale; when you have ten servers, being able to turn off one of them because of efficiency gains doesn't help that much; but it's nice to turn off ten percent of your 10,000 node cluster. It also leads to nice claims on resumes "made efficiency gain in X, leading to Y% cost savings on servers", and since you're likely to spend a bunch of time on a single issue, you'll be able to have a good discussion when it comes to 'discuss some of your interesting projects' in interviews.

The only downside is you'll be super frustrated about everyone else's software being slow. You'll likely gravitate towards classic games because they don't have load times; I hope you like pixel art. :D

For example, you could read the C++ spec forwards and backwards for a year and know your data structure big O notation just as well, but they will never tell you that inserting into the middle of a vector is faster than inserting into the middle of a linked list in a great many common cases.

Sure, ideally you'd measure, but you need to have some idea of what options to even compare with your measurements.

However, in practice, the O(n) operation will be faster - since doing a hundred reads and a hundred writes to shift a hundred numbers within a single cache line will be much faster than reading a single byte to an out-of-cache memory in the linked list case; the processor can do a lot while waiting for any new data from RAM.

C++ stl, LL appending is always O(1) and writing into a vector is O(n). There are plenty of caveats to this and the documentation should be referenced.

He's right btw that insert into a vector is generally faster in the C++ stl than insert into a LL. I don't recall off the top of my head why that is, just remember it's the case.

I guess we can split the difference and agree on Jedi. :-)

I just wish that there was a machine running OPENSTEP 5.0 (as I think of Mac OS X) which I wanted to buy --- I prefer pen computers (the highwater mark of my computing experience was an NCR-3125 running PenPoint paired w/ a NeXT Cube w/ a Wacom ArtZ graphics tablet).

--

From this small interview [1]:

"When I'm working on a research problem I generally begin by filling dozens of sheets of scratch paper with partial calculations. When I eventually get to a point where I can think about the problem while swimming, then I'm often ready to solve it."

--

From this other interview [2]:

"So I went to Case, and the Dean of Case says to us, says, it’s a all men’s school, says, “Men, look at, look to the person on your left, and the person on your right. One of you isn’t going to be here next year; one of you is going to fail.” So I get to Case, and again I’m studying all the time, working really hard on my classes, and so for that I had to be kind of a machine.

I, the calculus book that I had, in high school we — in high school, as I said, our math program wasn’t much, and I had never heard of calculus until I got to college. But the calculus book that we had was great, and in the back of the book there were supplementary problems that weren’t, you know, that weren’t assigned by the teacher. The teacher would assign, so this was a famous calculus text by a man named George Thomas, and I mention it especially because it was one of the first books published by Addison-Wesley, and I loved this calculus book so much that later I chose Addison-Wesley to be the publisher of my own book.

But Thomas’s Calculus would have the text, then would have problems, and our teacher would assign, say, the even numbered problems, or something like that. I would also do the odd numbered problems. In the back of Thomas’s book he had supplementary problems, the teacher didn’t assign the supplementary problems; I worked the supplementary problems. I was, you know, I was scared I wouldn’t learn calculus, so I worked hard on it, and it turned out that of course it took me longer to solve all these problems than the kids who were only working on what was assigned, at first. But after a year, I could do all of those problems in the same time as my classmates were doing the assigned problems, and after that I could just coast in mathematics, because I’d learned how to solve problems. So it was good that I was scared, in a way that I, you know, that made me start strong, and then I could coast afterwards, rather than always climbing and being on a lower part of the learning curve."

[1] http://authenticinquirymaths.blogspot.pt/2015/11/maths-in-sc....

[2] Transcript from here: https://github.com/kragen/knuth-interview-2006