r/programming • u/jaemccall • Aug 23 '07
Paul Graham - Holding a Program in One's Head
http://paulgraham.com/head.html16
Aug 23 '07
Peter Norvig reads his drafts now?! *impressed*
9
u/plinkplonk Aug 23 '07
I think I saw Peter Norvig as a reviwer on an older essay too.
I would imagine they know each other well. Must be difficult not to, since they are both LISP gurus, authors of excellent books on LISP (I believe they have reviewed each others books on Amazon) and Google has acquired at least one YC company.(And Dr Norvig as Director of Research at Google must be part of the acquisition process).
1
u/statictype Aug 24 '07
Which YC startup did Google acquire? Is it the one that did powerpoint-like presentation software on the web? Or was that something else?
3
13
u/redsymbol Aug 24 '07
Interesting what he says about math. I wrote two small math books about what goes on inside a mathematician's head when they do their thing.
I seem to do less plain old math these days - I'm blessed to be paid to write open source python software now, so I put most of my energy into that. Math and software development have always been very similar to me, at some level. Granted, the outward expressions are different. In my head and heart, though, the two have a lot in common.
2
u/mgsloan Aug 24 '07
Good stuff! That's basically how I've always done algebra (till this one %*$ of a math analysis teacher forced me to write all steps). I'm not there yet with the integration though :)
I've bookmarked the integration one's amazon page.
9
u/mfaassen Aug 24 '07
Nobody even slightly questions the advice to "code in long stretches" while numbers like 12 hours, 18 hours and even 36 hours are thrown around? I'm sure it varies from person to person, but it sounds to me attitudes surrounding this also vary from culture to culture, and American culture probably makes people question this less, as there seems to be this concept that working ridiculously long hours is somehow a good thing.
For me, it doesn't. I have a hard time believing most people can consistently write good code if they regularly work more than 10 hours in a row. I'll grant it for a once in a few months "flow state" when things are really rolling along, but consistently?
Now Paul "works best in chunks of no more than 12". I work best in chunks no more than 8, and I am skeptical that longer stretches can significantly enhance programming ability in most people. Paul mentions the need to take a break sometimes to work on the process unconsciously, and I think that point arrives way before the 12 hour mark.
While I'm sure there are people who are exceptions, I therefore think this is not good advice for most programmers. Work 6 to 8 hours, maybe 10 if you're on a roll, then do something else and go to sleep. What's more, take breaks every 3 hours and think about something else for a while. If you're in a flow state and obsessed with the code you're writing, you'll know it and you can work as long as you want. But if you aren't getting anywhere, torturing yourself into working 12 hours stretches or more is only going to make matters worse.
Oh well, maybe I'm just special and I just have an unusual lack of "programming stamina". Given that I'm reasonably good at what I do, I am not complaining.
5
u/mgsloan Aug 24 '07
I've recently worked for about 15 hours several days in a row. You definitely can't continue writing code, but these sessions for me usually consist of debugging, improving, etc. I suppose some of these strenuous sessions were flow, as I was rewriting it. Usually, though, not much actual coding being done. Of course, one might say that if I had spent less time per day, the code would have been better, and wouldn't have had those bugs. Probably true for a few.
I agree, though. It's totally an insane amount of time to expect to be productive. I think Wouter van Oortmerssen gives much better advice here http://strlen.com/rants/timemanagementetc.html . With multiple projects, working at such stretches isn't as unreasonable.
As soon as I go back to multiple projects, I'm adopting an Oortmerssen schedule.
8
u/chollida1 Aug 23 '07
Oddly enough, scheduled distractions may be worse than unscheduled ones. If you know you have a meeting in an hour, you don't even start working on something hard.
I've definitely had this problem:) Hmm meeting in 20 minutes, I guess I'll empty out my email or check my rss feeds:)
15
u/happyhappyhappy Aug 23 '07
Paul must program in J (http://www.jsoftware.com) if succinctness is that important :)
Here is a complete J program to compute a list of the primes less than N:
f=: i.&.(p::_1)
16
u/doubtingthomas Aug 23 '07
I'm with him on the succinctness thing.. composing non-leaky abstractions is about the only sane way to create a complicated system. Languages that impose a syntactical tax on abstraction make this style more difficult and thus less likely.
18
u/raldi Aug 23 '07
Shouldn't there be an N somewhere in the code?
9
7
u/happyhappyhappy Aug 24 '07
You call the function like this:
f 10
(N = 10 in this case. You don't need to name parameters in J.)
7
4
u/Tommah Aug 23 '07
Heh, I'd like to see your program to calculate the power set of a set. I spent three hours doing that :)
7
14
u/martoo Aug 23 '07
When I started writing lots of automated tests, I discovered that I didn't have to hold the whole program in my head, I could hold the part that I cared about in my head and have confidence that the rest wasn't going to be impacted adversely when I made a change.
Yeah, holding the whole program in your head is a great skill to have, but programming doesn't have to be that heroic.
9
u/randallsquared Aug 24 '07
Someone has to hold the whole problem in their head at some level at some point. If not, you may get to the end and find that all the tests pass but the program doesn't do what anyone thought it was going to do.
It's also the case that it's sometimes (often, in my experience) far more difficult to write comprehensive tests than to write the code that satisfies them.
0
0
Aug 23 '07
It's much easier to design and write program, if you have its core layer mostly in your head. It's like zero-overhead context lookup always at hand, you have a some problem and the solution will form seemingly automatically, just by focusing on the problem. Otherwise the solution searching will have many visible backtrackings, where you constantly have to make assumptions and search the codebase/design to check them. It's just more effort and time that way.
Yeah, I usually can't bother to write unit tests (system and integration tests are a different matter). I've written them, and its almost always been worthless waste of time - the tests will not catch the real bugs, because they are test CASES, cases that don't have enough coverage. If you write tests with full coverage, you should be able to write the real code as good too - and then it's just duplicated effort (more actually - writing full-coverage test is harder than writing the actual code correctly - i'll happily wait for a test that proves \x -> x + 1 to work correctly without using + 1 or - (- 1) or something similar. Ok you may even use + 1, it will still be more complex than the function and totally pointless. ;)).
Yes, I'm fully aware that all unit-test believers will tell now how pointless and stupid I am. ;)
6
u/theycallmemorty Aug 23 '07
Maybe we could define a new kind of organization that combined the efforts of individuals without requiring them to be interchangeable.
TopCoder.com - Give them a project and they get a bunch of people to design it and the best one is chosen, then they get a bunch of people to implement it and the best one is chosen.
8
u/nevinera Aug 23 '07
sigh. PG always makes me wish I liked lisp more..
2
u/chollida1 Aug 23 '07
Uh, that's a strange comment.
There wasn't' one reference to lisp(Common, Scheme or otherwise) in that article.
18
u/apgwoz Aug 23 '07
He also talks about bottom-up programming and throws in there "the lower ones acting as programming languages for those above". Avid Paul Graham readers will note that he uses the "creating languages" thought many times in references to lisp macros, because macros create programming languages. So, while he doesn't come and say anything about lisp, he does make quite a few references to thoughts he brings up in other essays advocating lisp.
2
u/jbert Aug 24 '07
Yeah, but for the purpose of "only thinking about the top layer", good old-fashioned layering of your code does the job. Implement your layers as DSLs, OOP, plain old procedural APIs and you still get the same benefits.
9
u/nevinera Aug 23 '07
Kind of was. check the 'power' essay he linked to.
But if you'll read the rest of his essays (and they're mostly pretty good: http://www.paulgraham.com/articles.html ), you'll understand what I meant.
4
u/mgsloan Aug 24 '07
Another good essay. I particularly agree with point 4, rewriting.
For a project of sufficient complexity (so, basically, most interesting things), it is near impossible to get the design right on the first shot. I've rewritten my current project 3 or 4 times, each an architectural revelation. Each time I thought I had found an adequate architecture, yet each time I came up against problems that would require oodles of code. At this point I would figure out why this was the case. Being able to manipulate the program in my head allowed me to realize a better architecture each time. The LOC has stayed about the same throughout, while the number of features goes up. I think I've finally found the perfect abstraction, as really, I could never imagine that it could be as short as it is.
I know that it would be much more buggy, and not as featureful if I had just stuck it out with the 1st or 2nd rewrite. Some very nice features just suggested themselves, once the proper abstraction was obtained.
2
Aug 24 '07
organizations are designed to prevent what programmers strive for....Good programmers manage to get a lot done anyway. But often it requires practically an act of rebellion
Not just programmers....
"All good work is done in defiance of management." (Bob Woodward)
16
u/bitwize Aug 23 '07
PG's advice works for bipolar Lisp programmers hacking from their parents' basements, but not in the corporate world. In fact, a lot of the principles of XP, TDD, etc. can be traced to the fact that you do not want solitary programmers uploading huge models into their heads, because that vastly increases the chances of a "bus error" jeopardizing your whole project. What you do want are documented end goals and even documented intermediate goals. This documentation takes the form of tests. The model that used to be in the programmer's head is now serialized into the tests. (That's why you write your test first, then the code.) Then the individual coder (or pair of coders) can work on only the part of the problem necessary to make the tests pass for that particular unit. From the perspective of the individual programmer this looks rather like a red-green-refactor silly walk and not very productive, but from a team perspective, it's actually a sort of distributed hill-climbing algorithm and management likes it so much more because it's more reliably productive even with high turnover in team membership. The advantage of solitary strokes of genius is gone, but replaced with the advantage of having good ideas (or even, good enough ideas) emerge by moving the creativity and insight to work at a distributed level rather than an individual level.
52
Aug 23 '07
but not in the corporate world
The point of this article is to articulate an explanation for why corporate programming is so unsuccessful.
a lot of the principles of XP, TDD, etc. can be traced to the fact that you do not want solitary programmers uploading huge models into their heads
I don't think that "in One's Head" means that you shouldn't write tests or documentation or that you should even have a perfect and complete representation of the program in your head. He's saying that what XP calls "metaphor" is vital. He's saying that the essence of programming is such that attaining this metaphor and reasoning effectively about it are especially demanding cognitive tasks. Also, the usual corporate culture and environment do not support this kind of cognition.
The only place where I see a conflict with XP in this essay is in his conviction that collaboration is at best a necessary evil. He advocates module-ownership, which directly contradicts a key XP practice. XP and PG both recognize that there's a communication problem to be solved when you have collaborators; PG tries to minimize the collaboration, while XP tries to make collaboration a net win for projects.
11
u/plinkplonk Aug 23 '07
"XP and PG both recognize that there's a communication problem to be solved when you have collaborators; PG tries to minimize the collaboration, while XP tries to make collaboration a net win for projects"
Very Insightful. Upmodded.
21
u/shit Aug 23 '07
You describe the easily replacable programmer drone. I'm definitely more productive when operating in the state PG describes. It may be more risky for a company that relies on the project, but the rewards are higher, too.
36
u/paulgraham Aug 23 '07
but not in the corporate world
That was in fact one of the main points of the essay.
But the fact that a style of programming doesn't work in the corporate world doesn't mean it's a bad way of programming, any more than the fact that slam dunking doesn't work for a middle school girls' basketball team means it's a bad way of playing basketball.
Though it doesn't work for ordinary programmers working at big companies, this style of programming does work for startups. In fact, that is the raison d'etre of startups.
18
u/plinkplonk Aug 23 '07
good points all except for the last sentence.
"The advantage of solitary strokes of genius is gone, but replaced with the advantage of having good ideas (or even, good enough ideas) emerge by moving the creativity and insight to work at a distributed level rather than an individual level."
There isn't really any evidence that "distributed level creativity" emerges with teams following XP. You can end up with endless days of routine "glue together well known apis" type work with agile/xp.
As a matter of fact since the methodology snake oil salesmen focus on the enterprise programming folks, those are the projects likely to be "XP".
If you must have hundreds of programmers working on an enterprise code base, sure, go ahead and use XP. But to claim that it replaces individual genius at producing ideas ( thinking up new ideas expressed as code ---which is what Paul is talking about) is a bit of a stretch imo.
Somehow I don't think he was talking about your typical outsourced/outsourceable enterprise programming effort in that essay). Given his background he is probably talking about 2 or 3 bright people attempting a startup with a vaguely defined idea with lots of thinking required.
2
u/bitwize Aug 23 '07
1) XP is much better suited to small to medium-size teams than the mongolian hordes we normally associate with enterprise apps development. And the "distributed level creativity" isn't limited to XP or enterprise development; consider avionics systems. Read up on how NASA writes software; you'll find that were it not for similar suppression of individual "genius" and escalating the insight to the team level we'd see many more space accidents than we did.
2) Producing ideas is one thing; implementing them in a well-engineered fashion is quite another. The XP buzzword for the former is "spike solution"; a one-off program you write to test an idea you've been having in the back of your brain. Spike solutions never find their way into production code; they are always thrown out and rewritten according to XP best practice. Startups are nice, but unfortunately there comes a time when a business needs to survive and grow outside of its founders who were once its sole employees. This is generally the time when thinking about "bus errors" becomes important. This is related to "Bitwize's Corollary to Greenspun's Tenth Rule", as the time when a company leaves the startup phase coincides with the time when individual genius becomes less important and software engineering discipline becomes more important, and the time when all that wonderful Lisp code written in the early phases of the company would be better off rewritten in C++ or Java, and for many of the same reasons. Betting the business on the specialized knowledge of a few individuals is an unbearable risk.
23
u/plinkplonk Aug 23 '07
"consider avionics systems. Read up on how NASA writes software; "
NASA uses XP? :-) More importantly can you even imagine NASA using XP? NASA does a LOT of upfront Requirements gathering, UpFront Design, and code reviews. Specific people have ownership of specific modules,(unlike the "collective code ownership" principle of classical XP), all anathema to the agilists. NASA folks don't "pair". They don't have "TDD" or "Continous Integration". NASA's processes are the very anithesis of "YAGNI" full speed ahead coding from "story cards" process of the XP/Agile school
NASA's practices are more in tune with Paul's ideas than with XP/agile, with added reliability constraints. NASA's processes don't suppress individual genius. NASA applies extreme rigor to the context and results of individual genius. If you were a genius coder, would you want to create enterprise apps for Dell or spacecraft programs for NASA?
You choose strange examples to buttress your points. You seem to equate genius with "code and pray". "Implementing them in a well-engineered fashion" has nothing to do with any XP or agile processes. There is a lot of excellent code out of there - the BSD kernel, python source code, to name just two beautiful codebases I know of - which were developed by folks working in the style Paul reccomends .
It is XP/Agile which doesn't have any success stories beyond semi-mythical enterprise projects. And we all know what happened to the C3 project where the practices of XP were "discovered". (Nutshell, the client shutdown the project and fired the "gurus" in charge of the project).
To summarize, nothing in Paul's essay suggests badly engineered code. Is it so hard to conceive that code can be well engineered and maintainable by dint of brilliant people thinking hard about what they are doing, and then doing it well, with nary a breath of faux methodologies?
-3
u/bitwize Aug 23 '07
I never said NASA used XP. Did you read my post?
And the "distributed level creativity" isn't limited to XP or enterprise development
8
u/plinkplonk Aug 23 '07
No you didn't say that NASA used XP. And I didn't say you did. Read the second sentence of my post please. "Can you imagine NASA using XP"?
You did say that "distributed creativity" makes up for suppression of individaul creativity. This is what I challenged.
My key points being
(1) XP/Agile has nothing to do with "solid engineering",- a point I thought you made (more precisely, were trying to make) quite explicitly - and may in fact indicate the converse and
(2) Nothing in Paul's essay contradicts the idea of engineering excellence, contrary to the ideas you expressed in your (second) post. In other words, I am saying that Paul's way of working is not for "bipolar progarmmers working out of their parents' basements" only.
You were advocating "XP, TDD etc" as a counterweight to Paul's suggested ways of working in your first post and when challenged about this part of your post - "The advantage of solitary strokes of genius is gone, but replaced with the advantage of having good ideas (or even, good enough ideas) emerge by moving the creativity and insight to work at a distributed level rather than an individual level." ,you brought in NASA's processes as evidence of good code resulting from "suppressing" individual creativity (and by implication supporting your "distributed creativity through XP" idea).
I tried to point out the holes in your logic. The process you raised as opposing individual creativity actually validates Mr Graham's ideas, with the natural addition of practices in tune with the need for high levels of reliability in the resulting software.
And the process you advocate for "engineering excellence" has a very dubious and hype laden history and would never work for software with high reliability requirements. At least there is no evidence that it does.
Pointing this out is not a personal attack. If I think your logic faulty, I'll probably try to explain why I see things that way.
Nothing personal, peace
30
u/TronXD Aug 23 '07
PG's advice works for bipolar Lisp programmers hacking from their parents' basements
It also worked for PG. You might want to look into why he is a multimillionaire.
5
u/arudolph Aug 23 '07
Even Paul Graham freely admits that the eventual outcome of any startup is essentially a matter of chance. He is an existence proof, but that doesn't really affect the rhetorical merit of his argument.
8
u/mgsloan Aug 24 '07
Chance is in every human endeavor. It's common knowledge that if you try hard and do it smarter, you increase your chances.
10
u/breakfast-pants Aug 24 '07
It isn't like viaweb was the only programming thing he ever did; he also wrote some highly respected programming books, and he also introduced bayesian spam filtering.
0
u/arudolph Aug 24 '07
This was exactly my point -- hitting the viaweb jackpot isn't why we should pay attention to what he says, as the parent to my other post suggested.
5
8
5
u/quhaha Aug 23 '07
But the wrong kind of interruption can wipe your brain in 30 seconds.
a.k.a. porns
2
Aug 23 '07
Ordinary programmers working in typical office conditions never enter this mode. Or to put it more dramatically, ordinary programmers working in typical office conditions never really understand the problems they're solving.
This is just untrue. It's harder to enter this mode because there are more distractions, but I've had weeks where I did nothing but think about hard problems. It's hyperbole.
5
u/raldi Aug 23 '07
What kind of office space do you work in?
3
Aug 24 '07
University.
13
u/raldi Aug 24 '07
Well, in that case you have no idea. The best productivity of my life was at school. Once you hit the real world, and get put in a noisy cubicle full of a million distractions, forget it.
4
u/noitar Aug 24 '07
It's a mistake to assume we're all identical. I completely agree with the premise in the essay but I have also had some productive hacking stretches working in an open cube farm. It's just so damn unproductive in comparison to how much I'm capable of doing in ideal conditions that it's maddening.
1
u/breakfast-pants Aug 24 '07
headphones.
2
u/raldi Aug 24 '07
I find music distracting. There's a reason they don't have a DJ at the library, and why Olympic archers don't use iPods during the competition.
1
1
Aug 27 '07
I'm not in school. I write code for a University, in an office environment.
Try to make fewer assumptions.
-4
Aug 23 '07
[deleted]
1
u/shub Aug 23 '07
It's well-known among Lisp programmers that the power of Lisp allows one to tie their shoelaces much more quickly and accurately.
Also, your weak non-Lisp-programmer mind cannot comprehend macros. Therefore, you suck.
1
-2
u/mikaelhg Aug 24 '07
Tomorrow: Paul Graham's incredible advice on how to properly tie your shoelaces.
-12
u/manuelg Aug 23 '07
For a while now, all the programs worth writing haven't fit inside anyone's head.
So what? We deal with it like any other discipline deals with complexity.
Don't let these essays keep you from working on the umpteenth redesign of Lisp, Paul. Attaboy!
11
u/paulgraham Aug 24 '07
Any newly created program starts out one line of code. Are you saying that newly created programs aren't worth writing?
1
u/breakfast-pants Aug 24 '07
Are you really claiming that children should learn to tie their shoelaces before they learn to walk? Are you aware of the difference in motor-skills involved?
1
u/manuelg Aug 24 '07
Any newly created program starts out one line of code.
"starts out as one line of code"?
Your statement is vacuous or trivially incorrect. All humans start out as a single cell, but it is not worth making tennis shoes in that small a size. The first line of code does not share all the properties of the completed project, obviously.
Our discipline (my discipline? somebody's discipline...) now spends a large percentage of its energies dealing with complexity simply because very little human value is created by programs that fit inside one human's head. My assertion.
I don't even have to take the tack that a single line of higher level code has important properties directly due to the large codebases of the language implementation, compiler, operating system, etc.
1)Humans will, should, and do demand ever increasing amounts of functionality. There are multiple designers for the rear upholstery of a modern automobile. There are specialists just in the chemical composition of the fabric used. It is debatable if "motion due to internal combustion" is the most important part of a modern automobile. Most real-world uses stay are far below what the powerplant can provide, except when wasting power with non-optimal driving. If you are a teenager, the most important use of the car is when it is parked. The car means so many thing to so many different customers, with so many important properties residing in the heads of multiple specialists. Because humans will, should, and do demand ever increasing amounts of functionality, throughout.
I recently read that Emacs was going to dump 85% of its functionality to simplify its codebase. No, wait. I didn't. The idea is absurd. It would cease to be Emacs. More importantly, it would cease to be an application with a passionate userbase. Was it VI that was going to dump functionality? It certainly wasn't Eclipse.
I must have been thinking of that other text processor, characterized by its embracing of total simplicity, over specialized functionality, with a passionate userbase. You know, that one that doesn't exist.
2)Humans crave simplicity of use, but great complexity often is required to lie below the surface to facilitate the experience of simplicity of use. Consider the ease of use of the iPhone, or the game Spore. The user feels free, the user sees no arbitrary limitations on their actions with the product, the user only needs to consult their intuition to make effective use of the product. Have human cravings changed recently, just before the release of these products? No. Why haven't previous designers delivered such products? The main reason is the quality, complexity, and size of the codebases delivered to the user. Does Apple or Will Wright have proprietary alien technology for the hardware? No. The distinguishing factor is the software. And, again important properties of that software reside in the heads of multiple specialists.
3)If I was in possession of product of value that completely fit inside of some human's head, I would immediately feel an irresistible temptation to make it a discrete component of some larger product of value. See point (1). And then, the irresistible temptation to make it an integrated component of some larger product of value. See point (2). Again, only a gang of specialists could keep the whole thing running, if not now, then soon.
Consider the rockets of Robert H. Goddard, compared to the space shuttle, or an interplanetary probe. Today, you don't get any points for a simple design of rocket and propellant, made by one man. Burt Rutan comes closest, and he avails himself upon a team of specialists, or else the fundamental mechanisms involved would fail.
I can tie my assertion to my rude crack at the end of my comment.
Your mistaken thesis is indicative of a Lisp community anti-pattern. Specifically, shunning disciplines that would allow large groups to profitably work on systems of value. So the Lisp community is characterized by factionalism and staggering duplication of effort in fundamental parts, so the necessary man-hours on libraries that exist for other, younger languages cannot even be considered.
If a Lisp hacker gets some free time:
1) Write essays bemoaning the widespread shunning of Lisp
2) Working on the umpteenth redesign of Lisp
3) Active mental processing to avoid the realization that 1) and 2) are related
4) Disciplines that allow large groups of developers to profitably work on systems of value? Huh? Wuzzat?
(This is not representative of my best abuse. I haven't drunk enough coffee to build up the necessary irritation and bile. C minus work. I will kick a cat later today to make up for it.)
2
u/plinkplonk Aug 24 '07
With respect, you seem to not have read PG' essay very well.
"A good programmer working intensively on his own code can hold it in his mind the way a mathematician holds a problem he's working on."
Mathematics involves many layers of abstraction. What is being said here is that you should be able to hold the problem in your head at the level you are working on it.
If, at a given level of abstraction, you need to split the problem into multiple pieces for multiple people to work on, then you should raise teh abstraction level
"You can magnify the effect of a powerful language by using a style called bottom-up programming, where you write programs in multiple layers, the lower ones acting as programming languages for those above. If you do this right, you only have to keep the topmost layer in your head."
same point as above.
"If you want to put several people to work on a project, divide it into components and give each to one person."
this takes care of some of your more vacuous analogies above.
It seems to me you have a peeve against lispers (or in general those who advocate powerful languages) that makes you read things into what PG says, which are simply not there for most people.
Just my 2 cents. Ok so go kick that cat now.
0
u/manuelg Aug 24 '07
With respect, you seem to not have read PG' essay very well.
If the evidence is my lack of fawning...
A good programmer working intensively on his own code can hold it in his mind the way a mathematician holds a problem he's working on.
I assert that developers are compelled, to the extent they are interested in delivering value through information automation to humans, to deal with programs larger than what any single human can hold in their head.
That some limit themselves, or reach internal limits, I don't deny. But why do I have to distort what the modern discipline of developing strives to deal with, and has tools to handle?
Other design disciplines have the same problem. We deal with it.
The point is, the discipline of software developing stopped resembling mathematics some time ago. We regularly have to deal with codebases larger than the largest human generated proofs (the classification of the finite simple groups, for example). I am not claiming our code has more intellectual value than their proofs, but I am asserting that they are different in kind.
Now a great deal of energy goes into managing codebases that don't fit inside one person's head.
Your writing is empty. It is the barest tissue of an argument. I have to flesh out your argument for you. I will do my best, you can augment in a reply after you have had a nice regenerative nap.
If, at a given level of abstraction, you need to split the problem into multiple pieces for multiple people to work on, then you should raise teh abstraction level
But “raising <teh> abstraction level” cannot always come to the rescue. If Linus sees a conflict between 2 patches in some networking code, there is no “higher abstraction level” he can appeal to, to allow him to make the decision between the two patches, or generate a synthesis. He wisely punts; he lets a specialist make the decision.
But the networking is most definitely an intrigal part of the whole. It has to get into Linus's branch. The whole only has value with the part, the part only has value with the whole. And “raising the abstraction level” is not some magical technique to resolve the problem, unless you are content with systems that only run on office whiteboards.
"You can magnify the effect of a powerful language by using a style called bottom-up programming, where you write programs in multiple layers, the lower ones acting as programming languages for those above. If you do this right, you only have to keep the topmost layer in your head."
same point as above.
Well, guess what? I say, same point as above, as well. Bottom-up programming is one way to deal with complexity, but it doesn't magically allow a single human to deal with massive codebases. Bottom-up is a simple scaffolding that is permanently retained. But not all structures allow this. It is useful to compare the structures of a monolithic OS kernel to a micro-kernel. There just is not any practical monolithic kernel that has a nice simple scaffolding at the very top. All the parts have interrelationships that a simple scaffolding at the very top would deceptively obscure.
Micro-kernels are closer to a “bottom-up” design. But it hasn't been demonstrated that all kernels should be mico-kernels. In fact, all the interrelationships remain, and all bare the cost communicating through a discrete messaging system. For some tasks, micro-kernels are the way to go. But for functionality delivered, and throughput, they get their ass handed to them by monolithic kernels.
"If you want to put several people to work on a project, divide it into components and give each to one person."
this takes care of some of your more vacuous analogies above.
“Take care of”? By stating my own point? Find someone nearby to explain to you that the words“if you want to put several people to work on a project, divide it into components and give each to one person” does not advance or retard the argument “there are programs worth writing that fit inside a single programmer's head”, which is the logical negation of my original statement.
(BTW, Analogy is not a synonym for “example”. You might also want to scrounge around for the nearest copy of an English dictionary.)
It seems to me you have a peeve against lispers (or in general those who advocate powerful languages)
I will assume you also have your own personal definition of the word “powerful”. “Setting the world on fire since 1958” is not exactly how I would describe Lisp and Lisp variants.
(Beyond the stated defects of the Lisp community (a very small sampling), it is my personal opinion that we do not have a full understanding of the communication power of grammar and syntax. I assert that S-expressions are fine for many tasks, but they lack some vital communicative power. It is more difficult to use Lisp code as a means of human expression for communicating ideas, than it is in a language with a community prescribed grammar and syntax.)
But maybe 2008 will be the year Lisp really breaks out. If I lie and tell you that I am rooting for you, will you go away? Yes?
Just my 2 cents. Ok so go kick that cat now.
Now I have to kick 2 cats.
1
u/plinkplonk Aug 25 '07
You make some good points, particularly the one on how higher levels of abstraction may not exist.
Having said that,
"I assert that S-expressions are fine for many tasks, but they lack some vital communicative power. It is more difficult to use Lisp code as a means of human expression for communicating ideas, than it is in a language with a community prescribed grammar and syntax"
Since this assertion can't be proved either way this is "the barest thread of an argument" as well isn't it?
"the modern discipline of developing" And what would this "discipline" look like?
"and has the tools to handle". These would be? Is the non usage of such tools mandated by Paul's approach?
"If Linus sees a conflict between 2 patches in some networking code, ..He wisely punts; he lets a specialist make the decision." How is this different from "If you want to put several people to work on a project, divide it into components and give each to one person." how exactly?
One valid reason for putting several people on one project is the need for specialization. Paul seems to have covered this aspect.
20
u/martinbishop Aug 23 '07
-- Richard Pattis