Having recently (last year) enrolled in a Computer Programming program, I can say this is likely the case. The class has a number of people who are simply enrolled to get a piece of paper that states they know how to code and then there are those that are breaking new ground in learning.
I have the logic down quite well but my biggest issue is the syntax - it would appear that Stanford thinks I'm doomed to fail.
One of the most touted advantages of Scheme (and other Lisps) as a teaching language is precisely the (lack of) syntax which allows the student to focus on logic.
One might as well claim that C++ has no syntax, as its source code is nothing but a linear series of undifferentiated octets. Scheme and other lisps have plenty of syntax; it's just expressed at a different level of abstraction than the one most languages use.
C++ is an abomination - it should not only be avoided when teaching, but we should make all the effort to avoid it whenever possible. Learning programming using C++ as a language is like learning to walk on a grass-covered minefield - almost every feature hides something that can blow your leg off.
Anyway, the fact remains that you need much fewer syntax lessons to get started with Scheme compared to anything else.
It seems to me that there are way too many variables that aren't really measurable for this to be completely accurate -- such as this example, some people probably had programmed before.
Anecdotally, at my university (graduated 2013 so still a recent experience), a lot of CS students took the intro to programming course. In fact, it was (and still is) a requirement for a BS in CS at the school. The only way you could get out of it was if you had taken AP CS in high school and passed the test for the credits.
A large majority of my CS friends in that class had all been programming for either a little bit or quite some time, and the class was mostly a joke for them. Lectures were optional, and a bunch of friends (including myself) only attended three classes the entire semester: the first class (where we found out we didn't need to go to class except for tests) and the mid term and final.
Of course on the other side of things, I had friends who had never programmed before and found the course grueling and struggled to pass.
I TA for this class. The class is optional for non-engineering majors but has enrollment >= 700 people per quarter. So many people at Stanford take the class that it's a bit of a cultural phenomenon.
There are three "intro" programming classes at Stanford. CS106A is "intro to java", CS106B is "intro to C++", CS106X is CS106A + CS106B + a bit more in one quarter. People who already know how to program are encouraged to take one of the latter two, although there is always a contingent of people with /some/ background in 106A.
For non-engineering people, Java seems like a slightly odd choice, is it not? So much overhead in learning, plus overhead if they did want to integrate coding into their lives in a light way. Why not Racket or a scripting language? As much as it pains me, even JS might be of better use.
Or is it just the case that these people just wanna take one course from the "real" CS for a non-coding reason?
Because the assignments, section material, grading infrastructure, handouts, teacher training material, and institutional knowledge for this class are all built around Java when the class switched over in the early 2000s. In the past, couple years the process of changing just one assignment out of the six or seven assignments was difficult for everyone involved (teaching staff and students). Changing the entire class would require monumental effort, and it would have to be driven by a majority contingent of the half dozen professors / lecturers involved in the introductory CS community at Stanford.
Given that the class has been successful at teaching programming methodology to tens of thousands of beginner students over the past decade there's no pressing need to restructure the entire course, even though everyone does acknowledge that Python or some other language could be better for new students. It's basically like any other large software project, yeah it could be rewritten from scratch and modernized to make small gains, but if it ain't broke, why fix it, especially when the professors and TAs have other projects that have far more interest in, like this paper.
Also one thing that many people don't know is that CS106A actually changes the language environment as well as providing a modified version of Eclipse to remove some of the less useful overhead in Java (for example removing main, and the beginner confusion surrounding keyword static, arrays and arguments). Also there's a much simpler introductory class called CS105/101 that uses Python.
Source: I taught sections and managed the TAs for the introductory classes over the course of four years while I was a student at Stanford.
From the actual article (and not the news article): "The results show that students’ change in programming patterns is only weakly predictive of course performance"
Not as "juicy" as the website - but then again, it's the actual results :)
Hey! I worked on the preliminary efforts of this project in 2012, and was also a grader for the course. (I'm "Gibbons, A." in the citation list)
Many people are making great points - there's a wealth of uncontrolled variables (ie some students already know how to program), but it was a fascinating dataset and I could even see the paradigms play out over the quarter during discussions with my students, both experienced programmers and students getting their first exposure to programming.
I would offer as a larger takeaway that work like this has the potential to vastly increase our understanding of learning and development with the rise of MOOCs, ipads-in-every-classroom, and of course IDEs that save every incremental compilation :-)
Oh and as another mention - working with Marcelo and Paulo was awesome, both are super passionate, intelligent dudes.
Fantastic! Good steps towards "artificial wisdom". Define wisdom-- skill in a craft (here: knowing how to think). Using ML to identify "wise" programming mindsets. Then (presumably) developing educational programs which teach these.
“Educational data mining should not be used to reinforce the existing ineffective forms of assessment, but to reimagine it completely,” Blikstein said. “Pre- and post-assessment is a black-box. We do not know much about what is happening in between.
“Ultimately, this work may help us better understand some forms of human cognition because you can see what people are doing and how they are thinking in real time.”
>> the authors did not find any correlation between the amount of “tinkering” — which is often thought to signify lack of programming expertise — and course performance. However, the authors found that students who changed their programming style — going from being a “planner” to a “tinkerer,” or vice versa — were the ones who performed best, suggesting that behavior change (rather than learning one single behavior) was determinant for course performance.
Not being flippant: The conclusion here seems to be evidence of an ability to learn something new, regardless of what, is strongly indicative of performance.
> "The discovery of these “sink states,” and how students
> got into them, offers opportunities beyond predicting
> grades: They open the door for developing systems to
> encourage students to go down more fruitful paths before
> they become lost in the programming weeds."
In the real world, my most valuable learning experiences have been those dead ends. Because I don't repeat those mistakes - better to make them and learn from them than to be steered away from them before they happen.
Relatedly, I have seen plenty of very smart people that were able to bowl over these "sink states" completely unaware of just how unmaintainable their ultimate solution was.
>> Alpha students, explained co-author Piech, moved efficiently from one point to another as they wrote code, creating a streamlined program. Beta and gamma students, on the other hand, wrote themselves into so-called “sink states” in which their programs slammed into dead ends from which they had to back out before they could resume progress.
Very interesting, and intuitively right to me. The difference is likely in the ability to create and maintain a consistent mental model of the program's structure and behavior during the implementation.
I think data mining editor/IDE use is a pretty useful idea overall and the "paradigm shift" of constant monitoring which allows immediate teacher feedback is much needed (not just for programming).
The other interesting approach I read about a while back was using tracing quiz data to identify "blind spots". A typical example was identifying students that had problems understanding loop constructs (they would always assume one iteration for example).
My gut instinct and thinking back to my university days make me think that a blind spot for recursion may very well be a thing. I'd be pretty interested in identifying students that struggle with loops and/or recursion and investigating that further. For example...is there a correlation? If not what happens if you give a loop-struggler group only recursive tasks and a recursion-struggler group only loop tasks? Etc.
It's not necessarily surprising that they might be able to detect students who will likely perform poorly in the course overall by monitoring their behavior early on. But the real challenge is not in determining which students are struggling, but rather in when and how you should intervene to improve these students' chances of success.
Begs the question: "Is there a 'right' way to think about programming problems?"
My knee-jerk reaction would be "No! Everyone learns and thinks differently"-- but being able to put programming styles into buckets and correlate final grades from it suggests otherwise, doesn't it?
It might suggest something, but it's just a hint. In order to take that approach as conclusive much more comprehensive studies in to highly diversified environments should take place.
All that leaving aside that it's a social science we're talking so... It's not exactly science as in scientific method. It's just models that (under X circumstances) might or might not apply... Go figure ;-)
I think the emphasis is more about the way people think about solving programming problems than the way they think about the problem itself. An analogy is that two people may differ in their mental models at the level of syntactic sugar rather than Turing completeness.
Completely anecdotal, but I vividly remember my own experience learning to program.
I learned in a vacuum, having nobody around with any software experience. I was given a PC and then saved up for an Amiga 500 (and later a $500 Lattice C compiler). My only link to software was through the bookstore and magazines.
Being a nerd, my main motivation at the time (of course) was to make a D&D game. I remember my first code was in Basic (no subroutines!) so I eventually learned to create subroutines based on line numbers (100-900 was main, 1000 was map, 1100 was character A, etc)
I managed to make about 50% of a single game before I ran out of memory. So I then set out to shrink (optimize) what I had made. After reading an article on dungeon map creation, I figured out that large arrays could replace the hand-coded drawing I had written. Check.
Ok, so now I start doing everything in arrays and pretty soon my code is smaller but "fixing stuff" (maintaining code) was stupidly hard. Oh okay, so I learned then some that data and code should be together rather than global (encapsulation).
I then needed to create random maps. After struggling with that, I read a paper on fractals, and implemented my first algorithm.
You can probably guess by now where I'm going with all this... I had crudely recreated many things by knocking up against hardware and brain limitations and figuring out how to work around them.
I progressed into Pascal, C, C++, assembly and eventually created my own assembler and graphical operating system at 16 years old. It was not a thing of beauty, but it did work. (Time was free, and compilers weren't cheap.)
I think that most people learn programming by failing, hitting traps, finding limits, crashing, burning and trying again. The only real identifier of outcomes, in the long term, is the willingness to continue trying and learning from mistakes.
So from my point of view, the outcomes of the paper are arbitrary in the long term. I would definitely had been a "Gamma" programmer. I hit every problem, and got terribly stuck. It took years for me to progress, learn good practices and become a proficient developer.
What was interesting is that the higher-performing group (Alpha) was stuck less often that the lower-performing group (Gamma). Is this the next programming motto after "Don't Repeat Yourself"?
Don't Get Stuck!
Now how does one avoid getting stuck while programming? I still get stuck a lot, but these really helped:
- Test what you write as quickly as you can.
- Hold it in your head! Or you won't have a clue what all your code, together, is doing.
- To get started, ask yourself, what is the simplest thing that could possibly work?
> Hold it in your head! Or you won't have a clue what all your code, together, is doing.
Do you mean to say: if you can't hold it all in your head, you're overcomplicating things? Because if so, I agree.
I just finished giving an introduction to programming course, using Processing. I tried teaching my students to write structured code from the start, splitting functionality into smaller methods as much as sensible. That way they never have to juggle too much functionality in their head at once.
On the first day I had them call built-in methods. Then I introduced the concept of variables and types, then I introduced defining your own methods, and only after that did I move on to booleans, if/else, for-loops and arrays.
It seems to have worked well for getting them to structure their code from the start, using methods more or less the same way you would use chapters, headers, sub-headers and paragraphs to structure a paper.
I know this isn't the best way to program, obviously, but this analogy is (relatively) easy for them to understand and loads better than the ball of muddy spaghetti you often see with people who just start out. It's also probably fairly easy to make the jump to other more advanced forms of code organisation.
I also completely agree with your other two points by the way, I kept repeating similar statements throughout the course:
- By splitting functionality into small methods, it's easy to test if something does what it is supposed to do (and I could easily correct them: "Hey, this method drawBall() is also doing hit detection. That's not what the method is supposed to do, restructure the code!").
- "The computer only does what it is literally instructed to do. What do you literally want to do? What are you literally instructing the computer right now?"
When I see students posting on /r/programming, what they usually seem to have a problem with is problem decomposition. i.e.: What are the steps needed to solve this part of the problem? Or algorithm design, critical thinking, or whatever you want to call it. How do I get from A to B, basically.
Remembering back to my first year, it seems that some people "get this" and some do not. Which is nothing against them -- everyone is different.
> - "The computer only does what it is literally instructed to do. What do you literally want to do? What are you literally instructing the computer right now?"
I feel like after you really know a system you almost start thinking only in terms of its components. "Hmm I want to do X...somehow I just automatically thought to modify function Y and call it in module Z."
Wow, this was a big wake-up call. It's ironic because I am in research and I always did a lot of theory and problem definition up front for whatever I did, but I am now in this "reckless rapid prototyping" phase where if I get some idea I have to try it out RIGHT NOW, stepping back be damned. Ironically it results in getting stuck more than I would have wanted!
I share your concern that this will result in a lot of sloppy solution to lots of little problems which in turn results in one big mess of sloppy-but-it-works solutions which can be a big problem (but doesn't have to be). However, even when working out a good solution I think it's important not to get stuck. Make sure you are making some sort of progress; it's okay that it takes time but it is not okay if someone is just moving in circles instead of inching closer to the solution. Basically if something takes "too long" to get working, try a different approach or come back to it at a later time to try again. I quite like the "Don't get stuck" motto if we look at it that way.
"If you make "don't get stuck" a motto, you can easily imagine people skipping over difficult problems and not thinking about a good solution."
It is also important to define, what "getting stuck" means. For the research the alpha performers where the ones who fulfilled the goal quickest. If you switch the point of view from the teachers to the learning student, instead of fulfilling the requested goal quickest, you might follow an interesting problem that you discovered on the way. Is that getting stuck? No! It is following your own path of learning which is a very effective method. It does not look best to the teacher, but it can be very good for the learning student.
Asking your own questions and answering them is an important part of a learning process. It also makes you independent, so you don't simply do most efficiently what someone told you to do.
Overcoming getting stuck is the difference between a highly productive dev and the average dev. This is especially the case if the problem being worked on is hard. I am a very average dev in regards coding skills, but due to my background I have learned strategies to “unstick” myself after ending up in the middle of a massive mud pit. This has allowed me to solve problems better devs have not been able to accomplish.
Until you run into that one problem where you can't test between the small steps, because you need the whole thing to be up (to some degree) and working for any test to work.
Operating system kernels would be an example of that: The best test for a *nix OS kernel is, if it can run a shell. You need all the essential syscalls to do something sensible and if any of the required parts doesn't work the whole thing fails.
Another example would be refactoring a complex library into something more manageable. If you keep working in small, testable steps you can move only along the gradient, bordered by "can execute" and "doesn't execute". So you'll be able to reach only a local extremum. Now if you're in the fog and don't know where to go, that's fine. And for most development this is exactly how it happens. But sometimes you can see that summit on the other side of that rift and you know you have to take a leap to get over there.
I spent the past 3 weeks doing exactly that, refactoring a code base. I knew exactly where I wanted to go, but eventually it meant working for about a week on code without being able to compile, not even think of testing it, because everything was moving around and getting reorganized. However now I'm enjoying the fruits of that week; a much cleaner codebase, easier to work with and I even managed to eliminate some Voodoo code nobody knew why it was there, except that it made things work and things broke if you touched it.
> - Hold it in your head! Or you won't have a clue what all your code, together, is doing.
Or, sometimes it's important to get it out of your head, take a week or two off and look at it again with a fresh mind and from a different angle. Often problems seem only hard because you're approaching them from that one angle and you're so stuck with wanting to get it done, that you don't see the better way.
Instead you should write code in a way that it's easy to get back into it.
> - To get started, ask yourself, what is the simplest thing that could possibly work?
And then wonder: What would it take to make this simple thing break. Make things as simple as necessary but not simpler.
> Operating system kernels would be an example of that: The best test for a *nix OS kernel is, if it can run a shell. You need all the essential syscalls to do something sensible and if any of the required parts doesn't work the whole thing fails.
So start with something simpler. Start by making a kernel that can run /bin/true, that never reclaims memory, that only boots on whichever VM you're using for testing. You absolutely can start with a kernel that's simple enough to write in a week, maybe even a day or hour, and work up from there. See http://www.hokstad.com/compiler for a good example of doing something in small pieces that you might think had to be written all at once before you could test any of it.
> I spent the past 3 weeks doing exactly that, refactoring a code base. I knew exactly where I wanted to go, but eventually it meant working for about a week on code without being able to compile, not even think of testing it, because everything was moving around and getting reorganized. However now I'm enjoying the fruits of that week; a much cleaner codebase, easier to work with and I even managed to eliminate some Voodoo code nobody knew why it was there, except that it made things work and things broke if you touched it.
Which is great until you put it back together and it doesn't work. Then what do you do? I've watched literally this happen at a previous job, and been called in to help fix it. It was a painful and terrifying experience that I never want to go through again.
In my experience with a little more thought you can do these things while keeping it working at the intermediate stage. It might mean writing a bit more code, writing shims and adapters and scaffolds that you know you're going to delete in a couple of weeks. But it's absolutely worth it.
If you haven't, you really need to see this post.[1] Starting simple and writing a test driven algorithm is not necessarily bad. However, realize that you are really just turning the act of finding the optimum solution into a search space where you have to assume mostly forward progress at all times. Not a safe assumption. At all.
And, because I love the solution, here is a link to my sudoku solver.[2] I will confess some more tests along the way would have been wise, though I was blessed by a problem I could just try out quickly. (That is, running the entire program is already fast. Not sure of the value on running the tiny parts faster.)
I've seen it, but it's just utterly alien to my experience. Partly the problems I encounter professionally don't look much like Sudoku; partly the things that are important on a large codebase are different from the things that are important in a small example. But mostly I think people realise they're not getting somewhere - and if they don't, others will point it out. That's partly why TDD tends to be found in the same place as agile with daily standups - you get that daily outside view that stops you just spinning your wheels the way that blogger did.
I have seen exactly this style of thinking happen in a large code base. Some of it was my own, sadly.
The odd thing to me, is you say this still of problem doesn't happen in a large code base. But, to me, this style of problem just happens many times in a large codebase. That is, large problems are just made up of smaller problems. Have I ever used the DLX problem? No. Do I appreciate that it is a good way to look at a problem you are working? Definitely. I wish I had more time to consider the implications there.
More subtly in your post, to me, is the idea that with the right people the problems don't happen. This leads me to this lovely piece.[1]
I think as you get more experienced, what is considered a "small step" changes as you're able to keep more context in your head (subconsciously of course). For the complex library example, that would be keeping the new vs the old architecture at the forefront of your mind instead of switching all thought to "const? maybe? what does that mean in this context?" and "how can I get rid of this reinterpret_cast?"
Absolutely agree with "Instead you should write code in a way that it's easy to get back into it" though!
> Operating system kernels would be an example of that: The best test for a *nix OS kernel is, if it can run a shell. You need all the essential syscalls to do something sensible and if any of the required parts doesn't work the whole thing fails.
How do you test kernel? Run a virtualised kernel? I think at least DragonflyBSD supports that.
Kernel debugging always has been kind of a dark art. What helps greatly is if you can tap into the CPU using some hardware debugging (JTAG or similar).
Today you can also exploit the busmaster DMA capabilities of IEEE1394 to peek into system memory.
However the still most widely used method is pushing debug messaged directly into a UART output register, bypassing any driver infrastructure, and hooking up a serial terminal on the other end.
It's also likely that these guys had already programmed before coming to the class. We all tend to get stuck when programming something for the first time. Second and third times are much smoother experiences.
> Before you can write your own subroutine, you have to know how. This means, to be practical, that you have written it before; which makes it difficult to get started. But give it a try. After writing the same subroutine a dozen times on as many computers and languages, you'll be pretty good at it.
Or, as Fred Brooks put it, “plan one to throw away”.
I'd offer two bits of marketable wisdom from this.
- think back. Students who change style do better. This implies that those students are reflecting on their own style and trying out techniques (performing experiments) to find out what works best for them.
- think ahead. Use lookahead to avoid backtracking. Some amount of top-down planning will make your life easier and avoid traps, even if your natural style is bottom-up.
My interpretation of that part of the article was "refactor early and reimplement often"
That grabs in the concept of switching between planning and experimenting modes with the end result of not getting stuck.
I may be biased in having learned this stuff a long time ago so when I get stuck its from getting hopelessly backed into a corner at which point the best solution is pretty much start over differently, and first semester programmers might get trapped a little more easily, like they don't even know common ancient anti-patterns so they don't even know where to start looking when debugging.
Reading between the lines it almost sounded like the A level programmers wrote their tests first (which is usually easy) then wrote code to pass the tests (usually not too awful) then they were done, whereas the lower grade level programmers wrote the code first, then tried to debug (and debugging is always harder than test writing or coding)
Yeah I really reckon that "what is the simplest thing that could possibly work" is the vital question. If you stray from that, getting stuck wastes what could otherwise be productive effort. I do have to wonder tho -- why is programming so complicated? Silicon and neurons so fundamentally different that we're always going to be fighting ourselves -- or one day we're going to have interfaces make programming way more natural?
> - To get started, ask yourself, what is the simplest thing that could possibly work?
I'm not sure I would go for what appear to be the simplest now ; I try to find the implementation which will have the simplest design to use/modify/delete in the future.
