It's frightening to see how many of these are errors which are solely created due to copy-pasting - they could have been avoided rather easily by a bit of thinking and applying common good practices (typically storing functionality in a function and/or using a loop would already fix like 90% of these, aka DRY), instead of the lazy but easy 'solution' of copy-pasting.

One of the first things a programming guru told me was basically: whenever you are ready to hit Crl-C/Ctrl-V (or Ctrl-D or whatever 'duplicate' is in your editor) you shall ask yourself if you are not going to violate the all-important DRY rule. Ignorant and stubborn as I was at the time being I was like 'pfffft what can be wrong with some copy-pasting'. As I learned through the years and is shown by this quiz: a whole lot obviously. It can not be stressed enough how important and widely applicable yet so simple DRY principle is. So as a call-out to all programmers who haven't learnt the habit yet: please think twice or more before even considering copy-pasting.

You're not wrong, per se, but C++'s memory model tends to make refactoring this sort of thing a lot more painful than needed. If you introduce function you have to spend a lot of time thinking about how do the variables copy over the stack, is this data structure safe to pass to a function? And do I pass by reference? And do you return values from the stack or in allocated memory or write into a passed reference? Or do you get really gross and write a macro? And so on, until you just think "fuck it, copy-paste".

It's not an excuse, but languages definitely shape how you think, and C++ definitely encourages long functions and copy pasta.

I don't disagree with you per se. But I think you may be overstating the case.

Once you are a C++ programmer for a while, you start to discern "standard" patterns for this sort of thing and tend to automatically use them. (The patterns might depend on the C++ version and to an extent the code base in question; but it is common to have such helper function take a const ref as input, return non-primitive return values via a out-parameter etc).

So while C++ might "encourage long functions", I think that may be the case only for newbies to the language. Once you have a some experience working with it, you no longer need to put that much thought in to every little decision.

As I've been following posts on PVS Studio for a while, I'm sort of familiar with the kind of errors it finds.

I was able to spot 13 of these errors -- although the page only counted that as 10 as it's very picky of which token constitutes the error.

PVS Studio (and CppCat) are great tools. I can't unfortunately use them as I don't write stuff for Windows.

Try to use CppCheck or LLVM's scan-build. They are pretty decent. I've been using them for a few weeks for now and I found quite a few errors and send patches to various open projects (Pidgin, util-linux, GHC).

Ah, yes, I'm familiar with these tools. They're indeed pretty good.

In addition to them, I also use Coverity (they offer free licenses for open source projects) and Coccinelle (although it's a tool to refactor/create semantic patches, can be also used to find defects if you know what you're looking for -- useful if you fix one bug and you're wondering if there are more like it lurking).

Having different opinions about your codebase is often good as the approach in finding defects is usually different between them. For instance, stuff by the PVS people seem pretty good to find copy/paste mistakes, which I've rarely found with other tools.

I say this as a C++ programmer: I think a lot of these errors are non obvious because of the sheer amount of syntactic noise surrounding them. I also write a lot of python, and a lot of these errors are still possible but way more obvious. C++ is seriously the worst about surprising syntax causing temporary blindness from overload, excluding maybe Perl.

Static code analysis is fantastic for a language like this (if you can afford it; not sure what the pricing on this product is -- can't say I've ever seen a cheap static code analyzer though :-) )

I wrote C++ for a few years professionally and found myself able to look through the noise quite a bit. I remember doing a side project in python to expand my horizons and I felt naked without blankets of parenthesis and brackets around all of my stuff.

So, I don't think it is the noise of the syntax; it is the noise of a large code base. I found 12/15 errors (it scored one as missed, though I did find it).

I did well, because I was actually looking for bugs; and I didn't write the code, so had no preconceived notions of what it was supposed to do.

When you have 10s or 100s of thousands of lines of code in your project, however, you aren't going to know where to look for a bug (because they could be anywhere or everywhere); and one can often be blinded by knowing how it is supposed to read, not how it actually reads.

clang's static analyzer performs well and is exceedingly inexpensive

The first question it asked me was:

  int SSL_shutdown(SSL *s)
  {
    if (s->handshake_func == 0)
    {
      SSLerr(SSL_F_SSL_SHUTDOWN, SSL_R_UNINITIALIZED);
      return -1;
    }

    if ((s != NULL) && !SSL_in_init(s))
      return(s->method->ssl_shutdown(s));
    else
      return(1);
    }
    ....
  }

so of course the first thing I noticed was the mismatched braces after the 'else'. The 'intended' error was elsewhere (and I agree with it) but to say that the brace isn't an error is a bit much given the sorts of problems that PVS is normally finding.

It's hard to say about that, since there is ... which can contain anything (like for example another opening { ).

It's a good practice to always use braces with if / else even if it's just one line in them.

I don't think you are right. Regardless if what is after that contained in the `...` wouldn't matter. A lone `{` or `}` would be a syntax error.

As I said, it wouldn't be a syntax error if ... contained the matching { somewhere inside.

That lone } which seemingly belongs to if in this case would be closing the function.

Yes, that's what I understood. That } after return matches to the function opening {. So it's correct syntactically, but is wrong according to intention.

Yes there's a typo / mispaste in the code there, but of course that wouldn't compile and they're not asking what wouldn't compile ('....' wouldn't compile either).

A lot of these aren't C or C++ specific. Many of them were typos, for example.

There are also some ambiguities. At least two questions I encountered contained more than one bug, or at least one explicit bug and one "occasional" bug due to a lack of testing in the code (say, for division by zero).

Also, it'd be a a good idea to put a disclaimer for the quiz taker to assume types, variables and other tokens are declared properly. It's easy to make the deduction after getting the first question which you get incorrect because you chose the "wrong" bug.

I agree. It's also worth noting that the title is wrong, this is strictly a C++ quiz, there are no C questions.

Though I know very little C++, a lot of the questions were applicable to a C programmer

In fact I think most of them are more applicable to a C programmer. Most C++ programs aren't going to be making extensive use of, say, snprintf, which several of the errors I was shown involved.

OpenSSL is written in plain C.

I can't see why the test say at the end "You should stay away from C++", when the real test should be "Are you smart enough to use a static analyzer to catch those errors that are truly hard to find just passively[0] reading/writing code?"

[0]Because of course, when you know there's a bug, it's already easier to find...

I got the first one they showed me right, in that I identified the issue as a mismatch between conditional and code, but I clicked on the code line instead of the conditional line and it was wrong, so peace out.

I spotted some errors but it always marked the answer as wrong... It's a fun exercise but that's a bummer. Still seems like a good tool, nonetheless.

It seems to be very particular about exactly what you clicked. For example in one bug I got the problem was that they incremented the wrong variable in an inner loop, but clicking on the variable name at that point in the program gave me a result of incorrect, but the bug description was spot on.

Also I can't figure out how to tell it that there's a missing format specifier in question 1 so it marked my answer as wrong. Missing items in code are impossible to click on.

On question 2 it gave me 60 seconds to visually deconstruct a statement, which is pretty unfair.

I gave up after that. I guess this thing exists to "show" you that you can't spot any errors and that you should just buy their tool?

I think you click on the argument that's being passed that doesn't have a matching format specifier.

I found the bug in the first question it asked me fairly quickly, but after trying about 6 times with the back button couldn't figure out where to click to get graded correct. (it was the same subexpression appearing twice in an or expression).

I got 13/15. Mostly copy-paste errors that really happen a lot in code.

I was mislead by

    int t1 = something.getArg(1);
    int t2 = something.getArg(); //I clicked here

    ...

    something else with the error they had in mind

What is the market for this tool? Is it for programmers? Is it for management that _doesn't_ know how to code, so they just run the program through this tool, if it passes, push to production?

Developers who want to find bugs in their code, preferrably before they blow up horribly in production. IMHO for C++ static analysis isn't an option, it's mandatory.

You have found 10 errors out of 15. But I found 3 others but clicked on the wrong piece of text as it isn't always clear which piece they've attached the trigger too.

It would be cool to have same for other languages. I'll see if I'll be able to do same for python

It's a nice quiz, but many of these problems are found by regular compilers (clang, anyway).

First must selected. See description.

Let's simplify the code to make the error more obvious:

         if (a & Min_Max) { Min[0] Min[1] }
    else if (a & Max_Min) { Max[0] Min[1] }
    else if (a & Max_Max) { Max[0] Max[1] }
    else if (a & Min_Max) { Min[0] Max[1] }

It should be evident now that every line except the first one contains the correlation between names of constants and names of arrays. So, in the first expression the 'sC0Min1Min' should have been used instead.

I've seen quizzes of lesser quality in interviews. One bad example that commes to mind: someone foratted a quiz in "Word", and "Word" auto-capitalized Java's "boolean". That's a rather big difference in behavior.

I promised myself I'd walk out on any quiz in the future. I haven't had to hold myself to that so far.

Refering to questions by number is useless since they are choosen randomly from a set number.

Mea culpa. Thanks.

So it had to be found again: https://news.ycombinator.com/item?id=8335217

2nd question I got

   CV_IMPL IplImage * icvCreateIsometricImage(....)
   {
   ....
    if (!dst || dst->depth != desired_depth ||
        dst->nChannels != desired_num_channels ||
        dst_size.width != src_size.width ||
        dst_size.height != dst_size.height)
       ....
    }

The website said the error was 'height was compared with itself', fair enough ... but there is a much much bigger crime here.

I clicked on the (!dst ||) as the code checks for a null pointer and then follows that with a dereference of that pointer which will get executed because of the || operator.

If the pointer is null the code will still crash.

The website said I was wrong .... I didn't bother with any more questions after that.

You're thinking of the '|' operator, which evaluates both arguments. The '||' operator is defined to never evaluate the 2nd if the first is true; its traditionally used exactly for this purpose e.g. if (!p || p->...) or more likely if (p && p->IsGood(...)) {edit: false -> true}

I have to put my hand up here and admit I got over excited and I was wrong. Thanks for correcting me.

As others have pointed out, || is short circuit.

I'm chiming in not to pile on, but to point out this is a canonical code pattern in C and C++. Check the pointer, then deference it, all in the same if expression.

   if (ptr && ptr->foo...

or

   if (!ptr || !ptr->foo...

The website was right.

http://en.wikipedia.org/wiki/Short-circuit_evaluation