Hacker News new | past | comments | ask | show | jobs | submit login

Someone tell me what CoPilot generates for this:

  // A Go function to swap the sixth bit and seventeenth bit of a 32-bit signed integer.
Here is a human solution:

  func swap(x int32) int32 {
      const mask = 1 << 5
      var (
          xor1 = (x>>11 ^ x) & mask
          xor2 = xor1 << 11
      )
      return x ^ xor1 ^ xor2
  }
I would be surprised if CoPilot can reason numerically like this (understand "seventeenth bit" and "sixth bit" and generate the right code for that combination).



For python it seems to generate a more reasonable result at least:

    # Swap the sixth bit and seventeenth bit of a 32-bit signed integer.
    def swap_bits(x):
        # Get the bit at position 6 and 17.
        bit6 = x & (1 << 6)
        bit17 = x & (1 << 17)
        # Swap the bits.
        x = x ^ (bit6 << 17)
        x = x ^ (bit17 << 6)
        return x


The function doesn't do the comment says it does. The code to "Swap the bits." just turns the bits on.

    >>> def swap_bits(x):
        # Get the bit at position 6 and 17.
        bit6 = x & (1 << 6)
        bit17 = x & (1 << 17)
        # Swap the bits.
        x = x ^ (bit6 << 17)
        x = x ^ (bit17 << 6)
        return x
    
    >>> x6 = (1 << 6)
    >>> f"{x6:b}"
    '1000000'
    >>> s6 = swap_bits(x6)
    >>> f"{s6:b}"
    '100000000000000001000000'
Here's one that correctly swap bits. It could be made more concise.

    >>> def swap_specific(x,i,j):
        def get(x,p): return 1 if x & (1 << p) else 0
        def set(x,p): return x ^  (1 << p)
        def clr(x,p): return x & ~(1 << p)
        bi, bj = get(x,i), get(x,j)
        x = set(x,j) if bi else clr(x,j)
        x = set(x,i) if bj else clr(x,i)
        return x

    >>> f"{x6:b}"
    '1000000'
    >>> b6 = swap_specific(x6,6,17)
    >>> f"{b6:b}"
    '100000000000000000'


almost but this part is swrong

    def set(x,p): return x ^  (1 << p)
should probably be

    def set(x,p): return x |  (1 << p)


I guess CoPilot has seen bit swapping in its Python training input but not in its Go training input.

The Python code is wrong because the 17th bit is shifted up, not down. Also, the bits are shifted by the wrong amount, not up/down by 11 (= 17 minus 6), but up by 6 and up by 17. What a joke.

Not only that, even if the shifts were correct, it's simply xoring the bits. The swap is completely wrong.

Garbage code, total fail.


Yeah, it seems to be pretty heavily trained on Python. It's honestly still (and should be used as) a glorified autocomplete, which is pretty useful from time to time.


You can open a vscode tab with copilot and have it synthesize many autocompletions.

If I add your comment, a newline and the "func" keyword it does this

https://gist.github.com/CapsAdmin/9fe57314ab9f77bec0445eb042...

If I add some more context, ie "func swap(" you get

https://gist.github.com/CapsAdmin/2c8757887c27f980d1a4c2edfd...

more context, "func swap(x int32) int32"

https://gist.github.com/CapsAdmin/64e0392fe6f3d406a607884583...

I use and like copilot, but when it comes to things like this I generally don't trust it.

I'm not very fluent with bit swapping either, so I would probably resort to google on this one.


All wrong. Every solution is garbage.


With just that prompt, Copilot keeps writing a comment about the function but never actually writes the function. Prompting to actually write the function by starting it with `func` gives:

    // A Go function to swap the sixth bit and seventeenth bit of a 32-bit signed integer.
    func swapBits(x int32) int32 {
        return ((x & 0x0F) << 28) | ((x & 0xF0000000) >> 28)
    }


Totally wrong, it's garbage.

And there you have it, the difference between real intelligence and regurgitation.

This is the kind of numerically specific coding that could be the basis of a CAPTCHA that CoPilot can't solve. Sixth bit, sixth byte, seventeenth bit, seventeenth byte, etc.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: