That is an awesome project, much better than the QR Clock I did.
My experience with the clock was that I really needed ECC mode 'M' to get the maximum ECC bits which worked around all sorts off issues with differences in scan rates between display and reader.
In my setup the QR clock is displayed on an array of 64 x 64 LEDs, when the LEDs were operating at a 60hz refresh rate you could induce some pretty interesting interference with the camera scan rate on phones with QR readers, once I boosted the refresh rate to 240 hz that interference went away, then it was a question of how much "dwell time" did the code need to be recognized and I got it down to 100mS as the fastest I could go and the QR libraries on the iPhone and Android could still pick it up.
So for reliability considerations maximize the display refresh rate and the ECC correction level.
oh how i wish Microsoft didnt 1] have a patent on color barcodes 2] choose to do nothing with it
Still, perhaps we could get a decent barcode format with some of its ideas, idk if they are all patented. It uses triangles which can be packed a lot more densely than squares, and has four- and eight-color modes. The density is nuts. You could theoretically download a decent size book via color barcodes in a couple seconds. It could revolutionize all kinds of things, being able to have large amounts of data in a barcode, rather than just a URL.
Let's assume a version 6 QR code, which is 41x41 (larger is unreasonable IMO). Now let's assume that using triangles is a twice as dense packing, and we use 8 colors. Also we don't need error correction.
Then we have 3x41x41x2/8 = 1260 bytes of storage. Even with compression that's still not a decent sized book.
---
EDIT: I'm guilty of not reading the article first. I assume now you meant a 'QR film' that you scan for a couple seconds, rather than a static image. In that case you can apply above calculation to '1260 bytes per frame', at 10FPS that'd be 100Kbps, which is reasonable.
Not a film, something printable, that lasts without battery or can be put on a poster at a protest. At 3kB/in^2 in a 3x3 grid of barcodes you'd scan left-to-right, top-to-bottom, if the text itself were compressed with PPMd down to 25% of its original size, that 3x3 barcode grid gives you 108kB of text
or 12kB in a single barcode (of plain text, compressed down to 3)
wikipedia says: "Microsoft claims that laboratory tests using standard off-the-shelf printers and scanners have yielded readable eight-color HCCBs equivalent to approximately 3,500 characters per square inch"
I've read about this a bit and I really wish there were an unpatented color barcode and a reliable/fast open source reader also, even if it's not quite as dense as my dreams. I think it would make for some neat possibilities.
Rabobank in NED uses QR-like square codes with color. While it generally works well, it breaks down when you try to scan from a screen running f.lux or night shift.
Flashing qr codes was just a quick hack and actually doesn't work super well. I think if I had more time I would try compressing and stripping down the message more so it would fit in a single qr code instead (like in https://webrtchacks.com/the-minimum-viable-sdp/).
Hey, how serendipitous... I've been trying to do a wiimote project myself and saw your post; do you have any pointers to /resources for troubleshooting the connection to the remote?
Very cool :) I wonder if you could potentially use colour to increase the data rate a bit? (although then it wouldn't be a standard QR code I guess)
http://www.jammarcade.net/backer-32/ -- I've got one of these devices, which was used back in the day, to store data on a VHS tape, in a similar way to what you're transmitting.
In the spirit of uncommon data transfer methods, here's my ridiculously slow version, using a cddrive:
Not OP here: Yes that would definitely work, but you have to be aware that color channels of the monitor and camera are not mapped 1:1, there will be some overlap.
Also be aware the light in the environment influences the light received by the camera (or emitted by the monitor surface, depending on the way you want to look at it). You may have to tweak the threshold if the light changes.
@OP: please do this, it will triple your data without additional costs. Don't ignore the color channels!
I think that even using more than one bit per RGB channel isn't hard. For example, 0 -64-128-192-255 for each RGB would increase the speed for 15 times. The 1-2kbps would become 30kbps, a modem from few years ago.
When using the real webcam (and not just passing files around), just one frame for calibration and no auto-white balance should be enough.
"First, you need some way to acknowledge that the other side read my message"
This is incorrect. Assume a protocol like UDP where the messages are unreliable. You do not get acknowledgement of packets.
Actually just removed that stuff and seems to be faster, there is a qrtun_async.py script now with no seq/ack. That being said, my concern was that if creating/displaying the qr code was much faster than detecting/decoding, and a lot of packets were coming in at once, you could maybe display multiple packets/qrcodes before the other side was able to decode the first one. That would result in drops of course which should be re transmitted though, so I guess its okay.
You could also decouple image capture and packet processing, so that if the former starts to run ahead, you can buffer until the latter catches up. NICs do this a lot, for much the same benefit.
I'm not great with Python, but your code seems quite legible, and it doesn't look like there's any buffering or backpressure in place right now - which is perfectly reasonable for a fantastic proof of concept like this! Just a thought for when you next get around to hacking on it.
It's funny I actually had the idea of doing this exact project yesterday, although I would probably have never gotten around to it.
Rather than just being a fun little project, I could see there being a few practical uses for this: If you could modify the protocol so that the 'QR code' could dynamicly resize down to be in the worst case 1 pixel for the whole screen, you could potentially do line of sight communication using just laptops between tens of kilometers away[1]. There aren't a lot of situations where you'd need that, but I could see scientists on a remote research mission finding in handy to be able to just pull out their laptops and send messages between mountain tops by just double-clicking some kind of application.
A crappy webcam that could do 10 frames per second that was looking for an entire monitor in the on/off state could send O(10) bits per second which would be useful enough to send small text messages. If you used a telescope, you could actually send real amounts of data.
Edit: Upon close inspection of the article I linked, it says later in the article that you could only see the flame 2.76 km away, but I don't think that changes my point very much since a laptop screen would be brighter.
In practice you're limited to sending information at half the theoretical maximum resolution[1], so 5 fps. Still has some very interesting possibilities.
Let's call it QR-Video, theoretically the world's most useless protocol as you only need a link and one still frame will suffice. However there is something nice about QR codes and it would be cool to stream information in this big visual way between phones, e.g. a business card sized piece of information.
The problem I have with QR codes is that they do not just work. You need to download a QR code reading app, this functionality should be hard coded at a deeper level, a notification appearing for QR codes no matter what you are doing with the device.
Many of the apps for QR codes do things like sneaking in affiliate marketing codes, this can malform a URL and do some redirection along the way for monetization purposes unknown.
Maybe QR codes deserve the video mode to become a protocol like this of sorts to be there for innovators with uses to be invented. QR codes are here to stay, they are too useful. They need acknowledgement as a thing and for phones to do a lot better out the box at hzving QR code readers baked into the OS.
I had one of those too. I was telling my 13 year old and 9 year old son about it last week. They looked at me like I was insane, then went back to their tablets.
Oh, funny, I'm working on something similar for a course at school, too: an IP over Avian Carrier modem. (I convinced some folks to build one with me :b). In our case we print the barcodes with a thermal printer, so there's an interesting constraint that we need really simple, easy to scan barcodes that fit on that paper but are still data rich. I have happily ignored the printing / scanning part of the project, but I think we're sticking with Data Matrix, and I've set an MTU of 300 bytes on our network interface just to be sure. We're using tun/tap, too! It's really handy.
Alas, carrier pigeons are difficult to employ, but that's okay. Part of our goal is to demonstrate network protocols in a tactile way, since you see every frame that gets sent and received as it happens. (And read them, too - we're including a hex dump with strings). I think it's a fun alternative use for projects like mine and like the article's :)
Haven't thought about performance much at all, so I'm curious to see what the author has in mind there.
I had a somewhat similar idea (which never materialized) where the sender would print QR codes (or something else) on a long paper roll (like those used for receipts in stores), which would then be scanned on the other end. The idea was about being about to visualize things like a ping or a TCP session progressing between computers.
If anyone wants to steal the idea, please go for it! And send me a link to the video when you're done. :)
Now, how about reducing the resolution and increasing the speed? E.g. a one pixel display and a one pixel camera that runs at MHz or GHz Speed? You'll have to add some modulation though, otherwise you can't sync the two devices.
My experience with the clock was that I really needed ECC mode 'M' to get the maximum ECC bits which worked around all sorts off issues with differences in scan rates between display and reader.
In my setup the QR clock is displayed on an array of 64 x 64 LEDs, when the LEDs were operating at a 60hz refresh rate you could induce some pretty interesting interference with the camera scan rate on phones with QR readers, once I boosted the refresh rate to 240 hz that interference went away, then it was a question of how much "dwell time" did the code need to be recognized and I got it down to 100mS as the fastest I could go and the QR libraries on the iPhone and Android could still pick it up.
So for reliability considerations maximize the display refresh rate and the ECC correction level.