Very cool! This'd be absolutely great for getting more value / use out of the pile of Everyman books that I [and, I'd hazard a guess, a solid chunk of chessplayers in general] have sitting on the shelf that don't get used as much as they should
Great question! There are lots of great beginner books out there, but if I had to pick three:
1) "A First Book of Morphy" by del Rosario -- takes the very well-considered principles of GM Reuben Fine and illustrates them with [mostly] Paul Morphy's games. Of all the books I've ingested on the subject of chess, this one has stuck with me most. Even my kid loves it!
2) "Winning Chess Strategy for Kids" by Coakley -- it's not just for kids! And it's not just strategy! Covers fundamental tactical concerns such as pins, forks, etc in a straightforward way. Similar in some regards to Pandolfini's "Weapons of Chess" but pedagogically superior in my view.
3) "Silman's Complete Endgame Course" by Silman -- the old saw that one should study the endgame first is pretty true -- Silman is the best at teaching it -- man, if I had a dollar for every game I was winning in the middlegame and then lost in the endgame, I'd have... well, a lot of dollars
cheater extra:
3.5) "Bobby Fischer Teaches Chess" -- the old standby -- this is a tactics book first and foremost, and the main value of it is that it gives you puzzles of gradually increasing complexity so that you can really feel your comprehension improving and say to your self, "I'm getting it!" Truth is that getting discouraged is the thing that stalls or scares off many / most novice chessplayers
When you start getting up in the 1100-1300 range, pick up "My System" by Nimzowitsch and "How to Reassess your Chess" by Silman, "Soviet Middlegame Technique" by Romanovsky -- and after then, start learning openings in more detail. The biggest mistake most beginners make, including me, is building up a repertoire of openings before having a solid grasp of the fundamentals. Truth is, below 1300, most players are "off book" by the ~tenth move anyway, so learning the intricacies of the Nimzo-Indian isn't gonna do a person much good at that stage.
It’d be great to convert this into a mobile app that lets you take a photo and then loads it into a game analysis board. We would use this all the time at my local chess club. Sometimes we just want to know what the best move is and other times we want to save the game so we can continue it later.
Although it is more difficult in 3D, it is still a very solvable problem, that won't work always, but in 99% of cases if the camera angle is not very low.
Indeed, the checkerboard itself gives valuable information about the pose of the checkerboard, it is even used for calibration in multi-view geometry: https://en.wikipedia.org/wiki/Chessboard_detection
I wonder how much easier it would be if you narrow down the likely set of pieces for each square. There are obviously some positions for some pieces that are invalid (bishop on the wrong colored square), but there are probably a lot of other positions that are so uncommon that they could be discounted.
What would make it easier IMHO is to make it a top-down version only. Take a photo from the top, then the program breaks down the board into 8x8 squares, feeds each square into a classification algorithm that you will train on a bunch of hand labeled images. Fine tune the model as you gather more data.
Probably not much. How many are actually impossible if you factor in pawn promotion though? The only one that comes to mind immediately for any individual piece is pawns on their own first row and a board without two kings.
Compared to all the other feats of machine learning that have blown my mind, parsing a photo of a limited set of a handful of different piece variants, in two colors, that located on a grid, doesn't seem too difficult.
Just to clarify an important detail that pkacprzak (the author) has been asked many times now by the streamers he is using the images of (GoldDustTori and GM Ben Finegold) - NONE of the streamers actually used this app during a game, as that would be considered outside assistance and cheating. The app is intended for viewers to follow along, not to enable cheating (even though the cheaters will use it for that purpose).
One feature that would be useful for me would be the ability to convert a video to a PGN of the game. This could, for example, be used to strip game data from streamed chess en masse and then analyze the chess games in aggregate.
Maybe it's just me, but I'd be wary of any browser extension that captures and analyzes my screen over-the-wire, especially if it is closed-source like this one.
Not to say that this is necessarily malicious. But I personally wouldn't recommend this to anybody as of right now, unless you want to risk leaking potentially sensitive data to a third-party.
Hey! I'm working on exactly this same problem for an undergraduate Computer Vision course right now. It's not going well! Chess piece recognition is hard. I definitely think the electronics approach described in the other comment is a much more reliable way to go.
If you ever do get it working, I think it'll be fun to push it to over-recognize chess games everywhere, like the deepdream stuff over-recognizing animals (or whatever) everywhere.
This might be hugely easier if you can keep state between turn analyses. IE, if a white rook disappears from A5 and some unidentified white piece appears at C5, then it’s probably the rook.
We weren't given many constraints related to the chess piece recognition itself. The course instead asks us to implement a CV research paper, and we chose an existing research project which focused on chess piece recognition.
That lack of constraints led us into running face first into issues of generalisation and variability within datasets. As in, exactly what you allude to with limiting the piece sets.
I think in my undergraduate naivety my aspirations were too high with what could reasonably be accomplished. I've spent a lot of time trying to improve an aspect of the project that really didn't need to be improved, which prevented meaningful progress.
Now finals are coming up and I feel terribly stressed. Having trouble functioning. Brain fog, etc. I feel so sad right now.
EDIT: I keep forgetting my password so apparently I have multiple throwaway now. Sorry.
It takes a lot of shortcuts, works with just the right lightning, etc., but worked great as a proof-of-concept :)
We got away with not identifying the pieces by just detecting the color, assuming the game started from the initial position, and assuming only legal moves (the whole game is unambiguous using these assumptions).
It's all old-school computer vision with hand-written features, and I'm pretty confident there is tons of low-hanging fruit, but who has the time.
If I understand correctly (watched the video) you start from the initial position and track moves that are made in order to update the position to reflect the move, is this correct?
That's correct. If you're curious about the architecture: the vision part detects a chessboard, then corrects the perspective and restricts the image to just the square of the board, then looks at each square and has some simple thresholds to decide if it's occupied by a piece, and of what color. From there, the camera is treated as a black box sensor that continuously streams two 64-bit masks, for where it thinks it sees white and black pieces. There is a second program (controller) that turns that stream into a stream of chess positions (and a Unix pipe in between). The sensor is faulty of course and the controller has logic for ignoring bits from the mask where there cannot possibly be pieces etc.
Those boards are ~$1000 each. High level tournaments use them, but low level tournaments don't unless some rich person has bought the club a bunch. Most tournaments are low level and are designed not to turn a profit: the entry fees either pay for the room rent, or go to the prize winnings: the more people who enter the more prize money there is.
Yeah, the DGT board is perfect for on-the-fly tournament analysis -- but a computer vision tool would be great for times when you don't have access to the/a DGT board or its data
Set one up at the park and get some insight on those hustled blitz games!
There are already available apps that allow cheating in more "convenient" (if I can even say that) way for cheaters. They analyze DOM and display the best move etc. without any latency and heavy processing
This is kind of a bummer for privacy reasons. Sending your application screenshots of my browser feels not-great.
To be honest, I trust you, even as a random stranger on hacker news. But I don't trust that you'll never sell this, and I don't trust whoever you you might sell it to.
I understand your concerns and would like to clarify any potential doubts. The client app (the extension itself) has only currentTab permission which means that it has access only the tab where user opened the extension and it takes a screenshot only when a user explicitly clicks Scan button. The source code of the extension is completely public - everyone can unpack it and verify that indeed it does exactly that. So this actually reduces to the same functionality as for example using any photo app with cloud processing - i.e. you take the photo/screenshot explicitly and no other information is sent to the server.
r/MachineLearning :https://www.reddit.com/r/MachineLearning/comments/b8jdho/p_d...
r/chess: https://www.reddit.com/r/chess/comments/b826h5/created_chess...
If you have any opinions and suggestions I'd love to hear them!