TCG Card Recognizer

Trading card play has moved onto camera. Players stream their games, play remotely against a webcam pointed at the table, and increasingly expect the software watching that table to understand the game rather than just relay it. The reference point for that behaviour is SpellTable, where clicking a card in a video feed tells you which card it is.

Girudo is a trading card game community and creator platform I had already designed end to end as a separate engagement. Its team had been trying to build the card recognition the platform depends on and had run into an accuracy wall they could not get past. I took it on as an independent project, built it separately from the platform design work, and delivered the working system to them.

Girudo's premise is that people play with their actual physical cards online, streaming their table to opponents and spectators. That premise only holds if the software knows what is on the table. A camera feed of a card game is just video, and video is what Twitch already does better than a startup can. The thing incumbents structurally cannot offer is hosting the game itself, and hosting the game starts with recognising the cards in it. Recognition was not one feature among the platform's modules. It was the capability that the differentiating half of the product rested on.

Everything downstream needs the same question answered first: which card is that. A spectator following a match cannot read a card held up to a webcam across a room, so the card played has to be identified on screen for the viewing experience to work at all. An opponent playing remotely needs the same thing to make a decision without asking someone to hold a card closer to the lens. Decklists and state tracking need to know what entered play. Without recognition, every one of those becomes manual typing, and a player who has to stop and type is a player who goes back to Discord and a webcam. The feature that justifies the platform quietly becomes the feature nobody uses.

The core move is to stop treating this as an OCR problem. Instead of one signal read well, the recognizer reads four independent signals and fuses them: OCR on the title band, a 256-bit perceptual hash of the artwork window verified by normalised cross-correlation against stored thumbnails, ORB keypoints with RANSAC homography verification, and a coarse colour and region feature vector. Each runs across four orientations, so a rotated or upside-down card costs nothing. A whole-card OCR rescue pass handles borderless and showcase frames that have no standard title band to read.

The signals fail in different ways, which is the point. Glare that ruins the title band leaves the artwork hash intact. A borderless frame with no title still has distinctive art and keypoints. Fusion scores agreement between signals higher than any single strong reading, and applies an explicit penalty to a result carried by one signal alone once the index grows past a few thousand cards, which is exactly the condition that produces confident wrong answers.

The accuracy wall was not where the team thought it was. They were trying to read cards better, and the answer was to stop depending on reading them at all. Reframing a recognition problem as a signal fusion problem rather than a text extraction problem is what unlocked it, and that reframing was worth more than any amount of tuning on the original approach. It is a reminder that when a technical effort stalls, the constraint is often in how the problem was framed rather than in the execution.

Designing for uncertainty is a product decision, not an engineering detail. The choice to abstain and offer candidates rather than assert a best guess shaped the confidence floor, the fusion penalty, the results panel, and the correction dialog, all of which exist because the system is allowed to be unsure. Building a product that admits doubt turned out to be what made it trustworthy enough to use.