The github counter is misleading - it is counting auto-generated C headers used for calling assembly code. The repo also contains a libaom submodule, but the only parts that are used are initialization tables and the transforms.
The stats aren't wrong. If it isn't 'real code' then you shouldn't store it in your repository, you should autogenerate it as part of your build process.