With the old-school plane transistors over the wafer's silicon design, you need about 10nm of doped material so it won't completely mix with everything else.

With theoretical organic 1-electron designs you can build a transistor out of 4 carbon atoms. But nobody knows how to mass produce those ones.

We are somewhere on the middle, modern designs did break the 10nm barrier, but are not nearly as small as those numbers you see around.