One time, during my company's development of a product, we kept encountering a problem while writing a lot of data to an eMMC chip. Was it a problem with the circuit design? Our eMMC controller? Software driver?
It wasn't until we discovered a unit that did not exhibit the problem that we discovered that the problem was isolated to a particular batch of eMMC chips from the vendor (like the OCaml team only realising it was a Skylake problem when they discovered their code worked fine on non-Skylake systems).
On a board, we were using an ADC chip. We bought the ICs from an official distributor. They sent us the chips, and we populated the boards (~200 of them).
None of the boards worked properly. We debugged for so long. After so much wasted time, we found out the root cause with the help of the manufacturer and the distributor : It turns out, the manufacturer actually produced another IC, but marked the packages with our part number. It just happened the power/gnd pins matched with our IC, so there were no electrical problems.
Another case:
We designed an ASIC and taped-out. The silicon arrived and we then sent it to a packaging house to get the silicon in a QFP package. The packaging house misread our specifications and placed the silicon with 90 degrees rotation in the package. That was fun to debug!
The 90 degree rotation thing is surprisingly common. I've seen it done at least twice. Also several instances of edge connectors that were entirely backwards due to being CAD placed on the wrong side of the board, or incorrectly pin nimbered.
One time, during my company's development of a product, we kept encountering a problem while writing a lot of data to an eMMC chip. Was it a problem with the circuit design? Our eMMC controller? Software driver?
It wasn't until we discovered a unit that did not exhibit the problem that we discovered that the problem was isolated to a particular batch of eMMC chips from the vendor (like the OCaml team only realising it was a Skylake problem when they discovered their code worked fine on non-Skylake systems).