The argument was the camps weren't built to kill people that rapidly, and there was something to that - but we know they died. We had to listen to how it worked during the months of accelerated death.

Their first few camps were inefficient, in terms of extermination. Like Dachau. I forget which was the first camp that was built employing efficient German engineering (Sobibor?). It wasn't built with labor in mind, just genocide. It had body chutes (similar to coal chutes) to deliver bodies for disposal, purpose-built gas chambers, etc. This was a response to the inefficiencies of the older camps. I believe the last several camps they built were on that model (including Auschwitz).

Yes, they did pack them tight on the trains, and many died there, too. I'm not sure "half", but it was a lot.

The primary reason we have any data at all is that the Germans were meticulous record-keepers. After all, they had to document the glory of the Reich for posterity! Even so, numbers do have some error bars. Auschwitz was about 1.1M; combined all the camps were about 12M (it wasn't just Jews, whom accounted for about 6M...another roughly 6M Soviet citizens of non-Jewish descent were killed, plus gypsies (Roma), non-Jewish Poles, Soviet POWs, Jehovah's Witnesses, and others). The error bars are perhaps 10%, resulting largely from records being lost/destroyed.