I simply put it down on the times and Patton's probable pattern of thinking.
We were starting to figure out the effects of long term and extreme stress on people. It was known at the time that even the best unit no longer operates at peak efficiency if kept in the front lines too long.
One often speaks of the horrors of WWI, but the static nature of the war in the West was a blessing in disguise. It allowed troops to be rotated away from the front and the average soldier only spent a few days a month effectively on the line and quite a bit of time in the rear, drilling and performing tasks and once in a while getting a bit of leave.
In WWII such a luxury was rare, both in Italy and Western Europe. Units were rotated in and out, but remained in the front lines for the duration, Italy proved to be a tough front with difficult conditions as one soldier described it, "If it was sunny it got unbearably hot, if it rained it drenched you to the bone, you could never advance more than a few miles before hitting a river, a mountain or a fortified village or town."
At the same time there was still a prevalent belief that soldiers would "Always grumble, but perform if you shouted hard enough at them." And the idea was if you coddle soldiers too much they would slack off even further. This is an age where failure is not so much considered a possible result of adverse conditions, but merely your failure to push onwards or the result of a willfully weak personality and bad character.
There is also a line of though or even a personality trait where people almost automatically assume that a person is guilty rather than innocent even if the evidence points the other way. This is often tied with what you could describe as a lack of empathy or the propensity to measure everyone by your own experience.
Patton certainly was under a huge amount of stress and had to oversee the arguably unenviable task of running of a whole army and it's easy from that kind of perspective to assume that a simple soldier has it easy, just march, dodge a few bullets and shoot the enemy and therefore has very little to grumble about.
And of course we don't know exactly what state these people were. People suffering from PTSD, depression or other mental issues may show few symptoms especially to an untrained eye, you may have seen shocking footage of shell-shocked soldiers in WWI shaking, freezing and convulsing, but hundreds of thousands more also suffered without exhibiting much external signs.
We can't exclude that they may have been shirkers and that Patton rightfully picked up on it.
In any other army you could get shot for such a thing. US society didn't quite accept this kind of thinking, considering soldiers as "citizens under arms" and even a slap in the face was too much for some …