Lee was the Speer of the Confederacy.
He's one of the few who escaped with his reputation intact and because of this he was held up as a shining beacon by both sides. The perfect example of a stalwart Southern Gentleman and that of the honorable man who through no real fault of his own became an antagonist of the Union and simply did his duty.
He was very useful to those who sought to promote the lost cause using his reputation to whitewash everyone else while those who wanted to heal the wounds found in him somebody who wasn't an undivided supporter of slavery, he was the "good southerner"
Lee was an extremely likeable man, but he had his flaws and he did end up joining a group of Southern people who had been obsessed with preserving their specific social system from the very day the US was founded. The concerns and fears are present in letters, speeches and political debates for decades and they do everything to preserve it, coming up with "states rights" as utterly absolute in every way above everything else, but they are also the first to throw out states rights when it comes to states trying to tackle slave-owning counties in Northern States. If that's not textbook hypocrisy and favouring one thing at the expense of self-proclaimed principles, I'm the next pope and editor of Hustler magazine.
However when we discuss Lee as a plantation and slave owner, we have to understand one thing that's that "a fair treatment" is one of those fantasies people have about slavery. As harsh and inhumane as it sounds, from a practical point of view, harsh treatment is pretty much mandatory because there is a huge pressure on slaves to try to escape or even to rise up. The idea that you can be kind and compassionate to your slaves and that they are willing to accept their status is pure fantasy. Yes, some slaves were devoted to their masters, but that was often because they were given preferential treatment and had it better than the other slaves. House slaves were often the target of retaliation from other slaves, some were murdered, beaten, whatever possessions they may gain were stolen etc.
The idea that slavery can be a benign thing is a pure fantasy. Those who out of christian charity or naivety attempted to give their slaves a better treatment often ended up in trouble or bankrupt, either their slaves would refuse to work, they would run away at the first opportunity or might even try to murder their masters because what was an attempt to show compassion was seen as a weakness and an opportunity for freedom. Harsh and difficult conditions will bring out the worst in people on both sides of the fence, there is no glossing it over and it's one of the main reasons why slavery is impossible to defend rationally.
Slavery is a horrible system which holds both sides hostage to a situation where they have to do terrible things to maintain a "workable" situation. There is no benign acceptance, only a watchful truce where things are reasonably quiet. But both planters and slaves lived in perpetual fear of each other, one sleeping with a pistol under their pillow and one open eye, the other fearful of punishment, reprisals or simply random violence aimed at keeping them in check.
When Lee is harsh to his slaves, he has to be, but he's not absolved of it, by the 1860's cotton was becoming problematic, while production was up, rising costs and increased competition means that many planters were in debt to foreign banks. The economy that had so heavily banked on King Cotton in the belief that it controlled the world was seeing that certainty erode faster than they could outrun it.
The plantation is the ultimate status symbol in the South, even if you live in near poverty with huge debts, you're part of an aristocracy, even if you or your parents were toiling in fields in the old country only a few decades earlier. It's the dream and aspiration of everyone in the South, wealth and success in every other field is insignificant next to being part of the plantation aristocracy.
Lee was a hard-working, successful man, but his real prestige comes from the plantation, he has to own slaves, he has to treat them harshly. If he steps away from it, he loses a huge amount of social credit.
If you look at the character of Rhett Butler in Gone With the Wind, he's rich, but disowned by his family. He's the 1860's equivalent of Marlon Brando in a leather jacket on a motorcycle in his sheer bad boy appeal. People who came from the background, must have seen him with a mix of admiration and revulsion for being so appealing an outsider, yet from the same stock, but with a sexy bad rep.
And this is why slavery is such an important cornerstone, it's easy to toss out a statistic saying that only a handful of people owned slaves, but owning slaves was like owning a Cadillac or that house in the Suburbs a few decades ago. It was a sign that you were going up in social status. Many white people in the South lived in abject poverty, the only distinction with slaves was the colour of their skin, creating a nominal distinction, take away that distinction and you're at the bottom of the ladder. Trying to distance yourself even marginally from someone else is one of our most basic human instincts. When a Jewish prisoner in a concentration camp was seen marching towards the area where the gypsies were held and screamed that at least the nazis cared about the Jews in that they hated them, the gypsies were nothing, not even human beings and were being wiped out in utter indifference like cockroaches. It wasn't a proclamation of hatred, it was a final painful cry of despair from a person who in one of the most inhuman conditions possible tried to give himself the illusion of dignity by trying to raise himself above others in order not to fall into utter despair.
The South is one of those systems which was a huge contradiction, caused far more problems than it solved, but where trying to reverse course could cause everything to collapse, so rather than fix the system they tried everything to preserve it even as it crumbled under their feet and the planters and their opulent mansions were just an illusion for a tiny minority of people who for the most part had been toiling the land themselves not so long ago.
Another huge issue that people didn't dare contemplate, was the big what if the of the age, what would happen if the slaves were freed ? Would they suddenly turn against their former masters ? Would they collectively leave the South and cause a huge manpower gap ? Would they make the price of cotton even more unprofitable and devastate the plantation economy ? Too many issues for comfort and something that was continually being pushed aside in politics.
Back to Lee, even post war his ideas about blacks were a product of his time and perspective. We find pretty much the same opinion in "Go set a Watchman" where Atticus Finch, that other "Perfect Southern Gentleman" is highly skeptical of the "Negro's" abilities in general, he sees them as children, uneducated, unable to think for themselves, prone to following trouble-makers and idle promises. Even in "Don't kill a Mockingbird" which despite it's reputation as a fine example of progressive thinking sees the failed trial of Tom Robinson as a victory. In the 1960's the idea that social progress would be a slow, gradual thing that would require good-hearted people to take pity on the "poor negro" and gently herd them towards a better future.
It becomes almost a romantic idea, and that's what was wrong and is still wrong with the South (thought it's by far certainly not an exclusively southern thing) it's that they still want to live in a romantic version of their own history.
If Lee is Speer, he's certainly not Himmler, he's at the more enlightened end of the spectrum of his day, but he is still seriously flawed by any standard, modern or otherwise. The fact that he is so deeply associated with slavery and slavery being a huge cornerstone of the South's very identity, despite all the claims to the contrary. We should acknowledge his more admirable qualities as well as his darker side. He wasn't a saint, nor a demon, but he was a very important figure in the Confederacy and we have to learn to understand his legacy and that of others and draw conclusions for today and tomorrow.