I've been reading a lot of both of these writers over the last couple of years, well more like the last 5 years.
Both write about similar things. The themes are a bit different for each. But they both write amazing ghost stories. Afterwards is one of the best ghost stories I've ever read, but so is Turn of the Screw.
Overall I think I prefer Wharton. She has more of an ironical view of the world which is reflected in her writing.
It is probably difficult to completely compare the two as while they are separated by only 20 years, they are an important 20 years and Wharton doesn't really start writing until James is almost finished.
Age of Innocence and Ethan Frome are excellent books and I don't think that even Wings of the Dove or Portrait of a Lady can compare.
Still I do prefer Turn of the Screw to anything Wharton wrote in the horror genre.
Where I think the largest difference lies between the two is in their short narratives. Edith Wharton seems far superior in getting a concept across in a short story than James. Xingu is a perfect example.
Compare that to something like Beast in the Jungle, which could have been probably half as long and gotten the point across.
So who do you prefer: James or Wharton?