I'll go with Buchanan as well, though Pierce and Fillmore were bad.
I think Grant gets a bad rap, as does Johnson.
McKinley was a product of his times. Not that it makes him a good President, but the imperialism was a given then.
Jackson was
complicated. He fought Indians, put them on the Trail of Tears, and defied the Supreme Court. He also raised an orphaned Indian child as his son, and threatened to hang every man in South Carolina if they seceded, defying federal authority over the Tariff of Abominations. Complicated.
I enjoy that folks recall that Mexico is part of North America.
We could nominate Mirabeau B. Lamar, for his willingness to kill every INdian in Texas. But he established public schools and attempted to secure European recognition of Texas rather than annexation by the United States.
Or Anson Jones. He let us get annexed.