Hi everyone!
I started in the AoS genre myself about a year and a half ago, so I've stood where you are. I've even got a thread somewhere here on TMP asking roughly the same question. : ) so here are my thoughts:
Reading your criteria, the closest rules to match all of them is Post Captain, which are also my favorite rules. The game is complicated enough to be interesting but simple enough to pick up. I've been able to teach the game to people easily. There are a few other choices besides which way to point the ship. It does single ships through small squadrons, and small ships through SoLs. It's a good AoS ‘skirmish' ruleset.
The only thing that might be a problem is the 1/600 scale. You'll have to take the extra step of downloading the 1/600 templates from the ODGW site, and that scale might require a larger play area. I play PC at the 1/1200 scale and a 6x4 table is sufficient.
Here's a review of PC that I wrote on my blog:
link
There's lots of post about PC here on TMP.
And while I'm plugging my blog, here's a review of KMH:
link
(for more blog AoS goodness, click the AoS label. : ) )
Here are some other thoughts on rulesets, but none of these I think use the variety of ships you want, it's all SoLs and Frigates. So take it for what it's worth. I'll divide these between those that use hexes and those that don't. All of these I have read and / or played…
No hexes:
Captaincy: and it's more streamlined version Enterprize: I think this game has some of the most interesting and creative sailing mechanics EVER. Worth checking out, lots of decisions in sailing the ship. You roll a pool of dice and then organize the roll on how you want to sail the ship. The drawback is that it gets complicated. I've tried to teach it once and just got confused myself. It'd be great for single ship actions.
Form Line of Battle: looks good for squadrons. Alternate activation by card kinda like KMH, You roll to see how far a ship will move. Ships move straight and then pivot. I've read these and liked them, but yet to play.
Fire as She Bears (2nd ed): This game looks like it'd be fun, but uses a strange octagon system for movement that just creates an extra step. However the mechanics look good and seem like fine rules for playing large actions.
Hex games: I haven't done a lot of hex games because I lack a sea mat with hexes.
Form on Admiral's Wake: I really like this game. It's best when each players has 3-5 ships, so it's a fleet game. A card deck is turned over and depending on the card and the status of your ship then the ship can activate. As ships get damaged they will activate less.
Bloody Broadsides: Another good game; the preplotted movement is the heart of this game. It tends to play pretty quick though, it's more of a ‘shoot each to splinters and then reset' type game.
Hopefully people who are more knowledgeable than me will chime in eventually. I was pretty much just buying rulesets until I got Post Captain, and then I was happy and stopped buying rulesets because one can't go on buying rulesets forever. Good luck on your search. : )