I'm currently working on something sort of like a beat 'em up. I won't go into the intricacies of its genre right now since it's kind of complicated, but basically it's on a top-down angle and everything moves in eight directions. Just think River City Ransom or another non-grid-based RPG or whatever. I'm using a custom movement engine with fastloops.
Everything was fine as far as the player was concerned. There are no detectors, but I'm binding movement to two alterable values so when the player is moving upward into an obstacle, it just pushes him down without checking to be sure there's actually an obstacle above him. It works fine, so I didn't think it was worth adding that yet.
But now I'm on enemy AI. They behave the same way - their movement is bound to two alterable values, and when they collide with a background, it pushes them in the opposite direction they happen to be moving in. Which is fine when they're running at the player and there's an obstacle in the way, in which case they happily bonk into it until the player helps guide them around (I should mention I am not very good with AI).
The problem is that, in some cases, they overlap with an obstacle while trying to move towards the player. They'll overlap with the ceiling and while trying to get to the player below. In that case, the game thinks they're moving down into an obstacle, and sends them up, so they just move through the ceiling.
I know the best way to deal with this would be with detectors, but there are going to be a lot of enemies and I don't know how to have them all get their own set without getting mixed up, and I don't even think it would be good on the engine to have so many objects.
Sorry for the long-winded explanation. I wanted to avoid having to give out the code because it's 10 MBs and there's a lot of terrible swearing in the comments. I see this sort of movement system a lot, so I was hoping people could fill in the blanks. If not, I'll just try to tear out the AI and movement code and put it in its own happy (PG-13) package so you can tell me exactly where my horrible coding went wrong.
Basically, how do I go about with the AI in this instance? I've done lots of searching on the forum but I can't seem to find many answers. I figured I might as well make my own topic based entirely on the system. Might have further questions, anyway.