Object Mover activated - How an object can avoid it?

Hi!

First, a little screenshot:

Please login to see this attachment.

The big square is the Object Mover. When I click on MAJ > it is activated > Destroy balls in collision and replace them by Green Balls. And I can move the square with these green balls inside it.
And the goal is to place these balls to another new empty space (white circle). By clicking on CTRL, catched ball(s) are released on the new empty place, and destroyed, and replaced by originals brown balls.

The problem:

When the Object Mover (square) is activated, it catches all the balls in collision with when I move it!
And I try to avoid that. It should keep the balls catched at first, and not catch the others until a new click on MAJ, only.

There is something I can do to make an object not catchable?

I'm a true beginner. I've tried a lot of things...
There is probably another option, as always... But I don't have it for 4 days

Thanks a lot for the help!

I've tried a looooot of things, and even put the active object under the Object Mover doesn't work.

Nothing? That Object Mover is really cool, but being stuck at this point make it useless for me... I can't continue my little game, this is frustrating

I still believe in a solution X)