Doesn't your email server log info about connections, that would be my first port of call to see what's going on?