I have only experienced the term in a humiliation context - the name being specifically used to engender feelings of embarressment in a male sub. As such it comes alongside such terms as 'whore', 'pretty boy', 'my little girl' and so on - reminders that Mistress has feminised the sub.

I think the ones who have been using it in the way you describe (i.e. without knowing you well enough to know if you like it/hate it) are, as stated, arseholes.