Your feeling are your feelings. They are understandable, and they are not wrong. However, that may not mean that you'll find what you want or need with your current master. Talk to him and tell him very clearly how you feel. He may decide that you're the only sub he wants, but if not, you may be faced with some difficult decisions. I wish you all the best.