Drop the idea that attachment and love are one thing. They are enemies. It is attachment that destroys all love. If you feed, if you nourish attachment, love will be destroyed; If you feed and nourish love, attachment will fall away by itself. They are not one; they are two separate entities, and antagonistic to each other.
It's true that over-apologizing interrupts the flow of conversation and irritates the person who has to stop and offer reassurance, like, "No, it's fine, don't worry about it. " But far greater than the challenge of toning down unnecessary "sorrys" is offering an apology when one is due.