No person has ever held all the power. There must be a balance between chaos and order, dark and light. With the Temple magic bound to you, the realms are no longer in balance. The power could change you. . . and you could change the magic.
The real axis of social change is not horizontal, but vertical. We don't need a whole bunch of people gathering to think shallow thoughts together. What we need is for as many people as are ready to gather and think deep thoughts together.