- Edited
IMO, if you can't find a better way, this would be a good enough reason to make a separate extension for the Summarizer. That extension can ask for more permissions.
It's a bit strange that Firefox doesn't provide a better solution -- there is a whole world of extensions that would need access to the webpage, but only if you click on them; Summarizer is similar to all kinds of "Reading Mode" extensions in this way. Creating a notification for such extensions on every page is bizarre, since the user shows their disinterest in the functionality of the extension by not clicking on it.
I'm pretty sure that if this problem persists, I'll have to uninstall the extension. Fortunately, I think that's workable thanks to search engine extensions like Search Engines Helper. (Not sure if it's the best one; it's a bit awkward to use, but it works well enough that I didn't look for another)