Ah, I understood how to save.
- Click "Kagi Search" icon of the extension
- Select "Japanese" as Output Language
- Click Configure button
- Click "Save settings"
The "Output Language" were not saved automatically, and it seemed that they would not be saved unless you pressed the "Save Settings" button.
I thought it was natural to remember the language I used last time.
However, this method may be inconvenient for people who switch frequently, so it may be better to keep it as it is.
(But, the current behavior is quite difficult to understand.)