With that setting off, the last language you used should be automatically selected when you reopen the app indeed.
Does the setting itself get reset when reopening the PWA back to on? And if you try using the site without the PWA (just from a browser), does it work as expected?