The suggestion is twofold, but deeply connected.
- For image translation to show over the text.
- For image translation to show over the text, and the processing to be done live without needing to take the picture.
The first feature is done in both Deepl and Google lens, the second one only by google. As for a more detailed explanation:
Explanation: the feature would be for a translation option where the translation itself is shown in front (or on top) of the image, spatially in front of the lines themselves. So in the result, you would have the original image, and a semi transparent translation right above the true text.
How it betters user experience: Especially N°1 is really important, because it allows the user to see if the translation skipped anything, especially a line, which in an unknown language it's almost impossible to notice. (I was testing the picture feature between English and Spanish using handwritten notes, and I noticed it worked really well, but it sometimes skipped a few lines at the top or bottom entirely, if I didn't know both languages that would be unnoticeable.)
Also it's really useful for understanding what each line says better without having to crop it (this is really useful for menu's in restaurants)
N°2 is a lot less necessary (and it's probably costly), but the advantage of doing the process as a live translation (suggestion N°2) is that it's fast and allows correction in the camera angle so that it picks up the text in its entirety, it is a lot simpler in translation. I have found it useful as a tourist looking at signs and stuff.
(I know this is a huge ask and complicated, the app and features are great already, just wanted to show my appreciation because I know how recent everything is. Didn't find it requested elsewhere, really sorry if I missed it)
User case example:
Through this, a user could easily distinguish for example between menu items, so that they know what translation corresponds to what part of the menu instead of needing to inferr what is what. A user could also check more easily if the tool missed any lines in their translation.
As for other apps that have implemented this:
Deepl does this, you take a picture and it shows the same image with a somewhat transparent translation in top. There is a "Select all text" button, when pressed it will show the text by itself, without the underlying image for better readability (it shows both, the original text and then the translated text besides it).
Google lens goes a step further and does it live. It will do it automatically by pressing the "camera button".
And as an option by pressing "Send to Translate Home" it also allows it to do what Kagi already does, which is send everything into a separate note (that shouldn't be lost since it allows to read everything neatly, they should be 2 separate things).
