Some filters on the image search are "Toggle" filters (vs dropdowns). These are "Latest", "GIF", and "HD". Clicking on these will "turn on" this filter. Clicking on these filters again, will not turn them off. They can only be turned on. To turn these off you must remove all filters with the "Clear" button.
Somewhat related - it looks like these are "shortcuts" to the more in-depth options. The relationship here is somewhat confusing. For example, you can click "GIF", then click on "Image Types > Clipart". The image results will show clipart, but the "GIF" toggle will still be selected. It seems like the toggle filters are not reflecting the actual filters being applied.
I would expect clicking on "Latest", "GIF", or "HD" after it's been toggled on, would then turn it off. Otherwise, you have to clear the filters, and add back all the filters you still wanted.