I am always getting “retries failed” from Gemini for even the most basic prompts + generally now responding as if made a change, but it has not (and takes a few attempts, pointing out it didn’t actually make the change before it does).
In my experience, one potential cause is a complex problem. The more complex the problem it has to consider, the more likely the failure. Since you mention it fails even with pretty basic prompts, perhaps your GEMINI.md and/or .idx/airules.md files contain complex instructions or require complex consideration.