Studio.firebase.google.com prototyper suddenly making lots of mistakes this week

Hi - Ive been working with the prototyper for a year, very successfully. However the last week has been very difficult, I assume you’ve done a release?

I totally understand that the AI makes mistakes, and goes down rabbit holes, so I use git to checkpoint working code, and rollback if I run into issues. That approach has been working fine for a year or so.

However this last week the prototyper isnt working anywhere near as well. It keeps on forgetting changes, re-applying changes, remembering old (disgarded) changes and applying them without any request from me.

Can I access the old version of Gemini? Am I amagining this? Have I hit some limit? I do have a couple of longish source files (3000 lines), but they are covered by a bunch of tests and the old version could cope. the new version not so ,much!!

Any ideas, comments?
Thanks
Duncan

BTW if this gets viewed by someone at google, my project is ~50,000 lines code typescript. Happy to share the project with you if it would help?

another “new” feature. When I ask for a small focused change in one area of the codebase gemini decides to reformat several code files.. Why is this suddenly happening?

Do you switch between Prototyper and Code Editor modes?

The reason I ask is that Prototyper and Code Editor chat agents don’t share context. They will not be aware of conversations you’ve had in each others chats. This can be really frustrating if you ask one agent to refactor something and then the other agent notices it’s missing and adds it back in.

Here’s my personal workflow and recommendation for success. I use prototyper to rapidly build my MVP. I switch to Code Editor mode when I’m ready to start fleshing out features in more detail. I start new chat conversations for each new feature to keep the agent and context lean and focused. I keep 2-3 chats alive at any given time, switching between them as I touch different parts of the code. (e.g. a chat for homepage, or sign-in, or user ratings, etc.). I delete old conversation immediately as soon as their goal is met. I don’t go back to prototyper mode once I’ve switched.

I intentionally don’t want the agent to remember everything from as way back as my first prompt. I don’t want it to have to sort through lengthy conversations to remember when I’ve changed my mind or backtracked on an idea. I want the agent to remain in the moment and focused on the task at hand using the current state of the code as truth, not what it thinks the code is from old conversations. Deleting chat conversations, starting new ones and forcing it to read SMALL segments of code as it starts new tasks has worked very well for me.

I’d love to hear strategies and workflows others are using.

@jamesor Do you have the agents maintain any docs, as a way to help onboard future agents? Or do you have a handoff process?

Hi James - thanks for the update. Yes you’re absolutely right, Ive been flipping between code editor and prototyper screens, and I hadnt been starting new conversations… I’ve just been working with 1 long conversation for almost a year (I guess I should have read the manual).

OK, so I understand that I’ve been using the UX incorrectly, but I have a couple of things tht dont make sense.

  1. why has it suddenly gone wrong now in such a catestrophic way?
  2. when Im on the Code screen whats the differece between new chat and new chat (classic)?
  3. In code → settings, why do I see 3 tabs of seetings (user, remote, workspace)?
  4. In New chat Im given a choice of models. How significant are the differences between models? Can I use different models in parallel?
  5. Im guessing that large code files are a problem? Is that true? How big is too big?

Sorry for all the questions!

Duncan

1 Like

1 more - so Ive started using new chats in the Gemini tab in Code… but the agent’s suggested changes are just printed on the screen, my codebase doesnt get updated. Ive set Agent (auto-run) to true, so I’d expect my code to get updated?

James - in reply to your question about workflows, apart from using Gemini AI I use github and regularly branch my codebase so I can quickly rewind to a know good commit and then try a new feature again.

1 Like

Hi Steve - I did start asking the agents to maintain a DB schema and an APP structure document but it was timeconsuming, error prone, and I was struggling to see the point of it, since the AI agents seem to good on anaklysing the codebase and understanding the relationships. What do you reccommend?

BTW most of my problems have been solved by using new chats in the code window and NOT using the prototyper any more :slight_smile:

1 Like

The reason I ask about documentation is as a way to minimize the amount of analysis the agent has to do. More analysis means more context, which means more to forget. This might be an issue for larger projects.

1 Like

I have been only using prototyper, which used to work perfectly. Just needed to clear chat when it became too long.

But now it’s completely broken, it has litteraly killed my fine working apps, and fails to implement a new gemini model for example.

Starting new chats in the code editor (built-in model) often results in chat failures, which never happens with prototype.

I am very worried if you don’t admit there is a bug.

1 Like