Teach AI like junior engineers, not magic tools

The biggest thing I would say is it really is just treat Devin like your new junior engineer. I think folks come in and they see the blank page and they think of all sorts of various things that they want to try out. But a lot of it is just like, 'Yeah, let's figure out what tickets we want to get done today or this week and let's have Devin get started on those and let's start with the easier ones and then work with Devin and understand what things Devin needs to get set up to be able to test its own code and do this well. And then let's scale up over time.'

You want to be giving Devin tasks, not problems. And a lot of these things like what you just saw, which was kind of like a quick front-end feature request or a bug fix or adding testing and documentation or things like that. One of the things that makes a loop really nice obviously is a quick way to iterate and test.

If it punches in the face, that's not a viable product. And so how do you design your products assuming that this thing will be squishy and not fully accurate and fully work?

Figuring out how to boil it down so that an agent can really understand what success and failure looks like is a lot of the game.

We can't just use that blindly now when we're using AI, as an example, because we have feedback loops much earlier and not even just at the local build and test phase. We have feedback loops throughout, and even sometimes in the middle of some of the pipeline.

We can't just put in a command and guess something back and accept it. We really need to evaluate it. Are we seeing hallucinations? What's the reliability? Does it meet the style that we would typically write?