Building an MCP-powered autonomous incident response ecosystem Transcript, AI Summary & Key Points

00:09 Building agents is incredibly easy, but how many of us are able to push the agents into production and run it successfully. Today we have Mir from PayPal and he's going to talk to us about his experience. Mir, tell us, what is the most impactful agent that you've built for PayPal. So what we have done is that we have created an autonomous agent at PayPal.

00:33 And by autonomous, I do not mean just that an AI assisted chatbot, but an agent workflow, which is going to help you to detect, triage, mitigate, and report of the incidents. A lot of companies are building things. They mostly survive and die in sandbox. Tell us about your journey from going from sandbox to production. We have been trying to build autonomous agent for a long time, and we have tried various different other tools and frameworks in the past.

01:02 We even had our own orchestration built in at PayPal. But when we started using edq and Gemini enterprise, we saw that it cut down our coding by 50% to 60% What did you think made the difference whether it cut down. Was it just the framework itself. So the framework actually came with various different other tools like for example session management, memory management, persistence, parallel execution of the agent, which is huge.

01:33 Before we had to think about all those things and hand code, all of that, now we don't have to. So it really supercharged our development process. You're a financial services company. So taking things from building to production, how did you jump from here to here. The advantage that we had with Gemini enterprise is that there was an observability sort of built in.

01:58 And each step of our way, when the agent is getting executed, we could see that we know what is the execution path. What is it that it's doing. What was the reasoning behind a certain path that it took that helped us to gain more confidence that, hey, if we put this agent out there in production, it will work. And were you able to get all of these capabilities right out of the box, or did you have to instrument them.

02:23 So there are several observability items that came out of the box. The execution path, as I mentioned, there are also on the model evaluation side that also came out of the box. The accuracy, if the model is hallucinating or not. How do you think of an autonomous agent with respect to an employee in your company. Where do they stand. So you can think of this agent as a returning intern.

02:49 They have some good context, understanding of what the company does or what is being asked of them, but they need some supervision. OK right. And they need somebody to oversee what they're doing, especially when they're making some changes. Agents that mutate the production systems. They have to be like supervised mode. Was there anything else that's helping you run the intern in production right now.

03:13 Our agent interns, they also get better. And the way they get better is because we humans are giving constant feedback. And in the Gemini enterprise, you have the persistent storage, which may include your knowledge base, which may include the SOP, ETR, and several other things in the world. Which agents will continue to use and get better. So is it fair to say that your success story is majorly because of how your infrastructure is already set up, where the agents already had a place to go fetch data from to be able

03:45 to operate independently. That is correct. So the agent needs to have the data. And the way we enable our PayPal data is via our MCP server. So the agents have access to this unified layer. Nice but they can access all our enterprise data and the incident information, past history and all of that gives it a very rich. Yeah data environment to work with.

04:11 The observability layer gave us step by step how the agent was executing, how the reasoning was happening. And those information against the particular model that we are using. Gemini app enterprise also gave us the ability to swap the models. That is huge. So you have the freedom to go pick any model underneath where you keep your agent coding as such.

04:35 That's right, that's right. Exactly so I don't have to change my code. I can just say, OK, this new model came out. Yeah, maybe it will be better in reasoning, and I can quickly try it out in QA and see how it performs. Can you tell us how many of these agents are running in production today, and what is the kind of impact they've made on your life.

04:53 So we are calling it autonomous agent. Now, this is not just one agent. This is actually an ecosystem of agent. If you think we have supervisor agent, then we have the agent, reporting agent, change analyst agent, service agent. So all of these agents work together to triage your incident. And find out the proper solution. Does your Autonomous agent take action today, or does it find things come back to you and report.

05:22 Production systems obviously are important. We cannot let it go down. At least initially. Everybody should start with read only type agents where they can see that, OK, what the agent is doing, how it's functioning, mutating agents or the agents which actually mutate your production are the most dangerous one, obviously. So they have to have block list or allow list.

05:45 Yeah they have to be put in a container or in a sandbox so that they cannot do any harm to the production. I want to say thank you so much for letting Google partner with you when you're on this pioneer journey, and it's been amazing working out and solving all of these use cases with you. So huge. Thank you. Thank you so much. And partnership with Google. Thanks