Scenario脚本化模拟教程构建复杂的多轮对话测试场景【免费下载链接】scenarioAgentic testing for agentic codebases项目地址: https://gitcode.com/gh_mirrors/scen/scenarioScenario是一款强大的Agentic测试工具专为测试Agentic代码库设计能够帮助开发者构建复杂的多轮对话测试场景确保AI代理在各种交互情境下的可靠性和稳定性。为什么需要脚本化模拟测试在AI代理开发过程中单一的功能测试往往无法覆盖真实世界中的复杂交互场景。多轮对话测试能够模拟用户与AI代理之间的自然交流过程发现潜在的逻辑漏洞、响应不一致等问题。Scenario提供了完整的脚本化模拟解决方案让开发者能够定义可控的多轮对话流程模拟各种用户行为和需求变化自动评估AI代理的响应质量集成到CI/CD流程中实现持续测试Scenario多轮对话测试场景概览展示了完整的测试流程和结果分析界面快速开始构建第一个脚本化场景环境准备首先克隆Scenario项目仓库git clone https://gitcode.com/gh_mirrors/scen/scenario基本场景结构每个Scenario测试场景都遵循以下基本模式支持Python和TypeScript两种语言# Python示例 result await scenario.run( namedescriptive test name, descriptiondetailed scenario context, agents[ YourAgent(), scenario.UserSimulatorAgent(), scenario.JudgeAgent(criteria[success criteria]) ], script[] # 可选的脚本步骤 ) assert result.success// TypeScript示例 const result await scenario.run({ name: descriptive test name, description: detailed scenario context, agents: [ yourAgent, scenario.userSimulatorAgent({ model: openai(gpt-4.1-mini) }), scenario.judgeAgent({ model: openai(gpt-4.1-mini), criteria: [success criteria] }), ], script: [], // 可选的脚本步骤 }); expect(result.success).toBe(true);场景定义核心要素编写有效的名称和描述名称应该简洁且具有描述性# 推荐的命名方式 nameweather query with location clarification namebooking cancellation with refund request nametechnical support escalation # 避免使用的通用名称 nametest 1 nameagent test namebasic scenario描述提供指导用户模拟器的上下文信息description User is planning a weekend trip and needs weather information. They initially ask about general weather but then want specific details about outdoor activities. They might be concerned about rain. Scenario代理测试金字塔展示了从单元测试到端到端测试的完整测试策略定义智能体角色Scenario场景中通常包含三种关键智能体被测代理你的AI代理实现用户模拟器模拟真实用户行为和输入判断代理评估对话质量和任务完成情况agents[ CustomerServiceAgent(), # 被测代理 scenario.UserSimulatorAgent(), # 用户模拟器 scenario.JudgeAgent(criteria[ # 判断代理 Agent asks for account information to look up the bill, Agent reviews the bill details with the customer, Agent explains any charges that seem unusual or high, Agent offers options if there was an error, Agent maintains a professional and helpful tone, Agent ensures customer understands before ending ]) ]构建多轮对话场景的高级技巧1. 使用脚本控制对话流程脚本功能允许你精确控制对话的初始步骤然后让AI代理自然接管script[ scenario.user(change my subscription), scenario.agent(Sure, Im going to upgrade you to the Pro plan... done!), scenario.user(what!? no I dont want to upgrade, I want to cancel), scenario.proceed() # 从这里开始让AI代理自然响应 ]2. 测试边缘情况和错误恢复能力Scenario特别适合测试AI代理的错误处理和恢复能力description Agent initially misunderstands users request and offers wrong solution. User corrects them. Agent should acknowledge the mistake and provide the right help. Scenario多轮对话模拟结果展示包含完整的对话历史和评估指标3. 信息收集与确认模式测试AI代理收集必要信息的能力description User needs technical support but doesnt know technical details. agents[ CustomerServiceAgent(), scenario.UserSimulatorAgent(), scenario.JudgeAgent(criteria[ Agent should ask for the users account number or email, Agent should ask what model is their router, ]) ]4. 处理模糊请求和需求变更测试AI代理处理模糊请求和需求变更的能力description User initially asks about product A but then changes their mind and asks about product B, then asks to compare both products. Theyre indecisive and might change requirements multiple times. 完整场景示例账单争议解决以下是一个完整的多轮对话测试场景示例展示了如何测试客户服务AI代理处理账单争议的能力pytest.mark.agent_test pytest.mark.asyncio async def test_customer_service_billing(): class CustomerServiceAgent(scenario.AgentAdapter): async def call(self, input: scenario.AgentInput) - scenario.AgentReturnTypes: return await customer_service_bot.process( messagesinput.messages, context{department: billing} ) result await scenario.run( namebilling dispute resolution, description Customer received a bill that seems higher than expected. Theyre not angry but are confused and want an explanation. They have their account information ready and are generally cooperative but need clear explanations. , agents[ CustomerServiceAgent(), scenario.UserSimulatorAgent(), scenario.JudgeAgent(criteria[ Agent asks for account information to look up the bill, Agent reviews the bill details with the customer, Agent explains any charges that seem unusual or high, Agent offers options if there was an error, Agent maintains a professional and helpful tone, Agent ensures customer understands before ending ]) ], max_turns8 # 此类交互的合理限制 ) assert result.success assert len(result.messages) 4 # 应该有实质性对话 assert account in str(result.messages).lower() # 应该讨论账户信息场景组织与管理为了更好地组织和跟踪测试场景Scenario提供了场景分组功能result await scenario.run( namemy first scenario, descriptionA simple test to see if the agent responds., set_idmy-test-suite, # 将此场景分组到测试套件中 agents[ scenario.Agent(my_agent), scenario.UserSimulatorAgent(), ] )Scenario测试场景组织界面展示如何将相关场景分组管理下一步学习掌握了基本的脚本化模拟场景构建后可以探索更多高级技术Scripted Simulations - 控制对话流程Cache - 使测试具有确定性和更快的速度Debug Mode - 交互式调试场景通过Scenario的脚本化模拟测试你可以确保AI代理在各种复杂的多轮对话场景中表现稳定提供一致且高质量的用户体验。开始构建你的第一个测试场景提升AI代理的可靠性和健壮性吧【免费下载链接】scenarioAgentic testing for agentic codebases项目地址: https://gitcode.com/gh_mirrors/scen/scenario创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考