AWS-Backed Strands Agents Framework Paired With Langfuse for AI Quality Evaluation

A proof-of-concept project demonstrates how to build a Python-based banking assistant using Strands Agents, an open-source LLM agent SDK released by AWS in May 2025. The agent simulates a customer support system for a fictional bank, handling tasks like card freezing, transaction lookups, and dispute management. Because AI applications can return confident but incorrect answers that traditional metrics like error rates and latency fail to detect, the project integrates Langfuse for tracing and evaluation. Langfuse, which is open-source and self-hostable via Docker Compose, enables both offline and online assessments of agent outputs, including LLM-as-judge scoring and human annotation queues. The full source code is available on GitHub, covering setup steps from agent configuration through CI/CD-ready evaluation pipelines.