Why is Humanloop shutting down?

The Humanloop team has been acquired by Anthropic. The Humanloop platform is being sunset and will be permanently deleted on September 8, 2025.

How do I export my data before the shutdown?

Use the official Humanloop Export Tool. You need a Humanloop API key, and the tool can be run with Python or uv. Detailed instructions are in the GitHub repository.

What happens to my billing and subscription?

All billing stopped on July 30, 2025. No further charges will be incurred. Prepaid credits and subscriptions end on the sunset date.

What alternatives does Humanloop recommend?

The announcement specifically recommends Langfuse and Braintrust as alternatives for AI evaluation and prompt management.

How does the export tool handle large datasets?

For datasets exceeding 1 million Logs, you must contact support@humanloop.com to arrange a custom export process.

What is the data format for exported data?

Data is exported in JSON and JSONL formats, making it compatible with most data processing tools. You may need to transform it for your new platform.

Directory/Coding/LLM Evaluation/Humanloop

Humanloop

Humanloop was an enterprise platform designed to help teams build, evaluate, and ship trustworthy LLM-powered applications. It provided a unified environment for collaborative prompt engineering, rigorous model evaluation with both UI and code-first workflows, and production observability for monitoring AI systems. The platform was built for both technical and non-technical users, enabling engineers to implement evaluations in code while product managers and domain experts could work on prompts and evaluation through a UI. Key capabilities included version-controlled prompt management, offline and online evaluators (including LLM-as-a-judge), dataset management, CI/CD integration, and comprehensive logging and tracing. Now, following its acquisition by Anthropic, the Humanloop platform has been sunset as of September 8, 2025. The team and core technology have been integrated into Anthropic to further its mission of building safe AI.

Coding

Visit website

Added on

Sep 21, 2026

LLM EvaluationsPrompt ManagementAI ObservabilityMulti-LLM PlaygroundEnterprise SSOVPC DeploymentSOC-2 Certified

Product information

Everything worth knowing about Humanloop

The complete picture — from everyday features to the technical detail builders and IT folks dig for.

Core features

What it actually does

LLM Evaluations

Build robust evaluation workflows for your LLM applications with code-based, AI-as-judge, and human review evaluators. Supports UI-first and code-first approaches, CI/CD integration, and dataset management.

Prompt Management

Version, test, and deploy prompts in a collaborative workspace. Includes a multi-LLM playground, role-based access, function calling support, and tagged deployments for controlled rollouts.

AI Observability

Monitor production LLM applications with tracing, logging, alerting, and end-user feedback capture. Helps track performance and diagnose issues in real-time.

Enterprise Security & Compliance

SOC-2 Type 2 certified, GDPR compliant, with options for custom SSO/SAML, VPC deployment, HIPAA compliance (with BAAs), and role-based access controls.

Multi-Model Playground

Compare and test prompts across different LLM providers within a collaborative UI, enabling rapid iteration and performance benchmarking.

Technical capabilities

SDKs & Integrations

Provides Python and JavaScript SDKs, a Vercel AI SDK integration, and APIs for logging, evaluation, and prompt management to embed into existing codebases and CI/CD pipelines.

Deployment Flexibility

Offers multiple deployment options: default cloud (AWS US-east), region-specific instances (EU/UK/US), dedicated VPC instances, and a self-hosted option for full control.

Must watch videos

How our AI Startup Grew 60x After a Pivot | Raza Habib, Jordan Burgess of Humanloop

11:36

Tutorial11:36

How our AI Startup Grew 60x After a Pivot | Raza Habib, Jordan Burgess of Humanloop

Meet Raza Habib and Jordan Burgess, co-founders of Humanloop, an innovative startup revolutionizing AI Engineering. Their blend of expertise in machine learning and software engineering drives their mission to create ethical AI solutions. With strategic partnerships and a growing user base, Humanloop leads the charge in transforming AI for privacy, fairness, and transparency. Join us as we explore their journey and commitment to reshaping the future of AI Engineering. EO stands for Entrepreneurship & Opportunities. We're looking for more inspiring stories of entrepreneurs all over the world, so don't hesitate to contact us! :) Feature Inquiry 💼 👉🏻👉🏻 [partner@eoeoeo.net](mailto:partner@eoeoeo.net) EO X | @eostudi0 LinkedIn | @EO STUDIO Instagram | @eostudio.official Newsletter | https://www.eomag.io/subscribe?utm_source=youtube&utm_medium=description Subtitles for this video were created using [XL8.ai](http://xl8.ai/) machine translation.

Full Walkthrough: Workflow for AI Coding — Matt Pocock

1:36:30

Tutorial1:36:30

Full Walkthrough: Workflow for AI Coding — Matt Pocock

A hands-on workshop covering the full lifecycle of AI-assisted development, from turning ambiguous requirements into agent-ready plans to running autonomous coding agents that ship production features. You'll learn to stress-test vague briefs into structured PRDs, slice work into thin "tracer bullet" vertical slices, and run an AI agent with TDD. You'll watch it select tasks, write tests, implement code, and commit. You'll then refine your prompts based on where it struggles, graduate to fully autonomous (AFK) runs, and learn to design codebases that maximize agent effectiveness. You'll walk away knowing how to: - Turn ambiguous requirements into agent-ready issues - Slice work into vertical tracer bullets an agent can grab independently - Run AI agents human-in-the-loop and autonomously with TDD - Design codebase architectures that AI agents love to work in For: Engineers ready to move beyond chat-based AI assistance and build a real workflow for shipping features with autonomous coding agents. Speaker info: - https://x.com/mattpocockuk - https://www.linkedin.com/in/mapocock/ - https://youtube.com/@mattpocockuk Timestamps: 00:00:00 - Introduction 00:00:14 - The Thesis of AI Engineering 00:04:20 - Phase 1: Research & Prototyping 00:12:45 - Phase 2: The Grill Session 00:22:10 - Phase 3: Writing the PRD 00:35:50 - Phase 4: Slicing Work into Issues 00:48:15 - Phase 5: Implementation with AI Agents 01:05:30 - Phase 6: Human-in-the-Loop Review 01:18:45 - Phase 7: Deployment & Monitoring 01:28:10 - Designing Codebases for AI Effectiveness 01:34:06 - Final Takeaways & Summary --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/-QFHIoCo-Ko-ai-coding-workflow Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIE New York, October 12–14 — the first AI Engineer Finance mainstage summit - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney ---

What is Human In The Loop with AI? How HITL Shapes AI Systems

10:44

Tutorial10:44

What is Human In The Loop with AI? How HITL Shapes AI Systems

Ready to become a certified watsonx Data Scientist? Register now and use code IBMTechYT20 for 20% off of your exam → https://ibm.biz/BdpZcg Learn more about Human-In-The-Loop here → https://ibm.biz/BdpZch 🚀 Should AI act alone or involve humans? Martin Keen explains human-in-the-loop AI, exploring HITL, RLHF, and how humans teach, tune, and monitor AI systems. Learn how active learning and confidence thresholds ensure safety and trust as AI evolves from training to autonomy. AI news moves fast. Sign up for a monthly newsletter for AI updates from IBM → https://ibm.biz/BdpZcV #ai #humanintheloop #aiworkflow #rlhf

Who uses it

Real use cases, no hype

AI Feature Evaluation

Systematically test and compare LLM prompts and model versions for quality.

Prompt Engineering

Collaboratively craft, version, and manage prompts across teams and environments.

Production Monitoring

Trace, log, and alert on real-world LLM application performance and errors.

CI/CD for AI

Automate evaluation of model/prompt changes before they reach production.

Compliance & Security

Maintain audit trails and deploy in VPCs or HIPAA-compliant environments.

Team Collaboration

Bridge engineers and domain experts with UI and API access to AI systems.

What's great

Pioneered the LLM eval platform category.
Seamless UI + code-first workflow for engineers and PMs.
Enterprise-grade security: SOC-2, HIPAA, VPC deployment.
Unified platform for prompt management, evaluation, and observability.

Technical strengths

Advanced evaluation tooling: LLM-as-a-judge, CI/CD integration, human review.
Proprietary .prompt and .agent file serialization for version control.
Flexible deployment: cloud, region-specific, dedicated, or self-hosted AWS VPC.

Where it falls short

Platform is being sunset; all data deleted Sept 8, 2025.
Migration required; team is joining Anthropic.
Free tier is limited: 2 members, 50 evals, 10K logs/mo.

Technical limitations

Bring Your Own API Key model means separate billing with LLM providers.
Export tool required for large datasets (>1M logs) and is a DIY process.

For technical folks

The deep-dive specs

FAQ

Includes technical Q&A

Humanloop is the LLM Evals Platform for Enterprises that enables product teams to build robust AI features. It focuses on evaluation, prompt management, and observability.