Peek-Then-Pay

The Open Licensing Standard for AI Crawlers

Giving publishers a voice and AI a smarter path forward — beyond scraping vs. paywalls to balanced collaboration.

Why Peek-Then-Pay?

Think of it as robots.txt for AI — but enforceable and auditable. Giving publishers a voice and AI a smarter path forward.

⚠️ Two Broken Extremes

Unrestricted Scraping

Disregards publisher rights, invites legal conflict, and provides no compensation for content creators.

Restrictive Walls

Hard paywalls and centralized tollbooths block discovery, fragment ecosystems, and concentrate control.

✅ Balanced Hybrid Model

Decentralized Control

Publishers maintain authority at their domain edge with standardized manifest advertising.

Centralized Coordination

Unified licensing marketplace handles payments and accounts across all publishers.

Protocol Flow

AI Agent

→

Discovery

→

HTTP 203 Preview

→

JWT License

→

Licensed Content

Automatic Discovery: AI agents who don't know the domain supports Peek-Then-Pay automatically get HTTP 203 responses with previews and pricing when landing on licensed content — enabling discovery without blocking access.

Composable Architecture: Build vs Buy

Publishers can mix and match components based on their technical capabilities and strategic preferences.

1. Edge Enforcer

Validates JWTs and serves content with minimal latency.

• 3P Plugin: CDN-based solutions

• SaaS: Managed enforcement service

• Home-built: Custom edge implementation

2. Tools

Transform and deliver domain-specific content.

• 3P Components: Plug-and-play tools

• Domain Assets: Existing agentic artifacts

• Brand Governance: Consistent quality

3. License Server

Centralized marketplace for AI agent discovery and payment.

• 3P SaaS: Recommended approach

• Marketplace: Discovery & payments

• Collective: Small publisher groups

How It Works

The Peek-Then-Pay Protocol

Discovery

AI agents discover publisher participation by either getting a more permissive 203 when requesting a licensed page, or by looking for /.well-known/peek.json

Preview

Content requests return HTTP 203 with preview + pricing for informed decisions

License

AI agents acquire JWT licenses for specific intents (read, summarize, embed, etc.) from the license server specified in the peek.json manifest

Access

Edge enforcers validate JWTs and serve licensed content with forensic manifests. Plug-and-play tools can deliver pre-transformed data, benefiting publishers through pay-per-use models while helping AI agents skip data cleaning and transformation steps

Report

Both parties report usage asynchronously for billing and dispute resolution

💡 Key Innovation: HTTP 203 responses let AI agents see and evaluate content quality before licensing, unlike traditional paywalls that block discovery.

⚡ Efficiency Gains: Publishers provide pre-transformed data (embeddings, summaries) that AI systems already need, reducing compute costs by 60–90% while ensuring quality and context preservation.

/.well-known/peek.json

{
  "version": "1.0",
  "meta": {
    "site_name": "TechNews Daily",
    "publisher": "TechNews Corp",
    "publisher_id": "01HQ2Z3Y4K5M6N7P8Q9R0S1T1X",
    "domains": ["technews.com", "*.technews.com"],
    "categories": ["technology", "news", "reviews"],
    "last_updated": "2025-09-01"
  },
  "enforcement": {
    "rate_limit_per_ip": 50,
    "grace_period_seconds": 300,
    "failover_mode": "deny",
    "bypass_paths": ["/robots.txt", "/sitemap.xml"]
  },
  "license": {
    "license_issuer": "https://api.fetchright.ai/v1/license",
    "terms_url": "https://technews.com/legal/ai-terms",
    "supported_intents": ["peek", "read", "summarize", "quote"],
    "content_hints": {
      "average_page_size_kb": 45,
      "content_types": ["text/html", "application/json"],
      "update_frequency": "daily",
      "cache_duration": 3600
    }
  }
}

Standard Intents

peek - Previewread - Full contentsummarize - Summaryquote - Citationsembed - Vectorsanalyze - Structure

Publishers can provide pre-computed transformations, eliminating redundant AI processing and ensuring domain-specific accuracy.

🔍 Discovery

AI agents automatically discover licensing through standardized manifest files and HTTP 203 previews

⚡ Edge Enforcement

Fast, distributed license validation at CDN/edge layer with assertion-only JWTs

📊 Bilateral Reporting

Both parties track usage for accuracy, billing reconciliation, and dispute resolution

Interactive Validation Demos

Try out the Peek-Then-Pay specification validation tools in real-time

1. Validate peek.json Manifest

Test the validation of a peek.json manifest file against the official specification schema.

peek.json Content

Validation Result

Click "Validate" to see results...

2. Validate Pricing Schema

Test the validation of a pricing scheme against the pricing schema specification.

Pricing JSON Content

{
  "pricing_scheme_id": "01HZXYZ123456789ABCDEFGHIJ",
  "pricing_digest": "sha256:a1b2c3d4e5f6789012345678901234567890123456789012345678901234abcd",
  "publisher_id": "01HZXYZ123456789ABCDEFGHIJ",
  "currency": "USD",
  "cache_ttl_seconds": 3600,
  "intents": {
    "read": {
      "intent": "read",
      "pricing_mode": "per_request",
      "enforcement_method": "trust",
      "usage": {
        "immediate": { "price_cents": 1 },
        "session": { "price_cents": 2, "max_ttl_seconds": 3600 },
        "index": { "price_cents": 5, "max_ttl_seconds": 86400, "requires_contract": true }
      }
    },
    "quote": {
      "intent": "quote",
      "pricing_mode": "per_1000_tokens",
      "enforcement_method": "tool_required",
      "usage": {
        "immediate": { "price_cents": 2 },
        "session": { "price_cents": 4, "max_ttl_seconds": 1800 }
      },
      "path_multipliers": {
        "/premium/**": 2.0,
        "/public/**": 0.5
      }
    },
    "summarize": {
      "intent": "summarize",
      "pricing_mode": "per_request",
      "enforcement_method": "tool_required",
      "usage": {
        "immediate": { "price_cents": 5 },
        "session": { "price_cents": 10, "max_ttl_seconds": 3600 },
        "train": { "price_cents": 500, "requires_contract": true },
        "distill": { "price_cents": 250, "requires_contract": true }
      },
      "model": {
        "id": "sum:gpt-4.1-mini@v2",
        "provider": "openai",
        "name": "gpt-4.1-mini",
        "version": "v2",
        "digest": "sha256:b2c3d4e5f6789012345678901234567890123456789012345678901234bcdef0"
      }
    }
  },
  "search": {
    "endpoint_url": "https://search.example.com/v1/search",
    "price_cents": 3
  },
  "rag_ingest": {
    "endpoint_url": "https://rag.example.com/v1/ingest",
    "pricing_mode": "per_1000_tokens",
    "price_cents": 25
  },
  "quotas": {
    "burst_rps": 100,
    "max_license_duration_seconds": 3600
  }
}

Validation Result

Click "Validate" to see results...

3. Generate Assertion-Only License

See how the license server generates JWT licenses and DPoP proofs for content access.

Generated License Response

Click "Generate License & DPoP Proof" to see the assertion-only license...

These demos use the official @peekthenpay/peek-json-spec package for validation and license generation.

Ecosystem

🔓

Peek-Then-Pay

Open Source Standard

The open schema and governance framework hosted on GitHub. Community-driven development with transparent RFC process.

• CC0-1.0 Licensed specification
• Community governance
• Open RFC process
• Reference implementations

⚡

FetchRight.ai

Reference Platform

The operational backbone for Peek-Then-Pay, providing secure infrastructure and optional transformation services.

• Secure license issuance & verification APIs
• Token-based accounting & payment processing
• Optional transformation services (embeddings, RAG)
• Analytics for publishers and operators

The ecosystem supports both open-source implementations and commercial services, ensuring widespread adoption while maintaining neutrality.

Learn More About FetchRight →

Get Involved

Peek-Then-Pay is community-driven. Help shape the future of AI content licensing.

💬

Join Discussions

Participate in discussions on GitHub Issues and RFCs. Help design new features and improvements.

View GitHub Issues →

🔧

Contribute Code

Contribute adapters for Cloudflare, Fastly, and Nginx. Build tools and libraries for the ecosystem.

Browse Repositories →

📢

Spread the Word

Join the conversation on LinkedIn and help evangelize responsible AI content licensing.

Follow #PeekThenPay →

Want to Implement Peek-Then-Pay?

We're looking for early adopters and implementation partners. Whether you're a publisher, AI company, or infrastructure provider, we'd love to work with you.

Governance & License

⚖️

Open Standard

Peek-Then-Pay is an open standard released under the CC0-1.0 Universal license. Specification and governance discussions live in the PeekThenPay GitHub organization.

Principles

•Open: Free to implement and use
•Neutral: No vendor lock-in
•Transparent: Public governance process
•Collaborative: Community-driven development

Governance

•RFC process for major changes
•Community consensus decision making
•Regular community calls and updates
•Transparent roadmap and priorities

Current Version: 1.0 (Draft) •License: CC0-1.0 Universal •Repository: github.com/PeekThenPay/peek-json-spec

View Full License →