OpenAPI Tooling Guide

Complete guide to validating, testing, and documenting the Bloqr Compiler API using the OpenAPI specification.

OpenAPI Tooling Guide

Complete guide to validating, testing, and documenting the Bloqr Compiler API using the OpenAPI specification.

📋 Table of Contents

Overview
Validation
Documentation Generation
Contract Testing
Postman Testing
CI/CD Integration
Best Practices

Overview

The Bloqr Compiler API is fully documented using the OpenAPI 3.0.3 specification (docs/api/openapi.yaml). This specification serves as the single source of truth for:

API endpoint definitions
Request/response schemas
Authentication requirements
Error responses
Examples and documentation

Validation

Validate OpenAPI Spec

Ensure your docs/api/openapi.yaml conforms to the OpenAPI specification:

# Run validation
deno task openapi:validate

# Or directly
./scripts/validate-openapi.ts

What it checks:

✅ YAML syntax
✅ OpenAPI version compatibility
✅ Required fields (info, paths, etc.)
✅ Unique operation IDs
✅ Response definitions
✅ Schema completeness
✅ Best practices compliance

Example output:

🔍 Validating OpenAPI specification...

✅ YAML syntax is valid
✅ OpenAPI version: 3.0.3
✅ Title: Bloqr Compiler API
✅ Version: 2.0.0
✅ Servers: 2 defined
✅ Paths: 10 endpoints defined
✅ Operations: 13 total
✅ Schemas: 30 defined
✅ Security schemes: 1 defined
✅ Tags: 5 defined

📋 Checking best practices...

✅ Request examples: 2 found
✅ Contact info provided
✅ License: MIT

============================================================
VALIDATION RESULTS
============================================================

✅ OpenAPI specification is VALID!

Summary: 0 errors, 0 warnings

Pre-commit Validation

Add to your git hooks:

#!/bin/sh
# .git/hooks/pre-commit
deno task openapi:validate || exit 1

Documentation Generation

Generate HTML Documentation

Create beautiful, interactive API documentation using Redoc:

# Generate docs
deno task openapi:docs

# Or directly
./scripts/generate-docs.ts

Output files:

docs/api/index.html - Interactive HTML documentation (Redoc)
docs/api/README.md - Markdown reference documentation

Generate Cloudflare API Shield Schema

Generate a Cloudflare-compatible schema for use with Cloudflare's API Shield Schema Validation:

# Generate Cloudflare schema
deno task schema:cloudflare

# Or directly
./scripts/generate-cloudflare-schema.ts

What it does:

✅ Filters out localhost servers (keeps only production/staging URLs)
✅ Removes non-standard x-* extension fields from operations
✅ Generates docs/api/cloudflare-schema.yaml ready for API Shield

Why use this: Cloudflare's API Shield Schema Validation provides request/response validation at the edge. The generated schema is optimized for Cloudflare's parser by removing development servers and custom ex[...]

Learn more: Cloudflare API Shield Schema Validation

CI/CD Integration: The schema generation is validated in CI to ensure it stays in sync with the main OpenAPI spec. If you update docs/api/openapi.yaml, you must regenerate the Cloudflare schema by running deno task schema:cloudflare and committing the result.

View Documentation

# Open HTML docs
open docs/api/index.html

# Or serve locally
python3 -m http.server 8000 --directory docs/api
# Then visit http://localhost:8000

Features

The generated HTML documentation includes:

🔍 Search functionality - Find endpoints quickly
📱 Responsive design - Works on mobile/tablet/desktop
🎨 Code samples - Request/response examples
📊 Schema explorer - Interactive schema browser
🔗 Deep linking - Share links to specific endpoints
📥 Download spec - Export OpenAPI YAML/JSON

Customization

Edit scripts/generate-docs.ts to customize:

Theme colors
Logo/branding
Sidebar configuration
Code sample languages

Contract Testing

Contract tests validate that your live API conforms to the OpenAPI specification.

Run Contract Tests

# Test against local server (default)
deno task test:contract

# Test against production
API_BASE_URL=https://api.bloqr.dev deno task test:contract

# Test specific scenarios
deno test --allow-read --allow-write --allow-net --allow-env worker/openapi-contract.test.ts --filter "Contract: GET /api"

What's Tested

Core Endpoints:

✅ GET /api - API info
✅ GET /metrics - Performance metrics
✅ POST /compile - Synchronous compilation
✅ POST /compile/stream - SSE streaming
✅ POST /compile/batch - Batch processing

Async Queue Operations (Cloudflare Queues):

✅ POST /compile/async - Queue async job
✅ POST /compile/batch/async - Queue batch job
✅ GET /queue/stats - Queue statistics
✅ GET /queue/results/{id} - Retrieve job results

Contract Validation:

✅ Response status codes match spec
✅ Response content types are correct
✅ Required fields are present
✅ Data types match schemas
✅ Headers conform to spec (X-Cache, X-Request-Deduplication)
✅ Error responses have proper structure

Async Testing with Queues

The contract tests properly validate Cloudflare Queue integration:

// Queues async compilation
const response = await apiRequest('/compile/async', {
    method: 'POST',
    body: JSON.stringify({ configuration, preFetchedContent }),
});

// Returns 202 if queues available, 500 if not configured
validateResponseStatus(response, [202, 500]);

if (response.status === 202) {
    const data = await response.json();
    // Validates requestId is returned
    validateBasicSchema(data, ['success', 'requestId', 'message']);
}

Queue Test Scenarios

Standard Priority Queue
- Tests default queue behavior
- Validates requestId generation
- Confirms job queuing
High Priority Queue
- Tests priority routing
- Validates faster processing (when implemented)
Batch Queue Operations
- Tests multiple jobs queued together
- Validates batch requestId tracking
Queue Statistics
- Validates queue depth metrics
- Confirms job status tracking
- Tests history retention

CI/CD Contract Testing

# .github/workflows/contract-tests.yml
name: Contract Tests

on: [push, pull_request]

jobs:
  contract-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - uses: denoland/setup-deno@v1
        with:
          deno-version: v2.x
      
      - name: Start local server
        run: deno task dev &
        
      - name: Wait for server
        run: sleep 5
        
      - name: Run contract tests
        run: deno task test:contract

Postman Testing

See POSTMAN_TESTING.md for complete Postman documentation.

Generate / Regenerate the Postman Collection

The Postman collection and environment files are auto-generated from docs/api/openapi.yaml. Do not edit them directly.

# Regenerate from the canonical OpenAPI spec
deno task postman:collection

This creates / updates:

docs/postman/postman-collection.json — all API requests with automated test assertions
docs/postman/postman-environment.json — local and production environment variables

The CI validate-postman-collection job regenerates the files and fails the build if the committed copies are out of sync with docs/api/openapi.yaml. Always run deno task postman:collection after editing the spec.

Schema Hierarchy

docs/api/openapi.yaml                 ← canonical source of truth (edit this)
docs/api/cloudflare-schema.yaml       ← auto-generated (deno task schema:cloudflare)
docs/postman/postman-collection.json  ← auto-generated (deno task postman:collection)
docs/postman/postman-environment.json ← auto-generated (deno task postman:collection)

Quick Start

# Import collection and environment into Postman
# - docs/postman/postman-collection.json
# - docs/postman/postman-environment.json

# Or use Newman CLI
npm install -g newman
newman run docs/postman/postman-collection.json -e docs/postman/postman-environment.json

Postman Features

🧪 25+ test requests
✅ Automated assertions
📊 Response validation
🔄 Dynamic variables
📈 Performance testing

CI/CD Integration

GitHub Actions

Complete pipeline for validation, testing, and documentation:

name: OpenAPI Pipeline

on: [push, pull_request]

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: denoland/setup-deno@v1
      
      - name: Validate OpenAPI spec
        run: deno task openapi:validate

  validate-cloudflare-schema:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: denoland/setup-deno@v1
      
      - name: Generate Cloudflare schema
        run: deno task schema:cloudflare
      
      - name: Check schema is up to date
        run: |
          if ! git diff --quiet docs/api/cloudflare-schema.yaml; then
            echo "❌ Cloudflare schema is out of date!"
            echo "Run 'deno task schema:cloudflare' and commit the result."
            exit 1
          fi

  generate-docs:
    needs: validate
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: denoland/setup-deno@v1
      
      - name: Generate documentation
        run: deno task openapi:docs
      
      - name: Upload docs
        uses: actions/upload-artifact@v3
        with:
          name: api-docs
          path: docs/api/

  contract-tests:
    needs: validate
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: denoland/setup-deno@v1
      
      - name: Start server
        run: deno task dev &
        
      - name: Wait for server
        run: sleep 10
      
      - name: Run contract tests
        run: deno task test:contract
        
  postman-tests:
    needs: validate
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Start server
        run: docker compose up -d
        
      - name: Install Newman
        run: npm install -g newman
        
      - name: Run Postman tests
        run: newman run docs/postman/postman-collection.json -e docs/postman/postman-environment.json --reporters cli,json
        
      - name: Upload results
        uses: actions/upload-artifact@v3
        with:
          name: newman-results
          path: newman/

Pre-deployment Checks

#!/bin/bash
# scripts/pre-deploy.sh

echo "🔍 Validating OpenAPI spec..."
deno task openapi:validate || exit 1

echo "☁️  Generating Cloudflare schema..."
deno task schema:cloudflare || exit 1

echo "📚 Generating documentation..."
deno task openapi:docs || exit 1

echo "🧪 Running contract tests..."
deno task test:contract || exit 1

echo "✅ All checks passed! Ready to deploy."

Best Practices

1. Keep Spec and Code in Sync

Problem: Spec drifts from actual implementation

Solution:

Run contract tests on every PR
Use CI/CD to block deployment if tests fail
Review OpenAPI changes alongside code changes

# Add to .git/hooks/pre-push
deno task openapi:validate
deno task test:contract

2. Version Your API

Current version: 2.0.0 in docs/api/openapi.yaml

When making breaking changes:

Increment major version (2.0.0 → 3.0.0)
Update info.version in docs/api/openapi.yaml
Document changes in CHANGELOG.md
Consider API versioning in URLs

3. Document Examples

Good:

requestBody:
  content:
    application/json:
      schema:
        $ref: '#/components/schemas/CompileRequest'
      examples:
        simple:
          summary: Simple compilation
          value:
            configuration:
              name: My Filter List
              sources:
                - source: test-rules

Why: Examples improve documentation and serve as test data.

4. Use Async Queues Appropriately

When to use Cloudflare Queues:

✅ Use queues for:

Long-running compilations (>5 seconds)
Large batch operations
Background processing
Rate limit avoidance
Retry-able operations

❌ Don't use queues for:

Quick operations (<1 second)
Real-time user interactions
Operations needing immediate feedback

Implementation:

// Queue job
const requestId = await queueCompileJob(env, configuration, preFetchedContent);

// Return immediately
return Response.json({
    success: true,
    requestId,
    message: 'Job queued for processing'
}, { status: 202 });

// Client polls for results
GET /queue/results/{requestId}

5. Test Queue Scenarios

Always test queue operations:

# Test queue availability
deno test --filter "Contract: POST /compile/async"

# Test queue stats
deno test --filter "Contract: GET /queue/stats"

# Test result retrieval
deno test --filter "Contract: GET /queue/results"

6. Monitor Queue Health

Track queue metrics:

Queue depth (pending jobs)
Processing rate (jobs/minute)
Average processing time
Failure rate
Retry rate

Access via: GET /queue/stats

7. Handle Queue Unavailability

Queues may not be configured in all environments:

if (!env.BLOQR_BACKEND_QUEUE) {
    return Response.json({
        success: false,
        error: 'Queue not available. Use synchronous endpoints instead.'
    }, { status: 500 });
}

Contract tests handle this gracefully:

validateResponseStatus(response, [202, 500]); // Both OK

Troubleshooting

Validation Fails

❌ Missing "operationId" for POST /compile

Fix: Add unique operationId to all operations in docs/api/openapi.yaml

Contract Tests Fail

Expected status 200, got 500

Fix:

Check server logs
Verify request body matches schema
Ensure queue bindings configured (for async endpoints)

Documentation Not Generating

Failed to parse YAML

Fix: Validate YAML syntax:

deno task openapi:validate

Queue Tests Always Return 500

Cause: Cloudflare Queues not configured locally

Expected: Queues are production-only. Tests accept 202 OR 500.

Fix: Deploy to Cloudflare Workers to test queue functionality.

Resources

Summary

The OpenAPI tooling provides:

Validation - Ensure spec quality (openapi:validate)
Documentation - Generate beautiful docs (openapi:docs)
Cloudflare Schema - Generate API Shield schema (schema:cloudflare)
Postman Collection - Regenerate from spec (postman:collection)
Contract Tests - Verify API compliance (test:contract)
Queue Support - Async operations via Cloudflare Queues

Schema Hierarchy

docs/api/openapi.yaml                 ← canonical source of truth (edit this)
docs/api/cloudflare-schema.yaml       ← auto-generated (deno task schema:cloudflare)
docs/postman/postman-collection.json  ← auto-generated (deno task postman:collection)
docs/postman/postman-environment.json ← auto-generated (deno task postman:collection)

All tools are designed to work together in a continuous integration pipeline, ensuring your API stays consistent, well-documented, and reliable.

Was this page helpful?