chatsbt/TESTING.md

Backend API Testing with Hurl

This document provides instructions for testing the chat backend API using Hurl.

Prerequisites

1. Install Hurl:

   • macOS: brew install hurl
   • Ubuntu/Debian: sudo apt update && sudo apt install hurl
   • Windows: Download from hurl.dev
   • Or use Docker: docker pull ghcr.io/orange-opensource/hurl:latest

2. Start the backend server:

   # Make sure you're in the project root directory
   python -m uvicorn app:application --host 0.0.0.0 --port 8000 --reload
   

Running the Tests

Basic Test Run

# Run all tests
hurl test-backend.hurl

# Run with verbose output
hurl --verbose test-backend.hurl

# Run with color output
hurl --color test-backend.hurl

Advanced Options

# Run with detailed report
hurl --report-html report.html test-backend.hurl

# Run specific tests (first 5 tests)
hurl --to-entry 5 test-backend.hurl

# Run tests with custom variables
hurl --variable host=localhost --variable port=8000 test-backend.hurl

# Run with retry on failure
hurl --retry 3 test-backend.hurl

Test Coverage

The test script covers:

✅ Happy Path Tests

• GET /api/models - Retrieves available AI models
• POST /api/chats - Creates new chat sessions
• GET /api/chats/{id} - Retrieves chat history
• POST /api/chats/{id}/messages - Sends messages to chat
• GET /api/chats/{id}/stream - Streams AI responses via SSE

✅ Error Handling Tests

• Invalid model names
• Non-existent chat IDs
• Missing required parameters

✅ Multi-turn Conversation Tests

• Multiple message exchanges
• Conversation history persistence
• Different model selection

Test Structure

The tests are organized in logical flow:

1. Model Discovery - Get available models
2. Chat Creation - Create new chat sessions
3. Message Exchange - Send messages and receive responses
4. History Verification - Ensure messages are persisted
5. Error Scenarios - Test edge cases and error handling

Environment Variables

You can customize the test environment:

# Set custom host/port
export HURL_host=localhost
export HURL_port=8000

# Or pass as arguments
hurl --variable host=127.0.0.1 --variable port=8080 test-backend.hurl

Troubleshooting

Common Issues

1. Connection refused

   • Ensure the backend is running on port 8000
   • Check firewall settings

2. Tests fail with 404 errors

   • Verify the backend routes are correctly configured
   • Check if database migrations have been run

3. SSE streaming tests timeout

   • Increase timeout: hurl --max-time 30 test-backend.hurl
   • Check if AI provider is responding

Debug Mode

Run tests with maximum verbosity:

hurl --very-verbose test-backend.hurl

Continuous Integration

Add to your CI/CD pipeline:

# GitHub Actions example
- name: Run API Tests
  run: |
    hurl --test --report-junit results.xml test-backend.hurl

Test Results

After running tests, you'll see:

• ✅ Green - Tests passed
• ❌ Red - Tests failed with details
• 📊 Summary - Total tests, duration, and success rate