chatsbt/TESTING.md

# Backend API Testing with Hurl

This document provides instructions for testing the chat backend API using Hurl.

## Prerequisites

1. **Install Hurl**: 
   - macOS: `brew install hurl`
   - Ubuntu/Debian: `sudo apt update && sudo apt install hurl`
   - Windows: Download from [hurl.dev](https://hurl.dev)
   - Or use Docker: `docker pull ghcr.io/orange-opensource/hurl:latest`

2. **Start the backend server**:
   ```bash
   # Make sure you're in the project root directory
   python -m uvicorn app:application --host 0.0.0.0 --port 8000 --reload
   ```

## Running the Tests

### Basic Test Run
```bash
# Run all tests
hurl test-backend.hurl

# Run with verbose output
hurl --verbose test-backend.hurl

# Run with color output
hurl --color test-backend.hurl
```

### Advanced Options
```bash
# Run with detailed report
hurl --report-html report.html test-backend.hurl

# Run specific tests (first 5 tests)
hurl --to-entry 5 test-backend.hurl

# Run tests with custom variables
hurl --variable host=localhost --variable port=8000 test-backend.hurl

# Run with retry on failure
hurl --retry 3 test-backend.hurl
```

## Test Coverage

The test script covers:

### ✅ Happy Path Tests
- **GET /api/models** - Retrieves available AI models
- **POST /api/chats** - Creates new chat sessions
- **GET /api/chats/{id}** - Retrieves chat history
- **POST /api/chats/{id}/messages** - Sends messages to chat
- **GET /api/chats/{id}/stream** - Streams AI responses via SSE

### ✅ Error Handling Tests
- Invalid model names
- Non-existent chat IDs
- Missing required parameters

### ✅ Multi-turn Conversation Tests
- Multiple message exchanges
- Conversation history persistence
- Different model selection

## Test Structure

The tests are organized in logical flow:

1. **Model Discovery** - Get available models
2. **Chat Creation** - Create new chat sessions
3. **Message Exchange** - Send messages and receive responses
4. **History Verification** - Ensure messages are persisted
5. **Error Scenarios** - Test edge cases and error handling

## Environment Variables

You can customize the test environment:

```bash
# Set custom host/port
export HURL_host=localhost
export HURL_port=8000

# Or pass as arguments
hurl --variable host=127.0.0.1 --variable port=8080 test-backend.hurl
```

## Troubleshooting

### Common Issues

1. **Connection refused**
   - Ensure the backend is running on port 8000
   - Check firewall settings

2. **Tests fail with 404 errors**
   - Verify the backend routes are correctly configured
   - Check if database migrations have been run

3. **SSE streaming tests timeout**
   - Increase timeout: `hurl --max-time 30 test-backend.hurl`
   - Check if AI provider is responding

### Debug Mode

Run tests with maximum verbosity:
```bash
hurl --very-verbose test-backend.hurl
```

## Continuous Integration

Add to your CI/CD pipeline:

```yaml
# GitHub Actions example
- name: Run API Tests
  run: |
    hurl --test --report-junit results.xml test-backend.hurl
```

## Test Results

After running tests, you'll see:
- ✅ **Green** - Tests passed
- ❌ **Red** - Tests failed with details
- 📊 **Summary** - Total tests, duration, and success rate