hurl api test script

TESTING.md

@@ -0,0 +1,131 @@
+# Backend API Testing with Hurl
+
+This document provides instructions for testing the chat backend API using Hurl.
+
+## Prerequisites
+
+1. **Install Hurl**: 
+   - macOS: `brew install hurl`
+   - Ubuntu/Debian: `sudo apt update && sudo apt install hurl`
+   - Windows: Download from [hurl.dev](https://hurl.dev)
+   - Or use Docker: `docker pull ghcr.io/orange-opensource/hurl:latest`
+
+2. **Start the backend server**:
+   ```bash
+   # Make sure you're in the project root directory
+   python -m uvicorn app:application --host 0.0.0.0 --port 8000 --reload
+   ```
+
+## Running the Tests
+
+### Basic Test Run
+```bash
+# Run all tests
+hurl test-backend.hurl
+
+# Run with verbose output
+hurl --verbose test-backend.hurl
+
+# Run with color output
+hurl --color test-backend.hurl
+```
+
+### Advanced Options
+```bash
+# Run with detailed report
+hurl --report-html report.html test-backend.hurl
+
+# Run specific tests (first 5 tests)
+hurl --to-entry 5 test-backend.hurl
+
+# Run tests with custom variables
+hurl --variable host=localhost --variable port=8000 test-backend.hurl
+
+# Run with retry on failure
+hurl --retry 3 test-backend.hurl
+```
+
+## Test Coverage
+
+The test script covers:
+
+### ✅ Happy Path Tests
+- **GET /api/models** - Retrieves available AI models
+- **POST /api/chats** - Creates new chat sessions
+- **GET /api/chats/{id}** - Retrieves chat history
+- **POST /api/chats/{id}/messages** - Sends messages to chat
+- **GET /api/chats/{id}/stream** - Streams AI responses via SSE
+
+### ✅ Error Handling Tests
+- Invalid model names
+- Non-existent chat IDs
+- Missing required parameters
+
+### ✅ Multi-turn Conversation Tests
+- Multiple message exchanges
+- Conversation history persistence
+- Different model selection
+
+## Test Structure
+
+The tests are organized in logical flow:
+
+1. **Model Discovery** - Get available models
+2. **Chat Creation** - Create new chat sessions
+3. **Message Exchange** - Send messages and receive responses
+4. **History Verification** - Ensure messages are persisted
+5. **Error Scenarios** - Test edge cases and error handling
+
+## Environment Variables
+
+You can customize the test environment:
+
+```bash
+# Set custom host/port
+export HURL_host=localhost
+export HURL_port=8000
+
+# Or pass as arguments
+hurl --variable host=127.0.0.1 --variable port=8080 test-backend.hurl
+```
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Connection refused**
+   - Ensure the backend is running on port 8000
+   - Check firewall settings
+
+2. **Tests fail with 404 errors**
+   - Verify the backend routes are correctly configured
+   - Check if database migrations have been run
+
+3. **SSE streaming tests timeout**
+   - Increase timeout: `hurl --max-time 30 test-backend.hurl`
+   - Check if AI provider is responding
+
+### Debug Mode
+
+Run tests with maximum verbosity:
+```bash
+hurl --very-verbose test-backend.hurl
+```
+
+## Continuous Integration
+
+Add to your CI/CD pipeline:
+
+```yaml
+# GitHub Actions example
+- name: Run API Tests
+  run: |
+    hurl --test --report-junit results.xml test-backend.hurl
+```
+
+## Test Results
+
+After running tests, you'll see:
+- ✅ **Green** - Tests passed
+- ❌ **Red** - Tests failed with details
+- 📊 **Summary** - Total tests, duration, and success rate

test-backend.hurl

@@ -0,0 +1,165 @@
+# Hurl Test Script for Chat Backend API
+# This script tests the complete flow of the backend API
+
+# Test 1: Get available models
+GET http://localhost:8000/api/models
+
+HTTP 200
+[Captures]
+models: jsonpath "$.models"
+
+# Validate models response
+[Asserts]
+jsonpath "$.models" count > 0
+
+# Test 2: Create a new chat
+POST http://localhost:8000/api/chats
+Content-Type: application/json
+
+{
+  "model": "qwen/qwen3-235b-a22b-2507"
+}
+
+HTTP 200
+[Captures]
+chat_id: jsonpath "$.id"
+model: jsonpath "$.model"
+# Validate chat creation response
+[Asserts]
+jsonpath "$.id" != null
+jsonpath "$.model" == "qwen/qwen3-235b-a22b-2507"
+
+# Test 3: Get chat history (should be empty initially)
+GET http://localhost:8000/api/chats/{{chat_id}}
+
+HTTP 200
+[Captures]
+messages: jsonpath "$.messages"
+# Validate empty history
+[Asserts]
+jsonpath "$.messages" count == 0
+
+# Test 4: Post a message to the chat
+POST http://localhost:8000/api/chats/{{chat_id}}/messages
+Content-Type: application/json
+{
+  "message": "Hello, this is a test message",
+  "model": "{{model}}"
+}
+
+HTTP 200
+[Captures]
+message_id: jsonpath "$.message_id"
+status: jsonpath "$.status"
+# Validate message posting
+[Asserts]
+jsonpath "$.status" == "queued"
+jsonpath "$.message_id" != null
+
+# Test 5: Stream the response (SSE)
+GET http://localhost:8000/api/chats/{{chat_id}}/stream?message_id={{message_id}}
+HTTP 200
+[Asserts]
+header "Content-Type" == "text/event-stream; charset=utf-8"
+
+# Test 6: Verify chat history now contains messages
+GET http://localhost:8000/api/chats/{{chat_id}}
+
+HTTP 200
+[Captures]
+updated_messages: jsonpath "$.messages"
+# Validate messages are stored
+[Asserts]
+jsonpath "$.messages" count == 2
+jsonpath "$.messages[0].role" == "human"
+jsonpath "$.messages[0].content" == "Hello, this is a test message"
+jsonpath "$.messages[1].role" == "assistant"
+
+# Test 7: Post another message to test multi-turn conversation
+POST http://localhost:8000/api/chats/{{chat_id}}/messages
+Content-Type: application/json
+
+{
+  "message": "Can you tell me a joke?",
+  "model": "{{model}}"
+}
+
+HTTP 200
+[Captures]
+message_id2: jsonpath "$.message_id"
+
+# Test 8: Stream second response
+GET http://localhost:8000/api/chats/{{chat_id}}/stream?message_id={{message_id2}}
+
+HTTP 200
+
+# Test 9: Verify multi-turn conversation history
+GET http://localhost:8000/api/chats/{{chat_id}}
+
+HTTP 200
+[Captures]
+final_messages: jsonpath "$.messages"
+# Validate 4 messages (2 human + 2 assistant)
+[Asserts]
+jsonpath "$.messages" count == 4
+
+# Test 10: Error handling - Invalid model
+POST http://localhost:8000/api/chats
+Content-Type: application/json
+
+{
+  "model": "invalid-model-name"
+}
+
+HTTP 400
+[Asserts]
+jsonpath "$.error" == "Unknown model"
+
+# Test 11: Error handling - Chat not found
+GET http://localhost:8000/api/chats/non-existent-chat-id
+
+HTTP 404
+[Asserts]
+jsonpath "$.error" == "Not found"
+
+# Test 12: Error handling - Invalid chat ID for messages
+POST http://localhost:8000/api/chats/non-existent-chat-id/messages
+Content-Type: application/json
+
+{
+  "message": "This should fail",
+  "model": "qwen/qwen3-235b-a22b-2507"
+}
+
+HTTP 404
+[Asserts]
+jsonpath "$.error" == "Chat not found"
+
+# Test 13: Error handling - Missing message in post
+POST http://localhost:8000/api/chats/{{chat_id}}/messages
+Content-Type: application/json
+
+{
+  "model": "{{model}}"
+}
+
+HTTP 200
+# Note: The backend seems to accept empty messages, so this might not fail
+
+# Test 14: Create another chat with different model
+POST http://localhost:8000/api/chats
+Content-Type: application/json
+
+{
+  "model": "openai/gpt-4.1"
+}
+
+HTTP 200
+[Captures]
+chat_id2: jsonpath "$.id"
+model2: jsonpath "$.model"
+
+# Test 15: Verify second chat has different ID
+[Asserts]
+variable "chat_id" != "chat_id2"
+variable "model2" == "openai/gpt-4.1"