AI RECEPTIONIST PLATFORM

Lab Overview

What You'll Do: Build a Flask MLOps service to track AI performance, integrate Prometheus for metrics monitoring, and implement comprehensive metrics collection for your AI receptionist

Lab Collaborators:

Edward Lampoh - Software Developer & Collaborator
Oluwafemi Adebayo, PhD - Academic Professor & Collaborator

🚨 Prerequisites Required

You must complete Lab 1 and have a working Next.js application before starting Lab 2.

Prerequisites from Lab 1

Before starting Lab 2, ensure you completed Lab 1 and have:

Working Next.js app running at http://localhost:3000
Environment file (.env) with these keys configured:

env

DATABASE_URL="your_neon_connection_string"
GOOGLE_GENERATIVE_AI_API_KEY="your_gemini_api_key"

Neon database connected and working (test by creating a business in the app)
Chat functionality working (you can ask questions and get AI responses)
Appointment booking working (AI can transition to booking mode)

🔍 Quick Test

1. Go to http://localhost:3000
2. Create a test business
3. Chat with the AI and try booking an appointment
4. If this works, you're ready for Lab 2!

Complete Database Setup

If you haven't set up Neon database yet, follow these steps:

Step 1: Create Neon Database (If Not Done)

Go to neon.tech and sign up for free
Create a new project called "ai-appointment-setter"
Copy your connection string from the dashboard
Add to your .env file:

env

DATABASE_URL="postgresql://username:password@host/database?sslmode=require"

Step 2: Create Required Tables

You need BOTH tables for Lab 2 to work:

Option A: Use the SQL files (Recommended)

bash

# Check if you have the SQL files
ls scripts/
# You should see: create-tables.sql and create-metrics-table.sql

# View the main tables SQL
cat scripts/create-tables.sql

# View the metrics table SQL  
cat scripts/create-metrics-table.sql

Option B: Copy-paste this SQL

Go to your Neon database console and run BOTH scripts:

First, create the main tables (if not done in Lab 1):

sql

-- Main application tables (from Lab 1) - Copy from scripts/create-tables.sql
-- This creates: businesses, documents, conversations, messages, appointments, business_settings
-- Run the entire contents of scripts/create-tables.sql in your Neon console

Then, create the metrics table for Lab 2:

sql

-- Create AI Metrics table for MLOps tracking
CREATE TABLE IF NOT EXISTS ai_metrics (
    id SERIAL PRIMARY KEY,
    business_id VARCHAR(255) NOT NULL,
    conversation_id VARCHAR(255),
    session_id VARCHAR(255) NOT NULL,
    
    -- Performance metrics
    response_time_ms INTEGER NOT NULL,
    success_rate DECIMAL(5,4) NOT NULL,
    
    -- AI performance metrics
    tokens_used INTEGER NOT NULL,
    prompt_tokens INTEGER,
    completion_tokens INTEGER,
    api_cost_usd DECIMAL(10,6) NOT NULL,
    model_name VARCHAR(100) NOT NULL,
    
    -- Business metrics
    intent_detected VARCHAR(50) NOT NULL,
    appointment_requested BOOLEAN DEFAULT FALSE,
    human_handoff_requested BOOLEAN DEFAULT FALSE,
    appointment_booked BOOLEAN DEFAULT FALSE,
    
    -- Message metrics
    user_message_length INTEGER NOT NULL,
    ai_response_length INTEGER NOT NULL,
    response_type VARCHAR(50) NOT NULL,
    
    -- Timestamp
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Create indexes for performance
CREATE INDEX IF NOT EXISTS idx_ai_metrics_business_id ON ai_metrics(business_id);
CREATE INDEX IF NOT EXISTS idx_ai_metrics_created_at ON ai_metrics(created_at);

Step 3: Verify Database Setup

Test your database connection:

sql

-- Check if all tables exist
SELECT table_name FROM information_schema.tables 
WHERE table_schema = 'public' 
ORDER BY table_name;

-- You should see: appointments, ai_metrics, businesses

Test inserting data:

sql

-- Test businesses table
INSERT INTO businesses (name, industry, description) 
VALUES ('Test Business', 'Healthcare', 'Test description');

-- Check it worked
SELECT * FROM businesses WHERE name = 'Test Business';

-- Test ai_metrics table structure
SELECT column_name, data_type FROM information_schema.columns 
WHERE table_name = 'ai_metrics' 
ORDER BY ordinal_position;

✅ Success indicators:

• All 3 tables exist (businesses, appointments, ai_metrics)
• You can insert and select from businesses table
• ai_metrics table has all required columns

❌ Still having issues?

• Check your DATABASE_URL format includes ?sslmode=require
• Ensure you're using the correct database name from Neon
• Try refreshing your Neon console and running SQL again

Step 4: Environment Configuration Check

Ensure your main .env file has:

env

# Required for Lab 2
DATABASE_URL="postgresql://username:password@host/database?sslmode=require"
GOOGLE_GENERATIVE_AI_API_KEY="your_gemini_api_key"

# App Settings
NEXT_PUBLIC_APP_URL=http://localhost:3000
SECRET_KEY=change_this_to_something_secure
DEBUG=true

Test your Next.js app works:

bash

# Start your Next.js app
npm run dev

# Go to http://localhost:3000
# Create a test business
# Try chatting with the AI
# Try booking an appointment

🎯 Pre-Lab 2 Checklist

Before starting Part A, verify you have:

✓ Neon database created and accessible
✓ All tables created (businesses, appointments, ai_metrics, etc.)
✓ DATABASE_URL in your .env file working
✓ Next.js app running at http://localhost:3000
✓ AI chat responding to messages
✓ Appointment booking working in the chat

Quick test command:

bash

# Test your complete database setup
node scripts/test-database-connection.js

This test will verify:

• Database connection works
• All required tables exist
• ai_metrics table has correct structure
• You can insert and read data

All checked? → Start Part A
Issues? → Fix the failing items above first

Part A: Build Flask MLOps Service

You'll create a separate Python service that tracks how well your AI is performing

1. Create Project Structure

Create the MLOps service folder:

Windows:

cmd

# In your project root directory
mkdir mlops-service
cd mlops-service

Mac/Linux:

bash

# In your project root directory
mkdir mlops-service
cd mlops-service

Create the basic file structure:

Windows:

cmd

# Create these files (we'll fill them in step by step)
echo. > app.py
echo. > requirements.txt
echo. > .env
echo. > start.bat

Mac/Linux:

bash

# Create these files (we'll fill them in step by step)
touch app.py
touch requirements.txt
touch .env
touch start.sh

Create the basic file structure:

Windows:

cmd

# Create these files (we'll fill them in step by step)
echo. > app.py
echo. > requirements.txt
echo. > .env
echo. > start.bat

Mac/Linux:

bash

# Create these files (we'll fill them in step by step)
touch app.py
touch requirements.txt
touch .env
touch start.sh

Your folder should now look like:

text

mlops-service/
├── app.py          # Main Flask application
├── requirements.txt # Python dependencies
├── .env            # Environment variables
└── start.sh        # Startup script

2. Python Dependencies

Create requirements.txt:

txt

# Flask MLOps Service Dependencies with Prometheus
# Lab 2: AI Lifecycle & MLOps Integration

# Core Flask dependencies
Flask==3.0.0
Flask-CORS==4.0.0

# Prometheus monitoring
prometheus-client==0.19.0

# Database connectivity - using requests for HTTP-based database access
# This avoids psycopg2 compilation issues on newer Python versions
requests==2.31.0

# Additional utilities
python-dotenv==1.0.0

# Development and testing (optional)
pytest==7.4.3
pytest-flask==1.3.0

Create Python virtual environment:

Windows:

cmd

# Create virtual environment
python -m venv venv

# Activate it
venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Mac/Linux:

bash

# Create virtual environment
python3 -m venv venv

# Activate it
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

✅ Success Check: Run pip list - you should see Flask, prometheus-client, and other packages installed.

💡 What You're Building

This Flask service will:

• Receive metrics from your Next.js chat application
• Track AI performance with Prometheus (industry standard)
• Provide real-time monitoring dashboard
• Store metrics for historical analysis

🪟 Windows Users - Important Notes

• Use Command Prompt or PowerShell (not Git Bash for Python commands)
• Python command is usually python (not python3)
• Virtual environment activation: venv\\Scripts\\activate
• Use start.bat instead of start.sh

3. Environment Variables

Create .env file in mlops-service directory:

env

# Database Configuration (same as your Next.js app)
DATABASE_URL=your_neon_database_url_here

# Flask Configuration
FLASK_ENV=development
FLASK_DEBUG=True

# Service Configuration
ENVIRONMENT=development
SERVICE_PORT=5001
PROMETHEUS_PORT=8001

🔑 Important Steps:

1. Copy your DATABASE_URL from your main project .env file
2. Use the EXACT same connection string - this connects to the same Neon database
3. Keep the quotes around the DATABASE_URL value

4. Flask Application

Create app.py with the basic Flask setup:

python

"""
Flask MLOps Service for AI Appointment Setter
This service tracks AI performance metrics and stores them for analysis
"""

from flask import Flask, request, jsonify
from flask_cors import CORS
from prometheus_client import Counter, Histogram, Gauge, generate_latest, CONTENT_TYPE_LATEST
import os
import json
import time
from datetime import datetime
import logging

# Configure logging so we can see what's happening
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

# Initialize Flask app
app = Flask(__name__)
CORS(app)  # Allow requests from Next.js app

# Database connection
DATABASE_URL = os.getenv('DATABASE_URL')

@app.route('/health', methods=['GET'])
def health_check():
    """Check if our service is running properly"""
    return jsonify({
        'status': 'healthy',
        'service': 'mlops-service',
        'timestamp': datetime.utcnow().isoformat(),
        'monitoring': 'prometheus'
    })

if __name__ == '__main__':
    # Run the Flask app
    app.run(host='0.0.0.0', port=5001, debug=True)

5. Startup Script

Create start.sh (Mac/Linux):

bash

#!/bin/bash

echo "🚀 Starting MLOps Service for AI Appointment Setter"
echo "=================================================="

# Check if Python is installed
if ! command -v python3 &> /dev/null; then
    echo "❌ Python 3 is not installed. Please install Python 3.8 or higher."
    exit 1
fi

# Create virtual environment if it doesn't exist
if [ ! -d "venv" ]; then
    echo "📦 Creating Python virtual environment..."
    python3 -m venv venv
fi

# Activate virtual environment
echo "🔧 Activating virtual environment..."
source venv/bin/activate

# Install dependencies
echo "📚 Installing Python dependencies..."
pip install -r requirements.txt

echo "✅ Setup complete!"
echo "🌐 Service will be available at: http://localhost:5001"
echo "Starting Flask application..."

# Start Flask application
python app.py

Make it executable:

bash

chmod +x start.sh

6. Test Flask Service

Start the service:

Mac/Linux:

bash

./start.sh

Windows:

cmd

start.bat

Test the health endpoint:

bash

# In a new terminal
curl http://localhost:5001/health

You should see:

json

{
  "status": "healthy",
  "service": "mlops-service",
  "timestamp": "2024-01-15T10:30:00.000000",
  "monitoring": "prometheus"
}

✅ Success Check: If you see the healthy response, your Flask service is working!

Part B: Prometheus Integration

Prometheus helps us track real-time metrics and see how our AI is performing over time

1. Prometheus Setup

Add Prometheus metrics to your app.py:

python

# Prometheus Metrics Setup
from prometheus_client import Counter, Histogram, Gauge, generate_latest, CONTENT_TYPE_LATEST

# Define Prometheus metrics
ai_requests_total = Counter('ai_requests_total', 'Total AI requests', ['business_id', 'intent', 'response_type'])
ai_response_time_seconds = Histogram('ai_response_time_seconds', 'AI response time in seconds', ['business_id'])
ai_tokens_used_total = Counter('ai_tokens_used_total', 'Total tokens used', ['business_id', 'model'])
ai_api_cost_usd_total = Counter('ai_api_cost_usd_total', 'Total API cost in USD', ['business_id'])
appointments_requested_total = Counter('appointments_requested_total', 'Total appointment requests', ['business_id'])

logger.info("Prometheus metrics initialized successfully")

2. Metrics Endpoint

Add this endpoint to your app.py:

python

@app.route('/metrics', methods=['GET'])
def prometheus_metrics():
    """Prometheus metrics endpoint - industry standard format"""
    try:
        return generate_latest(), 200, {'Content-Type': CONTENT_TYPE_LATEST}
    except Exception as e:
        logger.error(f"Error generating Prometheus metrics: {e}")
        return jsonify({'error': 'Failed to generate metrics'}), 500

3. Prometheus Logging

Add metrics tracking function:

python

def update_prometheus_metrics(metrics_data):
    """Update Prometheus metrics with new data"""
    try:
        business_id = metrics_data.get('business_id', 'unknown')

        # Update request counter
        ai_requests_total.labels(
            business_id=business_id,
            intent=metrics_data.get('intent_detected', 'unknown'),
            response_type=metrics_data.get('response_type', 'unknown')
        ).inc()

        # Update response time histogram
        if 'response_time_ms' in metrics_data:
            response_time_seconds = metrics_data['response_time_ms'] / 1000.0
            ai_response_time_seconds.labels(business_id=business_id).observe(response_time_seconds)

        # Update token usage
        if 'tokens_used' in metrics_data:
            ai_tokens_used_total.labels(
                business_id=business_id,
                model=metrics_data.get('model_name', 'gemini-1.5-flash')
            ).inc(metrics_data['tokens_used'])

        logger.info(f"Successfully updated Prometheus metrics for business {business_id}")
        return True

    except Exception as e:
        logger.error(f"Error updating Prometheus metrics: {e}")
        return False

4. Update Tracking

Add the main tracking endpoint:

python

@app.route('/track', methods=['POST'])
def track_metrics():
    """Receive metrics from the Next.js chat application"""
    try:
        metrics_data = request.get_json()

        if not metrics_data:
            return jsonify({'error': 'No metrics data provided'}), 400

        # Validate required fields
        required_fields = ['business_id', 'response_time_ms', 'tokens_used']
        for field in required_fields:
            if field not in metrics_data:
                return jsonify({'error': f'Missing required field: {field}'}), 400

        # Update Prometheus metrics
        prometheus_success = update_prometheus_metrics(metrics_data)

        if prometheus_success:
            logger.info(f"Successfully tracked metrics for business {metrics_data.get('business_id')}")
            return jsonify({
                'status': 'success',
                'message': 'Metrics tracked successfully',
                'timestamp': datetime.utcnow().isoformat()
            })
        else:
            return jsonify({'error': 'Failed to store metrics'}), 500

    except Exception as e:
        logger.error(f"Error tracking metrics: {e}")
        return jsonify({'error': 'Internal server error'}), 500

5. Test Prometheus

Send test metrics:

bash

curl -X POST http://localhost:5001/track \
  -H "Content-Type: application/json" \
  -d '{
    "business_id": "test-business",
    "response_time_ms": 1500,
    "tokens_used": 150,
    "intent_detected": "general",
    "response_type": "text"
  }'

Check Prometheus metrics:

bash

curl http://localhost:5001/metrics

💡 What is Prometheus?

Prometheus is an industry-standard monitoring system that tracks:

• How fast your AI responds (real-time metrics)
• How much each conversation costs
• Which conversations lead to appointments
• Performance trends over time
• System health and reliability

Part C: Next.js Integration

Now we'll modify your Next.js app to send performance data to your Flask service

Add MLOps Service URL to your main .env file:

env

# Add this line to your existing .env file
MLOPS_SERVICE_URL=http://localhost:5001

The metrics tracking is already implemented in your codebase! Check these files:

lib/mlops-tracking.ts - Contains all the metrics tracking functions
app/api/chat/route.ts - Already calls trackMetrics() after each AI response
lib/database.ts - Contains createAIMetrics() function for database storage

✅ Integration Complete!

Your Next.js app automatically tracks:

• Response times for each AI interaction
• Token usage and estimated API costs
• Intent detection and appointment requests
• Success rates and error tracking

Troubleshooting

Flask service won't start:

Check that you're in the mlops-service directory and have activated the virtual environment

Metrics not appearing:

Ensure both Next.js (port 3000) and Flask (port 5001) services are running

Database connection errors:

Verify your DATABASE_URL is correctly copied from the main .env file

Lab 2 Summary - What You Built

Congratulations! You've successfully implemented a complete MLOps monitoring system for your AI appointment setter. Here's what you accomplished:

✅ Core MLOps Infrastructure

Flask MLOps Service running on port 5001 with cross-platform compatibility
Prometheus Metrics Collection tracking real-time AI performance
Database Integration for historical metrics storage
Next.js Integration with automatic metrics collection

📊 Metrics You're Now Tracking

AI Performance: Response times, token usage, API costs
Business Metrics: Appointment requests, conversion rates, human handoffs
System Health: Service status, error rates, success rates

Note: This implementation uses HTTP-based database access to avoid psycopg2 compilation issues. Your metrics collection provides valuable insights for production AI systems.

🎯 Key URLs to Remember

Health Check: http://localhost:5001/health
Raw Metrics: http://localhost:5001/metrics (Prometheus format)

← Lab 1: Foundation

Back to Labs →