Remove legacy backend files and update microservices config

- Delete ARCHITECTURE.md and old services directory - Add sensor-service
and data-ingestion-service to Docker Compose - Comment out unused
services in docker-compose.yml - Update deploy.sh to use `docker
compose` command - Extend API gateway to proxy sensor-service routes and
WebSocket - Refactor health checks and service dependencies

ARCHITECTURE.md

@@ -1,139 +0,0 @@
-# Backend Architecture Restructuring
-
-## Overview
-
-The backend has been restructured from a monolithic approach to a clean **3-layer architecture** with proper separation of concerns.
-
-## Architecture Layers
-
-### 1. Infrastructure Layer (`layers/infrastructure/`)
-**Responsibility**: Data access, external services, and low-level operations
-
-- **`database_connection.py`** - MongoDB connection management and indexing
-- **`redis_connection.py`** - Redis connection and basic operations  
-- **`repositories.py`** - Data access layer with repository pattern
-
-**Key Principles**:
-- No business logic
-- Only handles data persistence and retrieval
-- Provides abstractions for external services
-
-### 2. Business Layer (`layers/business/`)
-**Responsibility**: Business logic, data processing, and core application rules
-
-- **`sensor_service.py`** - Sensor data processing and validation
-- **`room_service.py`** - Room metrics calculation and aggregation
-- **`analytics_service.py`** - Analytics calculations and reporting
-- **`cleanup_service.py`** - Data retention and maintenance
-
-**Key Principles**:
-- Contains all business rules and validation
-- Independent of presentation concerns
-- Uses infrastructure layer for data access
-
-### 3. Presentation Layer (`layers/presentation/`)
-**Responsibility**: HTTP endpoints, WebSocket handling, and user interface
-
-- **`api_routes.py`** - REST API endpoints and request/response handling
-- **`websocket_handler.py`** - WebSocket connection management
-- **`redis_subscriber.py`** - Real-time data broadcasting
-
-**Key Principles**:
-- Handles HTTP requests and responses
-- Manages real-time communications
-- Delegates business logic to business layer
-
-## File Comparison
-
-### Before (Monolithic)
-```
-main.py (203 lines)           # Mixed concerns
-api.py (506 lines)           # API + some business logic  
-database.py (220 lines)      # DB + Redis + cleanup
-persistence.py (448 lines)   # Business + data access
-models.py (236 lines)        # Data models
-```
-
-### After (Layered)
-```
-Infrastructure Layer:
-├── database_connection.py (114 lines)    # Pure DB connection
-├── redis_connection.py (89 lines)       # Pure Redis connection  
-└── repositories.py (376 lines)          # Clean data access
-
-Business Layer:
-├── sensor_service.py (380 lines)        # Sensor business logic
-├── room_service.py (242 lines)         # Room business logic
-├── analytics_service.py (333 lines)     # Analytics business logic
-└── cleanup_service.py (278 lines)       # Cleanup business logic
-
-Presentation Layer:
-├── api_routes.py (430 lines)           # Pure API endpoints
-├── websocket_handler.py (103 lines)    # WebSocket management
-└── redis_subscriber.py (148 lines)     # Real-time broadcasting
-
-Core:
-├── main_layered.py (272 lines)         # Clean application entry
-└── models.py (236 lines)               # Unchanged data models
-```
-
-## Key Improvements
-
-### 1. **Separation of Concerns**
-- Each layer has a single, well-defined responsibility
-- Infrastructure concerns isolated from business logic
-- Business logic separated from presentation
-
-### 2. **Testability**
-- Each layer can be tested independently
-- Business logic testable without database dependencies
-- Infrastructure layer testable without business complexity
-
-### 3. **Maintainability**
-- Changes in one layer don't affect others
-- Clear boundaries make code easier to understand
-- Reduced coupling between components
-
-### 4. **Scalability**
-- Layers can be scaled independently
-- Easy to replace implementations within layers
-- Clear extension points for new features
-
-### 5. **Dependency Management**
-- Clear dependency flow: Presentation → Business → Infrastructure
-- No circular dependencies
-- Infrastructure layer has no knowledge of business rules
-
-## Usage
-
-### Running the Layered Application
-```bash
-# Use the new layered main file
-conda activate dashboard
-uvicorn main_layered:app --reload
-```
-
-### Testing the Structure
-```bash
-# Validate the architecture
-python test_structure.py
-```
-
-## Benefits Achieved
-
-✅ **Clear separation of concerns**  
-✅ **Infrastructure isolated from business logic**  
-✅ **Business logic separated from presentation**  
-✅ **Easy to test individual layers**  
-✅ **Maintainable and scalable structure**  
-✅ **No layering violations detected**  
-✅ **2,290+ lines properly organized across 10+ files**  
-
-## Migration Path
-
-The original files are preserved, so you can:
-1. Test the new layered architecture with `main_layered.py`
-2. Gradually migrate consumers to use the new structure
-3. Remove old files once confident in the new architecture
-
-Both architectures can coexist during the transition period.

microservices/api-gateway/main.py

@@ -7,7 +7,7 @@ Port: 8000
 import asyncio
 import aiohttp
 from datetime import datetime
-from fastapi import FastAPI, HTTPException, Depends, Request, Response
+from fastapi import FastAPI, HTTPException, WebSocket, Depends, Request, Response
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.responses import JSONResponse
 from contextlib import asynccontextmanager
@@ -101,6 +101,12 @@ SERVICES = {
         base_url="http://localhost:8006",
         health_endpoint="/health",
         auth_required=True
+    ),
+    "sensor-service": ServiceConfig(
+        name="sensor-service",
+        base_url="http://localhost:8007",
+        health_endpoint="/health",
+        auth_required=True
     )
 }
 
@@ -204,6 +210,70 @@ async def iot_control_service_proxy(request: Request, path: str):
     """Proxy requests to IoT control service"""
     return await proxy_request(request, "iot-control-service", f"/{path}")
 
+# Sensor Service Routes (Original Dashboard Functionality)
+@app.api_route("/api/v1/sensors/{path:path}", methods=["GET", "POST", "PUT", "DELETE"])
+async def sensor_service_proxy(request: Request, path: str):
+    """Proxy requests to sensor service"""
+    return await proxy_request(request, "sensor-service", f"/{path}")
+
+@app.api_route("/api/v1/rooms/{path:path}", methods=["GET", "POST", "PUT", "DELETE"])
+async def room_service_proxy(request: Request, path: str):
+    """Proxy requests to sensor service for room management"""
+    return await proxy_request(request, "sensor-service", f"/rooms/{path}")
+
+@app.api_route("/api/v1/data/{path:path}", methods=["GET", "POST"])
+async def data_service_proxy(request: Request, path: str):
+    """Proxy requests to sensor service for data operations"""
+    return await proxy_request(request, "sensor-service", f"/data/{path}")
+
+@app.api_route("/api/v1/analytics/{path:path}", methods=["GET", "POST"])
+async def analytics_service_proxy(request: Request, path: str):
+    """Proxy requests to sensor service for analytics"""
+    return await proxy_request(request, "sensor-service", f"/analytics/{path}")
+
+@app.api_route("/api/v1/export", methods=["GET"])
+async def export_service_proxy(request: Request):
+    """Proxy requests to sensor service for data export"""
+    return await proxy_request(request, "sensor-service", "/export")
+
+@app.api_route("/api/v1/events", methods=["GET"])
+async def events_service_proxy(request: Request):
+    """Proxy requests to sensor service for system events"""
+    return await proxy_request(request, "sensor-service", "/events")
+
+# WebSocket proxy for real-time data
+@app.websocket("/ws")
+async def websocket_proxy(websocket: WebSocket):
+    """Proxy WebSocket connections to sensor service"""
+    try:
+        # Get sensor service URL
+        service_url = await load_balancer.get_service_url("sensor-service")
+        if not service_url:
+            await websocket.close(code=1003, reason="Sensor service unavailable")
+            return
+
+        # For simplicity, we'll just accept the connection and forward to sensor service
+        # In a production setup, you'd want a proper WebSocket proxy
+        await websocket.accept()
+
+        # For now, we'll handle this by having the sensor service manage WebSockets directly
+        # The frontend should connect to the sensor service WebSocket endpoint directly
+        await websocket.send_text(json.dumps({
+            "type": "proxy_info",
+            "message": "Connect directly to sensor service WebSocket at /ws",
+            "sensor_service_url": service_url.replace("http://", "ws://") + "/ws"
+        }))
+
+        # Keep connection alive
+        while True:
+            try:
+                await websocket.receive_text()
+            except:
+                break
+
+    except Exception as e:
+        logger.error(f"WebSocket proxy error: {e}")
+
 async def proxy_request(request: Request, service_name: str, path: str):
     """Generic request proxy function"""
     try:
@@ -299,6 +369,7 @@ async def get_system_overview():
                     async with aiohttp.ClientSession() as session:
                         # Try to get service-specific overview data
                         overview_endpoints = {
+                            "sensor-service": "/analytics/summary",
                             "battery-service": "/batteries",
                             "demand-response-service": "/flexibility/current",
                             "p2p-trading-service": "/market/status",

microservices/deploy.sh

@@ -42,7 +42,7 @@ check_dependencies() {
         exit 1
     fi
 
-    if ! command -v docker-compose &> /dev/null; then
+    if ! command -v docker compose &> /dev/null; then
         print_error "Docker Compose is not installed. Please install Docker Compose first."
         exit 1
     fi
@@ -132,7 +132,7 @@ EOF
 build_services() {
     print_status "Building all microservices..."
 
-    docker-compose -f $COMPOSE_FILE build
+    docker compose -f $COMPOSE_FILE build
 
     if [ $? -eq 0 ]; then
         print_success "All services built successfully"
@@ -146,7 +146,7 @@ build_services() {
 start_services() {
     print_status "Starting all services..."
 
-    docker-compose -f $COMPOSE_FILE up -d
+    docker compose -f $COMPOSE_FILE up -d
 
     if [ $? -eq 0 ]; then
         print_success "All services started successfully"
@@ -160,7 +160,7 @@ start_services() {
 stop_services() {
     print_status "Stopping all services..."
 
-    docker-compose -f $COMPOSE_FILE down
+    docker compose -f $COMPOSE_FILE down
 
     print_success "All services stopped"
 }
@@ -174,15 +174,15 @@ restart_services() {
 # Function to show service status
 show_status() {
     print_status "Service status:"
-    docker-compose -f $COMPOSE_FILE ps
+    docker compose -f $COMPOSE_FILE ps
 
     print_status "Service health checks:"
 
     # Wait a moment for services to start
     sleep 5
 
-    services=("api-gateway:8000" "token-service:8001" "battery-service:8002" "demand-response-service:8003")
+    # services=("api-gateway:8000" "token-service:8001" "battery-service:8002" "demand-response-service:8003")
-    
+    services=("api-gateway:8000" "token-service:8001")
     for service in "${services[@]}"; do
         name="${service%:*}"
         port="${service#*:}"
@@ -199,10 +199,10 @@ show_status() {
 view_logs() {
     if [ -z "$2" ]; then
         print_status "Showing logs for all services..."
-        docker-compose -f $COMPOSE_FILE logs -f
+        docker compose -f $COMPOSE_FILE logs -f
     else
         print_status "Showing logs for $2..."
-        docker-compose -f $COMPOSE_FILE logs -f $2
+        docker compose -f $COMPOSE_FILE logs -f $2
     fi
 }
 
@@ -212,7 +212,7 @@ cleanup() {
     read -r response
     if [[ "$response" =~ ^([yY][eE][sS]|[yY])$ ]]; then
         print_status "Cleaning up everything..."
-        docker-compose -f $COMPOSE_FILE down -v --rmi all
+        docker compose -f $COMPOSE_FILE down -v --rmi all
         docker system prune -f
         print_success "Cleanup completed"
     else

microservices/docker-compose.yml

@@ -1,4 +1,4 @@
-version: '3.8'
+version: "3.8"
 
 services:
   # Database Services
@@ -41,17 +41,19 @@ services:
       - MONGO_URL=mongodb://admin:password123@mongodb:27017/energy_dashboard?authSource=admin
       - REDIS_URL=redis://redis:6379
       - TOKEN_SERVICE_URL=http://token-service:8001
+      - SENSOR_SERVICE_URL=http://sensor-service:8007
       - BATTERY_SERVICE_URL=http://battery-service:8002
       - DEMAND_RESPONSE_SERVICE_URL=http://demand-response-service:8003
       - P2P_TRADING_SERVICE_URL=http://p2p-trading-service:8004
       - FORECASTING_SERVICE_URL=http://forecasting-service:8005
       - IOT_CONTROL_SERVICE_URL=http://iot-control-service:8006
+      - DATA_INGESTION_SERVICE_URL=http://data-ingestion-service:8008
     depends_on:
       - mongodb
       - redis
       - token-service
-      - battery-service
+      # - battery-service
-      - demand-response-service
+      # - demand-response-service
     networks:
       - energy-network
 
@@ -73,92 +75,134 @@ services:
       - energy-network
 
   # Battery Management Service
-  battery-service:
+  # battery-service:
-    build:
+  #   build:
-      context: ./battery-service
+  #     context: ./battery-service
-      dockerfile: Dockerfile
+  #     dockerfile: Dockerfile
-    container_name: energy-battery-service
+  #   container_name: energy-battery-service
-    restart: unless-stopped
+  #   restart: unless-stopped
-    ports:
+  #   ports:
-      - "8002:8002"
+  #     - "8002:8002"
-    environment:
+  #   environment:
-      - MONGO_URL=mongodb://admin:password123@mongodb:27017/energy_dashboard_batteries?authSource=admin
+  #     - MONGO_URL=mongodb://admin:password123@mongodb:27017/energy_dashboard_batteries?authSource=admin
-      - REDIS_URL=redis://redis:6379
+  #     - REDIS_URL=redis://redis:6379
-    depends_on:
+  #   depends_on:
-      - mongodb
+  #     - mongodb
-      - redis
+  #     - redis
-    networks:
+  #   networks:
-      - energy-network
+  #     - energy-network
 
   # Demand Response Service
-  demand-response-service:
+  # demand-response-service:
-    build:
+  #   build:
-      context: ./demand-response-service
+  #     context: ./demand-response-service
-      dockerfile: Dockerfile
+  #     dockerfile: Dockerfile
-    container_name: energy-demand-response-service
+  #   container_name: energy-demand-response-service
-    restart: unless-stopped
+  #   restart: unless-stopped
-    ports:
+  #   ports:
-      - "8003:8003"
+  #     - "8003:8003"
-    environment:
+  #   environment:
-      - MONGO_URL=mongodb://admin:password123@mongodb:27017/energy_dashboard_demand_response?authSource=admin
+  #     - MONGO_URL=mongodb://admin:password123@mongodb:27017/energy_dashboard_demand_response?authSource=admin
-      - REDIS_URL=redis://redis:6379
+  #     - REDIS_URL=redis://redis:6379
-      - IOT_CONTROL_SERVICE_URL=http://iot-control-service:8006
+  #     - IOT_CONTROL_SERVICE_URL=http://iot-control-service:8006
-    depends_on:
+  #   depends_on:
-      - mongodb
+  #     - mongodb
-      - redis
+  #     - redis
-    networks:
+  #   networks:
-      - energy-network
+  #     - energy-network
 
   # P2P Trading Service
-  p2p-trading-service:
+  # p2p-trading-service:
-    build:
+  #   build:
-      context: ./p2p-trading-service
+  #     context: ./p2p-trading-service
-      dockerfile: Dockerfile
+  #     dockerfile: Dockerfile
-    container_name: energy-p2p-trading-service
+  #   container_name: energy-p2p-trading-service
-    restart: unless-stopped
+  #   restart: unless-stopped
-    ports:
+  #   ports:
-      - "8004:8004"
+  #     - "8004:8004"
-    environment:
+  #   environment:
-      - MONGO_URL=mongodb://admin:password123@mongodb:27017/energy_dashboard_p2p?authSource=admin
+  #     - MONGO_URL=mongodb://admin:password123@mongodb:27017/energy_dashboard_p2p?authSource=admin
-      - REDIS_URL=redis://redis:6379
+  #     - REDIS_URL=redis://redis:6379
-    depends_on:
+  #   depends_on:
-      - mongodb
+  #     - mongodb
-      - redis
+  #     - redis
-    networks:
+  #   networks:
-      - energy-network
+  #     - energy-network
 
   # Forecasting Service
-  forecasting-service:
+  # forecasting-service:
+  #   build:
+  #     context: ./forecasting-service
+  #     dockerfile: Dockerfile
+  #   container_name: energy-forecasting-service
+  #   restart: unless-stopped
+  #   ports:
+  #     - "8005:8005"
+  #   environment:
+  #     - MONGO_URL=mongodb://admin:password123@mongodb:27017/energy_dashboard_forecasting?authSource=admin
+  #     - REDIS_URL=redis://redis:6379
+  #   depends_on:
+  #     - mongodb
+  #     - redis
+  #   networks:
+  #     - energy-network
+
+  # IoT Control Service
+  # iot-control-service:
+  #   build:
+  #     context: ./iot-control-service
+  #     dockerfile: Dockerfile
+  #   container_name: energy-iot-control-service
+  #   restart: unless-stopped
+  #   ports:
+  #     - "8006:8006"
+  #   environment:
+  #     - MONGO_URL=mongodb://admin:password123@mongodb:27017/energy_dashboard_iot?authSource=admin
+  #     - REDIS_URL=redis://redis:6379
+  #     - BATTERY_SERVICE_URL=http://battery-service:8002
+  #     - DEMAND_RESPONSE_SERVICE_URL=http://demand-response-service:8003
+  #   depends_on:
+  #     - mongodb
+  #     - redis
+  #   networks:
+  #     - energy-network
+
+  # Data Ingestion Service (FTP Monitoring & SA4CPS Integration)
+  data-ingestion-service:
     build:
-      context: ./forecasting-service
+      context: ./data-ingestion-service
       dockerfile: Dockerfile
-    container_name: energy-forecasting-service
+    container_name: energy-data-ingestion-service
     restart: unless-stopped
     ports:
-      - "8005:8005"
+      - "8008:8008"
     environment:
-      - MONGO_URL=mongodb://admin:password123@mongodb:27017/energy_dashboard_forecasting?authSource=admin
+      - MONGO_URL=mongodb://admin:password123@mongodb:27017/energy_dashboard_ingestion?authSource=admin
       - REDIS_URL=redis://redis:6379
+      - FTP_SA4CPS_HOST=ftp.sa4cps.pt
+      - FTP_SA4CPS_PORT=21
+      - FTP_SA4CPS_USERNAME=anonymous
+      - FTP_SA4CPS_PASSWORD=
+      - FTP_SA4CPS_REMOTE_PATH=/
     depends_on:
       - mongodb
       - redis
     networks:
       - energy-network
 
-  # IoT Control Service
+  # Sensor Management Service (Original Dashboard Functionality)
-  iot-control-service:
+  sensor-service:
     build:
-      context: ./iot-control-service
+      context: ./sensor-service
       dockerfile: Dockerfile
-    container_name: energy-iot-control-service
+    container_name: energy-sensor-service
     restart: unless-stopped
     ports:
-      - "8006:8006"
+      - "8007:8007"
     environment:
-      - MONGO_URL=mongodb://admin:password123@mongodb:27017/energy_dashboard_iot?authSource=admin
+      - MONGO_URL=mongodb://admin:password123@mongodb:27017/energy_dashboard_sensors?authSource=admin
       - REDIS_URL=redis://redis:6379
-      - BATTERY_SERVICE_URL=http://battery-service:8002
+      - TOKEN_SERVICE_URL=http://token-service:8001
-      - DEMAND_RESPONSE_SERVICE_URL=http://demand-response-service:8003
     depends_on:
       - mongodb
       - redis

services/__init__.py

@@ -1 +0,0 @@
-# Services package for dashboard backend

services/token_service.py

@@ -1,174 +0,0 @@
-"""
-Token management service for authentication and resource access control.
-Based on the tiocps JWT token implementation with resource-based permissions.
-"""
-
-import jwt
-import uuid
-from datetime import datetime, timedelta
-from typing import Dict, List, Optional, Any
-from pydantic import BaseModel
-from motor.motor_asyncio import AsyncIOMotorDatabase
-
-class TokenPayload(BaseModel):
-    """Token payload structure"""
-    name: str
-    list_of_resources: List[str]
-    data_aggregation: bool = False
-    time_aggregation: bool = False
-    embargo: int = 0  # embargo period in seconds
-    exp: int  # expiration timestamp
-
-class TokenRecord(BaseModel):
-    """Token database record"""
-    token: str
-    datetime: datetime
-    active: bool = True
-    created_at: datetime
-    updated_at: datetime
-
-class TokenService:
-    """Service for managing JWT tokens and authentication"""
-    
-    def __init__(self, db: AsyncIOMotorDatabase, secret_key: str = "dashboard-secret-key"):
-        self.db = db
-        self.secret_key = secret_key
-        self.tokens_collection = db.tokens
-    
-    def generate_token(self, name: str, list_of_resources: List[str], 
-                      data_aggregation: bool = False, time_aggregation: bool = False,
-                      embargo: int = 0, exp_hours: int = 24) -> str:
-        """Generate a new JWT token with specified permissions"""
-        
-        # Calculate expiration time
-        exp_timestamp = int((datetime.utcnow() + timedelta(hours=exp_hours)).timestamp())
-        
-        # Create token payload
-        payload = {
-            "name": name,
-            "list_of_resources": list_of_resources,
-            "data_aggregation": data_aggregation,
-            "time_aggregation": time_aggregation,
-            "embargo": embargo,
-            "exp": exp_timestamp,
-            "iat": int(datetime.utcnow().timestamp()),
-            "jti": str(uuid.uuid4())  # unique token ID
-        }
-        
-        # Generate JWT token
-        token = jwt.encode(payload, self.secret_key, algorithm="HS256")
-        return token
-    
-    def decode_token(self, token: str) -> Optional[Dict[str, Any]]:
-        """Decode and verify JWT token"""
-        try:
-            payload = jwt.decode(token, self.secret_key, algorithms=["HS256"])
-            return payload
-        except jwt.ExpiredSignatureError:
-            return {"error": "Token has expired"}
-        except jwt.InvalidTokenError:
-            return {"error": "Invalid token"}
-    
-    async def insert_token(self, token: str) -> Dict[str, Any]:
-        """Save token to database"""
-        now = datetime.utcnow()
-        
-        # Decode token to verify it's valid
-        decoded = self.decode_token(token)
-        if decoded and "error" not in decoded:
-            token_record = {
-                "token": token,
-                "datetime": now,
-                "active": True,
-                "created_at": now,
-                "updated_at": now,
-                "name": decoded.get("name", ""),
-                "resources": decoded.get("list_of_resources", []),
-                "expires_at": datetime.fromtimestamp(decoded.get("exp", 0))
-            }
-            
-            await self.tokens_collection.insert_one(token_record)
-            return {
-                "token": token,
-                "datetime": now.isoformat(),
-                "active": True
-            }
-        else:
-            raise ValueError("Invalid token cannot be saved")
-    
-    async def revoke_token(self, token: str) -> Dict[str, Any]:
-        """Revoke a token by marking it as inactive"""
-        now = datetime.utcnow()
-        
-        result = await self.tokens_collection.update_one(
-            {"token": token},
-            {
-                "$set": {
-                    "active": False,
-                    "updated_at": now,
-                    "revoked_at": now
-                }
-            }
-        )
-        
-        if result.matched_count > 0:
-            return {
-                "token": token,
-                "datetime": now.isoformat(),
-                "active": False
-            }
-        else:
-            raise ValueError("Token not found")
-    
-    async def get_tokens(self) -> List[Dict[str, Any]]:
-        """Get all tokens from database"""
-        cursor = self.tokens_collection.find({})
-        tokens = []
-        
-        async for token_record in cursor:
-            # Convert ObjectId to string and datetime to ISO format
-            token_record["_id"] = str(token_record["_id"])
-            for field in ["datetime", "created_at", "updated_at", "expires_at", "revoked_at"]:
-                if field in token_record and token_record[field]:
-                    token_record[field] = token_record[field].isoformat()
-            
-            tokens.append(token_record)
-        
-        return tokens
-    
-    async def is_token_valid(self, token: str) -> bool:
-        """Check if token is valid and active"""
-        # Check if token exists and is active in database
-        token_record = await self.tokens_collection.find_one({
-            "token": token,
-            "active": True
-        })
-        
-        if not token_record:
-            return False
-        
-        # Verify JWT signature and expiration
-        decoded = self.decode_token(token)
-        return decoded is not None and "error" not in decoded
-    
-    async def get_token_permissions(self, token: str) -> Optional[Dict[str, Any]]:
-        """Get permissions for a valid token"""
-        if await self.is_token_valid(token):
-            return self.decode_token(token)
-        return None
-    
-    async def cleanup_expired_tokens(self):
-        """Remove expired tokens from database"""
-        now = datetime.utcnow()
-        
-        # Find tokens that have expired
-        expired_cursor = self.tokens_collection.find({
-            "expires_at": {"$lt": now}
-        })
-        
-        expired_count = 0
-        async for token_record in expired_cursor:
-            await self.tokens_collection.delete_one({"_id": token_record["_id"]})
-            expired_count += 1
-        
-        return expired_count