# Lesson 1: The AI Engineering Landscape - From Theory to Production Systems ##

Detailed Description

The field of Artificial Intelligence has undergone a dramatic transformation over the past decade, evolving from academic research laboratories into production systems that power billions of daily interactions. This lesson establishes the foundation for understanding the modern AI engineering landscape, distinguishing between related but distinct disciplines: data science, machine learning engineering, and AI automation. While these fields overlap significantly, understanding their unique characteristics, responsibilities, and workflows is critical for anyone seeking to build production-ready AI systems.

The importance of this knowledge cannot be overstated. According to recent industry surveys, over 87% of data science projects never make it to production, often due to fundamental misunderstandings about the engineering requirements, stakeholder expectations, and operational complexities of AI systems. Data scientists may build brilliant models in notebooks, but without understanding the complete lifecycle—from data ingestion and model training to deployment, monitoring, and maintenance—these innovations remain isolated experiments. This lesson bridges that gap, providing a comprehensive view of how AI systems are conceptualized, developed, deployed, and maintained in real-world environments.

At the core of this lesson are several key concepts that will recur throughout your AI engineering journey. First, the ML lifecycle encompasses everything from problem formulation and data collection through model training, evaluation, deployment, and continuous monitoring. Unlike traditional software development, ML systems require ongoing attention to model drift, data quality, and performance degradation. Second, understanding stakeholder roles—including product managers, data engineers, ML engineers, DevOps specialists, and business analysts—helps you navigate organizational structures and communicate effectively across teams. Third, recognizing the integration points between AI and traditional software development workflows enables you to build systems that are maintainable, scalable, and aligned with established engineering practices.

Real-world applications of these concepts span virtually every industry. In e-commerce, recommendation systems must handle billions of user interactions daily while continuously learning from new behavioral patterns. In healthcare, diagnostic AI systems require not only high accuracy but also explainability, regulatory compliance, and seamless integration with existing electronic health record systems. Financial institutions deploy fraud detection models that must operate in real-time, adapt to evolving fraud patterns, and maintain strict audit trails. Each of these scenarios demands more than just algorithmic expertise—they require a comprehensive understanding of the engineering landscape.

This lesson also examines common failure modes through real-world case studies. You'll learn why projects fail due to poorly defined success metrics, inadequate data infrastructure, insufficient consideration of edge cases, or lack of monitoring capabilities. By studying both successful implementations and cautionary tales, you'll develop the judgment necessary to anticipate challenges and design robust solutions. We'll explore cases ranging from tech giants' AI platforms to startup deployments, identifying patterns and principles that transcend specific technologies or domains.

By the end of this lesson, you will master several critical competencies. You'll be able to articulate the differences between data science experimentation and production ML engineering, understand the complete end-to-end lifecycle of ML systems, identify key stakeholders and their responsibilities, set up professional Python development environments with proper dependency management, and critically evaluate AI project proposals for feasibility and risk. These foundational skills prepare you not just to build models, but to build systems that deliver sustainable business value in production environments.

Key Concepts and Terminology

Understanding the AI Engineering Roles

Before diving into practical implementation, it's essential to understand the distinct roles within AI engineering teams and how they collaborate:

Data Scientists: Focus on exploratory analysis, hypothesis testing, and model prototyping. They typically work in notebooks, experiment with algorithms, and communicate insights to stakeholders.
ML Engineers: Transform prototypes into production systems. They focus on scalability, reliability, model serving infrastructure, and integration with existing software systems.
Data Engineers: Build and maintain data pipelines, ensure data quality, manage data warehouses, and provide the infrastructure that feeds ML systems.
MLOps Engineers: Specialize in the operational aspects of ML systems, including deployment automation, monitoring, CI/CD for models, and infrastructure management.
AI Product Managers: Define product requirements, translate business needs into technical specifications, and ensure AI solutions deliver measurable value.

The ML Lifecycle Phases

Production ML systems follow a cyclical lifecycle rather than a linear development process:

Problem Formulation: Define business objectives, success metrics, and constraints
Data Collection & Preparation: Gather, clean, and prepare training data
Feature Engineering: Transform raw data into meaningful model inputs
Model Development: Select algorithms, train models, and optimize hyperparameters
Evaluation & Validation: Assess performance using appropriate metrics and validation strategies
Deployment: Integrate models into production environments
Monitoring & Maintenance: Track performance, detect drift, and retrain as needed

Practical Examples

Example 1: Setting Up a Professional Python Environment

The foundation of any AI engineering project begins with a properly configured development environment. This example demonstrates best practices for creating isolated, reproducible Python environments using virtual environments and requirements management.


"""
Professional Python Environment Setup
This script demonstrates best practices for initializing a Python project
with proper dependency management and environment isolation.
"""

import os
import subprocess
import sys
from pathlib import Path

class ProjectEnvironmentSetup:
    """
    Manages the creation and configuration of a Python project environment.
    
    This class encapsulates best practices including:
    - Virtual environment creation
    - Dependency management
    - Project structure initialization
    - Configuration file generation
    """
    
    def __init__(self, project_name, base_path="."):
        self.project_name = project_name
        self.base_path = Path(base_path)
        self.project_path = self.base_path / project_name
        self.venv_path = self.project_path / "venv"
        
    def create_project_structure(self):
        """Create standard project directory structure."""
        directories = [
            self.project_path / "src",
            self.project_path / "tests",
            self.project_path / "data" / "raw",
            self.project_path / "data" / "processed",
            self.project_path / "models",
            self.project_path / "notebooks",
            self.project_path / "configs",
        ]
        
        for directory in directories:
            directory.mkdir(parents=True, exist_ok=True)
            print(f"Created directory: {directory}")
            
    def create_virtual_environment(self):
        """Create isolated virtual environment."""
        print(f"Creating virtual environment at {self.venv_path}")
        subprocess.run([sys.executable, "-m", "venv", str(self.venv_path)])
        print("Virtual environment created successfully")
        
    def generate_requirements_file(self):
        """Generate requirements.txt with common ML dependencies."""
        requirements = [
            "# Core ML Libraries",
            "numpy==1.24.3",
            "pandas==2.0.3",
            "scikit-learn==1.3.0",
            "",
            "# Deep Learning",
            "torch==2.0.1",
            "tensorflow==2.13.0",
            "",
            "# Development Tools",
            "jupyter==1.0.0",
            "black==23.7.0",
            "pytest==7.4.0",
            "mlflow==2.5.0",
        ]
        
        req_file = self.project_path / "requirements.txt"
        req_file.write_text("\n".join(requirements))
        print(f"Generated requirements file: {req_file}")
        
    def generate_gitignore(self):
        """Create comprehensive .gitignore for ML projects."""
        gitignore_content = """
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
venv/
env/

# Jupyter
.ipynb_checkpoints
*.ipynb

# Data
data/raw/*
data/processed/*
!data/.gitkeep

# Models
models/*.pkl
models/*.h5
models/*.pt

# Environment
.env
.venv

# IDEs
.vscode/
.idea/
        """
        
        gitignore_file = self.project_path / ".gitignore"
        gitignore_file.write_text(gitignore_content.strip())
        print(f"Generated .gitignore: {gitignore_file}")
        
    def setup_complete_environment(self):
        """Execute complete environment setup."""
        print(f"\n{'='*50}")
        print(f"Setting up project: {self.project_name}")
        print(f"{'='*50}\n")
        
        self.create_project_structure()
        self.create_virtual_environment()
        self.generate_requirements_file()
        self.generate_gitignore()
        
        print(f"\n{'='*50}")
        print("Setup complete!")
        print(f"{'='*50}")
        print(f"\nNext steps:")
        print(f"1. Activate environment: source {self.venv_path}/bin/activate")
        print(f"2. Install dependencies: pip install -r requirements.txt")
        print(f"3. Start developing in {self.project_path}")

# Usage example
if __name__ == "__main__":
    setup = ProjectEnvironmentSetup("my_ml_project")
    setup.setup_complete_environment()

Output and Explanation:

This script creates a professional project structure with isolated dependencies. The virtual environment ensures that project dependencies don't conflict with system-wide packages. The standardized directory structure separates concerns (source code, tests, data, models) and follows industry conventions that make projects easier to understand and maintain.

Best Practices Demonstrated:

Use virtual environments to isolate dependencies
Pin dependency versions for reproducibility
Separate raw and processed data directories
Include comprehensive .gitignore to avoid committing large files
Use pathlib for cross-platform path handling

Example 2: ML Lifecycle Workflow Manager

This example implements a simple but complete ML lifecycle manager that tracks experiments, manages model versions, and logs key metrics—fundamental capabilities for production ML systems.


"""
ML Lifecycle Workflow Manager
Demonstrates tracking and managing the complete ML lifecycle from
data preparation through model deployment.
"""

import json
import pickle
from datetime import datetime
from pathlib import Path
from typing import Any, Dict, List
import hashlib

class MLLifecycleManager:
    """
    Manages the complete ML workflow with experiment tracking,
    model versioning, and metadata management.
    """
    
    def __init__(self, project_name: str, base_path: str = "./ml_projects"):
        self.project_name = project_name
        self.base_path = Path(base_path) / project_name
        self.experiments_path = self.base_path / "experiments"
        self.models_path = self.base_path / "models"
        self.metadata_path = self.base_path / "metadata"
        
        # Create necessary directories
        for path in [self.experiments_path, self.models_path, self.metadata_path]:
            path.mkdir(parents=True, exist_ok=True)
    
    def create_experiment(self, experiment_name: str, description: str = "") -> str:
        """
        Initialize a new experiment with unique ID and metadata.
        
        Returns:
            experiment_id: Unique identifier for the experiment
        """
        timestamp = datetime.now().isoformat()
        experiment_id = hashlib.md5(
            f"{experiment_name}{timestamp}".encode()
        ).hexdigest()[:8]
        
        metadata = {
            "experiment_id": experiment_id,
            "experiment_name": experiment_name,
            "description": description,
            "created_at": timestamp,
            "status": "initialized",
            "runs": []
        }
        
        metadata_file = self.experiments_path / f"{experiment_id}.json"
        with open(metadata_file, 'w') as f:
            json.dump(metadata, f, indent=2)
        
        print(f"Created experiment: {experiment_name} (ID: {experiment_id})")
        return experiment_id
    
    def log_run(self, experiment_id: str, params: Dict[str, Any], 
                metrics: Dict[str, float], tags: Dict[str, str] = None):
        """
        Log a training run with parameters, metrics, and optional tags.
        """
        metadata_file = self.experiments_path / f"{experiment_id}.json"
        
        if not metadata_file.exists():
            raise ValueError(f"Experiment {experiment_id} not found")
        
        with open(metadata_file, 'r') as f:
            metadata = json.load(f)
        
        run = {
            "run_id": len(metadata["runs"]) + 1,
            "timestamp": datetime.now().isoformat(),
            "parameters": params,
            "metrics": metrics,
            "tags": tags or {}
        }
        
        metadata["runs"].append(run)
        metadata["status"] = "active"
        
        with open(metadata_file, 'w') as f:
            json.dump(metadata, f, indent=2)
        
        print(f"Logged run #{run['run_id']} for experiment {experiment_id}")
        print(f"  Parameters: {params}")
        print(f"  Metrics: {metrics}")
    
    def save_model(self, experiment_id: str, run_id: int, 
                   model: Any, model_name: str):
        """
        Save trained model with versioning and metadata.
        """
        model_version = f"{experiment_id}_run{run_id}_{model_name}"
        model_path = self.models_path / f"{model_version}.pkl"
        
        with open(model_path, 'wb') as f:
            pickle.dump(model, f)
        
        # Update metadata
        metadata_file = self.experiments_path / f"{experiment_id}.json"
        with open(metadata_file, 'r') as f:
            metadata = json.load(f)
        
        # Find the specific run and add model info
        for run in metadata["runs"]:
            if run["run_id"] == run_id:
                run["model_path"] = str(model_path)
                run["model_name"] = model_name
                break
        
        with open(metadata_file, 'w') as f:
            json.dump(metadata, f, indent=2)
        
        print(f"Saved model: {model_path}")
    
    def get_best_run(self, experiment_id: str, metric_name: str, 
                     maximize: bool = True) -> Dict:
        """
        Retrieve the best run based on specified metric.
        """
        metadata_file = self.experiments_path / f"{experiment_id}.json"
        
        with open(metadata_file, 'r') as f:
            metadata = json.load(f)
        
        if not metadata["runs"]:
            raise ValueError("No runs found for this experiment")
        
        best_run = max(
            metadata["runs"],
            key=lambda x: x["metrics"].get(metric_name, float('-inf'))
            if maximize else -x["metrics"].get(metric_name, float('inf'))
        )
        
        return best_run
    
    def generate_report(self, experiment_id: str) -> str:
        """
        Generate comprehensive experiment report.
        """
        metadata_file = self.experiments_path / f"{experiment_id}.json"
        
        with open(metadata_file, 'r') as f:
            metadata = json.load(f)
        
        report = [
            f"\n{'='*60}",
            f"Experiment Report: {metadata['experiment_name']}",
            f"{'='*60}",
            f"ID: {metadata['experiment_id']}",
            f"Created: {metadata['created_at']}",
            f"Status: {metadata['status']}",
            f"Total Runs: {len(metadata['runs'])}",
            f"\n{'='*60}",
            "Run Summary:",
            f"{'='*60}"
        ]
        
        for run in metadata["runs"]:
            report.append(f"\nRun #{run['run_id']} - {run['timestamp']}")
            report.append(f"  Parameters: {run['parameters']}")
            report.append(f"  Metrics: {run['metrics']}")
            if 'model_path' in run:
                report.append(f"  Model: {run['model_name']}")
        
        return "\n".join(report)


# Demonstration usage
if __name__ == "__main__":
    # Initialize lifecycle manager
    manager = MLLifecycleManager("customer_churn_prediction")
    
    # Create experiment
    exp_id = manager.create_experiment(
        "initial_baseline",
        "Testing baseline models for customer churn"
    )
    
    # Simulate multiple training runs
    models_config = [
        {"model": "logistic_regression", "C": 1.0, "accuracy": 0.82, "f1": 0.78},
        {"model": "random_forest", "n_estimators": 100, "accuracy": 0.87, "f1": 0.84},
        {"model": "xgboost", "max_depth": 5, "accuracy": 0.89, "f1": 0.86},
    ]
    
    for config in models_config:
        params = {k: v for k, v in config.items() 
                  if k not in ["accuracy", "f1"]}
        metrics = {k: v for k, v in config.items() 
                   if k in ["accuracy", "f1"]}
        
        manager.log_run(exp_id, params, metrics)
    
    # Get best run
    best = manager.get_best_run(exp_id, "f1", maximize=True)
    print(f"\n\nBest run: #{best['run_id']} with F1 score: {best['metrics']['f1']}")
    
    # Generate report
    print(manager.generate_report(exp_id))

Key Learning Points:

Experiment tracking is essential for reproducibility
Metadata management enables comparing model versions
Systematic logging helps debug and optimize models
Model versioning prevents confusion in production deployments

Example 3: Data Pipeline with Quality Checks

Production ML systems require robust data pipelines with built-in quality checks. This example demonstrates a data pipeline that validates data quality at each stage.


"""
Data Pipeline with Quality Validation
Implements a production-grade data pipeline with comprehensive
quality checks and error handling.
"""

import pandas as pd
import numpy as np
from typing import List, Dict, Tuple, Callable
from dataclasses import dataclass
from enum import Enum

class DataQualityLevel(Enum):
    """Severity levels for data quality issues."""
    INFO = "info"
    WARNING = "warning"
    ERROR = "error"
    CRITICAL = "critical"

@dataclass
class QualityCheckResult:
    """Encapsulates results from a data quality check."""
    check_name: str
    passed: bool
    level: DataQualityLevel
    message: str
    details: Dict = None

class DataQualityValidator:
    """
    Validates data quality with configurable checks and thresholds.
    """
    
    def __init__(self, dataframe: pd.DataFrame):
        self.df = dataframe
        self.results: List[QualityCheckResult] = []
    
    def check_missing_values(self, max_missing_pct: float = 0.1) -> QualityCheckResult:
        """
        Check for missing values exceeding threshold.
        
        Args:
            max_missing_pct: Maximum allowed percentage of missing values
        """
        missing_stats = self.df.isnull().sum() / len(self.df)
        problematic_cols = missing_stats[missing_stats > max_missing_pct]
        
        if len(problematic_cols) > 0:
            result = QualityCheckResult(
                check_name="missing_values",
                passed=False,
                level=DataQualityLevel.WARNING,
                message=f"Found {len(problematic_cols)} columns with >{max_missing_pct*100}% missing",
                details={"columns": problematic_cols.to_dict()}
            )
        else:
            result = QualityCheckResult(
                check_name="missing_values",
                passed=True,
                level=DataQualityLevel.INFO,
                message="Missing values within acceptable range"
            )
        
        self.results.append(result)
        return result
    
    def check_data_types(self, expected_types: Dict[str, str]) -> QualityCheckResult:
        """
        Validate that columns have expected data types.
        
        Args:
            expected_types: Dictionary mapping column names to expected types
        """
        type_mismatches = []
        
        for col, expected_type in expected_types.items():
            if col not in self.df.columns:
                type_mismatches.append(f"{col}: column not found")
            elif str(self.df[col].dtype) != expected_type:
                actual_type = str(self.df[col].dtype)
                type_mismatches.append(f"{col}: expected {expected_type}, got {actual_type}")
        
        if type_mismatches:
            result = QualityCheckResult(
                check_name="data_types",
                passed=False,
                level=DataQualityLevel.ERROR,
                message=f"Found {len(type_mismatches)} type mismatches",
                details={"mismatches": type_mismatches}
            )
        else:
            result = QualityCheckResult(
                check_name="data_types",
                passed=True,
                level=DataQualityLevel.INFO,
                message="All data types match expectations"
            )
        
        self.results.append(result)
        return result
    
    def check_value_ranges(self, range_specs: Dict[str, Tuple[float, float]]) -> QualityCheckResult:
        """
        Check that numeric values fall within expected ranges.
        
        Args:
            range_specs: Dictionary mapping columns to (min, max) tuples
        """
        out_of_range = []
        
        for col, (min_val, max_val) in range_specs.items():
            if col in self.df.columns:
                out_of_bounds = (
                    (self.df[col] < min_val) | (self.df[col] > max_val)
                ).sum()
                
                if out_of_bounds > 0:
                    pct = (out_of_bounds / len(self.df)) * 100
                    out_of_range.append(f"{col}: {out_of_bounds} ({pct:.2f}%) out of range")
        
        if out_of_range:
            result = QualityCheckResult(
                check_name="value_ranges",
                passed=False,
                level=DataQualityLevel.WARNING,
                message=f"Found values outside expected ranges",
                details={"issues": out_of_range}
            )
        else:
            result = QualityCheckResult(
                check_name="value_ranges",
                passed=True,
                level=DataQualityLevel.INFO,
                message="All values within expected ranges"
            )
        
        self.results.append(result)
        return result
    
    def check_duplicates(self, subset: List[str] = None) -> QualityCheckResult:
        """
        Check for duplicate rows.
        
        Args:
            subset: List of columns to consider for duplicate detection
        """
        duplicates = self.df.duplicated(subset=subset).sum()
        duplicate_pct = (duplicates / len(self.df)) * 100
        
        if duplicates > 0:
            result = QualityCheckResult(
                check_name="duplicates",
                passed=False,
                level=DataQualityLevel.WARNING,
                message=f"Found {duplicates} duplicate rows ({duplicate_pct:.2f}%)",
                details={"count": duplicates, "percentage": duplicate_pct}
            )
        else:
            result = QualityCheckResult(
                check_name="duplicates",
                passed=True,
                level=DataQualityLevel.INFO,
                message="No duplicate rows found"
            )
        
        self.results.append(result)
        return result
    
    def generate_report(self) -> str:
        """Generate comprehensive quality report."""
        report_lines = [
            "\n" + "="*70,
            "DATA QUALITY VALIDATION REPORT",
            "="*70,
            f"Total Records: {len(self.df)}",
            f"Total Columns: {len(self.df.columns)}",
            f"Checks Performed: {len(self.results)}",
            "\n" + "-"*70,
            "Check Results:",
            "-"*70
        ]
        
        for result in self.results:
            status = "✓ PASS" if result.passed else "✗ FAIL"
            report_lines.append(f"\n[{result.level.value.upper()}] {status}: {result.check_name}")
            report_lines.append(f"  {result.message}")
            
            if result.details:
                report_lines.append(f"  Details: {result.details}")
        
        report_lines.append("\n" + "="*70)
        
        # Summary
        passed = sum(1 for r in self.results if r.passed)
        failed = len(self.results) - passed
        report_lines.append(f"SUMMARY: {passed} passed, {failed} failed")
        report_lines.append("="*70 + "\n")
        
        return "\n".join(report_lines)


# Demonstration
if __name__ == "__main__":
    # Create sample dataset with intentional quality issues
    np.random.seed(42)
    sample_data = pd.DataFrame({
        'customer_id': range(1000),
        'age': np.random.randint(18, 90, 1000),
        'income': np.random.normal(50000, 20000, 1000),
        'credit_score': np.random.randint(300, 850, 1000),
        'purchase_amount': np.random.exponential(100, 1000)
    })
    
    # Introduce quality issues
    sample_data.loc[sample_data.sample(50).index, 'age'] = np.nan
    sample_data.loc[sample_data.sample(10).index, 'income'] = -1000  # Invalid negative
    sample_data = pd.concat([sample_data, sample_data.head(20)])  # Add duplicates
    
    print("Created sample dataset with quality issues...")
    print(f"Shape: {sample_data.shape}\n")
    
    # Run quality checks
    validator = DataQualityValidator(sample_data)
    
    validator.check_missing_values(max_missing_pct=0.03)
    validator.check_data_types({
        'customer_id': 'int64',
        'age': 'float64',  # Will be float due to NaN
        'income': 'float64'
    })
    validator.check_value_ranges({
        'age': (18, 100),
        'income': (0, 500000),
        'credit_score': (300, 850)
    })
    validator.check_duplicates(subset=['customer_id'])
    
    # Print report
    print(validator.generate_report())

Best Practices Highlighted:

Automated quality checks catch issues before they reach models
Configurable thresholds adapt to different data requirements
Comprehensive logging enables debugging data issues
Structured results allow programmatic response to quality problems

Example 4: Model Monitoring and Drift Detection

Once models are deployed, continuous monitoring is crucial. This example implements basic monitoring for model performance and data drift detection.


"""
Model Monitoring and Drift Detection
Implements monitoring for production models including performance
tracking and data drift detection.
"""

import numpy as np
import pandas as pd
from scipy import stats
from typing import Dict, List, Tuple
from datetime import datetime, timedelta
from collections import deque

class ModelMonitor:
    """
    Monitors deployed models for performance degradation and data drift.
    """
    
    def __init__(self, model_name: str, baseline_data: pd.DataFrame, 
                 window_size: int = 1000):
        """
        Initialize model monitor.
        
        Args:
            model_name: Identifier for the model being monitored
            baseline_data: Reference data from training/validation
            window_size: Number of recent predictions to consider
        """
        self.model_name = model_name
        self.baseline_data = baseline_data
        self.window_size = window_size
        
        # Store recent predictions and features
        self.recent_predictions = deque(maxlen=window_size)
        self.recent_features = deque(maxlen=window_size)
        self.recent_actuals = deque(maxlen=window_size)
        self.timestamps = deque(maxlen=window_size)
        
        # Calculate baseline statistics
        self.baseline_stats = self._calculate_baseline_statistics()
        
    def _calculate_baseline_statistics(self) -> Dict:
        """Calculate statistical properties of baseline data."""
        stats_dict = {}
        
        for column in self.baseline_data.columns:
            if pd.api.types.is_numeric_dtype(self.baseline_data[column]):
                stats_dict[column] = {
                    'mean': self.baseline_data[column].mean(),
                    'std': self.baseline_data[column].std(),
                    'min': self.baseline_data[column].min(),
                    'max': self.baseline_data[column].max(),
                    'q25': self.baseline_data[column].quantile(0.25),
                    'q75': self.baseline_data[column].quantile(0.75)
                }
        
        return stats_dict
    
    def log_prediction(self, features: Dict, prediction: float, 
                       actual: float = None):
        """
        Log a single prediction for monitoring.
        
        Args:
            features: Input features dictionary
            prediction: Model prediction
            actual: Ground truth value (if available)
        """
        self.recent_predictions.append(prediction)
        self.recent_features.append(features)
        self.recent_actuals.append(actual)
        self.timestamps.append(datetime.now())
    
    def detect_feature_drift(self, feature_name: str, 
                            significance_level: float = 0.05) -> Tuple[bool, float]:
        """
        Detect distribution drift using Kolmogorov-Smirnov test.
        
        Args:
            feature_name: Name of feature to check
            significance_level: Statistical significance threshold
            
        Returns:
            (drift_detected, p_value)
        """
        if feature_name not in self.baseline_stats:
            return False, 1.0
        
        # Extract recent values for this feature
        recent_values = [f.get(feature_name) for f in self.recent_features 
                        if f.get(feature_name) is not None]