README

A high-performance Laravel package for tracking and detecting changes in Eloquent models using cryptographic hashes. Perfect for audit trails, cache invalidation, external system synchronization, and data integrity monitoring in large-scale applications.

Laravel Change Detection automatically calculates and stores MD5/SHA256 hashes of your model attributes, supports composite dependency tracking, provides MySQL-optimized bulk operations for 100k+ records, and includes comprehensive CLI tools for monitoring and maintenance.

Features

🚀 High Performance: MySQL-optimized hash calculations for 100k+ records with 1000-record batch processing
🔄 Composite Dependencies: Track changes across related models automatically
⚡ Bulk Operations: Efficient batch processing with configurable limits
🗃️ Cross-Database Support: Hash tables can be in different databases than models
🛠️ CLI Tools: Comprehensive commands for monitoring and maintenance
📊 Queue Integration: Background processing with retry logic
🔍 Change Detection: Instant detection of model modifications
💾 Soft Delete Support: Proper handling of deleted records
🧹 Cleanup Tools: Automatic orphaned hash detection and removal
📝 Debug Logging: Built-in LogPublisher for development
🎯 Smart Model Discovery: Automatically discovers models via Publishers and Morph Map
🔧 Dependency Tracking: Efficient dependency building with has_dependencies_built flag

Installation

You can install the package via composer:

composer require ameax/laravel-change-detection

Publish and run the migrations:

php artisan vendor:publish --tag="laravel-change-detection-migrations"
php artisan migrate

Optionally, publish the config file:

php artisan vendor:publish --tag="laravel-change-detection-config"

Configuration

The published config file (config/change-detection.php) includes:

return [
    // Database connection for hash tables (null = default)
    'database_connection' => null,

    // Custom table names
    'tables' => [
        'hashes' => 'hashes',
        'hash_dependents' => 'hash_dependents',
        'publishers' => 'publishers',
        'publishes' => 'publishes',
    ],

    // Queue configuration
    'queues' => [
        'publish' => 'default',
        'detect_changes' => 'default',
    ],

    // Hash algorithm: 'md5' or 'sha256'
    'hash_algorithm' => 'md5',

    // Retry intervals for failed publishes (in seconds)
    'retry_intervals' => [
        1 => 30,    // First retry after 30 seconds
        2 => 300,   // Second retry after 5 minutes
        3 => 21600, // Third retry after 6 hours
    ],
];

Quick Start

1. Make Your Model Hashable

use Ameax\LaravelChangeDetection\Contracts\Hashable;
use Ameax\LaravelChangeDetection\Traits\InteractsWithHashes;
use Illuminate\Database\Eloquent\Model;

class User extends Model implements Hashable
{
    use InteractsWithHashes;

    protected $fillable = ['name', 'email', 'status'];

    // Define which attributes to include in hash calculation
    public function getHashableAttributes(): array
    {
        return ['name', 'email', 'status'];
    }

    // Define child dependencies (optional - for composite hashing)
    public function getHashCompositeDependencies(): array
    {
        return ['posts']; // Hash changes when user's posts change
    }

    // Define parent relations to notify (optional)
    public function getHashParentRelations(): array
    {
        return []; // No parent relations for User
    }

    public function posts()
    {
        return $this->hasMany(Post::class);
    }
}

2. Automatic Hash Generation

Hashes are automatically calculated and stored when models are saved:

$user = User::create([
    'name' => 'John Doe',
    'email' => 'john@example.com',
    'status' => 'active'
]);

// Hash is automatically generated and stored
$currentHash = $user->getCurrentHash();
echo $currentHash->composite_hash; // e.g., "a1b2c3d4e5f6..."

3. Change Detection

// Check if model has changed since last hash calculation
if ($user->hasHashChanged()) {
    echo "User has been modified!";
}

// Manually recalculate hash
$user->updateHash();

// Force hash update regardless of changes
$newHash = $user->forceHashUpdate();

Advanced Usage

The Hashable Interface

The Hashable interface requires implementing these methods:

interface Hashable
{
    // Required: Define which attributes to include in the hash
    public function getHashableAttributes(): array;

    // Optional: Define child relations whose changes affect this model's hash
    public function getHashCompositeDependencies(): array;

    // Optional: Define parent relations to notify when this model changes
    public function getHashParentRelations(): array;

    // Optional: Define a scope to filter which records get hashed
    public function getHashableScope(): ?\Closure;
}

The InteractsWithHashes trait provides default implementations for optional methods.

Composite Dependencies

Track changes across related models:

class Article extends Model implements Hashable
{
    use InteractsWithHashes;

    public function getHashableAttributes(): array
    {
        return ['title', 'content', 'author'];
    }

    // Article hash changes when replies change
    public function getHashCompositeDependencies(): array
    {
        return ['replies'];
    }

    public function replies()
    {
        return $this->hasMany(Reply::class);
    }
}

class Reply extends Model implements Hashable
{
    use InteractsWithHashes;

    public function getHashableAttributes(): array
    {
        return ['content', 'author'];
    }

    // No child dependencies
    public function getHashCompositeDependencies(): array
    {
        return [];
    }

    // Define parent relations to notify when this model changes
    public function getHashParentRelations(): array
    {
        return ['article']; // Notify article when reply changes
    }

    public function article()
    {
        return $this->belongsTo(Article::class);
    }
}

When a reply is created/updated/deleted, the parent article's hash automatically updates through the parent relation notification.

Scope Filtering

Control which records get hashed using scopes:

class WeatherStation extends Model implements Hashable
{
    use InteractsWithHashes;

    public function getHashableAttributes(): array
    {
        return ['name', 'location', 'status'];
    }

    // Only active stations in specific regions get hashed
    public function getHashableScope(): ?\Closure
    {
        return function ($query) {
            $query->where('status', 'active')
                  ->whereIn('location', ['Bayern', 'Berlin']);
        };
    }
}

Important: The system ensures that:

Only models within their scope get hash records
Dependencies are only created for models that are in scope
When a model goes out of scope, its hash is soft-deleted

CLI Commands

Sync Command (Recommended)

The simplest way to synchronize all hash records with a single command:

# Synchronize all hashes (auto-discover, detect, cleanup, and update)
php artisan change-detection:sync

# Preview changes without making modifications
php artisan change-detection:sync --dry-run

# Show detailed report of operations
php artisan change-detection:sync --report

# Limit processing per model (useful for large datasets)
php artisan change-detection:sync --limit=1000

# Hard delete orphaned and soft-deleted hashes from database
php artisan change-detection:sync --purge

# Sync specific models only
php artisan change-detection:sync --models="App\Models\User,App\Models\Post"

This command combines auto-discovery, change detection, orphan cleanup, and hash updates in one operation.

Note about --purge option: By default, the sync command soft-deletes hashes (marks them with a deleted_at timestamp) when models are deleted or go out of scope. Using the --purge flag will:

Hard delete (completely remove) orphaned hashes where the model record no longer exists
Hard delete any previously soft-deleted hashes (where deleted_at is not null)
This permanently removes the hash records and their related data via cascade deletion

Model Discovery

The sync command automatically discovers models through multiple methods:

Publisher Records: Models registered via Publishers (supports models anywhere in your codebase)
Laravel Morph Map: Models registered in Laravel's morph map
App\Models Directory: Fallback scanning of the standard models directory

Models with dependencies are processed in the correct order: dependencies first, then main models.

Detect Changes Command

For fine-grained control over change detection:

# Auto-discover and check all hashable models
php artisan change-detection:detect --auto-discover

# Check specific models
php artisan change-detection:detect --models="App\Models\User,App\Models\Post"

# Update detected changes
php artisan change-detection:detect --auto-discover --update

# Clean up orphaned hashes
php artisan change-detection:detect --auto-discover --cleanup

# Show detailed report
php artisan change-detection:detect --auto-discover --report

# Limit processing (useful for large datasets)
php artisan change-detection:detect --auto-discover --limit=1000

Truncate Command

Reset the change detection system by clearing all tables:

# Truncate all change detection tables
php artisan change-detection:truncate

# Skip confirmation prompt
php artisan change-detection:truncate --force

# Only truncate specific tables
php artisan change-detection:truncate --only=hashes,publishes

Purge Deleted Hashes

Remove soft-deleted hash records:

# Purge all deleted hashes
php artisan change-detection:purge

# Purge hashes deleted more than 7 days ago
php artisan change-detection:purge --older-than=7

# Preview what would be purged
php artisan change-detection:purge --dry-run

# Skip confirmation
php artisan change-detection:purge --force

Background Processing

Queue change detection for large datasets:

use Ameax\LaravelChangeDetection\Jobs\DetectChangesJob;

// Queue change detection for a specific model
DetectChangesJob::dispatch(
    modelClass: User::class,
    updateHashes: true,
    cleanupOrphaned: true,
    limit: 5000
);

Bulk Operations

For high-performance scenarios with 100k+ records:

use Ameax\LaravelChangeDetection\Services\BulkHashProcessor;
use Ameax\LaravelChangeDetection\Services\ChangeDetector;

$processor = app(BulkHashProcessor::class);
$detector = app(ChangeDetector::class);

// Count changed models efficiently
$changedCount = $detector->countChangedModels(User::class);

// Process changes in batches (default: 1000 records per batch)
$updatedCount = $processor->processChangedModels(User::class, limit: 10000);

// Process specific model IDs
$modelIds = [1, 2, 3, 4, 5];
$updatedCount = $processor->updateHashesForIds(User::class, $modelIds);

// Build pending dependencies for models
$pendingCount = $processor->buildPendingDependencies(User::class);

Performance Optimizations

The package is optimized for large-scale data processing:

Batch Processing: All operations work in configurable chunks (default: 1000 records)
Bulk SQL Operations: Uses INSERT ... ON DUPLICATE KEY UPDATE for efficient hash updates
Smart Dependency Building: Dependencies are built after initial hash creation using the has_dependencies_built flag
N+1 Query Prevention: Batch updates for dependency flags to avoid multiple queries
Optimized Model Discovery: Models are discovered via Publishers and Laravel's Morph Map

Orphaned Hash Cleanup

Automatically detect and clean up hashes for deleted models:

use Ameax\LaravelChangeDetection\Services\OrphanedHashDetector;

$detector = app(OrphanedHashDetector::class);

// Count orphaned hashes
$orphanedCount = $detector->countOrphanedHashes(User::class);

// Get orphaned hash details
$orphanedHashes = $detector->detectOrphanedHashes(User::class);

// Clean up orphaned hashes
$cleanedCount = $detector->cleanupOrphanedHashes(User::class);

// Handle soft-deleted models
$softDeletedCount = $detector->cleanupSoftDeletedModelHashes(User::class);

Cross-Database Support

Store hash tables in a different database than your models:

// config/change-detection.php
return [
    'database_connection' => 'analytics', // Use different connection
    // ... other config
];

// database/config.php
'connections' => [
    'mysql' => [
        // Your main database
        'database' => 'main_app',
    ],
    'analytics' => [
        // Separate database for hash storage
        'driver' => 'mysql',
        'database' => 'analytics_db',
        'host' => 'analytics-server',
        // ... connection details
    ],
],

Publishing System

Publisher Configuration

Publishers implement the Publisher contract with comprehensive rate limiting and error handling configuration:

use Ameax\LaravelChangeDetection\Contracts\Publisher;

class CustomPublisher implements Publisher
{
    // Rate limiting configuration
    public function getBatchSize(): int
    {
        return 1000; // Process 1000 records per batch
    }

    public function getDelayMs(): int
    {
        return 100; // 100ms delay between each record
    }

    public function getRetryIntervals(): array
    {
        return [30, 300, 3600]; // Retry after 30s, 5m, 1h
    }

    // Error handling configuration
    public function getMaxValidationErrors(): int
    {
        return 50; // Stop job after 50 validation errors
    }

    public function getMaxInfrastructureErrors(): int
    {
        return 1; // Stop job after 1 infrastructure error
    }

    public function handlePublishException(\Throwable $exception): string
    {
        // Return 'stop_job', 'fail_record', or 'defer_record'
        if (str_contains($exception->getMessage(), 'Connection refused')) {
            return 'stop_job'; // Critical infrastructure issue
        }

        return 'defer_record'; // Retry later
    }

    // Publishing logic
    public function shouldPublish($model): bool
    {
        return $model->status === 'active';
    }

    public function getData($model): array
    {
        return $model->toArray();
    }

    public function publish($model, array $data): bool
    {
        // Your publishing logic here
        return $this->sendToExternalAPI($data);
    }
}

Error Handling & Recovery

The publishing system tracks errors with detailed categorization:

// Check publish status for a model
$user = User::find(1);
$status = $user->getPublishStatus();

// Returns:
// [
//     'publisher_name' => [
//         'status' => 'failed|deferred|completed',
//         'last_error' => 'Error message',
//         'error_type' => 'validation|infrastructure|data',
//         'response_code' => 404,
//         'attempt_count' => 3,
//         'last_attempted_at' => '2024-01-15 10:30:00'
//     ]
// ]

// Reset errors for debugging
$user->resetPublishErrors(); // Reset all publishers
$user->resetPublishErrorsForPublisher($publisherId); // Reset specific publisher

// Get error count
$errorCount = $user->getPublishErrorCount();

Processing Publishes

The BulkPublishJob processes publishes with intelligent error handling:

# Queue bulk publish job (recommended)
php artisan change-detection:process-publishes

# Process synchronously with real-time feedback
php artisan change-detection:process-publishes --sync

# Force processing even if another job is running
php artisan change-detection:process-publishes --force

# Process specific number of records
php artisan change-detection:process-publishes --limit=500

The job automatically:

Single Instance: Only one bulk job runs at a time using Laravel's ShouldBeUnique
Publisher Grouping: Groups records by publisher for optimal batch processing
Rate Limiting: Uses each publisher's getBatchSize() and getDelayMs() settings
Error Differentiation: Tracks validation vs infrastructure errors separately
Smart Stopping: Stops processing when error thresholds are reached
Response Tracking: Captures HTTP response codes for debugging
Automatic Chaining: Dispatches next batch if more records exist

Publisher Examples

API Publisher (with strict rate limiting):

class ApiPublisher implements Publisher
{
    public function getBatchSize(): int { return 50; }
    public function getDelayMs(): int { return 200; }
    public function getMaxValidationErrors(): int { return 10; }
    public function getMaxInfrastructureErrors(): int { return 1; }

    public function handlePublishException(\Throwable $exception): string
    {
        if (str_contains($exception->getMessage(), '429')) {
            return 'stop_job'; // Rate limit hit, stop job
        }
        return 'defer_record';
    }
}

SFTP Export Publisher (no limits):

class SftpExportPublisher implements Publisher
{
    public function getBatchSize(): int { return 0; }      // Unlimited
    public function getDelayMs(): int { return 0; }        // No delay
    public function getMaxValidationErrors(): int { return 0; } // No limit
    public function getMaxInfrastructureErrors(): int { return 1; } // Stop on connection issues
}

Email Publisher (moderate batching):

class EmailPublisher implements Publisher
{
    public function getBatchSize(): int { return 100; }
    public function getDelayMs(): int { return 50; }
    public function getMaxValidationErrors(): int { return 20; } // Invalid emails
    public function getMaxInfrastructureErrors(): int { return 3; } // SMTP issues
}

Immediate Publishing

For urgent records that bypass the queue system:

// Publish immediately (synchronous)
$success = $publishRecord->publishNow();

// Smart publishing - immediate if no bulk job running, else queue
$publishRecord->publishImmediatelyOrQueue();

// Model-level immediate publishing
$user = User::find(1);
$user->publishImmediately(); // Publishes to all configured publishers

Error Types & Response Codes

The system categorizes errors for targeted handling:

Validation Errors: Invalid data format, missing required fields
Infrastructure Errors: Network timeouts, connection failures, authentication
Data Errors: Missing models, empty data sets

Response codes are captured for HTTP-based publishers:

// Publish records store detailed error information
$publishRecord = Publish::find(1);
echo $publishRecord->last_error_message;     // "HTTP 404: Resource not found"
echo $publishRecord->last_response_code;     // 404
echo $publishRecord->error_type;             // "validation"
echo $publishRecord->attempt_count;          // 3

Bulk Processing Details

The BulkPublishJob processing flow:

Acquire Lock: Prevents multiple instances using cache locks
Group by Publisher: Processes each publisher type separately
Load Settings: Uses publisher-specific batch size and delays
Process Batch: Handles records with comprehensive error tracking
Error Monitoring: Stops when error thresholds are exceeded
Chain Next Batch: Automatically dispatches follow-up jobs
Release Lock: Ensures clean shutdown

Development & Debugging

LogPublisher for Development

use Ameax\LaravelChangeDetection\Publishers\LogPublisher;

// Configure log publisher
$publisher = new LogPublisher(
    logChannel: 'change-detection',
    logLevel: 'debug',
    includeHashData: true
);

// Get model data for debugging
$data = $publisher->getData($user);

// Publish to logs
$publisher->publish($user, $data);

Hash Information Methods

// Get current hash record
$hashRecord = $user->getCurrentHash();

// Get hash dependents (models that depend on this one)
$dependents = $user->getHashDependents();

// Get publish records for this hash
$publishes = $user->getHashPublishes();

// Check if hash is deleted
$isDeleted = $user->isHashDeleted();

// Get last hash update time
$lastUpdated = $user->getHashLastUpdated();

// Calculate hashes without storing
$attributeHash = $user->calculateAttributeHash();
$compositeHash = $user->calculateCompositeHash();

Error Management Methods

// Reset all publish errors for this model
$resetCount = $user->resetPublishErrors();

// Reset errors for specific publisher
$resetCount = $user->resetPublishErrorsForPublisher($publisherId);

// Get total error count across all publishers
$errorCount = $user->getPublishErrorCount();

// Get detailed publish status for all publishers
$status = $user->getPublishStatus();
// Returns array with publisher status details

// Model can be published immediately (bypassing queue)
$user->publishImmediately();

Performance Considerations

MySQL Optimization

This package is optimized for MySQL with direct SQL queries:

Hash calculations use MySQL's native MD5() and SHA2() functions
Bulk operations use INSERT...ON DUPLICATE KEY UPDATE for efficiency
Cross-database JOINs are properly qualified
Configurable batch sizes for memory management

Large Dataset Recommendations

For applications with 100k+ records:

// Use bulk operations instead of individual model methods
$processor = app(BulkHashProcessor::class);
$processor->setBatchSize(5000); // Adjust based on memory

// Use background jobs for large operations
DetectChangesJob::dispatch(User::class, limit: 10000);

// Use limits when detecting changes
php artisan change-detection:detect --auto-discover --limit=5000

Common Use Cases

1. Cache Invalidation

class ProductController extends Controller
{
    public function show(Product $product)
    {
        $cacheKey = "product.{$product->id}.{$product->getCurrentHash()->composite_hash}";

        return Cache::remember($cacheKey, 3600, function () use ($product) {
            return $this->generateProductView($product);
        });
    }
}

2. External System Synchronization

// Detect changed products for API sync
$changedProducts = app(ChangeDetector::class)
    ->detectChangedModels(Product::class, limit: 100);

foreach ($changedProducts as $product) {
    // Sync to external system
    $this->syncToExternalAPI($product);

    // Update hash after successful sync
    $product->updateHash();
}

3. Audit Trail Integration

class User extends Model implements Hashable
{
    use InteractsWithHashes;

    protected static function booted()
    {
        static::updated(function ($user) {
            if ($user->hasHashChanged()) {
                AuditLog::create([
                    'model_type' => 'user',
                    'model_id' => $user->id,
                    'old_hash' => $user->getOriginal('hash'),
                    'new_hash' => $user->getCurrentHash()->composite_hash,
                    'changed_at' => now(),
                ]);
            }
        });
    }
}

4. Data Integrity Monitoring

// Daily integrity check
$command = Artisan::call('change-detection:detect', [
    '--auto-discover' => true,
    '--report' => true,
]);

// Queue periodic cleanup
Schedule::job(new DetectChangesJob(User::class, false, true))
    ->daily()
    ->description('Clean up orphaned user hashes');

Testing

Run the test suite:

composer test

Run static analysis:

composer analyse

Format code:

composer format

The package includes comprehensive tests for:

Hash calculation and storage
Composite dependency tracking
Bulk operations and performance
Cross-database functionality
CLI commands and background jobs
Change detection algorithms

Requirements

PHP: 8.3+
Laravel: 12.0+
Database: MySQL 8.0+ (optimized for MySQL)
Extensions: PDO MySQL

Changelog

Please see CHANGELOG for more information on what has changed recently.

Contributing

Please see CONTRIBUTING for details.

Security Vulnerabilities

Please review our security policy on how to report security vulnerabilities.

Architecture

The package follows clean architecture principles:

Services: Core business logic (ChangeDetector, BulkHashProcessor, etc.)
Models: Database entities with proper relationships
Contracts: Interfaces for extensibility
Traits: Reusable functionality for models
Commands: CLI tools for management
Jobs: Background processing capabilities

All classes are kept under 200 lines for maintainability, and the codebase follows PSR-12 coding standards with comprehensive type safety.

Credits

License

The MIT License (MIT). Please see License File for more information.

ameax / laravel-change-detection

Maintainers

Details