production-evyos-systems-and-services-3/ServicesApi/Controllers/Redis/Broadcast/README.md

Redis Pub/Sub Chain Implementation

This module implements a chain of services communicating through Redis Pub/Sub channels. Each service in the chain subscribes to the previous service's channel and publishes to its own channel, creating a processing pipeline.

Architecture

The implementation follows a simple chain pattern:

READER → PROCESSOR → WRITER

• READER: Generates mock data with a "red" stage and publishes to chain:reader
• PROCESSOR: Subscribes to chain:reader, processes messages with "red" stage, updates stage to "processed", and publishes to chain:processor
• WRITER: Subscribes to chain:processor, processes messages with "processed" stage, updates stage to "completed", and publishes to chain:writer

Message Flow

Each message flows through the chain with a stage attribute that determines how it's processed:

1. READER generates a message with stage="red"
2. PROCESSOR receives the message, checks if stage="red", processes it, and sets stage="processed"
3. WRITER receives the message, checks if stage="processed", processes it, and sets stage="completed"

Performance

The implementation includes timing information to track how long messages take to flow through the entire chain. Sample output:

[READER] 1745176466.132082 | Published UUID: 74cf2312-25ec-4da8-bc0a-521b6ccd5206
[PROCESSOR] 1745176466.132918 | Received UUID: 74cf2312-25ec-4da8-bc0a-521b6ccd5206 | Published UUID: 74cf2312-25ec-4da8-bc0a-521b6ccd5206
[WRITER] 1745176466.133097 | Received UUID: 74cf2312-25ec-4da8-bc0a-521b6ccd5206 | Published UUID: 74cf2312-25ec-4da8-bc0a-521b6ccd5206 | Elapsed: 1.83ms
[READER] 1745176468.133018 | Published UUID: 2ffd217f-650f-4e10-bc16-317adcf7a59a
[PROCESSOR] 1745176468.133792 | Received UUID: 2ffd217f-650f-4e10-bc16-317adcf7a59a | Published UUID: 2ffd217f-650f-4e10-bc16-317adcf7a59a
[WRITER] 1745176468.134001 | Received UUID: 2ffd217f-650f-4e10-bc16-317adcf7a59a | Published UUID: 2ffd217f-650f-4e10-bc16-317adcf7a59a | Elapsed: 1.76ms
[READER] 1745176470.133841 | Published UUID: 87e1f3af-c6c2-4fa5-9a65-57e7327d3989
[PROCESSOR] 1745176470.134623 | Received UUID: 87e1f3af-c6c2-4fa5-9a65-57e7327d3989 | Published UUID: 87e1f3af-c6c2-4fa5-9a65-57e7327d3989
[WRITER] 1745176470.134861 | Received UUID: 87e1f3af-c6c2-4fa5-9a65-57e7327d3989 | Published UUID: 87e1f3af-c6c2-4fa5-9a65-57e7327d3989 | Elapsed: 1.68ms

The elapsed time shows the total time from when the READER publishes a message until the WRITER completes processing it. In the samples above, the end-to-end processing time ranges from 1.68ms to 1.83ms.

Usage

To run the demonstration:

python -m Controllers.Redis.Broadcast.implementations

This will start all three services in the chain and begin processing messages. Press Ctrl+C to stop the demonstration.

Implementation Details

The implementation uses:

1. A singleton Redis Pub/Sub handler with publisher and subscriber capabilities
2. Thread-based message processing
3. JSON serialization for message passing
4. Stage-based message processing to track progress through the chain
5. Timing information to measure performance

Each service in the chain follows these steps:

1. Subscribe to the appropriate channel
2. Define a message handler function
3. Process incoming messages based on their stage
4. Publish processed messages to the next channel in the chain