2.5 KiB
2.5 KiB
Writer Service
Overview
The Writer Service is the third and final component in the Redis pub/sub processing chain for bank-related email automation. It subscribes to messages with stage="parsed" from the Parser Service, writes the processed data to the database, and publishes a completion status with stage="written".
Features
Redis Integration
- Subscribes to the "parser" Redis channel for messages with stage="parsed"
- Processes parsed data and writes it to the database
- Publishes completion status to the "writer" channel with stage="written"
- Maintains message metadata and adds processing timestamps
Database Integration
- Writes parsed transaction data to AccountRecords database
- Links transactions to build information via IBAN
- Handles duplicate detection to prevent redundant entries
- Adds date components for easier querying (year, month, day, weekday)
Error Handling
- Robust error management for database operations
- Detailed logging of processing steps and errors
- Graceful handling of malformed messages
Configuration
Environment Variables
The service uses the same Redis configuration as the other services:
REDIS_HOST=10.10.2.15
REDIS_PORT=6379
REDIS_PASSWORD=your_strong_password_here
Deployment
Docker
The service is containerized using Docker and can be deployed using the provided Dockerfile and docker-compose configuration.
# Build and start the service
docker compose -f bank-services-docker-compose.yml up -d --build
# View logs
docker compose -f bank-services-docker-compose.yml logs -f writer_service
# Stop the service
docker compose -f bank-services-docker-compose.yml down
Service Management
The check_bank_services.sh script provides a simple way to restart the service:
./check_bank_services.sh
Architecture
Redis Pub/Sub Chain
This service is the third and final component in a multi-stage processing chain:
- Email Service: Reads emails, extracts attachments, publishes to "reader" channel with stage="red"
- Parser Service: Subscribes to "reader" channel, parses Excel data, publishes to "parser" channel with stage="parsed"
- Writer Service (this service): Subscribes to "parser" channel, writes data to database, publishes to "writer" channel with stage="written"
Development
Dependencies
- Python 3.12
- SQLAlchemy and PostgreSQL for database operations
- Redis for pub/sub messaging
- Arrow for date handling
- FastAPI for potential API endpoints