144 lines
3.7 KiB
Markdown
144 lines
3.7 KiB
Markdown
# Database Handlers
|
|
|
|
This directory contains database handlers for MongoDB and PostgreSQL used in the backend automate services.
|
|
|
|
## Overview
|
|
|
|
The database handlers provide a consistent interface for interacting with different database systems. They implement:
|
|
|
|
- Connection pooling
|
|
- Retry mechanisms
|
|
- Error handling
|
|
- Thread safety
|
|
- Context managers for resource management
|
|
|
|
## MongoDB Handler
|
|
|
|
The MongoDB handler is implemented as a singleton pattern to ensure efficient connection management across the application. It provides:
|
|
|
|
- Connection pooling via PyMongo's built-in connection pool
|
|
- Automatic retry capabilities for MongoDB operations
|
|
- Context manager for MongoDB collections to ensure connections are properly closed
|
|
- Thread safety for concurrent operations
|
|
|
|
### MongoDB Performance
|
|
|
|
The MongoDB handler has been tested with a concurrent load test:
|
|
|
|
```
|
|
Concurrent Operation Test Results:
|
|
Total threads: 100
|
|
Passed: 100
|
|
Failed: 0
|
|
Execution time: 0.73 seconds
|
|
Operations per second: 137.61
|
|
```
|
|
|
|
## PostgreSQL Handler
|
|
|
|
The PostgreSQL handler leverages SQLAlchemy for ORM capabilities and connection management. It provides:
|
|
|
|
- Connection pooling via SQLAlchemy's connection pool
|
|
- ORM models with CRUD operations
|
|
- Filter methods for querying data
|
|
- Transaction management
|
|
|
|
### PostgreSQL Performance
|
|
|
|
The PostgreSQL handler has been tested with a concurrent load test:
|
|
|
|
```
|
|
Concurrent Operation Test Results:
|
|
Total threads: 100
|
|
Passed: 100
|
|
Failed: 0
|
|
Execution time: 0.30 seconds
|
|
Operations per second: 332.11
|
|
```
|
|
|
|
## Usage Examples
|
|
|
|
### MongoDB Example
|
|
|
|
```python
|
|
from Controllers.Mongo.database import mongo_handler
|
|
|
|
# Using the context manager for automatic connection management
|
|
with mongo_handler.collection("users") as users_collection:
|
|
# Perform operations
|
|
users_collection.insert_one({"name": "John", "email": "john@example.com"})
|
|
user = users_collection.find_one({"email": "john@example.com"})
|
|
```
|
|
|
|
### PostgreSQL Example
|
|
|
|
```python
|
|
from Controllers.Postgres.schema import EndpointRestriction
|
|
|
|
# Using the session context manager
|
|
with EndpointRestriction.new_session() as db_session:
|
|
# Create a new record
|
|
new_endpoint = EndpointRestriction(
|
|
endpoint_code="TEST_API",
|
|
endpoint_name="Test API",
|
|
endpoint_method="GET",
|
|
endpoint_function="test_function",
|
|
endpoint_desc="Test description",
|
|
is_confirmed=True
|
|
)
|
|
new_endpoint.save(db=db_session)
|
|
|
|
# Query records
|
|
result = EndpointRestriction.filter_one(
|
|
EndpointRestriction.endpoint_code == "TEST_API",
|
|
db=db_session
|
|
).data
|
|
```
|
|
|
|
## Configuration
|
|
|
|
Both handlers are configured via environment variables:
|
|
|
|
### MongoDB Configuration
|
|
|
|
- `MONGO_ENGINE`: Database engine (mongodb)
|
|
- `MONGO_HOST`: Database host
|
|
- `MONGO_PORT`: Database port
|
|
- `MONGO_USER`: Database username
|
|
- `MONGO_PASSWORD`: Database password
|
|
- `MONGO_DB`: Database name
|
|
- `MONGO_AUTH_DB`: Authentication database
|
|
|
|
### PostgreSQL Configuration
|
|
|
|
- `POSTGRES_ENGINE`: Database engine (postgresql+psycopg2)
|
|
- `POSTGRES_HOST`: Database host
|
|
- `POSTGRES_PORT`: Database port
|
|
- `POSTGRES_USER`: Database username
|
|
- `POSTGRES_PASSWORD`: Database password
|
|
- `POSTGRES_DB`: Database name
|
|
- `POSTGRES_POOL_SIZE`: Connection pool size
|
|
- `POSTGRES_POOL_PRE_PING`: Whether to ping the database before using a connection
|
|
|
|
## Testing
|
|
|
|
Both handlers include comprehensive test suites that verify:
|
|
|
|
- Basic CRUD operations
|
|
- Complex queries
|
|
- Nested documents (MongoDB)
|
|
- Array operations (MongoDB)
|
|
- Aggregation (MongoDB)
|
|
- Index operations
|
|
- Concurrent operations
|
|
|
|
To run the tests:
|
|
|
|
```bash
|
|
# MongoDB tests
|
|
python -m Controllers.Mongo.implementations
|
|
|
|
# PostgreSQL tests
|
|
python -m Controllers.Postgres.implementations
|
|
```
|