Skip to content

Add Celery workers and Redis for concurrent training#1

Open
damiancodes wants to merge 2 commits intomainfrom
feature/concurrent-training-celery
Open

Add Celery workers and Redis for concurrent training#1
damiancodes wants to merge 2 commits intomainfrom
feature/concurrent-training-celery

Conversation

@damiancodes
Copy link
Collaborator

Infrastructure changes to support concurrent ML training.

Changes:

  • Added Redis service for message broker
  • Added 5 Celery worker containers (10 concurrency each = 50 capacity)
  • Created separate docker-compose-local.yml for local testing
  • Added worker Dockerfile and updated gitignore

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants