a2a475ba7a
* add bullmq redis for message queue processing * Update pnpm-lock.yaml * update queue manager * remove singleton patterns, add redis to cache pool * add bull board ui * update rate limit handler * update redis configuration * Merge add rate limit redis prefix * update rate limit queue events * update preview loader to queue * refractor namings to constants * update env variable for queue * update worker shutdown gracefully |
||
|---|---|---|
| .. | ||
| README.md | ||
| docker-compose.yml | ||
README.md
Flowise Worker
By utilizing worker instances when operating in queue mode, Flowise can be scaled horizontally by adding more workers to handle increased workloads or scaled down by removing workers when demand decreases.
Here’s an overview of the process:
- The primary Flowise instance sends an execution ID to a message broker, Redis, which maintains a queue of pending executions, allowing the next available worker to process them.
- A worker from the pool retrieves a message from Redis. The worker starts execute the actual job.
- Once the execution is completed, the worker alerts the main instance that the execution is finished.
How to use
Setting up Main Server:
- Follow setup guide
- In the
.env.example, setup all the necessary env variables forQUEUE CONFIGURATION
Setting up Worker:
- Copy paste the same
.envfile used to setup main server. Change thePORTto other available port numbers. Ex: 5566 docker compose up -d- Open http://localhost:5566
- You can bring the worker container down by
docker compose stop