Skip to content

Conversation

@blanch0t
Copy link
Collaborator

  • feat: add user priority management and integrate Celery for task execution
  • add new metrics for observability
  • wip 1
  • wip 2

Julien Almarcha and others added 4 commits October 24, 2025 15:08
…ution

- Introduced a new `priority` field in the User model to manage user priorities for scheduling and rate limiting.
- Updated the IdentityAccessManager to handle user priority during user creation and updates.
- Enhanced the BaseModelRouter to implement a least-loaded routing strategy for model invocation, selecting the provider with the fewest inflight requests.
- Added Celery configuration for asynchronous task execution, allowing model invocations to be processed in the background.
- Implemented a new `invoke_model_request` function to handle both eager and Celery-based invocations based on configuration.
- Updated API schemas to include priority in user-related requests and responses.
- Added necessary database migrations to support the new priority field in the User table.
- Enhanced unit tests to cover new functionality and ensure proper integration with Celery.

fix iam tests

update config to run test without workers

Update coverage badge

Update configuration documentation

update alembic

add least loaded router

Add documentation

Update diagram

Refactor Redis connection handling for thread safety and performance

Refactor chat completions to use invoke_model_request for non-streaming responses

fix : base model router lock

fix: lock release

feat: add additional_data parameter to invoke_model_request and invoke_model_task

fix: handle SoftTimeLimitExceeded in invoke_model_task for structured response

refactor: enhance streaming response handling with safe client access routing

fix: ensure client lock is released properly in delete_client method

aristote dispatcher philosophy (untested

fix celery task

fix round robin

fix receive task result

task failure handling

fix requeuing

remarques pr

tests

simplify celery task by removing async
@leoguillaume leoguillaume changed the title priorite locale Model provider first (aka local priority) Nov 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants