Try to run 2 Ollam

Thank you for your amazing work. I am trying to use two different OLLama models locally as strong (Phi4) and weak (Llama3 8b). I can run both models using method A which is also known as python sdk. But I want to try using method B.My method B never worked. This is my input `python -m routellm.openai_server --routers mf --config config.example.yaml --base-url http://localhost:11434/v1/ --api-key ollama --strong-model phi4 --weak-model llama3` Also I change the router chat `client = OpenAI( api_key=openai_api_key, base_url=openai_api_base, ) ` When I run it in CMD it shows `INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:6060 (Press CTRL+C to quit)` After that I using `python -m examples.router_chat --router mf --threshold 0.11593` to start the server However when I input anything in server it shows `Provider List: https://docs.litellm.ai/docs/providers INFO: 127.0.0.1:53951 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last): File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 403, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\uvicorn\middleware\proxy_headers.py", line 60, in __call__ return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\fastapi\applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\applications.py", line 113, in __call__ await self.middleware_stack(scope, receive, send) File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\middleware\errors.py", line 187, in __call__ raise exc File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\middleware\errors.py", line 165, in __call__ await self.app(scope, receive, _send) File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\middleware\exceptions.py", line 62, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app raise exc File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\routing.py", line 715, in __call__ await self.middleware_stack(scope, receive, send) File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\routing.py", line 735, in app await route.handle(scope, receive, send) File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\routing.py", line 288, in handle await self.app(scope, receive, send) File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\routing.py", line 76, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app raise exc File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\routing.py", line 73, in app response = await f(request) ^^^^^^^^^^^^^^^^ File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\fastapi\routing.py", line 301, in app raw_response = await run_endpoint_function( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\fastapi\routing.py", line 212, in run_endpoint_function return await dependant.call(**values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\dzm10\OneDrive\大学\CSC490\RouteLLM-main\routellm\openai_server.py", line 124, in create_chat_completion res = await CONTROLLER.acompletion( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\dzm10\OneDrive\大学\CSC490\RouteLLM-main\routellm\controller.py", line 170, in acompletion return await acompletion(api_base=self.api_base, api_key=self.api_key, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\litellm\utils.py", line 1241, in wrapper_async raise e File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\litellm\utils.py", line 1093, in wrapper_async result = await original_function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\litellm\main.py", line 426, in acompletion _, custom_llm_provider, _, _ = get_llm_provider( ^^^^^^^^^^^^^^^^^ File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\litellm\litellm_core_utils\get_llm_provider_logic.py", line 356, in get_llm_provider raise e File "C:\Users\dzm10\AppData\Local\Programs\Python\Python312\Lib\site-packages\litellm\litellm_core_utils\get_llm_provider_logic.py", line 333, in get_llm_provider raise litellm.exceptions.BadRequestError( # type: ignore litellm.exceptions.BadRequestError: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=llama3 Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers` But at the same time, even if I follow the readme and put gbt4 and llama3 together, it doesn't work. Do you have any solutions? Thanks