Get inference endpoint model settings via client

### Feature request Enable commands via clients such as `OpenAI` that would get model settings from an inference endpoint. Does this exist and I just can't find it? ### Motivation There is currently no clear way to get inference model settings directly from an endpoint. Individual base models have their original settings, but this does not necessarily translate to an endpoint. As an example, [Microsoft's Phi-3 model](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct) supports 128k context length as input, but if instantiated as an endpoint on a 24GB gpu the allowed input context length is less (48k). The only way I have found to access the information regarding an individual endpoint is via `huggingface_hub`, specifically: ``` from huggingface_hub import get_inference_endpoint endpoint = get_inference_endpoint(ENDPOINT_NAME, namespace=USERNAME, token=api_key) ``` To get the general settings, you can then access the `raw` dict of the endpoint's image. For example, if I want to get the context length of a specific model at an endpoint, I can do it this way: ``` # the settings/specs of the endpoint in a 'llamacpp' image settings = endpoint.raw['model']['image']['llamacpp'] # this allows me to get info like context length (via the ['ctxSize']) key >>> print(settings['ctxSize']) 48000 ``` This is problematic when sending prompts to an endpoint - if it were easier to query model properties programmatically, then I could write code to adjust queries on the fly appropriately depending on the target model. As it is, the sender needs to know the properties of a particular endpoint beforehand. IMO what is needed is to be able to get this info directly from a client. In the OpenAI client in the Huggingface Inference API there seems to be some functionality for this, i.e. I can instantiate a client: ``` client = OpenAI( base_url=endpoint, # AWS/server URL api_key=api_key, # huggingface token ) ``` Then I can get a list of models at that url: ``` print(client.models.list()) ``` But this only prints out basic information, which doesn't include such things as context length. Is there a way to get this info from the client that I'm just missing? I have noticed when there are errors related to input length, the client returns an error with the key `n_ctx`. For example, if a model I'm working with has a 12k context window and I send 13k tokens, the error is: ``` openai.BadRequestError: Error code: 400 - {'error': {'code': 400, 'message': 'the request exceeds the available context size, try increasing it', 'type': 'exceed_context_size_error', 'n_prompt_tokens': 13954, 'n_ctx': 12032}} ``` This tells me that the client has access to the overall settings, but it's not clear to me how to get them. ### Your contribution Happy to work on this if someone can point me where to look for relevant code that would pass inference endpoint settings info to the client, perhaps via the `client.models.list()` method.