Keycloak - Infinite loop

## An infinite loop causes too many requests to be sent, which makes the Keycloak module to crash. ### Expected behavior The infinite loop should not exist so cartography doesn't crash. ### Actual behavior Cartography is trying to get paginated results from a REST endpoint that does not support it. The condition to stop can never been met depending on the Keycloak realm population. ### To Reproduce uv run cartography --selected-modules keycloak --keycloak-client-id your-client-id --keycloak-client-secret-env-var KEYCLOAK_CLIENT_SECRET --keycloak-url your-instance-url You need a keycloak realm with more than 100 client-scopes. ### Logs ``` INFO:cartography.intel.keycloak.identityproviders:Loading 1 Keycloak IdentityProviders (<realm>) into Neo4j. INFO:cartography.client.core.tx:Loaded 1 KeycloakIdentityProvider nodes INFO:cartography.graph.statement:Completed KeycloakIdentityProvider statement #1 INFO:cartography.graph.statement:Completed KeycloakIdentityProvider statement #2 INFO:cartography.graph.statement:Completed KeycloakIdentityProvider statement #3 INFO:cartography.graph.job:Finished job KeycloakIdentityProvider ERROR:cartography.sync:Unhandled exception during sync stage 'keycloak' Traceback (most recent call last): File "/home/anon/Documents/gitRepo/cartography/.venv/lib/python3.10/site-packages/urllib3/connection.py", line 204, in _new_conn sock = connection.create_connection( File "/home/anon/Documents/gitRepo/cartography/.venv/lib/python3.10/site-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): File "/home/anon/snap/code/226/.local/share/uv/python/cpython-3.10.20-linux-x86_64-gnu/lib/python3.10/socket.py", line 967, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -3] Temporary failure in name resolution The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/anon/Documents/gitRepo/cartography/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( File "/home/anon/Documents/gitRepo/cartography/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/home/anon/Documents/gitRepo/cartography/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) File "/home/anon/Documents/gitRepo/cartography/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() File "/home/anon/Documents/gitRepo/cartography/.venv/lib/python3.10/site-packages/urllib3/connection.py", line 759, in connect self.sock = sock = self._new_conn() File "/home/anon/Documents/gitRepo/cartography/.venv/lib/python3.10/site-packages/urllib3/connection.py", line 211, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: HTTPSConnection(host='<keycloak-instance>', port=443): Failed to resolve '<keycloak-instance>' ([Errno -3] Temporary failure in name resolution) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/anon/Documents/gitRepo/cartography/.venv/lib/python3.10/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( File "/home/anon/Documents/gitRepo/cartography/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( File "/home/anon/Documents/gitRepo/cartography/.venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 535, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='<keycloak-instance>', port=443): Max retries exceeded with url: **/admin/realms/<realm>/client-scopes?briefRepresentation=False&first=923440&max=100** (Caused by NameResolutionError("HTTPSConnection(host='<keycloak-instance>', port=443): Failed to resolve '<keycloak-instance>' ([Errno -3] Temporary failure in name resolution)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/anon/Documents/gitRepo/cartography/cartography/sync.py", line 244, in run stage_func(neo4j_session, config) File "/home/anon/Documents/gitRepo/cartography/cartography/util.py", line 471, in timed return method(*args, **kwargs) File "/home/anon/Documents/gitRepo/cartography/cartography/intel/keycloak/__init__.py", line 85, in start_keycloak_ingestion scopes = cartography.intel.keycloak.scopes.sync( File "/home/anon/Documents/gitRepo/cartography/cartography/util.py", line 471, in timed return method(*args, **kwargs) File "/home/anon/Documents/gitRepo/cartography/cartography/intel/keycloak/scopes.py", line 25, in sync scopes = get( File "/home/anon/Documents/gitRepo/cartography/cartography/util.py", line 471, in timed return method(*args, **kwargs) File "/home/anon/Documents/gitRepo/cartography/cartography/intel/keycloak/scopes.py", line 47, in get return list(get_paginated(api_session, url, params={"briefRepresentation": False})) File "/home/anon/Documents/gitRepo/cartography/cartography/intel/keycloak/util.py", line 39, in get_paginated req = api_session.get(endpoint, params=payload, timeout=_TIMEOUT) File "/home/anon/Documents/gitRepo/cartography/.venv/lib/python3.10/site-packages/requests/sessions.py", line 602, in get return self.request("GET", url, **kwargs) File "/home/anon/Documents/gitRepo/cartography/.venv/lib/python3.10/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) File "/home/anon/Documents/gitRepo/cartography/.venv/lib/python3.10/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) File "/home/anon/Documents/gitRepo/cartography/.venv/lib/python3.10/site-packages/requests/adapters.py", line 677, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='<keycloak-instance>', port=443): Max retries exceeded with url: /admin/realms/<realm>/client-scopes?briefRepresentation=False&first=923440&max=100 (Caused by NameResolutionError("HTTPSConnection(host='<keycloak-instance>', port=443): Failed to resolve '<keycloak-instance>' ([Errno -3] Temporary failure in name resolution)")) ERROR:cartography.cli:Cartography failed: HTTPSConnectionPool(host='<keycloak-instance>', port=443): Max retries exceeded with url: /admin/realms/<realm>/client-scopes?briefRepresentation=False&first=923440&max=100 (Caused by NameResolutionError("HTTPSConnection(host='<keycloak-instance>', port=443): Failed to resolve '<keycloak-instance>' ([Errno -3] Temporary failure in name resolution)")) ``` ### Screenshots > If applicable, add screenshots to help explain your problem. <img width="151" height="518" alt="Image" src="https://github.com/user-attachments/assets/2a59a6f6-c7dc-4597-9c38-b0717c77ff6d" /> Data being always the same size because of the one time return of the endpoint. ### Please complete the following information: > - Cartography release version or commit hash [e.g. 0.12.0 or 95e8e11913e2a44a4d4682506d8364a638ceac69] cartography release 0.133.1.dev11+g2ea34287b > - Python version: [e.g. 3.10.0] Python 3.10.20 > - OS (feel free to omit this if you don't think it's relevant to your issue): [e.g. Ubuntu bla bla, OSX bla bla] ### Additional context > Add any other context about the problem here. Keycloak admin API endpoint `/admin/realms/{realm}/client-scopes` ([ref](https://github.com/cartography-cncf/cartography/blob/9b19e0054df6ce4523876964dfbe23b973816583/cartography/intel/keycloak/scopes.py#L46) does not support '**first**' and '**max**' params : [Official documentation](https://www.keycloak.org/docs-api/latest/rest-api/index.html#_get_adminrealmsrealmclient_scopes) Using **get_paginated()** causes the data length to never get lower than 100 (items_per_page), because the full list of client-scopes is sent at every request, which causes the infinite loop followed by the cartography crash. Editing ` return list(get_paginated(api_session, url, params={"briefRepresentation": False})) ` To `req = api_session.get(url, params={"briefRepresentation": False}, timeout=_TIMEOUT) req.raise_for_status() return req.json() ` Fixes the issue.