ITADN

Duplicate entries in Prometheus metrics

#7393ClosedThomasHineXYZ 创建于 2026-05-13
help
T
ThomasHineXYZcommented
### 📑 I have found these related issues/pull requests None that I was able to find ### 🛡️ Security Policy - [x] I have read and agree to Uptime Kuma's [Security Policy](https://github.com/louislam/uptime-kuma/security/policy). ### 📝 Description When using the Metrics page with both Prometheus and a custom script, it is reporting multiple of the monitors as being down. With the custom script I have had to assume if one is showing as Up, just assume that it is up. However, I can't exactly do this with Prometheus. It looks like it is being doubled, where the doubled one doesn't have the groups showing. Here is a redacted snippet from the metrics page from my install: ``` monitor_status{production="",teamnetworks="",monitor_id="177",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 0 monitor_status{production="",teamnetworks="",monitor_id="178",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 0 monitor_status{production="",teamnetworks="",monitor_id="179",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 0 monitor_status{production="",teamnetworks="",monitor_id="180",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 0 monitor_status{production="",teamnetworks="",monitor_id="181",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 0 monitor_status{production="",teamnetworks="",monitor_id="182",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 0 monitor_status{production="",teamnetworks="",monitor_id="183",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 0 monitor_status{production="",teamnetworks="",monitor_id="184",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 0 monitor_status{production="",teamnetworks="",monitor_id="185",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 0 monitor_status{production="",teamnetworks="",monitor_id="186",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 0 monitor_status{monitor_id="191",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 1 monitor_status{monitor_id="174",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 1 monitor_status{monitor_id="178",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 1 monitor_status{monitor_id="179",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 1 monitor_status{monitor_id="180",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 1 monitor_status{monitor_id="181",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 1 monitor_status{monitor_id="182",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 1 monitor_status{monitor_id="177",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 1 monitor_status{monitor_id="183",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 1 monitor_status{monitor_id="184",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 1 monitor_status{monitor_id="185",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 1 monitor_status{monitor_id="186",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 1 monitor_status{monitor_id="176",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 1 monitor_status{production="",teamnetworks="",monitor_id="191",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 0 monitor_status{production="",teamenterprise="",monitor_id="190",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 0 monitor_status{production="",teamenterprise="",monitor_id="187",monitor_name="[REDACTED]",monitor_type="push",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 0 monitor_status{monitor_id="107",monitor_name="[REDACTED]",monitor_type="group",monitor_url="https://",monitor_hostname="null",monitor_port="null"} 1 ``` Here is a screenshot showing the comparison between the Uptime Kuma dashboard and a [Grafana dashboard](https://grafana.com/grafana/dashboards/18667-uptime-kuma-metrics/): <img width="1673" height="680" alt="Image" src="https://github.com/user-attachments/assets/ca1165e4-50d8-4200-b9c0-fb7fedcec201" /> I haven't run in to this issue with a second install of Uptime Kuma that I run that only has 83 monitors. The script I wrote helped me diagnose the issue. I did try the official Python library for reading a metrics endpoint, and ended up with the same issue. Here's part of the Python code that I put together just in case that helps anything. ```python def grab_metrics(): """Grabs the data from the Uptime Kuma metrics endpoint Returns: list: A list of dictionaries of all the monitors """ try: url = f"{kuma_server_url}/metrics" basic_auth = HTTPBasicAuth("api", kuma_api_key) response = requests.get(url, auth=basic_auth, timeout=10) metrics = {} # Iterate through every line of the response and grab the desired lines and # convert them in to a dictionary monitor_status_list = {} for line in response.text.splitlines(): # Remove leading or trailing whitespace, just in case line = line.strip() parsed_line = parse_prometheus_line(line) # monitor_status lines # Creates a dictionary of all the monitor_status lines, with the monitor_id # as the key and the rest of the data as the value if line.startswith("monitor_status"): # Check if the monitor is already in the list, if it is # log a debug message do some logic to determine whether 0 or 1 if parsed_line["labels"]["monitor_id"] in monitor_status_list: logging.debug( f"Duplicate monitor_id {parsed_line['labels']['monitor_id']} found, skipping." ) # NOTE: I don't like this fix, but it works for the time being # Check if the new status is 1, and overwrite the old status if it is, otherwise keep the old status if parsed_line["value"] == 1: monitor_status_list[parsed_line["labels"]["monitor_id"]][ "status" ] = 1 else: monitor_status_list[parsed_line["labels"]["monitor_id"]] = ( parsed_line["labels"] ) monitor_status_list[parsed_line["labels"]["monitor_id"]][ "status" ] = parsed_line["value"] metrics["monitor_status"] = monitor_status_list logging.debug(f"Metrics: {metrics}") return metrics ``` ### 👟 Reproduction steps Unsure, but even after multiple restarts of the container and even the host OS, the issue still occurs. ### 👀 Expected behavior Prometheus reads the metrics API, and reports back the same data that is being reported on the Uptime Kuma dashboard. ### 😓 Actual Behavior Prometheus (and custom script) reads services being offline when they aren't ### 🐻 Uptime-Kuma Version 2.2.1 ### 💻 Operating System and Arch Red Hat 8.10 ### 🌐 Browser Mozilla Firefox 140.10.2esr ### 🖥️ Deployment Environment - **Runtime Environment**: - Docker: Version `26.1.3, build b72abbb` - **Database**: - SQLite: Embedded - **Database Storage**: - **Filesystem**: - Linux: ext4 - **Storage Medium**: SSD - **Uptime Kuma Setup**: - Number of monitors: `177` ### 📝 Relevant log output ```bash session ```
关闭于 2026-05-13 4 条评论