feat:add grounded_sam2_tracking_camera_with_continuous_id.py (closes …
## 🔧 feat: add `grounded_sam2_tracking_camera_with_continuous_id.py` (closes #74)
### Overview
This PR introduces a new script and two key methods to support **real-time or incremental frame-by-frame processing** with Grounded-SAM-2. These additions do **not affect** existing functionalities.
#### New Functions
* **`add_new_frame`**:
Adds a new frame to the internal inference state and caches extracted visual features.
This enables real-time frame ingestion from streaming sources such as webcams or robot cameras.
* **`infer_single_frame`**:
Performs segmentation mask inference using the cached visual features and current object prompts.
These functions allow for dynamic input frame processing **without needing to preload all frames into memory**, making it ideal for long videos or live streams.
---
### Motivation
In many robotic and streaming applications, such as real-time perception or autonomous tours, frames are captured sequentially from a camera feed.
The original API required all frames to be preloaded, which is memory-intensive and not feasible for long or continuous video sources.
By decoupling **frame ingestion** and **mask inference**, this design enables:
* ✅ Memory-efficient processing for long-horizon streams
* ✅ Per-frame inference with fast turnaround
* ✅ On-the-fly prompt updates and instance ID tracking
This improvement aligns with the needs of **online perception systems** in robotics and embedded applications.
---
### Usage
To run real-time inference with live video or camera stream:
```bash
python grounded_sam2_tracking_camera_with_continuous_id.py
```
By default, it opens the webcam:
```python
# Open the camera (or replace with a local video file)
cap = cv2.VideoCapture(0)
# or:
cap = cv2.VideoCapture("your_video.mp4")
```
---
### Output
The following outputs will be saved to the `./outputs` directory:
* `mask_data/` – segmentation masks in `.npy` format
* `json_data/` – object metadata in `.json` format
* `result/` – visualization frames with bounding boxes and instance masks
These outputs are useful for downstream analysis, training data generation, or visual debugging.
---
### Real-time Visualization
The script supports **live visualization** using `cv2.imshow()` during execution.
After each frame is processed, the annotated image will be displayed in a pop-up window for monitoring purposes.
Press `q` to quit the window and stop the script:
```python
cv2.imshow("Live Inference", annotated_frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
```
This helps users monitor inference quality and tracked objects in real time.
合并状态:已合并 合并于 2025-05-08 关闭于 2025-05-08 0 条评论