ITADN

feat:add grounded_sam2_tracking_camera_with_continuous_id.py (closes …

#97Pull RequestHITSZ-Robotics 创建于 2025-05-06已合并
H
HITSZ-Roboticscommented
## 🔧 feat: add `grounded_sam2_tracking_camera_with_continuous_id.py` (closes #74) ### Overview This PR introduces a new script and two key methods to support **real-time or incremental frame-by-frame processing** with Grounded-SAM-2. These additions do **not affect** existing functionalities. #### New Functions * **`add_new_frame`**: Adds a new frame to the internal inference state and caches extracted visual features. This enables real-time frame ingestion from streaming sources such as webcams or robot cameras. * **`infer_single_frame`**: Performs segmentation mask inference using the cached visual features and current object prompts. These functions allow for dynamic input frame processing **without needing to preload all frames into memory**, making it ideal for long videos or live streams. --- ### Motivation In many robotic and streaming applications, such as real-time perception or autonomous tours, frames are captured sequentially from a camera feed. The original API required all frames to be preloaded, which is memory-intensive and not feasible for long or continuous video sources. By decoupling **frame ingestion** and **mask inference**, this design enables: * ✅ Memory-efficient processing for long-horizon streams * ✅ Per-frame inference with fast turnaround * ✅ On-the-fly prompt updates and instance ID tracking This improvement aligns with the needs of **online perception systems** in robotics and embedded applications. --- ### Usage To run real-time inference with live video or camera stream: ```bash python grounded_sam2_tracking_camera_with_continuous_id.py ``` By default, it opens the webcam: ```python # Open the camera (or replace with a local video file) cap = cv2.VideoCapture(0) # or: cap = cv2.VideoCapture("your_video.mp4") ``` --- ### Output The following outputs will be saved to the `./outputs` directory: * `mask_data/` – segmentation masks in `.npy` format * `json_data/` – object metadata in `.json` format * `result/` – visualization frames with bounding boxes and instance masks These outputs are useful for downstream analysis, training data generation, or visual debugging. --- ### Real-time Visualization The script supports **live visualization** using `cv2.imshow()` during execution. After each frame is processed, the annotated image will be displayed in a pop-up window for monitoring purposes. Press `q` to quit the window and stop the script: ```python cv2.imshow("Live Inference", annotated_frame) if cv2.waitKey(1) & 0xFF == ord('q'): break ``` This helps users monitor inference quality and tracked objects in real time.
合并状态:已合并 合并于 2025-05-08 关闭于 2025-05-08 0 条评论