Fix: align the source frame and first driving frame, fix the bad case cause by misalign.
**Problem Description:**
- When use i2v, if the source frame is not align with the first driven video frame, it will raise bad case. For example, if the first frame is mouth-openning, and the first frame is moutyh-close, when trying to use the driven-video driven the source frame, the result attemps to cloth mouth. This is a common issue.
**Workaround:**
- Try to align the source frame to driven-video first frame. Base on the driven-video first frame's mouth ration, we can use the warpper function to warp the source mouth ot be same as the driven-video first frame.
- try to modify the code in https://github.com/KwaiVGI/LivePortrait/blob/aa749159bda43b458349bcd978c007d51ca6659e/src/live_portrait_pipeline.py#L259 and https://github.com/KwaiVGI/LivePortrait/blob/aa749159bda43b458349bcd978c007d51ca6659e/src/live_portrait_pipeline.py#L288
- Result comapre:
- When don't do normalize
https://github.com/user-attachments/assets/ecf28474-9383-4378-8771-22b75dab4c93
- Do normalized as modified
https://github.com/user-attachments/assets/cfc6f7d5-533e-4a54-a785-e4275ab73de8
- Do normalize as original, warp source base on mouth_ratio=0
https://github.com/user-attachments/assets/6a35bdeb-60ee-4376-bd9d-09275e0fb66b
\
I am not'sure if this can be a workaround to break the limited in `Make sure the first frame of driving video is a frontal face with neutral expression.` mentioned in https://github.com/KwaiVGI/LivePortrait?tab=readme-ov-file#driving-video-auto-cropping-
合并状态:未合并 0 条评论