Adversarial Video Moment Retrieval by Jointly Modeling Ranking and Localization (AVMR)

This is our implementation for the paper:

Da Cao, Yawen Zeng, Xiaochi Wei, Liqiang Nie, Richang Hong, Zheng Qin. 2020. Adversarial Video Moment Retrieval by Jointly Modeling Ranking and Localization. In The ACM International Conference on Multimedia (ACM MM '20). ACM, Seattle, United States.

Environment Settings

We use the framework pytorch.

• pytorch version: '1.2.0'
• python version: '3.5'

AVMR

The released code consists of the following files.

--workplaceTest
--workplaceTrain
--feature_all
--main.py
--MADDPG.py
--model.py
--memory.py
--utils.py
--randomProcess.py
--readme

Example to run the codes

Run AVMR：

python main.py

Example demo

AVMR_demo_YVKIV.mp4

The visualization of five generated video moments on five iterations is revealed above. The green line and green box indicate the ground truth, the black line and black box represent the generated video moment candidates.

Dataset

We provide two processed datasets: Charades-STA && TACoS The strategy of multi-scale sliding windows is utilized to segment each video with the size of [64, 128, 256, 512] frames with 80% overlap and we randomly selected 80% and 20% of them for training and testing, respectively.

All features are saved in ./feature_all_train, ./feature_all_test.

• These two processed features are available for downloading here: https://drive.google.com/open?id=1-AMToMuTlPRY1C2n0ZoyKrwBsVPehbFK
• The original videos and their corresponding caption annotations/querys: https://github.com/jiyanggao/TALL and http://www.coli.uni-saarland.de/projects/smile/tacos

Workspace

/workplaceTest and /workplaceTrain that processes videos in advance as a runtime workspace.

Last Update Date: Jul 28, 2020