GD-ML/Peak-End-Net

🎬 Peak-End Net

Peak-End-Net: A Peak-End Rule Inspired Framework for Generalizable Video Aesthetic Assessment

Accepted to ACM Multimedia 2026

📄 Paper · 💻 Code · 🤗 Model

Peak-End Net is a video aesthetic assessment framework inspired by the peak-end rule: people tend to judge an experience disproportionately by its most salient moments and its ending, rather than by uniformly averaging the entire experience. Peak-End Net translates this insight into a learnable temporal model that predicts an overall aesthetic score together with ten fine-grained attribute scores.

✨ Overview

The framework contains five main components:

• Frame Aesthetic Perceiver — a frozen CLIP ViT-L/14 encoder and an AVA-pretrained aesthetic head produce a 10-bin score distribution and an expected aesthetic score for each frame.
• Key Moment Discovery — learnable peak-, valley-, and end-aware signals are combined into a unified temporal attention distribution.
• Peak-End Aggregation — attention-weighted pooling summarizes the frame features into a video-level representation.
• Rhythm Encoder — a multi-scale 1D CNN with kernel sizes 3, 5, and 7 captures local fluctuations and longer-range trends in the frame-score sequence.
• Gated Fusion — a lightweight second-stage module adaptively combines the learned video-level score with the mean frame-level AVA score.

🚀 Pretrained Model and Inference

The self-contained Peak-End-Net.pth checkpoint includes the CLIP ViT-L/14 encoder, AVA aesthetic head, Peak-End modules, and gated-fusion module. No separate AVA or Stage 1 checkpoint is required for inference.

Clone the code repository and install dependencies:

git clone https://github.com/AMAP-ML/Peak-End-Net.git
cd Peak-End-Net

conda create -n peak-end-net python=3.10 -y
conda activate peak-end-net
pip install -r requirements.txt

Download the checkpoint:

hf download GD-ML/Peak-End-Net Peak-End-Net.pth --local-dir ./checkpoints

Run inference on a video:

python inference.py \
    --checkpoint ./checkpoints/Peak-End-Net.pth \
    --video /path/to/video.mp4

The script reports the overall score, ten attribute scores (composition, shotsize, lighting, visualtone, color, depthoffield, expression, movement, costume, makeup), the fusion gate, and the two scores combined by the gate.

🏋️ Training

Full training instructions — pretraining the AVA aesthetic head, Stage 1 (Peak-End Net), and Stage 2 (Gated Fusion) — are provided in the GitHub repository.

📖 Citation

If you find this work useful, please cite:

@misc{li2026peakendnetpeakendruleinspired,
      title={Peak-End-Net: A Peak-End Rule Inspired Framework for Generalizable Video Aesthetic Assessment},
      author={Geng Li and Haiwen Li and Rui Chen and Jing Tang and Lei Sun and Xiangxiang Chu},
      year={2026},
      eprint={2607.13941},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.13941},
}

🤝 Acknowledgments

This project builds on CLIP and CLIP4Clip.

📄 License

This project is released under the MIT License.