|
Gang YU (俞刚)
I am a Principal Research Scientist at StepFun
(阶跃星辰). My research interests focus on the computer vision and artificical
intelligence, specifically on the topic of generative AI, object detection,
segmentation, human keypoint, human action recognition, and 3D reconstruction. I
obtained PhD from NTU in 2014
supervised by Prof. Junsong Yuan. Before
joining StepFun, I worked as a research director at Tencent for four years and another five years at Megvii (Face++).
Google Scholar / CV / Zhihu |
|
|
News
Five papers accepted by CVPR2026. Six papers accepted by ICLR2026. I will serve as an Area Chair for CVPR2026. Five papers accepted by Neurips2025. Three papers accepted by ICCV2025. Two papers accepted by CVPR2025. I will serve as an Area Chair for CVPR2025. One paper accepted by Neurips2024. Two papers accepted by ECCV2024. Two papers accepted by CVPR2024. I will serve as an Innovation Program (Industry) Chair for ICME2024. I will serve as an Area Chair for CVPR2024. Three papers accepted by NeurIPS2023. Three papers accepted by ICCV2023. Three papers accepted by CVPR2023. We have organized a tutorial Mobile Visual Analytics in CVPR2021. Our team obtained the first place of Mobile AI Challenge in the Depth Estimation Challenge (CVPR 2021). We have organized a tutorial Human Pose Estimation and Action Recognition [Skeleton, Action] (ICIP 2019). We have organized a tutorial Object Detection in Recent Three Years [Detection, AutoML, Fine-Grained] (ICME 2019). We organized the Detection In the Wild (DIW2019) Challenge in CVPR2019. Our team obtained the first place of nuScenes 3D Detection and BDD100K & D²-City Detection Domain Adaptation in the Workshop on Autonomous Driving (CVPR 2019). Our team obtained the first place of COCO Detecction, COCO Keypoint Detection, COCO Panoptic Segmentation, Mapillary Panoptic Segmentation (four Champions) in the COCO + Mapillary Joint Challenge. ChinaMedia 1 ChinaMedia 2(ECCV 2018). Our team obtained the first place of WiderFace Detecction in the Wider Challenge (ECCV 2018). Our team obtained the first place of Video Instance Segmentation [Report][Slides] in the WAD2018 (Workshop on Autonomous Driving) Challenge (CVPR 2018). Our team obtained the first place of AVA [Report][Slides] and second place of Moments in time [Report] in the ActivityNet2018 Challenge (CVPR 2018). Presentation: Beyond RetinaNet and Mask R-CNN, Jiangmen (将门), 2018 Presentation: Introduction to Object Detection, PKU & CAS, 2018
Our team obtained the first place of COCO 2017 Challenge (Detection
Track & Keypoint
Track) (ICCV 2017). |
| Pre-prints |
|
Step-Audio-R1 Technical
Report |
|
Step-Audio-EditX Technical
Report |
|
Step-Audio 2 Technical
Report |
|
Step1X-Edit: A Practical
Framework for General Image Editing
|
|
Step-Video-T2V Technical
Report: The Practice, Challenges, and Future of Video Foundation Model
|
|
AppAgent: Multimodal
Agents as Smartphone Users |
| Conference |
|
ReasonEdit: Towards
Reasoning-Enhanced Image Editing Models
|
|
OmniLottie: Generating Vector Animations via Parameterized
Lottie Tokens
|
|
ViStoryBench:
Comprehensive Benchmark Suite for Story Visualization
|
|
Dropping Anchor and Spherical Harmonics for Gaussian
Splatting
|
|
iMontage: Unified,
Versatile, Highly Dynamic Many-to-many Image Generation
|
|
RegionE: Adaptive
Region-Aware Generation for Efficient Image Editing
|
|
WithAnyone: Toward
Controllable and ID Consistent Image Generation
|
|
Training-Free Text-Guided
Color Editing with Multi-Modal Diffusion Transformer
|
|
LazyDrag: Enabling Stable
Drag-Based Editing on Multi-Modal Diffusion Transformers via Explicit
Correspondence
|
|
IGGT: Instance-Grounded
Geometry Transformer for Semantic 3D Reconstruction
|
|
SpeakerVid-5M: A
Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human
Generation
|
|
Sparse-vDiT: Unleashing
the Power of Sparse Attention to Accelerate Video Diffusion Transformers
|
|
OmniSVG: A Unified
Scalable Vector Graphics Generation Model
|
|
Vision Foundation Models
as Effective Visual Tokenizers for Autoregressive Generation |
|
KRIS-Bench: Benchmarking
Next-Level Intelligent Image Editing Models
|
|
FAVOR-Bench: A
Comprehensive Benchmark for Fine-Grained Video Motion Understanding
|
|
OneIG-Bench:
Omni-dimensional Nuanced Evaluation for Image Generation
|
|
MeshAnything:
Artist-Created Mesh Generation with Autoregressive Transformers |
|
MotionAgent: Fine-grained
Controllable Video Generation via Motion Field Agent |
|
MikuDance: Animating
Character Art with Mixed Motion Dynamics
|
|
SC-Captioner: Improving
Image Captioning with Self-Correction by Reinforcement Learning
|
|
DeRS: Towards Extremely
Efficient Upcycled Mixture-of-Experts Models
|
|
MVPaint: 3D Texture
Generation with Multi-View Consistency
|
|
MeshXL: Neural Coordinate
Field for Generative 3D Foundation Models
|
|
M3DBench: Towards Omni 3D
Assistant with Interleaved Multi-modal Instructions |
|
MotionChain:
Conversational Motion Controllers via Multimodal Prompts
|
|
Generative Motion
Stylization of Cross-structure Characters within Canonical Motion Space
|
|
Enhanced Visual
Instruction Tuning with Synthesized Image-Dialogue Data |
|
Paint3D: Paint Anything 3D
with Lighting-Less Texture Diffusion Models |
|
LL3DA: Visual Interactive
Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning
|
|
TapMo: Shape-aware Motion
Generation of Skeleton-free Characters |
|
IT3D: Improved Text-to-3D
Generation with Explicit View Synthesis |
|
PM-INR: Prior-Rich Multi-Modal Implicit Large-Scale Scene Neural
Representation |
|
MotionGPT: Human Motion as
a Foreign Language
|
|
Michelangelo: Conditional
3D Shape Generation based on Shape-Image-Text Aligned Latent Representation
|
|
PDF: Point Diffusion Implicit Function for Large-scale
Scene Neural Representation
|
|
A Large-Scale Outdoor
Multi-modal Dataset and Benchmark for Novel View Synthesis and Implicit Scene
Reconstruction
|
|
Metric3D: Towards
Zero-shot Metric 3D Prediction from A Single Image
|
|
Robust Geometry-Preserving Depth Estimation Using
Differentiable Rendering
|
|
Executing your Commands
via Motion Diffusion in Latent Space
|
|
STAR Loss: Reducing
Semantic Ambiguity in Facial Landmark Detection
|
|
End-to-End 3D Dense
Captioning with Vote2Cap-DETR
|
|
Capturing the motion
of every joint: 3D human pose and shape estimation with independent tokens
|
|
SeaFormer:
Squeeze-enhanced Axial Transformer for Mobile Semantic Segmentation
|
|
Hierarchical
Normalization for Robust Monocular Depth Estimation |
|
Coordinates Are NOT
Lonely - Codebook Prior Helps Implicit Neural 3D Representations |
|
D&D: Learning Human
Dynamics from Dynamic Camera |
|
TopFormer: Token
Pyramid Transformer for Mobile Semantic Segmentation |
|
Object-aware
Long-short-range Spatial Alignment for Few-Shot Fine-Grained Image
Classification |
|
Attribute-specific
Control Units in StyleGAN for Fine-grained Image Manipulation
|
|
State-Aware Tracker for
Real-Time Video Object Segmentation |
|
High-Order Information
Matters: Learning Relation and Topology for Occluded Person
Re-Identification |
|
Context Prior for Scene
Segmentation |
|
SiamFC++: Towards Robust
and Accurate Visual Tracking with Target Estimation Guidelines |
|
Learnable Tree Filter for
Structure-preserving Feature Transform |
|
ThunderNet: Towards
Real-time Generic Object Detection |
|
Efficient and
Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network
|
|
Objects365: A Large-scale,
High-quality Dataset for Object Detection |
|
An End-to-end Network
for Panoptic Segmentation |
|
TACNet: Transition-Aware Context
Network for Spatio-Temporal Action Detection |
|
Modeling Local
Geometric Structure of 3D Point Clouds using Geo-CNN |
|
Shape Robust Text
Detection with Progressive Scale Expansion Network |
|
Scene Text Detection
with Supervised Pyramid Context Network |
|
Attention-based
Multi-Context Guiding for Few-Shot Semantic Segmentation
|
|
DetNet: A Backbone
network for Object
Detection
|
|
BiSeNet: Bilateral
Segmentation Network for Real-time Semantic Segmentation
|
|
Associating
Inter-Image Salient Instances forWeakly Supervised Semantic Segmentation
|
|
MegDet: A Large Mini-Batch Object
Detector |
|
Cascaded Pyramid Network for
Multi-Person Pose Estimation [Code] |
|
Learning a Discriminative Feature
Network for Semantic Segmentation |
|
R-FCN++: Towards Accurate
Region-based Fully Convolutional Networks for Object Detection |
|
Large Kernel Matters -- Improve
Semantic Segmentation by Global Convolutional Network |
|
Fast
Action Proposals for Human Action Detection and Search |
|
Discriminative
Orderlet Mining For Real-time Recognition of Human-Object Interaction
[Project]
|
|
Scalable
Forest Hashing for Fast Similarity Search
|
|
Propagative
Hough Voting for Human Activity Recognition
|
|
Randomized
Spatial Partition for Scene Recognition
|
|
Predicting
Human Activities using Spatio-Temporal Structure of Interest Points
|
|
Unsupervised
Random Forest Indexing for Fast Action Search
|
|
Real-time
HumanAction Search using Random Forest based Hough Voting
|
| Journal |
|
Lightweight Model
Pre-Training Via Language Guided Knowledge Distillation |
|
Vote2cap-detr++:
Decoupling localization and describing for end-to-end 3d dense
captioning |
|
BiSeNet V2: Bilateral
Network with Guided Aggregation for Real-time Semantic Segmentation |
|
Propagative
Hough Voting for Human Activity Detection and Recognition |
|
Action
Search by Example using Randomized Visual Vocabularies |
|
Fast
Action Detection via Discriminative Random Forest Voting and Top-K Subvolume
Search |
| Book |
|
Human
Action Analysis with Randomized Trees |