|
Gang YU (俞刚)
I am a Principal Research Scientist at StepFun (阶跃星辰). My research interests focus on the computer vision and artificical intelligence, specifically on the topic of generative AI, object detection, segmentation, human keypoint, human action recognition, and 3D reconstruction. I obtained PhD from NTU in 2014 supervised by Prof. Junsong Yuan. Before joining StepFun, I worked as a research director at Tencent for four years and another five years at Megvii (Face++).
Google Scholar / CV / Zhihu |
|
|
News
I will serve as an Area Chair for CVPR2025. One paper accepted by Neurips2024. Two papers accepted by ECCV2024. Two papers accepted by CVPR2024. I will serve as an Innovation Program (Industry) Chair for ICME2024. I will serve as an Area Chair for CVPR2024. Three papers accepted by NeurIPS2023. Three papers accepted by ICCV2023. Three papers accepted by CVPR2023. We have organized a tutorial Mobile Visual Analytics in CVPR2021. Our team obtained the first place of Mobile AI Challenge in the Depth Estimation Challenge (CVPR 2021). We have organized a tutorial Human Pose Estimation and Action Recognition [Skeleton, Action] (ICIP 2019). We have organized a tutorial Object Detection in Recent Three Years [Detection, AutoML, Fine-Grained] (ICME 2019). We organized the Detection In the Wild (DIW2019) Challenge in CVPR2019. Our team obtained the first place of nuScenes 3D Detection and BDD100K & D²-City Detection Domain Adaptation in the Workshop on Autonomous Driving (CVPR 2019). Our team obtained the first place of COCO Detecction, COCO Keypoint Detection, COCO Panoptic Segmentation, Mapillary Panoptic Segmentation (four Champions) in the COCO + Mapillary Joint Challenge. ChinaMedia 1 ChinaMedia 2(ECCV 2018) . Our team obtained the first place of WiderFace Detecction in the Wider Challenge (ECCV 2018). Our team obtained the first place of Video Instance Segmentation [Report][Slides] in the WAD2018 (Workshop on Autonomous Driving) Challenge (CVPR 2018). Our team obtained the first place of AVA [Report][Slides] and second place of Moments in time [Report] in the ActivityNet2018 Challenge (CVPR 2018). Presentation: Beyond RetinaNet and Mask R-CNN, Jiangmen (将门), 2018 Presentation: Introduction to Object Detection, PKU & CAS, 2018
Our team obtained the first place of COCO 2017 Challenge (Detection Track & Keypoint Track) (ICCV 2017). |
| Pre-prints |
|
AppAgent: Multimodal Agents as Smartphone Users |
|
ChartLlama: A Multimodal LLM for Chart Understanding and Generation |
|
FaceStudio: Put Your Face Everywhere in Seconds |
|
ShapeGPT: 3D Shape Generation with A Unified Multi-modal Language Model |
|
Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer |
| Conference |
|
MeshXL: Neural Coordinate Field for Generative 3D Foundation Models
|
|
M3DBench: Towards Omni 3D Assistant with Interleaved Multi-modal Instructions |
|
MotionChain: Conversational Motion Controllers via Multimodal Prompts
|
|
Generative Motion Stylization of Cross-structure Characters within Canonical Motion Space
|
|
Enhanced Visual Instruction Tuning with Synthesized Image-Dialogue Data |
|
Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models |
|
LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning |
|
TapMo: Shape-aware Motion Generation of Skeleton-free Characters |
|
IT3D: Improved Text-to-3D Generation with Explicit View Synthesis |
|
PM-INR: Prior-Rich Multi-Modal Implicit Large-Scale Scene Neural
Representation |
|
MotionGPT: Human Motion as a Foreign Language
|
|
Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representation
|
|
PDF: Point Diffusion Implicit Function for Large-scale Scene Neural Representation
|
|
A Large-Scale Outdoor Multi-modal Dataset and Benchmark for Novel View Synthesis and Implicit Scene Reconstruction
|
|
Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image
|
|
Robust Geometry-Preserving Depth Estimation Using Differentiable Rendering
|
|
Executing your Commands via Motion Diffusion in Latent Space
|
|
STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detection
|
|
End-to-End 3D Dense Captioning with Vote2Cap-DETR
|
|
Capturing the motion of every joint: 3D human pose and shape estimation with independent tokens
|
|
SeaFormer: Squeeze-enhanced Axial Transformer for Mobile Semantic Segmentation
|
|
Hierarchical Normalization for Robust Monocular Depth Estimation |
|
Coordinates Are NOT Lonely - Codebook Prior Helps Implicit Neural 3D Representations |
|
D&D: Learning Human Dynamics from Dynamic Camera |
|
TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation |
|
Object-aware Long-short-range Spatial Alignment for Few-Shot Fine-Grained Image Classification |
|
Attribute-specific Control Units in StyleGAN for Fine-grained Image Manipulation
|
|
State-Aware Tracker for Real-Time Video Object Segmentation |
|
High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification |
|
Context Prior for Scene Segmentation |
|
SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines |
|
Learnable Tree Filter for Structure-preserving Feature Transform |
|
ThunderNet: Towards Real-time Generic Object Detection |
|
Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network |
|
Objects365: A Large-scale, High-quality Dataset for Object Detection |
|
An End-to-end Network for Panoptic Segmentation |
|
TACNet: Transition-Aware Context Network for Spatio-Temporal Action Detection |
|
Modeling Local Geometric Structure of 3D Point Clouds using Geo-CNN |
|
Shape Robust Text Detection with Progressive Scale Expansion Network |
|
Scene Text Detection with Supervised Pyramid Context Network |
|
Attention-based Multi-Context Guiding for Few-Shot Semantic Segmentation
|
|
DetNet: A Backbone network for Object
Detection
|
|
BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation
|
|
Associating Inter-Image Salient Instances forWeakly Supervised Semantic Segmentation
|
|
MegDet: A Large Mini-Batch Object Detector |
|
Cascaded Pyramid Network for Multi-Person Pose Estimation [Code] |
|
Learning a Discriminative Feature Network for Semantic Segmentation |
|
R-FCN++: Towards Accurate Region-based Fully Convolutional Networks for Object Detection |
|
Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Network |
|
Fast Action Proposals for Human Action Detection and Search |
|
Discriminative Orderlet Mining For Real-time Recognition of Human-Object Interaction [Project]
|
|
Scalable Forest Hashing for Fast Similarity Search
|
|
Propagative Hough Voting for Human Activity Recognition
|
|
Randomized Spatial Partition for Scene Recognition
|
|
Predicting Human Activities using Spatio-Temporal Structure of Interest Points
|
|
Unsupervised Random Forest Indexing for Fast Action Search
|
|
Real-time HumanAction Search using Random Forest based Hough Voting
|
| Journal |
|
Lightweight Model Pre-Training Via Language Guided Knowledge Distillation |
|
Vote2cap-detr++: Decoupling localization and describing for end-to-end 3d dense captioning |
|
BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation |
|
Propagative Hough Voting for Human Activity Detection and Recognition |
|
Action Search by Example using Randomized Visual Vocabularies |
|
Fast Action Detection via Discriminative Random Forest Voting and Top-K Subvolume Search |
| Book |
|
Human Action Analysis with Randomized Trees |