https://mengmouxu.github.io/SceneGen/
Paper: https://arxiv.org/abs/2508.15769
Code: https://github.com/Mengmouxu/SceneGen

To summarize, we make the following contributions in this paper:
(i) we present SceneGen, a novel framework that takes a scene image and corresponding object masks as input, simultaneously producing multiple 3D assets with geometry and texture. Notably, SceneGen operates with no need for optimization or asset retrieval;
(ii) we introduce a novel feature aggregation module that integrates local and global scene information from visual and geometric encoders within the feature extraction module. Coupled with a position head, this enables the generation of 3D assets and their relative spatial positions in a single feedforward pass;
(iii) we demonstrate SceneGen's direct extensibility to multi-image input scenarios. Despite being trained solely on single-image inputs, our architectural design enables improved generation performance with multi-image inputs;
(iv) extensive quantitative and qualitative evaluations confirm the efficiency and robust generation abilities of our approach.
We believe this paradigm offers a novel solution for high-quality 3D content generation, potentially advancing its practical applications in downstream tasks.

We are organizing our code, data, and checkpoints, and will gradually open-source them in the near future. Please stay tuned!!! Feel free to reach out for discussions!

\n","updatedAt":"2025-08-22T02:00:58.968Z","author":{"_id":"632c7a0d1d303f5f9acf01b8","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/632c7a0d1d303f5f9acf01b8/T010IFuCp6UaOeIyWhbCk.jpeg","fullname":"Haoning Wu","name":"haoningwu","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":5}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8494384288787842},"editors":["haoningwu"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/632c7a0d1d303f5f9acf01b8/T010IFuCp6UaOeIyWhbCk.jpeg"],"reactions":[{"reaction":"🔥","users":["linoyts"],"count":1}],"isReport":false}},{"id":"68a91ab24a1f0871dd89001f","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":264},"createdAt":"2025-08-23T01:34:42.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Towards Geometric and Textural Consistency 3D Scene Generation via Single Image-guided Model Generation and Layout Optimization](https://huggingface.co/papers/2507.14841) (2025)\n* [DepR: Depth Guided Single-view Scene Reconstruction with Instance-level Diffusion](https://huggingface.co/papers/2507.22825) (2025)\n* [Video Perception Models for 3D Scene Synthesis](https://huggingface.co/papers/2506.20601) (2025)\n* [Matrix-3D: Omnidirectional Explorable 3D World Generation](https://huggingface.co/papers/2508.08086) (2025)\n* [DeOcc-1-to-3: 3D De-Occlusion from a Single Image via Self-Supervised Multi-View Diffusion](https://huggingface.co/papers/2506.21544) (2025)\n* [Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Survey](https://huggingface.co/papers/2507.14501) (2025)\n* [SIU3R: Simultaneous Scene Understanding and 3D Reconstruction Beyond Feature Alignment](https://huggingface.co/papers/2507.02705) (2025)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend

\n","updatedAt":"2025-08-23T01:34:42.460Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":264}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6954857707023621},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2508.15769","authors":[{"_id":"68a7cd5939413c456c05b09e","user":{"_id":"666a5ca53315aa1fa8061ef1","avatarUrl":"/avatars/da5ce98959f553c7d25675acae55b6fe.svg","isPro":false,"fullname":"Mengmouxu","user":"Mengmouxu","type":"user"},"name":"Yanxu Meng","status":"claimed_verified","statusLastChangedAt":"2025-08-26T19:06:35.644Z","hidden":false},{"_id":"68a7cd5939413c456c05b09f","user":{"_id":"632c7a0d1d303f5f9acf01b8","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/632c7a0d1d303f5f9acf01b8/T010IFuCp6UaOeIyWhbCk.jpeg","isPro":false,"fullname":"Haoning Wu","user":"haoningwu","type":"user"},"name":"Haoning Wu","status":"claimed_verified","statusLastChangedAt":"2025-08-22T07:22:01.228Z","hidden":false},{"_id":"68a7cd5939413c456c05b0a0","name":"Ya Zhang","hidden":false},{"_id":"68a7cd5939413c456c05b0a1","user":{"_id":"63f1841f2f7c0152e8748ec4","avatarUrl":"/avatars/5398658c7d0cf556531a625a4ca5d18a.svg","isPro":false,"fullname":"Xie","user":"Weidi","type":"user"},"name":"Weidi Xie","status":"admin_assigned","statusLastChangedAt":"2025-08-22T08:27:57.412Z","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/632c7a0d1d303f5f9acf01b8/rmLnc0_Wu60sfTJ6zBQdi.jpeg"],"publishedAt":"2025-08-21T17:59:16.000Z","submittedOnDailyAt":"2025-08-22T00:30:58.941Z","title":"SceneGen: Single-Image 3D Scene Generation in One Feedforward Pass","submittedOnDailyBy":{"_id":"632c7a0d1d303f5f9acf01b8","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/632c7a0d1d303f5f9acf01b8/T010IFuCp6UaOeIyWhbCk.jpeg","isPro":false,"fullname":"Haoning Wu","user":"haoningwu","type":"user"},"summary":"3D content generation has recently attracted significant research interest\ndue to its applications in VR/AR and embodied AI. In this work, we address the\nchallenging task of synthesizing multiple 3D assets within a single scene\nimage. Concretely, our contributions are fourfold: (i) we present SceneGen, a\nnovel framework that takes a scene image and corresponding object masks as\ninput, simultaneously producing multiple 3D assets with geometry and texture.\nNotably, SceneGen operates with no need for optimization or asset retrieval;\n(ii) we introduce a novel feature aggregation module that integrates local and\nglobal scene information from visual and geometric encoders within the feature\nextraction module. Coupled with a position head, this enables the generation of\n3D assets and their relative spatial positions in a single feedforward pass;\n(iii) we demonstrate SceneGen's direct extensibility to multi-image input\nscenarios. Despite being trained solely on single-image inputs, our\narchitectural design enables improved generation performance with multi-image\ninputs; and (iv) extensive quantitative and qualitative evaluations confirm the\nefficiency and robust generation abilities of our approach. We believe this\nparadigm offers a novel solution for high-quality 3D content generation,\npotentially advancing its practical applications in downstream tasks. The code\nand model will be publicly available at: https://mengmouxu.github.io/SceneGen.","upvotes":19,"discussionId":"68a7cd5939413c456c05b0a2","projectPage":"https://mengmouxu.github.io/SceneGen/","githubRepo":"https://github.com/Mengmouxu/SceneGen","ai_summary":"SceneGen generates multiple 3D assets from a single scene image using a novel framework that integrates local and global scene information, enabling efficient and robust 3D content creation.","ai_keywords":["SceneGen","feature aggregation module","visual encoders","geometric encoders","position head","multi-image input","3D content generation","VR/AR","embodied AI"],"githubStars":149},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"632c7a0d1d303f5f9acf01b8","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/632c7a0d1d303f5f9acf01b8/T010IFuCp6UaOeIyWhbCk.jpeg","isPro":false,"fullname":"Haoning Wu","user":"haoningwu","type":"user"},{"_id":"6342796a0875f2c99cfd313b","avatarUrl":"/avatars/98575092404c4197b20c929a6499a015.svg","isPro":false,"fullname":"Yuseung \"Phillip\" Lee","user":"phillipinseoul","type":"user"},{"_id":"6707ed652e5427a4c8ffef10","avatarUrl":"/avatars/b6537a916a6f1f42aa143380608d8705.svg","isPro":false,"fullname":"ruwwww","user":"ruwwww","type":"user"},{"_id":"6683fc5344a65be1aab25dc0","avatarUrl":"/avatars/e13cde3f87b59e418838d702807df3b5.svg","isPro":false,"fullname":"hjkim","user":"hojie11","type":"user"},{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user"},{"_id":"648eb1eb59c4e5c87dc116e0","avatarUrl":"/avatars/c636cea39c2c0937f01398c94ead5dad.svg","isPro":false,"fullname":"fdsqefsgergd","user":"T-representer","type":"user"},{"_id":"64ea59beb36ed038b6638ece","avatarUrl":"/avatars/a74b3c8b63b5ca8ebb3a00455f6f803f.svg","isPro":false,"fullname":"Slava","user":"wertlon","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"6487e158f675b4a7867f45fa","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6487e158f675b4a7867f45fa/J0sls6zZ682o-SH7iQs7B.jpeg","isPro":false,"fullname":"Zilong Huang","user":"SereinH","type":"user"},{"_id":"638f308fc4444c6ca870b60a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/638f308fc4444c6ca870b60a/Q11NK-8-JbiilJ-vk2LAR.png","isPro":true,"fullname":"Linoy Tsaban","user":"linoyts","type":"user"},{"_id":"638546d0a179f856005ae310","avatarUrl":"/avatars/f7f2fface336c8e168a1daaf9fd4d40c.svg","isPro":true,"fullname":"yashkant","user":"yashkant","type":"user"},{"_id":"663ccbff3a74a20189d4aa2e","avatarUrl":"/avatars/83a54455e0157480f65c498cd9057cf2.svg","isPro":false,"fullname":"Nguyen Van Thanh","user":"NguyenVanThanhHust","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0}">

Papers

arxiv:2508.15769

SceneGen: Single-Image 3D Scene Generation in One Feedforward Pass

Published on Aug 21

· Submitted by

Haoning Wu on Aug 22

Upvote

Authors:

Yanxu Meng ,

Haoning Wu ,

Weidi Xie

Abstract

SceneGen generates multiple 3D assets from a single scene image using a novel framework that integrates local and global scene information, enabling efficient and robust 3D content creation.

AI-generated summary

3D content generation has recently attracted significant research interest due to its applications in VR/AR and embodied AI. In this work, we address the challenging task of synthesizing multiple 3D assets within a single scene image. Concretely, our contributions are fourfold: (i) we present SceneGen, a novel framework that takes a scene image and corresponding object masks as input, simultaneously producing multiple 3D assets with geometry and texture. Notably, SceneGen operates with no need for optimization or asset retrieval; (ii) we introduce a novel feature aggregation module that integrates local and global scene information from visual and geometric encoders within the feature extraction module. Coupled with a position head, this enables the generation of 3D assets and their relative spatial positions in a single feedforward pass; (iii) we demonstrate SceneGen's direct extensibility to multi-image input scenarios. Despite being trained solely on single-image inputs, our architectural design enables improved generation performance with multi-image inputs; and (iv) extensive quantitative and qualitative evaluations confirm the efficiency and robust generation abilities of our approach. We believe this paradigm offers a novel solution for high-quality 3D content generation, potentially advancing its practical applications in downstream tasks. The code and model will be publicly available at: https://mengmouxu.github.io/SceneGen.

View arXiv page View PDF Project page GitHub 149 Add to collection

Community

haoningwu

Paper author Paper submitter Aug 22

Project Page: https://mengmouxu.github.io/SceneGen/
Paper: https://arxiv.org/abs/2508.15769
Code: https://github.com/Mengmouxu/SceneGen

We are organizing our code, data, and checkpoints, and will gradually open-source them in the near future. Please stay tuned!!! Feel free to reach out for discussions!