Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

\n","updatedAt":"2024-01-03T14:07:25.464Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":264}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7140737175941467},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2312.13964","authors":[{"_id":"6584f9100f26133956091593","user":{"_id":"64ccaa4687ec96aa4752e754","avatarUrl":"/avatars/d2dd2040a521de4f55c7335cb7771c75.svg","isPro":false,"fullname":"Yiming Zhang","user":"ymzhang319","type":"user"},"name":"Yiming Zhang","status":"admin_assigned","statusLastChangedAt":"2024-07-02T20:08:34.819Z","hidden":false},{"_id":"6584f9100f26133956091594","user":{"_id":"62fb2a9dc95d426ff8f74c8d","avatarUrl":"/avatars/25c1a68ee7b7d0cc7e9f56bde37f4914.svg","isPro":false,"fullname":"Zhening Xing","user":"Leoxing","type":"user"},"name":"Zhening Xing","status":"admin_assigned","statusLastChangedAt":"2023-12-22T12:30:04.341Z","hidden":false},{"_id":"6584f9100f26133956091595","user":{"_id":"63d4b843df01ef426a0f79fb","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1676365795587-63d4b843df01ef426a0f79fb.jpeg","isPro":false,"fullname":"Yanhong Zeng","user":"zengyh1900","type":"user"},"name":"Yanhong Zeng","status":"claimed_verified","statusLastChangedAt":"2023-12-22T13:00:39.833Z","hidden":false},{"_id":"6584f9100f26133956091596","name":"Youqing Fang","hidden":false},{"_id":"6584f9100f26133956091597","name":"Kai Chen","hidden":false}],"publishedAt":"2023-12-21T15:51:12.000Z","submittedOnDailyAt":"2023-12-22T00:18:53.440Z","title":"PIA: Your Personalized Image Animator via Plug-and-Play Modules in\n Text-to-Image Models","submittedOnDailyBy":{"_id":"60f1abe7544c2adfd699860c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg","isPro":false,"fullname":"AK","user":"akhaliq","type":"user"},"summary":"Recent advancements in personalized text-to-image (T2I) models have\nrevolutionized content creation, empowering non-experts to generate stunning\nimages with unique styles. While promising, adding realistic motions into these\npersonalized images by text poses significant challenges in preserving distinct\nstyles, high-fidelity details, and achieving motion controllability by text. In\nthis paper, we present PIA, a Personalized Image Animator that excels in\naligning with condition images, achieving motion controllability by text, and\nthe compatibility with various personalized T2I models without specific tuning.\nTo achieve these goals, PIA builds upon a base T2I model with well-trained\ntemporal alignment layers, allowing for the seamless transformation of any\npersonalized T2I model into an image animation model. A key component of PIA is\nthe introduction of the condition module, which utilizes the condition frame\nand inter-frame affinity as input to transfer appearance information guided by\nthe affinity hint for individual frame synthesis in the latent space. This\ndesign mitigates the challenges of appearance-related image alignment within\nand allows for a stronger focus on aligning with motion-related guidance.","upvotes":20,"discussionId":"6584f9150f26133956091678","ai_summary":"PIA enhances personalized T2I models to generate animations by integrating temporal alignment and condition modules for motion control and appearance consistency.","ai_keywords":["personalized T2I models","temporal alignment layers","condition module","condition frame","inter-frame affinity","latent space","image animation model","motion controllability","appearance-related image alignment"]},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"6032802e1f993496bc14d9e3","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6032802e1f993496bc14d9e3/w6hr-DEQot4VVkoyRIBiy.png","isPro":false,"fullname":"Omar Sanseviero","user":"osanseviero","type":"user"},{"_id":"63d4b843df01ef426a0f79fb","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1676365795587-63d4b843df01ef426a0f79fb.jpeg","isPro":false,"fullname":"Yanhong Zeng","user":"zengyh1900","type":"user"},{"_id":"63c5d43ae2804cb2407e4d43","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1673909278097-noauth.png","isPro":false,"fullname":"xziayro","user":"xziayro","type":"user"},{"_id":"648eb1eb59c4e5c87dc116e0","avatarUrl":"/avatars/c636cea39c2c0937f01398c94ead5dad.svg","isPro":false,"fullname":"fdsqefsgergd","user":"T-representer","type":"user"},{"_id":"6538119803519fddb4a17e10","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6538119803519fddb4a17e10/ffJMkdx-rM7VvLTCM6ri_.jpeg","isPro":false,"fullname":"samusenps","user":"samusenps","type":"user"},{"_id":"63d4c8ce13ae45b780792f32","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1675778487155-63d4c8ce13ae45b780792f32.jpeg","isPro":false,"fullname":"Ohenenoo","user":"PeepDaSlan9","type":"user"},{"_id":"609653c1146ef3bfe2fc7392","avatarUrl":"/avatars/1639b6552a419209ae67b6562183bc2f.svg","isPro":false,"fullname":"Inui","user":"Norm","type":"user"},{"_id":"6576b99d58ce19fa1e33eb1d","avatarUrl":"/avatars/b533e776aa3d95d722b46ef0cd381acd.svg","isPro":false,"fullname":"Jihyong Oh","user":"ozbro","type":"user"},{"_id":"642924f916d4d8293c93af08","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/642924f916d4d8293c93af08/Olp4zFPgGu2E6mFFPXfQO.jpeg","isPro":false,"fullname":"Yixin Song","user":"yixinsong","type":"user"},{"_id":"62fb2a9dc95d426ff8f74c8d","avatarUrl":"/avatars/25c1a68ee7b7d0cc7e9f56bde37f4914.svg","isPro":false,"fullname":"Zhening Xing","user":"Leoxing","type":"user"},{"_id":"64ccaa4687ec96aa4752e754","avatarUrl":"/avatars/d2dd2040a521de4f55c7335cb7771c75.svg","isPro":false,"fullname":"Yiming Zhang","user":"ymzhang319","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0}">

Papers

arxiv:2312.13964

PIA: Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Models

Published on Dec 21, 2023

· Submitted by

AK on Dec 22, 2023

Upvote

Authors:

Yiming Zhang ,

Zhening Xing ,

Yanhong Zeng ,

Abstract

PIA enhances personalized T2I models to generate animations by integrating temporal alignment and condition modules for motion control and appearance consistency.

AI-generated summary

Recent advancements in personalized text-to-image (T2I) models have revolutionized content creation, empowering non-experts to generate stunning images with unique styles. While promising, adding realistic motions into these personalized images by text poses significant challenges in preserving distinct styles, high-fidelity details, and achieving motion controllability by text. In this paper, we present PIA, a Personalized Image Animator that excels in aligning with condition images, achieving motion controllability by text, and the compatibility with various personalized T2I models without specific tuning. To achieve these goals, PIA builds upon a base T2I model with well-trained temporal alignment layers, allowing for the seamless transformation of any personalized T2I model into an image animation model. A key component of PIA is the introduction of the condition module, which utilizes the condition frame and inter-frame affinity as input to transfer appearance information guided by the affinity hint for individual frame synthesis in the latent space. This design mitigates the challenges of appearance-related image alignment within and allows for a stronger focus on aligning with motion-related guidance.