To generate the images I used code that similar to the provided example.\n","updatedAt":"2022-09-21T13:07:53.319Z","author":{"_id":"62fbbc5c21c444a56f7a564b","avatarUrl":"/avatars/bc5afad3cad6af8b58840b7aebb1acf5.svg","fullname":"Ori Matityahu","name":"ori-m-2022a","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"editors":["ori-m-2022a"],"editorAvatarUrls":["/avatars/bc5afad3cad6af8b58840b7aebb1acf5.svg"],"reactions":[],"isReport":false}},{"id":"63adbfc18b50b36a189bd305","author":{"_id":"6144d541fa30a4ba3ef7ef15","avatarUrl":"/avatars/eeb849eb29e249fd1b40e5c10d9bf9e5.svg","fullname":"Alex Havrilla","name":"Dahoas","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":73,"isOwner":false,"isOrgMember":false},"createdAt":"2022-12-29T16:26:41.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Any update on this? I also observe the same FID.","html":"
Any update on this? I also observe the same FID.
\n","updatedAt":"2022-12-29T16:26:41.472Z","author":{"_id":"6144d541fa30a4ba3ef7ef15","avatarUrl":"/avatars/eeb849eb29e249fd1b40e5c10d9bf9e5.svg","fullname":"Alex Havrilla","name":"Dahoas","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":73}},"numEdits":0,"editors":["Dahoas"],"editorAvatarUrls":["/avatars/eeb849eb29e249fd1b40e5c10d9bf9e5.svg"],"reactions":[],"isReport":false}},{"id":"63bd791408409827334d07d7","author":{"_id":"6144d541fa30a4ba3ef7ef15","avatarUrl":"/avatars/eeb849eb29e249fd1b40e5c10d9bf9e5.svg","fullname":"Alex Havrilla","name":"Dahoas","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":73,"isOwner":false,"isOrgMember":false},"createdAt":"2023-01-10T14:41:24.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"@patrickvonplaten @anton-l can you perhaps comment?","html":"\n\n@patrickvonplaten\n\t \n\n@anton-l\n\t can you perhaps comment?
\n","updatedAt":"2023-01-10T14:44:38.544Z","author":{"_id":"6144d541fa30a4ba3ef7ef15","avatarUrl":"/avatars/eeb849eb29e249fd1b40e5c10d9bf9e5.svg","fullname":"Alex Havrilla","name":"Dahoas","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":73}},"numEdits":1,"editors":["Dahoas"],"editorAvatarUrls":["/avatars/eeb849eb29e249fd1b40e5c10d9bf9e5.svg"],"reactions":[],"isReport":false}},{"id":"63ec5d64308bd1f7fcbff129","author":{"_id":"63ec5cde59a46fd5f4b739ca","avatarUrl":"/avatars/efb1ce91a4ae5e42da4951e2aac8d91b.svg","fullname":"Jaeyun_Lee","name":"TurtleNeck","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2023-02-15T04:19:48.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Any update for this? I trained improved-ddpm code from https://github.com/openai/improved-diffusion and check FID score which is much worse than reported in the paper... :(","html":"Any update for this? I trained improved-ddpm code from https://github.com/openai/improved-diffusion and check FID score which is much worse than reported in the paper... :(
\n","updatedAt":"2023-02-15T04:19:48.911Z","author":{"_id":"63ec5cde59a46fd5f4b739ca","avatarUrl":"/avatars/efb1ce91a4ae5e42da4951e2aac8d91b.svg","fullname":"Jaeyun_Lee","name":"TurtleNeck","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"editors":["TurtleNeck"],"editorAvatarUrls":["/avatars/efb1ce91a4ae5e42da4951e2aac8d91b.svg"],"reactions":[],"isReport":false}},{"id":"63ff06a75caf40b7417fd318","author":{"_id":"5dfcb1aada6d0311fd3d5448","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg","fullname":"Patrick von Platen","name":"patrickvonplaten","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":725,"isOwner":false,"isOrgMember":false},"createdAt":"2023-03-01T08:02:47.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"Hey @TurtleNeck,\n\nDiffusers DDPM is an exact clone of: https://github.com/pesser/pytorch_diffusion\n\nwhich is the PyTorch implementation of orig TF Version","html":"Hey \n\n@TurtleNeck\n\t,
\nDiffusers DDPM is an exact clone of: https://github.com/pesser/pytorch_diffusion
\nwhich is the PyTorch implementation of orig TF Version
\n","updatedAt":"2023-03-01T08:03:00.268Z","author":{"_id":"5dfcb1aada6d0311fd3d5448","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg","fullname":"Patrick von Platen","name":"patrickvonplaten","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":725}},"numEdits":1,"editors":["patrickvonplaten"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg"],"reactions":[],"isReport":false}},{"id":"63ff06d46bb959b94e6970a7","author":{"_id":"5dfcb1aada6d0311fd3d5448","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg","fullname":"Patrick von Platen","name":"patrickvonplaten","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":725,"isOwner":false,"isOrgMember":false},"createdAt":"2023-03-01T08:03:32.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Can you check whether you get better results with presser/pytorch_diffusion?","html":"Can you check whether you get better results with presser/pytorch_diffusion?
\n","updatedAt":"2023-03-01T08:03:32.869Z","author":{"_id":"5dfcb1aada6d0311fd3d5448","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg","fullname":"Patrick von Platen","name":"patrickvonplaten","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":725}},"numEdits":0,"editors":["patrickvonplaten"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg"],"reactions":[],"isReport":false}},{"id":"63ff075625b278f4117d8ab6","author":{"_id":"5dfcb1aada6d0311fd3d5448","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg","fullname":"Patrick von Platen","name":"patrickvonplaten","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":725,"isOwner":false,"isOrgMember":false},"createdAt":"2023-03-01T08:05:42.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Also note that this is the non ema checkpoint, maybe they use the ema checkpoint in their paper for FID evaluation.","html":"Also note that this is the non ema checkpoint, maybe they use the ema checkpoint in their paper for FID evaluation.
\n","updatedAt":"2023-03-01T08:05:42.794Z","author":{"_id":"5dfcb1aada6d0311fd3d5448","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg","fullname":"Patrick von Platen","name":"patrickvonplaten","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":725}},"numEdits":0,"editors":["patrickvonplaten"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg"],"reactions":[],"isReport":false}},{"id":"6423104bae7fe472976f2899","author":{"_id":"62fbbc5c21c444a56f7a564b","avatarUrl":"/avatars/bc5afad3cad6af8b58840b7aebb1acf5.svg","fullname":"Ori Matityahu","name":"ori-m-2022a","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2023-03-28T16:05:31.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Is there an ema checkpoint for this model?","html":"Is there an ema checkpoint for this model?
\n","updatedAt":"2023-03-28T16:05:31.277Z","author":{"_id":"62fbbc5c21c444a56f7a564b","avatarUrl":"/avatars/bc5afad3cad6af8b58840b7aebb1acf5.svg","fullname":"Ori Matityahu","name":"ori-m-2022a","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"editors":["ori-m-2022a"],"editorAvatarUrls":["/avatars/bc5afad3cad6af8b58840b7aebb1acf5.svg"],"reactions":[],"isReport":false}},{"id":"6464947b1054f08ddba099f1","author":{"_id":"63ce831481065f5e3ffff2e3","avatarUrl":"/avatars/4d0b9d0d58ab3cb7097fe714a5377a7d.svg","fullname":"Peter Lorenz","name":"j35t3r","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2023-05-17T08:46:51.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"here is one ema checkpoint: https://github.com/pesser/pytorch_diffusion","html":"here is one ema checkpoint: https://github.com/pesser/pytorch_diffusion
\n","updatedAt":"2023-05-17T08:46:51.182Z","author":{"_id":"63ce831481065f5e3ffff2e3","avatarUrl":"/avatars/4d0b9d0d58ab3cb7097fe714a5377a7d.svg","fullname":"Peter Lorenz","name":"j35t3r","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"editors":["j35t3r"],"editorAvatarUrls":["/avatars/4d0b9d0d58ab3cb7097fe714a5377a7d.svg"],"reactions":[],"isReport":false}},{"id":"646a1aec96cfe72aef787baf","author":{"_id":"646a1939c37ca1e12308fe81","avatarUrl":"/avatars/752e9d86018e7d33ad8bcd741203fd86.svg","fullname":"Gongfan Fang","name":"Vinnnf","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6,"isOwner":false,"isOrgMember":false},"createdAt":"2023-05-21T13:21:48.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"Hey guys. We converted the [official EMA checkpoint]( https://github.com/pesser/pytorch_diffusion) to Diffusers. Hope this will be helpful: https://github.com/VainF/Diff-Pruning/releases/download/v0.0.1/ddpm_ema_cifar10.zip\n\nA bash script for converting is available at https://github.com/VainF/Diff-Pruning/blob/main/tools/convert_cifar10_ddpm_ema.sh \n\nThis checkpoint achieves an FID score of 4.5 with 100 DDIM steps.","html":"Hey guys. We converted the official EMA checkpoint to Diffusers. Hope this will be helpful: https://github.com/VainF/Diff-Pruning/releases/download/v0.0.1/ddpm_ema_cifar10.zip
\nA bash script for converting is available at https://github.com/VainF/Diff-Pruning/blob/main/tools/convert_cifar10_ddpm_ema.sh
\nThis checkpoint achieves an FID score of 4.5 with 100 DDIM steps.
\n","updatedAt":"2023-05-21T15:45:43.911Z","author":{"_id":"646a1939c37ca1e12308fe81","avatarUrl":"/avatars/752e9d86018e7d33ad8bcd741203fd86.svg","fullname":"Gongfan Fang","name":"Vinnnf","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6}},"numEdits":3,"editors":["Vinnnf"],"editorAvatarUrls":["/avatars/752e9d86018e7d33ad8bcd741203fd86.svg"],"reactions":[{"reaction":"👍","users":["Mqleet"],"count":1}],"isReport":false}},{"id":"64f99c34a5067f6b6552f064","author":{"_id":"64b94bf05067873176e4d49b","avatarUrl":"/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg","fullname":"peiqin sun","name":"speiqin","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2023-09-07T09:47:32.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"I use the ema checkpoint, but I get an FID score of 5.23 in 1000 DDPM steps. can you give me a link to you FID scripts? ","html":"I use the ema checkpoint, but I get an FID score of 5.23 in 1000 DDPM steps. can you give me a link to you FID scripts?
\n","updatedAt":"2023-09-07T09:47:32.962Z","author":{"_id":"64b94bf05067873176e4d49b","avatarUrl":"/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg","fullname":"peiqin sun","name":"speiqin","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8925860524177551},"editors":["speiqin"],"editorAvatarUrls":["/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg"],"reactions":[],"isReport":false}},{"id":"64f9d253615b4ffedf390467","author":{"_id":"646a1939c37ca1e12308fe81","avatarUrl":"/avatars/752e9d86018e7d33ad8bcd741203fd86.svg","fullname":"Gongfan Fang","name":"Vinnnf","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6,"isOwner":false,"isOrgMember":false},"createdAt":"2023-09-07T13:38:27.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"> I use the ema checkpoint, but I get an FID score of 5.23 in 1000 DDPM steps. can you give me a link to you FID scripts?\n\nHi @speiqin, I modified the original diffusers codebase to fix [this issue](https://github.com/huggingface/diffusers/issues/2585). You can try [this script](https://github.com/VainF/Diff-Pruning/blob/main/scripts/sample_ddpm_cifar10_pretrained.sh) to reproduce my results (50k samples, FID=4.5, 100-step DDIM). But I think DDPM sampling is not affected by this issue. How many samples did you use for FID estimation?","html":"\n\nI use the ema checkpoint, but I get an FID score of 5.23 in 1000 DDPM steps. can you give me a link to you FID scripts?
\n
Hi \n\n@speiqin\n\t, I modified the original diffusers codebase to fix this issue. You can try this script to reproduce my results (50k samples, FID=4.5, 100-step DDIM). But I think DDPM sampling is not affected by this issue. How many samples did you use for FID estimation?
\n","updatedAt":"2023-09-07T13:38:27.361Z","author":{"_id":"646a1939c37ca1e12308fe81","avatarUrl":"/avatars/752e9d86018e7d33ad8bcd741203fd86.svg","fullname":"Gongfan Fang","name":"Vinnnf","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8265745043754578},"editors":["Vinnnf"],"editorAvatarUrls":["/avatars/752e9d86018e7d33ad8bcd741203fd86.svg"],"reactions":[],"isReport":false}},{"id":"64faf75ada4429f25ddce8e2","author":{"_id":"64b94bf05067873176e4d49b","avatarUrl":"/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg","fullname":"peiqin sun","name":"speiqin","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2023-09-08T10:28:42.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Hi, @Vinnnf . I get the FID result which close to ddpm paper. the most importance error in my experiment( FID only 5.23) is that I only use 1w images to calculate the FID score. Now, I use 5w is get the right result. Thanks for your checkpoint. ","html":"Hi, \n\n@Vinnnf\n\t . I get the FID result which close to ddpm paper. the most importance error in my experiment( FID only 5.23) is that I only use 1w images to calculate the FID score. Now, I use 5w is get the right result. Thanks for your checkpoint.
\n","updatedAt":"2023-09-08T10:28:42.242Z","author":{"_id":"64b94bf05067873176e4d49b","avatarUrl":"/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg","fullname":"peiqin sun","name":"speiqin","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8345819115638733},"editors":["speiqin"],"editorAvatarUrls":["/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg"],"reactions":[{"reaction":"👍","users":["j35t3r"],"count":1}],"isReport":false}},{"id":"653fe07a01b0d514c8978893","author":{"_id":"6537815a62dd8126cf97d4af","avatarUrl":"/avatars/3f20b124ac6f62343d2085c5e43765fa.svg","fullname":"Tang Fuzhi","name":"the-weakest","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2023-10-30T16:57:30.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"@ori-m-2022a Hello. Could you tell me the finally FID score of 50k samples of the checkpoints \"google/ddpm-cifar10-32\" when T = 1000? I got 12.9786970050562 I don't know whether it 's correct. Thank you very much!","html":"\n\n@ori-m-2022a\n\t Hello. Could you tell me the finally FID score of 50k samples of the checkpoints \"google/ddpm-cifar10-32\" when T = 1000? I got 12.9786970050562 I don't know whether it 's correct. Thank you very much!
\n","updatedAt":"2023-10-30T17:02:30.044Z","author":{"_id":"6537815a62dd8126cf97d4af","avatarUrl":"/avatars/3f20b124ac6f62343d2085c5e43765fa.svg","fullname":"Tang Fuzhi","name":"the-weakest","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.7941372990608215},"editors":["the-weakest"],"editorAvatarUrls":["/avatars/3f20b124ac6f62343d2085c5e43765fa.svg"],"reactions":[],"isReport":false}},{"id":"65505c384c6f4e3300234ce3","author":{"_id":"64b94bf05067873176e4d49b","avatarUrl":"/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg","fullname":"peiqin sun","name":"speiqin","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2023-11-12T05:01:44.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"hi, @Vinnnf. I found the dropout rate in converted checkpoint is 0.0, but the offical model is setting 0.1 in the training. This is a wrong?","html":"hi, \n\n@Vinnnf\n\t. I found the dropout rate in converted checkpoint is 0.0, but the offical model is setting 0.1 in the training. This is a wrong?
\n","updatedAt":"2023-11-12T05:01:44.098Z","author":{"_id":"64b94bf05067873176e4d49b","avatarUrl":"/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg","fullname":"peiqin sun","name":"speiqin","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.964205801486969},"editors":["speiqin"],"editorAvatarUrls":["/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg"],"reactions":[],"isReport":false}},{"id":"65cc1fb39a4ee02bbbbaab58","author":{"_id":"6382bdcb2a85b6714b84a619","avatarUrl":"/avatars/b36fc22c2290934820c2c0faebb55087.svg","fullname":"isaacNingLee","name":"ilee0022","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2024-02-14T02:04:35.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Hi all, I am sampling 10k examples with DDPM(T=1000) and 50k DDIM(T=100) and 10k PNDM(T=100). The best FID score I can get is just around 36. I am computing the sampled image in respect to the images from torchvision cifar10 dataset saved in a directory. The script that I use to compute FID is https://github.com/mseitzer/pytorch-fid . I am very new to FID and I am not too sure which part did I made some mistakes. I really appreciate any help and advices.\n\nCheers","html":"Hi all, I am sampling 10k examples with DDPM(T=1000) and 50k DDIM(T=100) and 10k PNDM(T=100). The best FID score I can get is just around 36. I am computing the sampled image in respect to the images from torchvision cifar10 dataset saved in a directory. The script that I use to compute FID is https://github.com/mseitzer/pytorch-fid . I am very new to FID and I am not too sure which part did I made some mistakes. I really appreciate any help and advices.
\nCheers
\n","updatedAt":"2024-02-14T02:04:35.477Z","author":{"_id":"6382bdcb2a85b6714b84a619","avatarUrl":"/avatars/b36fc22c2290934820c2c0faebb55087.svg","fullname":"isaacNingLee","name":"ilee0022","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9119212031364441},"editors":["ilee0022"],"editorAvatarUrls":["/avatars/b36fc22c2290934820c2c0faebb55087.svg"],"reactions":[],"isReport":false}},{"id":"65df674b8f6ea1ca7079457b","author":{"_id":"65095f5af99720fea1297b9b","avatarUrl":"/avatars/d3c1afa1e004b8bff91b6ede48d65a48.svg","fullname":"Cassie Li","name":"Cassie-0121","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2024-02-28T17:03:07.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"> Hi all, I am sampling 10k examples with DDPM(T=1000) and 50k DDIM(T=100) and 10k PNDM(T=100). The best FID score I can get is just around 36. I am computing the sampled image in respect to the images from torchvision cifar10 dataset saved in a directory. The script that I use to compute FID is https://github.com/mseitzer/pytorch-fid . I am very new to FID and I am not too sure which part did I made some mistakes. I really appreciate any help and advices.\n> \n> Cheers\n\nsame process, how did you solve this? many thanks\n","html":"\n\nHi all, I am sampling 10k examples with DDPM(T=1000) and 50k DDIM(T=100) and 10k PNDM(T=100). The best FID score I can get is just around 36. I am computing the sampled image in respect to the images from torchvision cifar10 dataset saved in a directory. The script that I use to compute FID is https://github.com/mseitzer/pytorch-fid . I am very new to FID and I am not too sure which part did I made some mistakes. I really appreciate any help and advices.
\nCheers
\n
same process, how did you solve this? many thanks
\n","updatedAt":"2024-02-28T17:03:07.231Z","author":{"_id":"65095f5af99720fea1297b9b","avatarUrl":"/avatars/d3c1afa1e004b8bff91b6ede48d65a48.svg","fullname":"Cassie Li","name":"Cassie-0121","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9296927452087402},"editors":["Cassie-0121"],"editorAvatarUrls":["/avatars/d3c1afa1e004b8bff91b6ede48d65a48.svg"],"reactions":[],"isReport":false}},{"id":"667298be0560603efd36d4d5","author":{"_id":"6448b2f53e7b3c11be684348","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6448b2f53e7b3c11be684348/QvlUQG3pWf8ZyEVBV6F7w.jpeg","fullname":"Qianli Ma","name":"Mqleet","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2024-06-19T08:37:18.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Any one have solved this problem? I generated 50K images using 100 steps DDIMSampler, and I got a FID of 11.214451002914018 which is far away from 3.17....","html":"Any one have solved this problem? I generated 50K images using 100 steps DDIMSampler, and I got a FID of 11.214451002914018 which is far away from 3.17....
\n","updatedAt":"2024-06-19T08:37:18.076Z","author":{"_id":"6448b2f53e7b3c11be684348","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6448b2f53e7b3c11be684348/QvlUQG3pWf8ZyEVBV6F7w.jpeg","fullname":"Qianli Ma","name":"Mqleet","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8826470971107483},"editors":["Mqleet"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6448b2f53e7b3c11be684348/QvlUQG3pWf8ZyEVBV6F7w.jpeg"],"reactions":[],"isReport":false}}],"pinned":false,"locked":false,"collection":"discussions","isPullRequest":false,"isReport":false},"repo":{"name":"google/ddpm-cifar10-32","type":"model"},"activeTab":"discussion","discussionRole":0,"watched":false,"muted":false,"repoDiscussionsLocked":false}">FID Score
To generate the images I used code that similar to the provided example.\n","updatedAt":"2022-09-21T13:07:53.319Z","author":{"_id":"62fbbc5c21c444a56f7a564b","avatarUrl":"/avatars/bc5afad3cad6af8b58840b7aebb1acf5.svg","fullname":"Ori Matityahu","name":"ori-m-2022a","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"editors":["ori-m-2022a"],"editorAvatarUrls":["/avatars/bc5afad3cad6af8b58840b7aebb1acf5.svg"],"reactions":[],"isReport":false}},{"id":"63adbfc18b50b36a189bd305","author":{"_id":"6144d541fa30a4ba3ef7ef15","avatarUrl":"/avatars/eeb849eb29e249fd1b40e5c10d9bf9e5.svg","fullname":"Alex Havrilla","name":"Dahoas","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":73,"isOwner":false,"isOrgMember":false},"createdAt":"2022-12-29T16:26:41.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Any update on this? I also observe the same FID.","html":"
Any update on this? I also observe the same FID.
\n","updatedAt":"2022-12-29T16:26:41.472Z","author":{"_id":"6144d541fa30a4ba3ef7ef15","avatarUrl":"/avatars/eeb849eb29e249fd1b40e5c10d9bf9e5.svg","fullname":"Alex Havrilla","name":"Dahoas","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":73}},"numEdits":0,"editors":["Dahoas"],"editorAvatarUrls":["/avatars/eeb849eb29e249fd1b40e5c10d9bf9e5.svg"],"reactions":[],"isReport":false}},{"id":"63bd791408409827334d07d7","author":{"_id":"6144d541fa30a4ba3ef7ef15","avatarUrl":"/avatars/eeb849eb29e249fd1b40e5c10d9bf9e5.svg","fullname":"Alex Havrilla","name":"Dahoas","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":73,"isOwner":false,"isOrgMember":false},"createdAt":"2023-01-10T14:41:24.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"@patrickvonplaten @anton-l can you perhaps comment?","html":"\n\n@patrickvonplaten\n\t \n\n@anton-l\n\t can you perhaps comment?
\n","updatedAt":"2023-01-10T14:44:38.544Z","author":{"_id":"6144d541fa30a4ba3ef7ef15","avatarUrl":"/avatars/eeb849eb29e249fd1b40e5c10d9bf9e5.svg","fullname":"Alex Havrilla","name":"Dahoas","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":73}},"numEdits":1,"editors":["Dahoas"],"editorAvatarUrls":["/avatars/eeb849eb29e249fd1b40e5c10d9bf9e5.svg"],"reactions":[],"isReport":false}},{"id":"63ec5d64308bd1f7fcbff129","author":{"_id":"63ec5cde59a46fd5f4b739ca","avatarUrl":"/avatars/efb1ce91a4ae5e42da4951e2aac8d91b.svg","fullname":"Jaeyun_Lee","name":"TurtleNeck","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2023-02-15T04:19:48.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Any update for this? I trained improved-ddpm code from https://github.com/openai/improved-diffusion and check FID score which is much worse than reported in the paper... :(","html":"Any update for this? I trained improved-ddpm code from https://github.com/openai/improved-diffusion and check FID score which is much worse than reported in the paper... :(
\n","updatedAt":"2023-02-15T04:19:48.911Z","author":{"_id":"63ec5cde59a46fd5f4b739ca","avatarUrl":"/avatars/efb1ce91a4ae5e42da4951e2aac8d91b.svg","fullname":"Jaeyun_Lee","name":"TurtleNeck","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"editors":["TurtleNeck"],"editorAvatarUrls":["/avatars/efb1ce91a4ae5e42da4951e2aac8d91b.svg"],"reactions":[],"isReport":false}},{"id":"63ff06a75caf40b7417fd318","author":{"_id":"5dfcb1aada6d0311fd3d5448","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg","fullname":"Patrick von Platen","name":"patrickvonplaten","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":725,"isOwner":false,"isOrgMember":false},"createdAt":"2023-03-01T08:02:47.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"Hey @TurtleNeck,\n\nDiffusers DDPM is an exact clone of: https://github.com/pesser/pytorch_diffusion\n\nwhich is the PyTorch implementation of orig TF Version","html":"Hey \n\n@TurtleNeck\n\t,
\nDiffusers DDPM is an exact clone of: https://github.com/pesser/pytorch_diffusion
\nwhich is the PyTorch implementation of orig TF Version
\n","updatedAt":"2023-03-01T08:03:00.268Z","author":{"_id":"5dfcb1aada6d0311fd3d5448","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg","fullname":"Patrick von Platen","name":"patrickvonplaten","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":725}},"numEdits":1,"editors":["patrickvonplaten"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg"],"reactions":[],"isReport":false}},{"id":"63ff06d46bb959b94e6970a7","author":{"_id":"5dfcb1aada6d0311fd3d5448","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg","fullname":"Patrick von Platen","name":"patrickvonplaten","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":725,"isOwner":false,"isOrgMember":false},"createdAt":"2023-03-01T08:03:32.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Can you check whether you get better results with presser/pytorch_diffusion?","html":"Can you check whether you get better results with presser/pytorch_diffusion?
\n","updatedAt":"2023-03-01T08:03:32.869Z","author":{"_id":"5dfcb1aada6d0311fd3d5448","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg","fullname":"Patrick von Platen","name":"patrickvonplaten","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":725}},"numEdits":0,"editors":["patrickvonplaten"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg"],"reactions":[],"isReport":false}},{"id":"63ff075625b278f4117d8ab6","author":{"_id":"5dfcb1aada6d0311fd3d5448","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg","fullname":"Patrick von Platen","name":"patrickvonplaten","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":725,"isOwner":false,"isOrgMember":false},"createdAt":"2023-03-01T08:05:42.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Also note that this is the non ema checkpoint, maybe they use the ema checkpoint in their paper for FID evaluation.","html":"Also note that this is the non ema checkpoint, maybe they use the ema checkpoint in their paper for FID evaluation.
\n","updatedAt":"2023-03-01T08:05:42.794Z","author":{"_id":"5dfcb1aada6d0311fd3d5448","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg","fullname":"Patrick von Platen","name":"patrickvonplaten","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":725}},"numEdits":0,"editors":["patrickvonplaten"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1584435275418-5dfcb1aada6d0311fd3d5448.jpeg"],"reactions":[],"isReport":false}},{"id":"6423104bae7fe472976f2899","author":{"_id":"62fbbc5c21c444a56f7a564b","avatarUrl":"/avatars/bc5afad3cad6af8b58840b7aebb1acf5.svg","fullname":"Ori Matityahu","name":"ori-m-2022a","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2023-03-28T16:05:31.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Is there an ema checkpoint for this model?","html":"Is there an ema checkpoint for this model?
\n","updatedAt":"2023-03-28T16:05:31.277Z","author":{"_id":"62fbbc5c21c444a56f7a564b","avatarUrl":"/avatars/bc5afad3cad6af8b58840b7aebb1acf5.svg","fullname":"Ori Matityahu","name":"ori-m-2022a","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"editors":["ori-m-2022a"],"editorAvatarUrls":["/avatars/bc5afad3cad6af8b58840b7aebb1acf5.svg"],"reactions":[],"isReport":false}},{"id":"6464947b1054f08ddba099f1","author":{"_id":"63ce831481065f5e3ffff2e3","avatarUrl":"/avatars/4d0b9d0d58ab3cb7097fe714a5377a7d.svg","fullname":"Peter Lorenz","name":"j35t3r","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2023-05-17T08:46:51.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"here is one ema checkpoint: https://github.com/pesser/pytorch_diffusion","html":"here is one ema checkpoint: https://github.com/pesser/pytorch_diffusion
\n","updatedAt":"2023-05-17T08:46:51.182Z","author":{"_id":"63ce831481065f5e3ffff2e3","avatarUrl":"/avatars/4d0b9d0d58ab3cb7097fe714a5377a7d.svg","fullname":"Peter Lorenz","name":"j35t3r","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"editors":["j35t3r"],"editorAvatarUrls":["/avatars/4d0b9d0d58ab3cb7097fe714a5377a7d.svg"],"reactions":[],"isReport":false}},{"id":"646a1aec96cfe72aef787baf","author":{"_id":"646a1939c37ca1e12308fe81","avatarUrl":"/avatars/752e9d86018e7d33ad8bcd741203fd86.svg","fullname":"Gongfan Fang","name":"Vinnnf","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6,"isOwner":false,"isOrgMember":false},"createdAt":"2023-05-21T13:21:48.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"Hey guys. We converted the [official EMA checkpoint]( https://github.com/pesser/pytorch_diffusion) to Diffusers. Hope this will be helpful: https://github.com/VainF/Diff-Pruning/releases/download/v0.0.1/ddpm_ema_cifar10.zip\n\nA bash script for converting is available at https://github.com/VainF/Diff-Pruning/blob/main/tools/convert_cifar10_ddpm_ema.sh \n\nThis checkpoint achieves an FID score of 4.5 with 100 DDIM steps.","html":"Hey guys. We converted the official EMA checkpoint to Diffusers. Hope this will be helpful: https://github.com/VainF/Diff-Pruning/releases/download/v0.0.1/ddpm_ema_cifar10.zip
\nA bash script for converting is available at https://github.com/VainF/Diff-Pruning/blob/main/tools/convert_cifar10_ddpm_ema.sh
\nThis checkpoint achieves an FID score of 4.5 with 100 DDIM steps.
\n","updatedAt":"2023-05-21T15:45:43.911Z","author":{"_id":"646a1939c37ca1e12308fe81","avatarUrl":"/avatars/752e9d86018e7d33ad8bcd741203fd86.svg","fullname":"Gongfan Fang","name":"Vinnnf","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6}},"numEdits":3,"editors":["Vinnnf"],"editorAvatarUrls":["/avatars/752e9d86018e7d33ad8bcd741203fd86.svg"],"reactions":[{"reaction":"👍","users":["Mqleet"],"count":1}],"isReport":false}},{"id":"64f99c34a5067f6b6552f064","author":{"_id":"64b94bf05067873176e4d49b","avatarUrl":"/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg","fullname":"peiqin sun","name":"speiqin","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2023-09-07T09:47:32.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"I use the ema checkpoint, but I get an FID score of 5.23 in 1000 DDPM steps. can you give me a link to you FID scripts? ","html":"I use the ema checkpoint, but I get an FID score of 5.23 in 1000 DDPM steps. can you give me a link to you FID scripts?
\n","updatedAt":"2023-09-07T09:47:32.962Z","author":{"_id":"64b94bf05067873176e4d49b","avatarUrl":"/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg","fullname":"peiqin sun","name":"speiqin","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8925860524177551},"editors":["speiqin"],"editorAvatarUrls":["/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg"],"reactions":[],"isReport":false}},{"id":"64f9d253615b4ffedf390467","author":{"_id":"646a1939c37ca1e12308fe81","avatarUrl":"/avatars/752e9d86018e7d33ad8bcd741203fd86.svg","fullname":"Gongfan Fang","name":"Vinnnf","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6,"isOwner":false,"isOrgMember":false},"createdAt":"2023-09-07T13:38:27.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"> I use the ema checkpoint, but I get an FID score of 5.23 in 1000 DDPM steps. can you give me a link to you FID scripts?\n\nHi @speiqin, I modified the original diffusers codebase to fix [this issue](https://github.com/huggingface/diffusers/issues/2585). You can try [this script](https://github.com/VainF/Diff-Pruning/blob/main/scripts/sample_ddpm_cifar10_pretrained.sh) to reproduce my results (50k samples, FID=4.5, 100-step DDIM). But I think DDPM sampling is not affected by this issue. How many samples did you use for FID estimation?","html":"\n\nI use the ema checkpoint, but I get an FID score of 5.23 in 1000 DDPM steps. can you give me a link to you FID scripts?
\n
Hi \n\n@speiqin\n\t, I modified the original diffusers codebase to fix this issue. You can try this script to reproduce my results (50k samples, FID=4.5, 100-step DDIM). But I think DDPM sampling is not affected by this issue. How many samples did you use for FID estimation?
\n","updatedAt":"2023-09-07T13:38:27.361Z","author":{"_id":"646a1939c37ca1e12308fe81","avatarUrl":"/avatars/752e9d86018e7d33ad8bcd741203fd86.svg","fullname":"Gongfan Fang","name":"Vinnnf","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8265745043754578},"editors":["Vinnnf"],"editorAvatarUrls":["/avatars/752e9d86018e7d33ad8bcd741203fd86.svg"],"reactions":[],"isReport":false}},{"id":"64faf75ada4429f25ddce8e2","author":{"_id":"64b94bf05067873176e4d49b","avatarUrl":"/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg","fullname":"peiqin sun","name":"speiqin","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2023-09-08T10:28:42.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Hi, @Vinnnf . I get the FID result which close to ddpm paper. the most importance error in my experiment( FID only 5.23) is that I only use 1w images to calculate the FID score. Now, I use 5w is get the right result. Thanks for your checkpoint. ","html":"Hi, \n\n@Vinnnf\n\t . I get the FID result which close to ddpm paper. the most importance error in my experiment( FID only 5.23) is that I only use 1w images to calculate the FID score. Now, I use 5w is get the right result. Thanks for your checkpoint.
\n","updatedAt":"2023-09-08T10:28:42.242Z","author":{"_id":"64b94bf05067873176e4d49b","avatarUrl":"/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg","fullname":"peiqin sun","name":"speiqin","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8345819115638733},"editors":["speiqin"],"editorAvatarUrls":["/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg"],"reactions":[{"reaction":"👍","users":["j35t3r"],"count":1}],"isReport":false}},{"id":"653fe07a01b0d514c8978893","author":{"_id":"6537815a62dd8126cf97d4af","avatarUrl":"/avatars/3f20b124ac6f62343d2085c5e43765fa.svg","fullname":"Tang Fuzhi","name":"the-weakest","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2023-10-30T16:57:30.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"@ori-m-2022a Hello. Could you tell me the finally FID score of 50k samples of the checkpoints \"google/ddpm-cifar10-32\" when T = 1000? I got 12.9786970050562 I don't know whether it 's correct. Thank you very much!","html":"\n\n@ori-m-2022a\n\t Hello. Could you tell me the finally FID score of 50k samples of the checkpoints \"google/ddpm-cifar10-32\" when T = 1000? I got 12.9786970050562 I don't know whether it 's correct. Thank you very much!
\n","updatedAt":"2023-10-30T17:02:30.044Z","author":{"_id":"6537815a62dd8126cf97d4af","avatarUrl":"/avatars/3f20b124ac6f62343d2085c5e43765fa.svg","fullname":"Tang Fuzhi","name":"the-weakest","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.7941372990608215},"editors":["the-weakest"],"editorAvatarUrls":["/avatars/3f20b124ac6f62343d2085c5e43765fa.svg"],"reactions":[],"isReport":false}},{"id":"65505c384c6f4e3300234ce3","author":{"_id":"64b94bf05067873176e4d49b","avatarUrl":"/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg","fullname":"peiqin sun","name":"speiqin","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2023-11-12T05:01:44.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"hi, @Vinnnf. I found the dropout rate in converted checkpoint is 0.0, but the offical model is setting 0.1 in the training. This is a wrong?","html":"hi, \n\n@Vinnnf\n\t. I found the dropout rate in converted checkpoint is 0.0, but the offical model is setting 0.1 in the training. This is a wrong?
\n","updatedAt":"2023-11-12T05:01:44.098Z","author":{"_id":"64b94bf05067873176e4d49b","avatarUrl":"/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg","fullname":"peiqin sun","name":"speiqin","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.964205801486969},"editors":["speiqin"],"editorAvatarUrls":["/avatars/ad322d0bcf5a6ac3bcca228b1b10f4c0.svg"],"reactions":[],"isReport":false}},{"id":"65cc1fb39a4ee02bbbbaab58","author":{"_id":"6382bdcb2a85b6714b84a619","avatarUrl":"/avatars/b36fc22c2290934820c2c0faebb55087.svg","fullname":"isaacNingLee","name":"ilee0022","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2024-02-14T02:04:35.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Hi all, I am sampling 10k examples with DDPM(T=1000) and 50k DDIM(T=100) and 10k PNDM(T=100). The best FID score I can get is just around 36. I am computing the sampled image in respect to the images from torchvision cifar10 dataset saved in a directory. The script that I use to compute FID is https://github.com/mseitzer/pytorch-fid . I am very new to FID and I am not too sure which part did I made some mistakes. I really appreciate any help and advices.\n\nCheers","html":"Hi all, I am sampling 10k examples with DDPM(T=1000) and 50k DDIM(T=100) and 10k PNDM(T=100). The best FID score I can get is just around 36. I am computing the sampled image in respect to the images from torchvision cifar10 dataset saved in a directory. The script that I use to compute FID is https://github.com/mseitzer/pytorch-fid . I am very new to FID and I am not too sure which part did I made some mistakes. I really appreciate any help and advices.
\nCheers
\n","updatedAt":"2024-02-14T02:04:35.477Z","author":{"_id":"6382bdcb2a85b6714b84a619","avatarUrl":"/avatars/b36fc22c2290934820c2c0faebb55087.svg","fullname":"isaacNingLee","name":"ilee0022","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9119212031364441},"editors":["ilee0022"],"editorAvatarUrls":["/avatars/b36fc22c2290934820c2c0faebb55087.svg"],"reactions":[],"isReport":false}},{"id":"65df674b8f6ea1ca7079457b","author":{"_id":"65095f5af99720fea1297b9b","avatarUrl":"/avatars/d3c1afa1e004b8bff91b6ede48d65a48.svg","fullname":"Cassie Li","name":"Cassie-0121","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2024-02-28T17:03:07.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"> Hi all, I am sampling 10k examples with DDPM(T=1000) and 50k DDIM(T=100) and 10k PNDM(T=100). The best FID score I can get is just around 36. I am computing the sampled image in respect to the images from torchvision cifar10 dataset saved in a directory. The script that I use to compute FID is https://github.com/mseitzer/pytorch-fid . I am very new to FID and I am not too sure which part did I made some mistakes. I really appreciate any help and advices.\n> \n> Cheers\n\nsame process, how did you solve this? many thanks\n","html":"\n\nHi all, I am sampling 10k examples with DDPM(T=1000) and 50k DDIM(T=100) and 10k PNDM(T=100). The best FID score I can get is just around 36. I am computing the sampled image in respect to the images from torchvision cifar10 dataset saved in a directory. The script that I use to compute FID is https://github.com/mseitzer/pytorch-fid . I am very new to FID and I am not too sure which part did I made some mistakes. I really appreciate any help and advices.
\nCheers
\n
same process, how did you solve this? many thanks
\n","updatedAt":"2024-02-28T17:03:07.231Z","author":{"_id":"65095f5af99720fea1297b9b","avatarUrl":"/avatars/d3c1afa1e004b8bff91b6ede48d65a48.svg","fullname":"Cassie Li","name":"Cassie-0121","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9296927452087402},"editors":["Cassie-0121"],"editorAvatarUrls":["/avatars/d3c1afa1e004b8bff91b6ede48d65a48.svg"],"reactions":[],"isReport":false}},{"id":"667298be0560603efd36d4d5","author":{"_id":"6448b2f53e7b3c11be684348","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6448b2f53e7b3c11be684348/QvlUQG3pWf8ZyEVBV6F7w.jpeg","fullname":"Qianli Ma","name":"Mqleet","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2024-06-19T08:37:18.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Any one have solved this problem? I generated 50K images using 100 steps DDIMSampler, and I got a FID of 11.214451002914018 which is far away from 3.17....","html":"Any one have solved this problem? I generated 50K images using 100 steps DDIMSampler, and I got a FID of 11.214451002914018 which is far away from 3.17....
\n","updatedAt":"2024-06-19T08:37:18.076Z","author":{"_id":"6448b2f53e7b3c11be684348","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6448b2f53e7b3c11be684348/QvlUQG3pWf8ZyEVBV6F7w.jpeg","fullname":"Qianli Ma","name":"Mqleet","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8826470971107483},"editors":["Mqleet"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6448b2f53e7b3c11be684348/QvlUQG3pWf8ZyEVBV6F7w.jpeg"],"reactions":[],"isReport":false}}],"pinned":false,"locked":false,"collection":"discussions","isPullRequest":false,"isReport":false},"primaryEmailConfirmed":false,"repo":{"name":"google/ddpm-cifar10-32","type":"model"},"discussionRole":0,"acceptLanguages":["*"],"hideComments":true,"repoDiscussionsLocked":false,"isDiscussionAuthor":false}">Calculating the FID score of the model on 50k generated images gives: 12.437098284576962.
This FID score is much higher than what reported on the original paper (3.17).
Hi @study-work-2022a, could you please share the script you've used to calculate FID? I'd like to reproduce your result to debug this.
I used this code to calculate the FID: https://github.com/mseitzer/pytorch-fid.
To generate the images I used code that similar to the provided example.
Any update on this? I also observe the same FID.
@patrickvonplaten @anton-l can you perhaps comment?
Any update for this? I trained improved-ddpm code from https://github.com/openai/improved-diffusion and check FID score which is much worse than reported in the paper... :(
Hey @TurtleNeck ,
Diffusers DDPM is an exact clone of: https://github.com/pesser/pytorch_diffusion
which is the PyTorch implementation of orig TF Version
Can you check whether you get better results with presser/pytorch_diffusion?
Also note that this is the non ema checkpoint, maybe they use the ema checkpoint in their paper for FID evaluation.
Is there an ema checkpoint for this model?
here is one ema checkpoint: https://github.com/pesser/pytorch_diffusion
Hey guys. We converted the official EMA checkpoint to Diffusers. Hope this will be helpful: https://github.com/VainF/Diff-Pruning/releases/download/v0.0.1/ddpm_ema_cifar10.zip
A bash script for converting is available at https://github.com/VainF/Diff-Pruning/blob/main/tools/convert_cifar10_ddpm_ema.sh
This checkpoint achieves an FID score of 4.5 with 100 DDIM steps.
I use the ema checkpoint, but I get an FID score of 5.23 in 1000 DDPM steps. can you give me a link to you FID scripts?
I use the ema checkpoint, but I get an FID score of 5.23 in 1000 DDPM steps. can you give me a link to you FID scripts?
Hi @speiqin , I modified the original diffusers codebase to fix this issue. You can try this script to reproduce my results (50k samples, FID=4.5, 100-step DDIM). But I think DDPM sampling is not affected by this issue. How many samples did you use for FID estimation?
@ori-m-2022a Hello. Could you tell me the finally FID score of 50k samples of the checkpoints "google/ddpm-cifar10-32" when T = 1000? I got 12.9786970050562 I don't know whether it 's correct. Thank you very much!
Hi all, I am sampling 10k examples with DDPM(T=1000) and 50k DDIM(T=100) and 10k PNDM(T=100). The best FID score I can get is just around 36. I am computing the sampled image in respect to the images from torchvision cifar10 dataset saved in a directory. The script that I use to compute FID is https://github.com/mseitzer/pytorch-fid . I am very new to FID and I am not too sure which part did I made some mistakes. I really appreciate any help and advices.
Cheers
Hi all, I am sampling 10k examples with DDPM(T=1000) and 50k DDIM(T=100) and 10k PNDM(T=100). The best FID score I can get is just around 36. I am computing the sampled image in respect to the images from torchvision cifar10 dataset saved in a directory. The script that I use to compute FID is https://github.com/mseitzer/pytorch-fid . I am very new to FID and I am not too sure which part did I made some mistakes. I really appreciate any help and advices.
Cheers
same process, how did you solve this? many thanks
Any one have solved this problem? I generated 50K images using 100 steps DDIMSampler, and I got a FID of 11.214451002914018 which is far away from 3.17....