Spaces:

---

big-vision

/

paligemma-hf

like 83

Running

https://huggingface.co/spaces/big-vision/paligemma - the Jax model works well.

\n

But the https://huggingface.co/spaces/big-vision/paligemma-hf space just selects the entire image (on the same input). I'm trying to reproduce the (better) Jax behavior on HF transformers, but I can't figure out what's being done differently on the Jax side. Any tips would be appreciated!

\n","updatedAt":"2024-05-20T19:07:17.564Z","author":{"_id":"6310eaa407a7682790242b8d","avatarUrl":"/avatars/51a9109ac81a962fe2a9641821c9e2b7.svg","fullname":"NessAiver","name":"Shalev","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8643802404403687},"editors":["Shalev"],"editorAvatarUrls":["/avatars/51a9109ac81a962fe2a9641821c9e2b7.svg"],"reactions":[{"reaction":"👍","users":["codelion","mastercallum"],"count":2}],"isReport":false}},{"id":"664f0e58aa0f682b3b72a93d","author":{"_id":"62f32eab52ad88c930bb3f3b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1677134945205-62f32eab52ad88c930bb3f3b.png","fullname":"Asankhaya Sharma","name":"codelion","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":350,"isOwner":false,"isOrgMember":false},"createdAt":"2024-05-23T09:37:28.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Seeing similar issues, is there a difference in the HF version?","html":"

Seeing similar issues, is there a difference in the HF version?

\n","updatedAt":"2024-05-23T09:37:28.159Z","author":{"_id":"62f32eab52ad88c930bb3f3b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1677134945205-62f32eab52ad88c930bb3f3b.png","fullname":"Asankhaya Sharma","name":"codelion","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":350}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9507095217704773},"editors":["codelion"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1677134945205-62f32eab52ad88c930bb3f3b.png"],"reactions":[],"isReport":false}},{"id":"664f1a7e60ddd85b1967514c","author":{"_id":"6141a88b3a0ec78603c9e784","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6141a88b3a0ec78603c9e784/DJsxSmWV39M33JFheLobC.jpeg","fullname":"merve","name":"merve","type":"user","isPro":true,"isHf":true,"isHfAdmin":false,"isMod":false,"followerCount":9206,"isOwner":false,"isOrgMember":false},"createdAt":"2024-05-23T10:29:18.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"@Shalev @codelion I will debug and come back to you on this","html":"

\n\n@Shalev\n\t \n\n@codelion\n\t I will debug and come back to you on this

\n","updatedAt":"2024-05-23T10:29:18.256Z","author":{"_id":"6141a88b3a0ec78603c9e784","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6141a88b3a0ec78603c9e784/DJsxSmWV39M33JFheLobC.jpeg","fullname":"merve","name":"merve","type":"user","isPro":true,"isHf":true,"isHfAdmin":false,"isMod":false,"followerCount":9206}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8836972713470459},"editors":["merve"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6141a88b3a0ec78603c9e784/DJsxSmWV39M33JFheLobC.jpeg"],"reactions":[],"isReport":false}},{"id":"666acc9df9f493152f560132","author":{"_id":"63a61ba064f470027818efca","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1671830348493-noauth.png","fullname":"Anxhelo Diko","name":"D-Anel","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2024-06-13T10:40:29.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Hi, how can we decode the segmentation tokens into binary mask for object segmentation?","html":"

Hi, how can we decode the segmentation tokens into binary mask for object segmentation?

\n","updatedAt":"2024-06-13T10:40:29.394Z","author":{"_id":"63a61ba064f470027818efca","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1671830348493-noauth.png","fullname":"Anxhelo Diko","name":"D-Anel","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8098273277282715},"editors":["D-Anel"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1671830348493-noauth.png"],"reactions":[],"isReport":false}},{"id":"666ad11fe564b5bdbb581881","author":{"_id":"62f32eab52ad88c930bb3f3b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1677134945205-62f32eab52ad88c930bb3f3b.png","fullname":"Asankhaya Sharma","name":"codelion","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":350,"isOwner":false,"isOrgMember":false},"createdAt":"2024-06-13T10:59:43.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"@D-Anel you can check the code here - https://huggingface.co/spaces/big-vision/paligemma-hf/blob/main/app.py#L43","html":"

\n\n@D-Anel\n\t you can check the code here - https://huggingface.co/spaces/big-vision/paligemma-hf/blob/main/app.py#L43

\n","updatedAt":"2024-06-13T10:59:43.512Z","author":{"_id":"62f32eab52ad88c930bb3f3b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1677134945205-62f32eab52ad88c930bb3f3b.png","fullname":"Asankhaya Sharma","name":"codelion","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":350}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.5831394195556641},"editors":["codelion"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1677134945205-62f32eab52ad88c930bb3f3b.png"],"reactions":[],"isReport":false}},{"id":"666ad27daa40cf5abecbe0ac","author":{"_id":"63a61ba064f470027818efca","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1671830348493-noauth.png","fullname":"Anxhelo Diko","name":"D-Anel","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2024-06-13T11:05:33.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"@codelion Thank you","html":"

\n\n@codelion\n\t Thank you

\n","updatedAt":"2024-06-13T11:05:33.400Z","author":{"_id":"63a61ba064f470027818efca","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1671830348493-noauth.png","fullname":"Anxhelo Diko","name":"D-Anel","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.5507365465164185},"editors":["D-Anel"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1671830348493-noauth.png"],"reactions":[],"isReport":false}},{"id":"666b45540c8ae90e196e5258","author":{"_id":"63a61ba064f470027818efca","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1671830348493-noauth.png","fullname":"Anxhelo Diko","name":"D-Anel","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2024-06-13T19:15:32.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"@merve Did you find any solution on why the HF version does not perform? I am having the same issue as @Shalev but in segmentation. It would return a mas of zeros in HF version while works pretty well on jax.","html":"

\n\n@merve\n\t Did you find any solution on why the HF version does not perform? I am having the same issue as \n\n@Shalev\n\t but in segmentation. It would return a mas of zeros in HF version while works pretty well on jax.

\n","updatedAt":"2024-06-13T19:15:32.886Z","author":{"_id":"63a61ba064f470027818efca","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1671830348493-noauth.png","fullname":"Anxhelo Diko","name":"D-Anel","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9643650054931641},"editors":["D-Anel"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1671830348493-noauth.png"],"reactions":[],"isReport":false}}],"pinned":false,"locked":false,"collection":"discussions","isPullRequest":false,"isReport":false},"primaryEmailConfirmed":false,"repo":{"name":"big-vision/paligemma-hf","type":"space"},"discussionRole":0,"acceptLanguages":["*"],"hideComments":true,"repoDiscussionsLocked":false,"isDiscussionAuthor":false}">