rhaymisoncristian@gmail.com or call me on linkedIn and i will share the notebook with you.

\n","updatedAt":"2024-03-25T20:53:05.056Z","author":{"_id":"64a15716ddcdc3438ed93bde","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64a15716ddcdc3438ed93bde/lVhYKhLKOwCpI-b1PbVt6.jpeg","fullname":"Rhaymison Cristian","name":"rhaymison","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":52}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8502079844474792},"editors":["rhaymison"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/64a15716ddcdc3438ed93bde/lVhYKhLKOwCpI-b1PbVt6.jpeg"],"reactions":[],"isReport":false}},{"id":"6658237f1436864bccd5aee4","author":{"_id":"64a71c211e4dd9f3548c2786","avatarUrl":"/avatars/d3d541b05e2a8e17dd89a0d0dd46b0c1.svg","fullname":"Dawid Ewald ","name":"Wielebnyd","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2024-05-30T06:58:07.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"@rhaymison Is it possible to share the script you used?","html":"

\n\n@rhaymison\n\t Is it possible to share the script you used?

\n","updatedAt":"2024-05-30T06:58:07.597Z","author":{"_id":"64a71c211e4dd9f3548c2786","avatarUrl":"/avatars/d3d541b05e2a8e17dd89a0d0dd46b0c1.svg","fullname":"Dawid Ewald ","name":"Wielebnyd","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7903512716293335},"editors":["Wielebnyd"],"editorAvatarUrls":["/avatars/d3d541b05e2a8e17dd89a0d0dd46b0c1.svg"],"reactions":[{"reaction":"🔥","users":["rhaymison"],"count":1}],"isReport":false}},{"id":"665823f26e1aa2b59f730109","author":{"_id":"64a15716ddcdc3438ed93bde","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64a15716ddcdc3438ed93bde/lVhYKhLKOwCpI-b1PbVt6.jpeg","fullname":"Rhaymison Cristian","name":"rhaymison","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":52,"isOwner":false,"isOrgMember":false},"createdAt":"2024-05-30T07:00:02.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"@Wielebnyd sure, could you send me a mail to share with you the full notebook ?","html":"

\n\n@Wielebnyd\n\t sure, could you send me a mail to share with you the full notebook ?

\n","updatedAt":"2024-05-30T07:00:02.304Z","author":{"_id":"64a15716ddcdc3438ed93bde","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64a15716ddcdc3438ed93bde/lVhYKhLKOwCpI-b1PbVt6.jpeg","fullname":"Rhaymison Cristian","name":"rhaymison","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":52}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9023318290710449},"editors":["rhaymison"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/64a15716ddcdc3438ed93bde/lVhYKhLKOwCpI-b1PbVt6.jpeg"],"reactions":[],"isReport":false}},{"id":"6658285a58ce7efee6deeac7","author":{"_id":"64a71c211e4dd9f3548c2786","avatarUrl":"/avatars/d3d541b05e2a8e17dd89a0d0dd46b0c1.svg","fullname":"Dawid Ewald ","name":"Wielebnyd","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2024-05-30T07:18:50.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"@rhaymison -Thank you, I just sent you an email.","html":"

\n\n@rhaymison\n\t -Thank you, I just sent you an email.

\n","updatedAt":"2024-05-30T07:18:50.875Z","author":{"_id":"64a71c211e4dd9f3548c2786","avatarUrl":"/avatars/d3d541b05e2a8e17dd89a0d0dd46b0c1.svg","fullname":"Dawid Ewald ","name":"Wielebnyd","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8521878719329834},"editors":["Wielebnyd"],"editorAvatarUrls":["/avatars/d3d541b05e2a8e17dd89a0d0dd46b0c1.svg"],"reactions":[{"reaction":"❤️","users":["rhaymison"],"count":1}],"isReport":false}},{"id":"666364ce5c4cd59ad5d29011","author":{"_id":"65c4dc611f4d1a70a90979d4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65c4dc611f4d1a70a90979d4/ry3ozBsPnymBY4fPWf1F-.jpeg","fullname":"Soufyane Moudabbir","name":"soufyane","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isOwner":false,"isOrgMember":false},"createdAt":"2024-06-07T19:51:42.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"@rhaymison hello sir I'm intersting in your work can you give some informations about the prompt you used and lora rank you used please \n\nI want to fine tune gemma 2b on 40k row english and darija(moroccan language)\n","html":"

\n\n@rhaymison\n\t hello sir I'm intersting in your work can you give some informations about the prompt you used and lora rank you used please

I want to fine tune gemma 2b on 40k row english and darija(moroccan language)

\n","updatedAt":"2024-06-07T19:51:42.073Z","author":{"_id":"65c4dc611f4d1a70a90979d4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65c4dc611f4d1a70a90979d4/ry3ozBsPnymBY4fPWf1F-.jpeg","fullname":"Soufyane Moudabbir","name":"soufyane","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.879887044429779},"editors":["soufyane"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/65c4dc611f4d1a70a90979d4/ry3ozBsPnymBY4fPWf1F-.jpeg"],"reactions":[{"reaction":"🚀","users":["rhaymison"],"count":1}],"isReport":false}},{"id":"6663f5ee127474a70872b582","author":{"_id":"64a15716ddcdc3438ed93bde","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64a15716ddcdc3438ed93bde/lVhYKhLKOwCpI-b1PbVt6.jpeg","fullname":"Rhaymison Cristian","name":"rhaymison","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":52,"isOwner":false,"isOrgMember":false},"createdAt":"2024-06-08T06:10:54.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"@SaadManzur \nI received your email, i will give your more info. Thanks\n","html":"

\n\n@SaadManzur\n\t
I received your email, i will give your more info. Thanks

\n","updatedAt":"2024-06-08T06:10:54.098Z","author":{"_id":"64a15716ddcdc3438ed93bde","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64a15716ddcdc3438ed93bde/lVhYKhLKOwCpI-b1PbVt6.jpeg","fullname":"Rhaymison Cristian","name":"rhaymison","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":52}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9755904674530029},"editors":["rhaymison"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/64a15716ddcdc3438ed93bde/lVhYKhLKOwCpI-b1PbVt6.jpeg"],"reactions":[{"reaction":"❤️","users":["soufyane"],"count":1}],"isReport":false}},{"id":"6780a81db5e1f9a449c7c855","author":{"_id":"6657bebbb27e330219468bf0","avatarUrl":"/avatars/1ab8c0976606ddc4575fad7a539f9e8f.svg","fullname":"Manit Roy","name":"manitroy","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2025-01-10T04:54:53.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"@rhaymison Can you share the notebook with me?\n","html":"

\n\n@rhaymison\n\t Can you share the notebook with me?

\n","updatedAt":"2025-01-10T04:55:37.827Z","author":{"_id":"6657bebbb27e330219468bf0","avatarUrl":"/avatars/1ab8c0976606ddc4575fad7a539f9e8f.svg","fullname":"Manit Roy","name":"manitroy","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.9324789047241211},"editors":["manitroy"],"editorAvatarUrls":["/avatars/1ab8c0976606ddc4575fad7a539f9e8f.svg"],"reactions":[],"isReport":false}}],"pinned":false,"locked":false,"collection":"discussions","isPullRequest":false,"isReport":false},"repo":{"name":"google/gemma-7b","type":"model"},"activeTab":"discussion","discussionRole":0,"watched":false,"muted":false,"repoDiscussionsLocked":false}">

Finetuning Genna for Foreign Language

#78

by deleted - opened Mar 21, 2024

Discussion

rhaymisoncristian@gmail.com or call me on linkedIn and i will share the notebook with you.

\n\n@rhaymison\n\t Is it possible to share the script you used?

\n\n@Wielebnyd\n\t sure, could you send me a mail to share with you the full notebook ?

\n\n@rhaymison\n\t -Thank you, I just sent you an email.

\n\n@rhaymison\n\t hello sir I'm intersting in your work can you give some informations about the prompt you used and lora rank you used please

I want to fine tune gemma 2b on 40k row english and darija(moroccan language)

\n\n@SaadManzur\n\t
I received your email, i will give your more info. Thanks

\n\n@rhaymison\n\t Can you share the notebook with me?

\n","updatedAt":"2025-01-10T04:55:37.827Z","author":{"_id":"6657bebbb27e330219468bf0","avatarUrl":"/avatars/1ab8c0976606ddc4575fad7a539f9e8f.svg","fullname":"Manit Roy","name":"manitroy","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.9324789047241211},"editors":["manitroy"],"editorAvatarUrls":["/avatars/1ab8c0976606ddc4575fad7a539f9e8f.svg"],"reactions":[],"isReport":false}}],"pinned":false,"locked":false,"collection":"discussions","isPullRequest":false,"isReport":false},"primaryEmailConfirmed":false,"repo":{"name":"google/gemma-7b","type":"model"},"discussionRole":0,"acceptLanguages":["*"],"hideComments":true,"repoDiscussionsLocked":false,"isDiscussionAuthor":false}">

deleted

Mar 21, 2024

I am attempting to fine-tune Gemma for one of the languages on which it has been pretrained. Could you provide any suggestions regarding the optimal size of the dataset to ensure a noticeable improvement in performance? The best format for the training files? Any other recommendations? Thank you.

Ekabilisim

Mar 22, 2024

This comment has been hidden

rhaymison

Mar 23, 2024

•

edited Mar 23, 2024

@user1357925

hello friend. I had a good response from the gemma 2b model using this format to pass to the dataset. I did the Fine tuning for Brazilian Portuguese. He follows
I have 2 datasets. One for mental 36k (gemma 2b )
another with 100k for instruct ( gemma 7b )

def formatting_func(example):
     instruction = example['question']
     output = example['answer']
     text = f"<start_of_turn>user\n{instruction}<end_of_turn> <start_of_turn>model\n{output}<end_of_turn>"
     return text