https://huggingface.co/NexaAIDev/Dolphin\n","updatedAt":"2024-08-29T01:56:10.976Z","author":{"_id":"60f1abe7544c2adfd699860c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg","fullname":"AK","name":"akhaliq","type":"user","isPro":false,"isHf":true,"isHfAdmin":false,"isMod":false,"followerCount":8259}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.3712664246559143},"editors":["akhaliq"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg"],"reactions":[],"isReport":false}},{"id":"66d02b793c5faabc37bb6d7e","author":{"_id":"62d1ddfac58f969c1528f1b5","avatarUrl":"/avatars/75c372a831cde3c7c6dce3bc875488a7.svg","fullname":"Kalle Hilsenbek","name":"Bachstelze","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":8},"createdAt":"2024-08-29T08:04:09.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"What are the results of common benchmarks on the leaderboard?","html":"
What are the results of common benchmarks on the leaderboard?
\n","updatedAt":"2024-08-29T08:04:09.385Z","author":{"_id":"62d1ddfac58f969c1528f1b5","avatarUrl":"/avatars/75c372a831cde3c7c6dce3bc875488a7.svg","fullname":"Kalle Hilsenbek","name":"Bachstelze","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":8}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9582390785217285},"editors":["Bachstelze"],"editorAvatarUrls":["/avatars/75c372a831cde3c7c6dce3bc875488a7.svg"],"reactions":[{"reaction":"👀","users":["heliark"],"count":1}],"isReport":false}},{"id":"66d12184092cc13d0f379622","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":264},"createdAt":"2024-08-30T01:33:56.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [MobileQuant: Mobile-friendly Quantization for On-device Language Models](https://huggingface.co/papers/2408.13933) (2024)\n* [Enhancing Model Performance: Another Approach to Vision-Language Instruction Tuning](https://huggingface.co/papers/2407.17813) (2024)\n* [FocusLLM: Scaling LLM's Context by Parallel Decoding](https://huggingface.co/papers/2408.11745) (2024)\n* [CROME: Cross-Modal Adapters for Efficient Multimodal LLM](https://huggingface.co/papers/2408.06610) (2024)\n* [LowCLIP: Adapting the CLIP Model Architecture for Low-Resource Languages in Multimodal Image Retrieval Task](https://huggingface.co/papers/2408.13909) (2024)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
\nThe following papers were recommended by the Semantic Scholar API
\n- \n
- MobileQuant: Mobile-friendly Quantization for On-device Language Models (2024) \n
- Enhancing Model Performance: Another Approach to Vision-Language Instruction Tuning (2024) \n
- FocusLLM: Scaling LLM's Context by Parallel Decoding (2024) \n
- CROME: Cross-Modal Adapters for Efficient Multimodal LLM (2024) \n
- LowCLIP: Adapting the CLIP Model Architecture for Low-Resource Languages in Multimodal Image Retrieval Task (2024) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
Please see our HF space at :
https://huggingface.co/spaces/NexaAIDev/dolphin
Papers
arxiv:2408.15518
Dolphin: Long Context as a New Modality for Energy-Efficient On-Device Language Models
Abstract
Dolphin, a novel decoder-decoder architecture, enables efficient processing of long contexts in language models by distilling contextual information into memory embeddings, thereby improving energy efficiency and latency without sacrificing accuracy.
AI-generated summary
This paper presents Dolphin, a novel decoder-decoder architecture for energy-efficient processing of long contexts in language models. Our approach addresses the significant energy consumption and latency challenges inherent in on-device models. Dolphin employs a compact 0.5B parameter decoder to distill extensive contextual information into a memory embedding, substantially reducing the input length for the primary 7B parameter decoder model. Inspired by vision-language models, we repurpose the image embedding projector to encode long textual contexts, effectively treating extended context as a distinct modality. This innovative method enables processing of substantially longer contexts without the typical computational overhead associated with extended input sequences. Empirical evaluations demonstrate a 10-fold improvement in energy efficiency and a 5-fold reduction in latency compared to conventional full-length context processing methods without losing quality of the response. Our work contributes to the development of more sustainable and scalable language models for on-device applications, addressing the critical need for energy-efficient and responsive AI technologies in resource-constrained environments while maintaining the accuracy to understand long contexts. This research has implications for the broader field of natural language processing, particularly in the domain of efficient model design for resource-limited settings. By enabling more sophisticated AI capabilities on edge devices, Dolphin paves the way for advanced language processing in a wide range of applications where computational resources are at a premium. The Dolphin model is publicly available at https://huggingface.co/NexaAIDev/Dolphin.
Community
What are the results of common benchmarks on the leaderboard?
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- MobileQuant: Mobile-friendly Quantization for On-device Language Models (2024)
- Enhancing Model Performance: Another Approach to Vision-Language Instruction Tuning (2024)
- FocusLLM: Scaling LLM's Context by Parallel Decoding (2024)
- CROME: Cross-Modal Adapters for Efficient Multimodal LLM (2024)
- LowCLIP: Adapting the CLIP Model Architecture for Low-Resource Languages in Multimodal Image Retrieval Task (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 1
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2408.15518 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2408.15518 in a Space README.md to link it from this page.