https://github.com/YangLing0818/SuperCorrect-llm\n","updatedAt":"2024-10-14T01:51:43.061Z","author":{"_id":"64fde4e252e82dd432b74ce9","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64fde4e252e82dd432b74ce9/-CQZbBP7FsPPyawYrsi4z.jpeg","fullname":"Ling Yang","name":"Lingaaaaaaa","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":10}},"numEdits":0,"editors":["Lingaaaaaaa"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/64fde4e252e82dd432b74ce9/-CQZbBP7FsPPyawYrsi4z.jpeg"],"reactions":[],"isReport":false}},{"id":"670c7955a48acae935ee594f","author":{"_id":"64fde4e252e82dd432b74ce9","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64fde4e252e82dd432b74ce9/-CQZbBP7FsPPyawYrsi4z.jpeg","fullname":"Ling Yang","name":"Lingaaaaaaa","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":10},"createdAt":"2024-10-14T01:52:21.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"","html":"
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
\nThe following papers were recommended by the Semantic Scholar API
\n- \n
- S3c-Math: Spontaneous Step-level Self-correction Makes Large Language Models Better Mathematical Reasoners (2024) \n
- Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification (2024) \n
- Subtle Errors Matter: Preference Learning via Error-injected Self-editing (2024) \n
- BEATS: Optimizing LLM Mathematical Capabilities with BackVerify and Adaptive Disambiguate based Efficient Tree Search (2024) \n
- Self-Correction is More than Refinement: A Learning Framework for Visual and Language Reasoning Tasks (2024) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
Papers
arxiv:2410.09008
SuperCorrect: Supervising and Correcting Language Models with Error-Driven Insights
Published on Oct 11, 2024
Abstract
SuperCorrect uses a large teacher model to guide and correct a smaller student model's reasoning and reflection processes, significantly improving its performance in complex mathematical tasks.
AI-generated summary
Large language models (LLMs) like GPT-4, PaLM, and LLaMA have shown significant improvements in various reasoning tasks. However, smaller models such as Llama-3-8B and DeepSeekMath-Base still struggle with complex mathematical reasoning because they fail to effectively identify and correct reasoning errors. Recent reflection-based methods aim to address these issues by enabling self-reflection and self-correction, but they still face challenges in independently detecting errors in their reasoning steps. To overcome these limitations, we propose SuperCorrect, a novel two-stage framework that uses a large teacher model to supervise and correct both the reasoning and reflection processes of a smaller student model. In the first stage, we extract hierarchical high-level and detailed thought templates from the teacher model to guide the student model in eliciting more fine-grained reasoning thoughts. In the second stage, we introduce cross-model collaborative direct preference optimization (DPO) to enhance the self-correction abilities of the student model by following the teacher's correction traces during training. This cross-model DPO approach teaches the student model to effectively locate and resolve erroneous thoughts with error-driven insights from the teacher model, breaking the bottleneck of its thoughts and acquiring new skills and knowledge to tackle challenging problems. Extensive experiments consistently demonstrate our superiority over previous methods. Notably, our SuperCorrect-7B model significantly surpasses powerful DeepSeekMath-7B by 7.8%/5.3% and Qwen2.5-Math-7B by 15.1%/6.3% on MATH/GSM8K benchmarks, achieving new SOTA performance among all 7B models. Code: https://github.com/YangLing0818/SuperCorrect-llm
Community
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- S3c-Math: Spontaneous Step-level Self-correction Makes Large Language Models Better Mathematical Reasoners (2024)
- Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification (2024)
- Subtle Errors Matter: Preference Learning via Error-injected Self-editing (2024)
- BEATS: Optimizing LLM Mathematical Capabilities with BackVerify and Adaptive Disambiguate based Efficient Tree Search (2024)
- Self-Correction is More than Refinement: A Learning Framework for Visual and Language Reasoning Tasks (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 10
Browse 10 models citing this paperDatasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2410.09008 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2410.09008 in a Space README.md to link it from this page.