https://github.com/NovaSky-AI/SkyThought\n","updatedAt":"2025-02-21T03:04:42.649Z","author":{"_id":"60f1abe7544c2adfd699860c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg","fullname":"AK","name":"akhaliq","type":"user","isPro":false,"isHf":true,"isHfAdmin":false,"isMod":false,"followerCount":8255}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7095736861228943},"editors":["akhaliq"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg"],"reactions":[{"reaction":"❤️","users":["InvidFlower","DachengLi","sugatoray"],"count":3}],"isReport":false}},{"id":"67b8d1a00409462b123b39f9","author":{"_id":"63715b25ffc0489ed7d1f415","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63715b25ffc0489ed7d1f415/xZJepbs0LRqFbW1knnBKR.jpeg","fullname":"Dacheng Li","name":"DachengLi","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":12},"createdAt":"2025-02-21T19:18:56.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Thank you very much @akhaliq for featuring our work! \nThe code has been available at the repo!","html":"
Thank you very much \n\n@akhaliq\n\t for featuring our work!
The code has been available at the repo!
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
\nThe following papers were recommended by the Semantic Scholar API
\n- \n
- Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning (2025) \n
- MetaSC: Test-Time Safety Specification Optimization for Language Models (2025) \n
- Benchmarking Prompt Engineering Techniques for Secure Code Generation with GPT Models (2025) \n
- Dynamic Scaling of Unit Tests for Code Reward Modeling (2025) \n
- Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling (2025) \n
- MCTS-Judge: Test-Time Scaling in LLM-as-a-Judge for Code Correctness Evaluation (2025) \n
- FairCode: Evaluating Social Bias of LLMs in Code Generation (2025) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
Papers
arxiv:2502.14382
S*: Test Time Scaling for Code Generation
Published on Feb 20
Abstract
A hybrid test-time scaling framework improves code generation coverage and accuracy across various models and domains.
AI-generated summary
Increasing test-time compute for LLMs shows promise across domains but remains underexplored in code generation, despite extensive study in math. In this paper, we propose S*, the first hybrid test-time scaling framework that substantially improves the coverage and selection accuracy of generated code. S* extends the existing parallel scaling paradigm with sequential scaling to push performance boundaries. It further leverages a novel selection mechanism that adaptively generates distinguishing inputs for pairwise comparison, combined with execution-grounded information to robustly identify correct solutions. We evaluate across 12 Large Language Models and Large Reasoning Model and show: (1) S* consistently improves performance across model families and sizes, enabling a 3B model to outperform GPT-4o-mini; (2) S* enables non-reasoning models to surpass reasoning models - GPT-4o-mini with S* outperforms o1-preview by 3.7% on LiveCodeBench; (3) S* further boosts state-of-the-art reasoning models - DeepSeek-R1-Distill-Qwen-32B with S* achieves 85.7% on LiveCodeBench, approaching o1 (high) at 88.5%. Code will be available under https://github.com/NovaSky-AI/SkyThought.
Community
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning (2025)
- MetaSC: Test-Time Safety Specification Optimization for Language Models (2025)
- Benchmarking Prompt Engineering Techniques for Secure Code Generation with GPT Models (2025)
- Dynamic Scaling of Unit Tests for Code Reward Modeling (2025)
- Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling (2025)
- MCTS-Judge: Test-Time Scaling in LLM-as-a-Judge for Code Correctness Evaluation (2025)
- FairCode: Evaluating Social Bias of LLMs in Code Generation (2025)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2502.14382 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2502.14382 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2502.14382 in a Space README.md to link it from this page.