Skip to content

Commit 612c3f0

Browse files
github-actions[bot]github-actions[bot]
committed
Deployed 0d1b966 with MkDocs version: 1.6.0
1 parent 571c3f1 commit 612c3f0

File tree

2 files changed

+54
-42
lines changed

2 files changed

+54
-42
lines changed

‎index.html

Lines changed: 53 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -605,48 +605,60 @@
605605

606606

607607
<h1 id="scicode-a-research-coding-benchmark-curated-by-scientists">SciCode: A Research Coding Benchmark Curated by Scientists</h1>
608-
<p><strong>Minyang Tian<sup>1,2<em></sup>, Luyu Gao<sup>3</em></sup>, Shizhuo Dylan Zhang<sup>1</sup>, Xinan Chen<sup>1†</sup>, Cunwei Fan<sup>1†</sup>, Xuefei Guo<sup>1†</sup>, Roland Haas<sup>1†</sup>, Pan Ji<sup>4†</sup>, Kittithat Krongchon<sup>1†</sup>, Yao Li<sup>1†</sup>, Shengyan Liu<sup>1†</sup>, Di Luo<sup>5,6,11†</sup>, Yutao Ma<sup>7†</sup>, Hao Tong<sup>1†</sup>, Kha Trinh<sup>7†</sup>, Chenyu Tian<sup>8†</sup>, Zihan Wang<sup>1†</sup>, Bohao Wu<sup>1†</sup>, Yanyu Xiong<sup>9†</sup>, Shengzhu Yin<sup>1†</sup>, Minhui Zhu<sup>1†</sup>, Kilian Lieret<sup>10</sup>, Yanxin Lu<sup>1</sup>, Genglin Liu<sup>1</sup>, Yufeng Du<sup>1</sup>, Tianhua Tao<sup>1</sup>, Ofir Press<sup>10</sup>, Jamie Callan<sup>3</sup>, Eliu Huerta<sup>1,2,7</sup>, Hao Peng<sup>1</sup></strong></p>
609-
<p><sup>1</sup>University of Illinois Urbana-Champaign<br />
610-
<sup>2</sup>Argonne National Laboratory<br />
611-
<sup>3</sup>Carnegie Mellon University<br />
612-
<sup>4</sup>University of North Carolina at Chapel Hill<br />
613-
<sup>5</sup>Massachusetts Institute of Technology<br />
614-
<sup>6</sup>Harvard University<br />
615-
<sup>7</sup>University of Chicago<br />
616-
<sup>8</sup>University of Texas at Austin<br />
617-
<sup>9</sup>Stanford University<br />
618-
<sup>10</sup>Princeton University<br />
619-
<sup>11</sup>The NSF AI Institute for Artificial Intelligence and Fundamental Interactions </p>
620-
<ul>
621-
<li>Equal contribution lead authors. † Data curation, alphabetical order.</li>
622-
</ul>
623-
<div class="grid cards">
624-
<ul>
625-
<li>
626-
<p><span class="twemoji lg middle"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M18 22a2 2 0 0 0 2-2V4a2 2 0 0 0-2-2h-6v7L9.5 7.5 7 9V2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12Z"/></svg></span> <strong>Leaderboard</strong></p>
627-
<hr />
628-
<p>How good are LMs at science, really?</p>
629-
<p><a href="leaderboard/"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M13.22 19.03a.75.75 0 0 1 0-1.06L18.19 13H3.75a.75.75 0 0 1 0-1.5h14.44l-4.97-4.97a.749.749 0 0 1 .326-1.275.749.749 0 0 1 .734.215l6.25 6.25a.75.75 0 0 1 0 1.06l-6.25 6.25a.75.75 0 0 1-1.06 0Z"/></svg></span> Browse the results</a></p>
630-
</li>
631-
<li>
632-
<p><span class="twemoji lg middle"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M18 22a2 2 0 0 0 2-2V4a2 2 0 0 0-2-2h-6v7L9.5 7.5 7 9V2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12Z"/></svg></span> <strong>Preprint</strong></p>
633-
<hr />
634-
<p>Learn all the details</p>
635-
<p><a href="https://arxiv.com"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M13.22 19.03a.75.75 0 0 1 0-1.06L18.19 13H3.75a.75.75 0 0 1 0-1.5h14.44l-4.97-4.97a.749.749 0 0 1 .326-1.275.749.749 0 0 1 .734.215l6.25 6.25a.75.75 0 0 1 0 1.06l-6.25 6.25a.75.75 0 0 1-1.06 0Z"/></svg></span> Read the preprint</a></p>
636-
</li>
637-
</ul>
608+
<p align="center">
609+
**Minyang Tian<sup>1,2*</sup>, Luyu Gao<sup>3*</sup>, Shizhuo Dylan Zhang<sup>1</sup>, Xinan Chen<sup>1†</sup>, Cunwei Fan<sup>1†</sup>, Xuefei Guo<sup>1†</sup>, Roland Haas<sup>1†</sup>, Pan Ji<sup>4†</sup>, Kittithat Krongchon<sup>1†</sup>, Yao Li<sup>1†</sup>, Shengyan Liu<sup>1†</sup>, Di Luo<sup>5,6,11†</sup>, Yutao Ma<sup>7†</sup>, Hao Tong<sup>1†</sup>, Kha Trinh<sup>7†</sup>, Chenyu Tian<sup>8†</sup>, Zihan Wang<sup>1†</sup>, Bohao Wu<sup>1†</sup>, Yanyu Xiong<sup>9†</sup>, Shengzhu Yin<sup>1†</sup>, Minhui Zhu<sup>1†</sup>, Kilian Lieret<sup>10</sup>, Yanxin Lu<sup>1</sup>, Genglin Liu<sup>1</sup>, Yufeng Du<sup>1</sup>, Tianhua Tao<sup>1</sup>, Ofir Press<sup>10</sup>, Jamie Callan<sup>3</sup>, Eliu Huerta<sup>1,2,7</sup>, Hao Peng<sup>1</sup>**
610+
<p align="center">
611+
<sup>1</sup>University of Illinois Urbana-Champaign
612+
<sup>2</sup>Argonne National Laboratory
613+
<sup>3</sup>Carnegie Mellon University
614+
<sup>4</sup>University of North Carolina at Chapel Hill
615+
<sup>5</sup>Massachusetts Institute of Technology
616+
<sup>6</sup>Harvard University
617+
<sup>7</sup>University of Chicago
618+
<sup>8</sup>University of Texas at Austin
619+
<sup>9</sup>Stanford University
620+
<sup>10</sup>Princeton University
621+
<sup>11</sup>The NSF AI Institute for Artificial Intelligence and Fundamental Interactions
622+
<p align="center">
623+
* Equal contribution lead authors. † Data curation, alphabetical order.
624+
625+
626+
627+
628+
<div class="grid cards" markdown>
629+
630+
- :material-book:{ .lg .middle } __Leaderboard__
631+
632+
---
633+
634+
How good are LMs at science, really?
635+
636+
[:octicons-arrow-right-24: Browse the results](leaderboard.md)
637+
638+
- :material-book:{ .lg .middle } __Preprint__
639+
640+
---
641+
642+
Learn all the details
643+
644+
[:octicons-arrow-right-24: Read the preprint](https://arxiv.com)
638645
</div>
639-
<p>commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, pellentesque eu, pretium quis, sem. Nulla consequat massa quis enim. Donec pede justo, fringilla vel, aliquet nec, vulputate eget, arcu. In enim justo, rhoncus ut, imperdiet a, venenatis vitae, justo. Nullam dictum felis eu pede mollis pretium. Integer tincidunt. Cras dapibus. Vivamus elementum semper nisi. Aenean vulputate eleifend tellus. </p>
640-
<p>Aenean leo ligula, porttitor eu, consequat vitae, eleifend ac, enim. Aliquam lorem ante, dapibus in, viverra quis, feugiat a, tellus. Phasellus viverra nulla ut metus varius laoreet. Quisque rutrum. Aenean imperdiet. Etiam ultricies nisi vel augue. Curabitur ullamcorper ultricies nisi. Nam eget dui. Etiam rhoncus. Maecenas tempus, tellus eget condimentum rhoncus, sem quam semper libero, sit amet adipiscing sem neque sed ipsum. Nam quam nunc, blandit vel, luctus pulvinar, hendrerit id, lorem. Maecenas nec odio et ante tincidunt tempus. Donec vitae sapien ut libero venenatis faucibus. Nullam quis ante. Etiam sit amet orci eget eros faucibus tincidunt. Duis leo. Sed fringilla mauris sit amet nibh. Donec sodales sagittis magna. Sed consequat, leo eget bibendum sodales, augue velit cursus nunc.</p>
641-
<div class="grid cards">
642-
<ul>
643-
<li>
644-
<p><span class="twemoji lg middle"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M8 5.14v14l11-7-11-7Z"/></svg></span> <strong>Installation &amp; usage</strong></p>
645-
<hr />
646-
<p>Learn how to evaluate your model</p>
647-
<p><a href="docs/"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M13.22 19.03a.75.75 0 0 1 0-1.06L18.19 13H3.75a.75.75 0 0 1 0-1.5h14.44l-4.97-4.97a.749.749 0 0 1 .326-1.275.749.749 0 0 1 .734.215l6.25 6.25a.75.75 0 0 1 0 1.06l-6.25 6.25a.75.75 0 0 1-1.06 0Z"/></svg></span> Read the docs</a></p>
648-
</li>
649-
</ul>
646+
647+
commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, pellentesque eu, pretium quis, sem. Nulla consequat massa quis enim. Donec pede justo, fringilla vel, aliquet nec, vulputate eget, arcu. In enim justo, rhoncus ut, imperdiet a, venenatis vitae, justo. Nullam dictum felis eu pede mollis pretium. Integer tincidunt. Cras dapibus. Vivamus elementum semper nisi. Aenean vulputate eleifend tellus.
648+
649+
Aenean leo ligula, porttitor eu, consequat vitae, eleifend ac, enim. Aliquam lorem ante, dapibus in, viverra quis, feugiat a, tellus. Phasellus viverra nulla ut metus varius laoreet. Quisque rutrum. Aenean imperdiet. Etiam ultricies nisi vel augue. Curabitur ullamcorper ultricies nisi. Nam eget dui. Etiam rhoncus. Maecenas tempus, tellus eget condimentum rhoncus, sem quam semper libero, sit amet adipiscing sem neque sed ipsum. Nam quam nunc, blandit vel, luctus pulvinar, hendrerit id, lorem. Maecenas nec odio et ante tincidunt tempus. Donec vitae sapien ut libero venenatis faucibus. Nullam quis ante. Etiam sit amet orci eget eros faucibus tincidunt. Duis leo. Sed fringilla mauris sit amet nibh. Donec sodales sagittis magna. Sed consequat, leo eget bibendum sodales, augue velit cursus nunc.
650+
651+
<div class="grid cards" markdown>
652+
653+
654+
- :material-play:{ .lg .middle } __Installation & usage__
655+
656+
---
657+
658+
Learn how to evaluate your model
659+
660+
[:octicons-arrow-right-24: Read the docs](docs/index.md)
661+
650662
</div>
651663

652664

‎search/search_index.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"SciCode: A Research Coding Benchmark Curated by Scientists","text":"<p>Minyang Tian<sup>1,2, Luyu Gao<sup>3</sup>, Shizhuo Dylan Zhang<sup>1</sup>, Xinan Chen<sup>1\u2020</sup>, Cunwei Fan<sup>1\u2020</sup>, Xuefei Guo<sup>1\u2020</sup>, Roland Haas<sup>1\u2020</sup>, Pan Ji<sup>4\u2020</sup>, Kittithat Krongchon<sup>1\u2020</sup>, Yao Li<sup>1\u2020</sup>, Shengyan Liu<sup>1\u2020</sup>, Di Luo<sup>5,6,11\u2020</sup>, Yutao Ma<sup>7\u2020</sup>, Hao Tong<sup>1\u2020</sup>, Kha Trinh<sup>7\u2020</sup>, Chenyu Tian<sup>8\u2020</sup>, Zihan Wang<sup>1\u2020</sup>, Bohao Wu<sup>1\u2020</sup>, Yanyu Xiong<sup>9\u2020</sup>, Shengzhu Yin<sup>1\u2020</sup>, Minhui Zhu<sup>1\u2020</sup>, Kilian Lieret<sup>10</sup>, Yanxin Lu<sup>1</sup>, Genglin Liu<sup>1</sup>, Yufeng Du<sup>1</sup>, Tianhua Tao<sup>1</sup>, Ofir Press<sup>10</sup>, Jamie Callan<sup>3</sup>, Eliu Huerta<sup>1,2,7</sup>, Hao Peng<sup>1</sup> <p><sup>1</sup>University of Illinois Urbana-Champaign <sup>2</sup>Argonne National Laboratory <sup>3</sup>Carnegie Mellon University <sup>4</sup>University of North Carolina at Chapel Hill <sup>5</sup>Massachusetts Institute of Technology <sup>6</sup>Harvard University <sup>7</sup>University of Chicago <sup>8</sup>University of Texas at Austin <sup>9</sup>Stanford University <sup>10</sup>Princeton University <sup>11</sup>The NSF AI Institute for Artificial Intelligence and Fundamental Interactions </p> <ul> <li>Equal contribution lead authors. \u2020 Data curation, alphabetical order.</li> </ul> <ul> <li> <p> Leaderboard</p> <p>How good are LMs at science, really?</p> <p> Browse the results</p> </li> <li> <p> Preprint</p> <p>Learn all the details</p> <p> Read the preprint</p> </li> </ul> <p>commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, pellentesque eu, pretium quis, sem. Nulla consequat massa quis enim. Donec pede justo, fringilla vel, aliquet nec, vulputate eget, arcu. In enim justo, rhoncus ut, imperdiet a, venenatis vitae, justo. Nullam dictum felis eu pede mollis pretium. Integer tincidunt. Cras dapibus. Vivamus elementum semper nisi. Aenean vulputate eleifend tellus. </p> <p>Aenean leo ligula, porttitor eu, consequat vitae, eleifend ac, enim. Aliquam lorem ante, dapibus in, viverra quis, feugiat a, tellus. Phasellus viverra nulla ut metus varius laoreet. Quisque rutrum. Aenean imperdiet. Etiam ultricies nisi vel augue. Curabitur ullamcorper ultricies nisi. Nam eget dui. Etiam rhoncus. Maecenas tempus, tellus eget condimentum rhoncus, sem quam semper libero, sit amet adipiscing sem neque sed ipsum. Nam quam nunc, blandit vel, luctus pulvinar, hendrerit id, lorem. Maecenas nec odio et ante tincidunt tempus. Donec vitae sapien ut libero venenatis faucibus. Nullam quis ante. Etiam sit amet orci eget eros faucibus tincidunt. Duis leo. Sed fringilla mauris sit amet nibh. Donec sodales sagittis magna. Sed consequat, leo eget bibendum sodales, augue velit cursus nunc.</p> <ul> <li> <p> Installation &amp; usage</p> <p>Learn how to evaluate your model</p> <p> Read the docs</p> </li> </ul>"},{"location":"_footer/","title":"footer","text":"<ul> <li> <p> Something broken? Report bug</p> </li> <li> <p> Something unclear? Ask question</p> </li> </ul>"},{"location":"leaderboard/","title":"Leaderboard","text":"<p> date author model score 240712 scicode gpt4 0.8 240712 scicode gpt4o 0.8 <p></p> <p>How to submit</p> <p>Want to submit your own model? Head over to the documentation.</p>"},{"location":"leaderboard_table/","title":"Leaderboard table","text":"date author model score 240712 scicode gpt4 0.8 240712 scicode gpt4o 0.8"}]}
1+
{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"SciCode: A Research Coding Benchmark Curated by Scientists","text":"<p> **Minyang Tian<sup>1,2*</sup>, Luyu Gao<sup>3*</sup>, Shizhuo Dylan Zhang<sup>1</sup>, Xinan Chen<sup>1\u2020</sup>, Cunwei Fan<sup>1\u2020</sup>, Xuefei Guo<sup>1\u2020</sup>, Roland Haas<sup>1\u2020</sup>, Pan Ji<sup>4\u2020</sup>, Kittithat Krongchon<sup>1\u2020</sup>, Yao Li<sup>1\u2020</sup>, Shengyan Liu<sup>1\u2020</sup>, Di Luo<sup>5,6,11\u2020</sup>, Yutao Ma<sup>7\u2020</sup>, Hao Tong<sup>1\u2020</sup>, Kha Trinh<sup>7\u2020</sup>, Chenyu Tian<sup>8\u2020</sup>, Zihan Wang<sup>1\u2020</sup>, Bohao Wu<sup>1\u2020</sup>, Yanyu Xiong<sup>9\u2020</sup>, Shengzhu Yin<sup>1\u2020</sup>, Minhui Zhu<sup>1\u2020</sup>, Kilian Lieret<sup>10</sup>, Yanxin Lu<sup>1</sup>, Genglin Liu<sup>1</sup>, Yufeng Du<sup>1</sup>, Tianhua Tao<sup>1</sup>, Ofir Press<sup>10</sup>, Jamie Callan<sup>3</sup>, Eliu Huerta<sup>1,2,7</sup>, Hao Peng<sup>1</sup>** <p> <sup>1</sup>University of Illinois Urbana-Champaign <sup>2</sup>Argonne National Laboratory <sup>3</sup>Carnegie Mellon University <sup>4</sup>University of North Carolina at Chapel Hill <sup>5</sup>Massachusetts Institute of Technology <sup>6</sup>Harvard University <sup>7</sup>University of Chicago <sup>8</sup>University of Texas at Austin <sup>9</sup>Stanford University <sup>10</sup>Princeton University <sup>11</sup>The NSF AI Institute for Artificial Intelligence and Fundamental Interactions <p> * Equal contribution lead authors. \u2020 Data curation, alphabetical order. - :material-book:{ .lg .middle } __Leaderboard__ --- How good are LMs at science, really? [:octicons-arrow-right-24: Browse the results](leaderboard.md) - :material-book:{ .lg .middle } __Preprint__ --- Learn all the details [:octicons-arrow-right-24: Read the preprint](https://arxiv.com) commodo ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Donec quam felis, ultricies nec, pellentesque eu, pretium quis, sem. Nulla consequat massa quis enim. Donec pede justo, fringilla vel, aliquet nec, vulputate eget, arcu. In enim justo, rhoncus ut, imperdiet a, venenatis vitae, justo. Nullam dictum felis eu pede mollis pretium. Integer tincidunt. Cras dapibus. Vivamus elementum semper nisi. Aenean vulputate eleifend tellus. Aenean leo ligula, porttitor eu, consequat vitae, eleifend ac, enim. Aliquam lorem ante, dapibus in, viverra quis, feugiat a, tellus. Phasellus viverra nulla ut metus varius laoreet. Quisque rutrum. Aenean imperdiet. Etiam ultricies nisi vel augue. Curabitur ullamcorper ultricies nisi. Nam eget dui. Etiam rhoncus. Maecenas tempus, tellus eget condimentum rhoncus, sem quam semper libero, sit amet adipiscing sem neque sed ipsum. Nam quam nunc, blandit vel, luctus pulvinar, hendrerit id, lorem. Maecenas nec odio et ante tincidunt tempus. Donec vitae sapien ut libero venenatis faucibus. Nullam quis ante. Etiam sit amet orci eget eros faucibus tincidunt. Duis leo. Sed fringilla mauris sit amet nibh. Donec sodales sagittis magna. Sed consequat, leo eget bibendum sodales, augue velit cursus nunc. - :material-play:{ .lg .middle } __Installation &amp; usage__ --- Learn how to evaluate your model [:octicons-arrow-right-24: Read the docs](docs/index.md)"},{"location":"_footer/","title":"footer","text":"<ul> <li> <p> Something broken? Report bug</p> </li> <li> <p> Something unclear? Ask question</p> </li> </ul>"},{"location":"leaderboard/","title":"Leaderboard","text":"<p> date author model score 240712 scicode gpt4 0.8 240712 scicode gpt4o 0.8 <p></p> <p>How to submit</p> <p>Want to submit your own model? Head over to the documentation.</p>"},{"location":"leaderboard_table/","title":"Leaderboard table","text":"date author model score 240712 scicode gpt4 0.8 240712 scicode gpt4o 0.8"}]}

0 commit comments

Comments
 (0)