Skip to content

Commit ccd4f05

Browse files
Deployed e4c065b with MkDocs version: 1.6.0
1 parent 7b3b833 commit ccd4f05

File tree

2 files changed

+2
-2
lines changed

2 files changed

+2
-2
lines changed

index.html

+1-1
Original file line numberDiff line numberDiff line change
@@ -622,7 +622,7 @@ <h2 id="overview">Overview</h2>
622622
<p align="justify">
623623
SciCode sources challenging and realistic research-level coding problems across 6 natural science disciplines, covering a total of 16 subfields. This diverse selection ensures a comprehensive representation of the natural sciences, where extensive code development is essential. SciCode is mainly drawn from the scripts that scientists use in their everyday workflow. Many of these have been used in one or more publications, demonstrating their robustness and correctness.
624624

625-
Among various coding necessities, Scicode mainly focuses on: 1. Numerical methods 2. Simulation of systems 3. Scientific calculation. These are the tasks we believe require intense scientific knowledge and reasoning to optimally test LM’s science capability. The below figure is an example of the combination of 1 and 3.
625+
Among various coding necessities, Scicode mainly focuses on: 1. Numerical methods. 2. Simulation of systems. 3. Scientific calculation. These are the tasks we believe require intense scientific knowledge and reasoning to optimally test LM’s science capability. The below figure is an example of the combination of 1 and 3.
626626

627627
In designing test cases for evaluation, we incorporate domain-specific test cases in addition to numerical cases. These tests are extracted from real scientific workflows: scientists must design domain-specific test cases to verify code accuracy by reproducing results published in papers or matching analytical solutions derived from theoretical models. Each problem goes through <b>3</b> rounds of validation (i.e. by in-domain scientists, out-of-domain scientists, GPT4) for quality control.
628628
</p>

0 commit comments

Comments
 (0)