Deployed 69dac59 with MkDocs version: 1.6.0

Author: github-actions[bot]

index.html

@@ -440,7 +440,16 @@
     </span>
   </a>
 
-    <nav class="md-nav" aria-label="Benchmark Statistics">
+</li>
+      
+        <li class="md-nav__item">
+  <a href="#experiment-results" class="md-nav__link">
+    <span class="md-ellipsis">
+      Experiment Results
+    </span>
+  </a>
+  
+    <nav class="md-nav" aria-label="Experiment Results">
       <ul class="md-nav__list">
 
           <li class="md-nav__item">
@@ -863,7 +872,16 @@
     </span>
   </a>
 
-    <nav class="md-nav" aria-label="Benchmark Statistics">
+</li>
+      
+        <li class="md-nav__item">
+  <a href="#experiment-results" class="md-nav__link">
+    <span class="md-ellipsis">
+      Experiment Results
+    </span>
+  </a>
+  
+    <nav class="md-nav" aria-label="Experiment Results">
       <ul class="md-nav__list">
 
           <li class="md-nav__item">
@@ -1170,6 +1188,11 @@ <h2 id="benchmark-statistics">Benchmark Statistics</h2>
 </table>
 <p><a class="glightbox" href="figures/SciCode_chart.png" data-type="image" data-width="auto" data-height="auto" data-desc-position="bottom"><img alt="Image Title" src="figures/SciCode_chart.png" /></a>
 <strong>Left:</strong> Distribution of Main Problems   <strong>Right:</strong> Distribution of Subproblems</p>
+<h2 id="experiment-results">Experiment Results</h2>
+<p>We evaluate our model using zero-shot prompts. We keep the prompts general and design different ones for different evaluation setups only to inform the model about the tasks. We keep prompts the same across models and fields, and they contain the model’s main and sub-problem instructions and code for previous subproblems. The standard setup means the model is tested without background knowledge and carrying over generated solutions to previous subproblems. The scientists' annotated background provides the necessary knowledge and reasoning steps to solve the problems, shifting the evaluation’s focus more towards the models’ coding and instruction-following capabilities.
+<a class="glightbox" href="figures/Standard_Setup.png" data-type="image" data-width="auto" data-height="auto" data-desc-position="bottom"><img alt="Image Title" src="figures/Standard_Setup.png" /></a>
+<a class="glightbox" href="figures/Standard_Background.png" data-type="image" data-width="auto" data-height="auto" data-desc-position="bottom"><img alt="Image Title" src="figures/Standard_Background.png" /></a>
+<a class="glightbox" href="figures/Performance_Gain.png" data-type="image" data-width="auto" data-height="auto" data-desc-position="bottom"><img alt="Image Title" src="figures/Performance_Gain.png" /></a></p>
 <h3 id="numerical-linear-algebra">Numerical Linear Algebra</h3>
 <p>1_Conjugate_Gradient</p>
 <p>3_Gauss_Seidel</p>