Skip to content

Commit 2d43ddb

Browse files
committed
feat(courselab): update PKU Pintos course details to include link and year
1 parent 6c97b73 commit 2d43ddb

File tree

6 files changed

+18
-14
lines changed

6 files changed

+18
-14
lines changed

benchmarks/courseexam_bench/data/raw/cs162_operating_systems_and_system_programming_summer_2022_final/exam.md

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ Your answer should be either "True" or "False", followed by a brief explanation
6767
"type": "Freeform",
6868
"tags": ["operating-systems", "virtual-memory", "page-faults"],
6969
"reference_materials": ["rust_reference.md"],
70-
"answer": "False. FIFO and Belady's anomaly",
70+
"answer": "False. FIFO and Belady's anomaly.",
7171
"llm_judge_instructions": "Award 1 point for correctly answering False. Award 1 point for a valid explanation. Longer explanations may get no credit."
7272
}
7373
```
@@ -147,7 +147,7 @@ Your answer should be either "True" or "False", followed by a brief explanation
147147
"type": "Freeform",
148148
"tags": ["distributed-systems", "idempotency"],
149149
"reference_materials": ["rust_reference.md"],
150-
"answer": "False. f(f(x)) != f(x)",
150+
"answer": "False. f(f(x)) != f(x).",
151151
"llm_judge_instructions": "Award 1 point for correctly answering False. Award 1 point for a valid explanation. Longer explanations may get no credit."
152152
}
153153
```
@@ -503,7 +503,7 @@ Please answer in THREE SENTENCES OR LESS. Longer explanations may get no credit
503503
"type": "Freeform",
504504
"tags": ["operating-systems", "virtual-memory", "caching", "tlb"],
505505
"reference_materials": ["rust_reference.md"],
506-
"answer": "Cache lookups can be overlapped when the page offset in the virtual address contains the cache index. See lecture 15, slide 26.",
506+
"answer": "Cache lookups can be overlapped when the page offset in the virtual address contains the cache index.",
507507
"llm_judge_instructions": "Award full points for a fully correct explanation. award partial credit for partially correct explanations"
508508
}
509509
```
@@ -598,7 +598,10 @@ give partial credit if you show your work.
598598

599599
Suppose we have resources A, B, C and threads T1, T2, T3, T4. The total number of each resource as well as the current/max allocations for each thread are as follows:
600600

601-
Total Allocation: A=5, B=6, C=7
601+
Total Allocation:
602+
| A | B | C |
603+
|---|---|---|
604+
| 5 | 6 | 7 |
602605

603606
Current Allocation:
604607
| Thread | A | B | C |
@@ -734,7 +737,7 @@ Fill in sections [A], [B], and [C]:
734737
"tags": ["operating-systems", "virtual-memory", "page-tables"],
735738
"reference_materials": ["rust_reference.md"],
736739
"answer": "[A]:\nvpn[0] = vaddr >> 22;\nvpn[1] = zero(vaddr >> 12, 10, 32);\n\n[B]:\nptentry = get_word(pt + 4*vpn[i]);\nif ((ptentry & 1) == 0) {\n page_fault_handler(vaddr);\n return 0;\n}\npt = zero(ptentry, 0, 12);\n\n[C]:\nreturn pt + zero(vaddr, 12, 32);",
737-
"llm_judge_instructions": "Grade each section separately and sum the points"
740+
"llm_judge_instructions": "A is worth 2 points, B is worth 6 points and C is worth 2 points. Grade each section separately and sum the points"
738741
}
739742
```
740743

benchmarks/courselab_bench/data/courses.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,8 +36,9 @@
3636
"num_tasks": 1
3737
},
3838
{
39-
"course_id": "pku_pintos",
39+
"course_id": "pku_pintos_2025",
4040
"name": "PKU Operating Systems: Pintos Labs",
41+
"link": "https://pkuflyingpig.gitbook.io/pintos",
4142
"institution": "Peking University",
4243
"year": 2024,
4344
"num_tasks": 4

benchmarks/courselab_bench/data/pku_pintos/task_1_threads/config.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
2-
"instance_id": "pku_pintos__threads",
3-
"course_id": "pku_pintos",
2+
"instance_id": "pku_pintos_2025__threads",
3+
"course_id": "pku_pintos_2025",
44
"timeout_minutes": 60,
55
"tags": ["operating-systems", "c", "threads", "synchronization", "scheduling", "pintos"],
66
"artifacts": [

benchmarks/courselab_bench/data/pku_pintos/task_2_userprog/config.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
2-
"instance_id": "pku_pintos__userprog",
3-
"course_id": "pku_pintos",
2+
"instance_id": "pku_pintos_2025__userprog",
3+
"course_id": "pku_pintos_2025",
44
"timeout_minutes": 90,
55
"tags": ["operating-systems", "c", "system-calls", "processes", "user-programs", "pintos"],
66
"artifacts": [

benchmarks/courselab_bench/data/pku_pintos/task_3a_vm/config.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
2-
"instance_id": "pku_pintos__vm",
3-
"course_id": "pku_pintos",
2+
"instance_id": "pku_pintos_2025__vm",
3+
"course_id": "pku_pintos_2025",
44
"timeout_minutes": 120,
55
"tags": ["operating-systems", "c", "virtual-memory", "demand-paging", "page-replacement", "pintos"],
66
"artifacts": [

benchmarks/courselab_bench/data/pku_pintos/task_3b_mmap/config.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
2-
"instance_id": "pku_pintos__mmap",
3-
"course_id": "pku_pintos",
2+
"instance_id": "pku_pintos_2025__mmap",
3+
"course_id": "pku_pintos_2025",
44
"timeout_minutes": 120,
55
"tags": ["operating-systems", "c", "virtual-memory", "memory-mapped-files", "stack-growth", "pintos"],
66
"artifacts": [

0 commit comments

Comments
 (0)