Skip to content

Commit 43fc186

Browse files
author
David Thrower
committed
Remove quotes on shell command for serialization test. See if it fixes the problem in the .
1 parent 33e8905 commit 43fc186

File tree

1 file changed

+6
-1
lines changed

1 file changed

+6
-1
lines changed

train_a_generative_llm_docker.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1096,8 +1096,13 @@ def create_dataset(raw_text_samples, tokenizer, sample_expansion_batch_size=50,
10961096
generator.save(MODEL_SAVE_PATH)
10971097
print(f"Final model saved to {MODEL_SAVE_PATH}")
10981098

1099+
print("File system sync")
1100+
os.sync() # Force write of buffer to disk
1101+
print("Waiting 15 seconds to make sure sync is complete.")
1102+
time.sleep(15) # Allow time for network/disk I/O to complete
1103+
10991104
print(f"🧪 Running serialization test for Stage I-b trial {meta_trial_number}...")
1100-
ser_test_cmd = f"""python3 test_llm_serialization.py "{TOKENIZER_SAVE_PATH}" "{MODEL_SAVE_PATH}" """
1105+
ser_test_cmd = f"python3 test_llm_serialization.py {TOKENIZER_SAVE_PATH} {MODEL_SAVE_PATH}"
11011106
print(f"""Running command: "{ser_test_cmd}" """)
11021107
result = subprocess.run(
11031108
ser_test_cmd,

0 commit comments

Comments
 (0)