Add evaluation, multimodal, native tests to run-readme-pr-mps.yml

[mikekgfb]

Add evaluation, multimodal, native tests to run-readme-pr-mps.yml

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1410

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit a66062d with merge base f4ae60f ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Jack-Khuu]

Just realized that none of the test in this file are being ran:

Invalid workflow file: .github/workflows/run-readme-pr-mps.yml#L13
The workflow is not valid. .github/workflows/run-readme-pr-mps.yml (Line: 13, Col: 24): Invalid input, timeout-minutes is not defined in the referenced workflow.

[mikekgfb]

Just realized that none of the test in this file are being ran:

Invalid workflow file: .github/workflows/run-readme-pr-mps.yml#L13
The workflow is not valid. .github/workflows/run-readme-pr-mps.yml (Line: 13, Col: 24): Invalid input, timeout-minutes is not defined in the referenced workflow.

Should be fixed now. Please rerun tests?

[mikekgfb]

Just realized that none of the test in this file are being ran:

Invalid workflow file: .github/workflows/run-readme-pr-mps.yml#L13
The workflow is not valid. .github/workflows/run-readme-pr-mps.yml (Line: 13, Col: 24): Invalid input, timeout-minutes is not defined in the referenced workflow.

Should be fixed now. Please rerun tests?

Still timed out @ 50 minutes, increased timeout further.

Beyond the "let's extend timeout" bandaid, I'm wondering whether this may be due to us trying to update llvm+openmp with the command conda create -y -n test-readme-mps-macos python=3.10.11 llvm-openmp which may be adding significant runtime to the test? This might benefit from somebody looking at it who is familiar with pytorch ci/cd setup on github? (i.e., the pt releng team.

Also, MPS is the only set of doc tests using conda, which on the one hand is good because we're testing setup with conda as well, but otoh makes MPS the odd man platform out at the moment.