Testing for parity between cli and server

[zphoenixrises]

To test cli script:

python3 -m shortfin_apps.llm.cli --device hip --tokenizer_json=/data/Mistral-Nemo-Instruct-2407-FP8/tokenizer.json  --model_config=../artifacts/quark_mistral_config.json  --vmfb=../artifacts/quark_mistral_nemo.vmfb  --prompt "one two three four five"  --parameters /data/Mistral-Nemo-Instruct-2407-FP8/quark_mistral_nemo.irpa  --benchmark  --benchmark_tasks=64  --decode_steps=64 --token_selection_strategy=multi_greedy --num_beams=8 --device_ids 0 --workers_offline=16

To test server

 python -m shortfin_apps.llm.server   --tokenizer_json /data/Mistral-Nemo-Instruct-2407-FP8/tokenizer.json   --model_config ../artifacts/quark_mistral_config.json   --vmfb ../artifacts/quark_mistral_nemo.vmfb   --device=hip   --device_ids 0   --parameters /data/Mistral-Nemo-Instruct-2407-FP8/quark_mistral_nemo.irpa   --token_selection_strategy multi_greedy   --num_beams 8   --port 8081

Then run benchmark script with the following:

      OUTPUT="${MODEL}_gpu${NUM_GPUS}_bo${BON}_${OUTPUT_TOKENS}_out_${INPUT_TOKENS}_in.json"
      python3 profiles.py profile --samples 64 \
        --output $OUTPUT \
        --output_tokens $OUTPUT_TOKENS \
        --input_tokens $INPUT_TOKENS \
        --best_of_n $BON \
        --batches "8" \
        --server_ports "8081"