Code

TensorRT Cuda Context Sharing#106

Code

Open

ashishk98 wants to merge 17 commits intotriton-inference-server:maintriton-inference-server/tensorrt_backend:mainfrom

ashishk98:mainashishk98/tensorrt_backend:mainCopy head branch name to clipboard

Commits

Commits on Aug 28, 2024

• Enable CiG support in Tensorrt backend

  Akarale98

  committed
  134bf33

  Copy full SHA for 134bf33

• Creating scoped runtime context structure for better management of CiG context

  Akarale98

  committed
  7353671

  Copy full SHA for 7353671

• instance_state null check

  Akarale98

  committed
  ed1296d

  Copy full SHA for ed1296d

Commits on Sep 5, 2024

• Minor bug fix

  Akarale98

  committed
  05e3786

  Copy full SHA for 05e3786

Commits on Sep 19, 2024

• pre-commit fixes

  Akarale98

  committed
  9ae5a09

  Copy full SHA for 9ae5a09

• Added new cmake flag TRITON_ENABLE_CIG to make the CiG support build conditional

  Akarale98

  committed
  89ab580

  Copy full SHA for 89ab580

Commits on Sep 24, 2024

• pre-commit fixes

  Akarale98

  committed
  b624b98

  Copy full SHA for b624b98

Commits on Oct 30, 2024

• CiG->Cuda. Making the changes more generic to cuda context sharing + hidden ifdefs

  Akarale98

  committed
  1f1ae7e

  Copy full SHA for 1f1ae7e

• remove todo

  Akarale98

  committed
  b42a8d6

  Copy full SHA for b42a8d6

• Add GetParameter to fetch string params

  Akarale98

  committed
  d9aff26

  Copy full SHA for d9aff26

Commits on Jan 6, 2025

• Merge remote-tracking branch 'main/main'

  Akarale98

  committed
  80ba393

  Copy full SHA for 80ba393

• Cig->Cuda + comment updates

  Akarale98

  committed
  2ca19ac

  Copy full SHA for 2ca19ac

• Use RETURN_IF_ERROR macro

  Akarale98

  committed
  20283c6

  Copy full SHA for 20283c6

Commits on Jan 27, 2025

• Handle Multi-GPU failure case

  Akarale98

  committed
  e2a9336

  Copy full SHA for e2a9336

Commits on Jan 28, 2025

• typo + styling fixes

  Akarale98

  committed
  625d19e

  Copy full SHA for 625d19e

Commits on Feb 3, 2025

• Linux build fix

  Akarale98

  committed
  950962a

  Copy full SHA for 950962a

Commits on Feb 17, 2025

• Optionally check for CUDA_CONTEXT_PTR

  Akarale98

  committed
  cdc5ece

  Copy full SHA for cdc5ece