Raspberry Pi OS (64-bit) - Unable to import FoundryLocalManager from foundry_local_sdk

[MaheshSQL]

Hi team,

Steps:

• Installed python 3.13 on Raspberry Pi OS (64-bit)
• Created python virtual environment
• pip install foundry-local-sdk openai

test.py

from foundry_local_sdk import FoundryLocalManager
print('Imports successful')

Import error suggesting cannot import FoundryLocalManager from foundry_local_sdk

I tried to repro the issue that I faced on Raspberry Pi OS (64-bit) on Windows with steps below and it seems to produce no error on Windows.
Thanks.

[prathikr]

Are you able to repro the issue on a linux-x64 vm? I am unable to repro on my end...

[bmehta001]

Hi @MaheshSQL, Just to confirm - is your OS arm64?

[MaheshSQL]

Hi team - sharing updates:

Are you able to repro the issue on a linux-x64 vm?

• Switch to Ubuntu VM (Azure ML Compute Instance)

Is OS x64?

• ran: uname -m
• output: x86_64

Setup

Install the venv package

sudo apt update
sudo apt install python3-venv -y

Create virtual env (Python 3.10)

python3 -m venv myenv1

Activate env

source myenv1/bin/activate

Install foundry-local-sdk as per public docs.

pip install foundry-local-sdk openai

Further investigation

This installed the packages to globally not in virtual env

pip show foundry_local_sdk

Location: /home/azureuser/.local/lib/python3.10/site-packages

Navigated to above path

cd /home/azureuser/.local/lib/python3.10/site-packages
cd foundry_local
ls
nano __init__.py

Prints this

import logging
import sys

from foundry_local.api import FoundryLocalManager
from foundry_local.version import __version__

_logger = logging.getLogger(__name__)
_logger.setLevel(logging.WARNING)

_sc = logging.StreamHandler(stream=sys.stdout)
_formatter = logging.Formatter(
    "[foundry-local] | %(asctime)s | %(levelname)-8s | %(message)s", datefmt="%Y-%m-%d %H:%M:%S"
)
_sc.setFormatter(_formatter)
_logger.addHandler(_sc)
_logger.propagate = False

__all__ = ["FoundryLocalManager", "__version__"]

Interpretation

• Q 1: Does this essentially mean the information in public docs need to be updated?

from foundry_local_sdk import Configuration, FoundryLocalManager

Public docs (https://learn.microsoft.com/en-us/azure/foundry-local/get-started?tabs=xplatform&pivots=programming-language-python#use-native-chat-completions-api)

from foundry_local_sdk import Configuration, FoundryLocalManager

• Q 2: Does it need to be?
  from foundry_local import FoundryLocalManager

• Q 3: Where would Configuration come from?

[prathikr]

@MaheshSQL from foundry_local is the old python API that was a simple wrapper over the CLI. from foundry_local_sdk is the new python API which is the recommended method. We have GA'd this product to the API is stable and should not change. The public documentation is correct.

You can see more details on the Configuration settings at https://github.com/microsoft/Foundry-Local/blob/main/sdk/python/src/configuration.py

[MaheshSQL]

Thanks, I not sure why old python API/sdk gets installed upon running pip install foundry-local-sdk openai as shown above per the content of the nano __init__.py

Not sure if this is specific to use of python venv and not conda env

What I see -

import logging
import sys

from foundry_local.api import FoundryLocalManager
from foundry_local.version import __version__

_logger = logging.getLogger(__name__)
_logger.setLevel(logging.WARNING)

_sc = logging.StreamHandler(stream=sys.stdout)
_formatter = logging.Formatter(
    "[foundry-local] | %(asctime)s | %(levelname)-8s | %(message)s", datefmt="%Y-%m-%d %H:%M:%S"
)
_sc.setFormatter(_formatter)
_logger.addHandler(_sc)
_logger.propagate = False

__all__ = ["FoundryLocalManager", "__version__"]

What it should be -

import logging
import sys

from .configuration import Configuration
from .foundry_local_manager import FoundryLocalManager
from .version import __version__

_logger = logging.getLogger(__name__)
_logger.setLevel(logging.WARNING)

_sc = logging.StreamHandler(stream=sys.stdout)
_formatter = logging.Formatter(
    "[foundry-local] | %(asctime)s | %(levelname)-8s | %(message)s", datefmt="%Y-%m-%d %H:%M:%S"
)
_sc.setFormatter(_formatter)
_logger.addHandler(_sc)
_logger.propagate = False

__all__ = ["Configuration", "FoundryLocalManager", "__version__"]