[Feat] Add Microsoft Outlook - Mail Actions

[Tanuj-Taneja1]

Description

Fixes #3201
Hi @Wendong-Fan
I have completed the mail actions part of Outlook toolkit and this pr is ready to review.
Also I wanted to split other outlook related actions (like calender) into seperate file, so if that is ok we can rename this file to something mail related.

Reference:
msgraph
msgraph (some docs like send mail are in users section for some reason)
authorizationcodecredential for auth

(This is a edited conversation, older conversation was draft related)

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feat/outlook-toolkit

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Tanuj-Taneja1]

Hi @Wendong-Fan
Sorry to bother you but i was getting some error during setup env in the above tests
Is this related to me pushing uv.lock or in general ( I didnt see anyone else getting this error)

Run source .venv/bin/activate
  × No solution found when resolving dependencies:
  ╰─▶ Because fish-audio-sdk==2024.12.5 was yanked (reason: Break Change) and
      only the following versions of fish-audio-sdk are available:
          fish-audio-sdk<=2024.12.5
          fish-audio-sdk>2025
      we can conclude that fish-audio-sdk>=2024.12.5,<2025 cannot be used.
      And because camel-ai[all]==0.2.79 depends on
      fish-audio-sdk>=2024.12.5,<2025, we can conclude that
      camel-ai[all]==0.2.79 cannot be used.
      And because only camel-ai[all]==0.2.79 is available and you require
      camel-ai[all], we can conclude that your requirements are unsatisfiable.

[Tanuj-Taneja1]

Hi @Wendong-Fan Sorry to bother you but i was getting some error during setup env in the above tests Is this related to me pushing uv.lock or in general ( I didnt see anyone else getting this error)

Run source .venv/bin/activate
  × No solution found when resolving dependencies:
  ╰─▶ Because fish-audio-sdk==2024.12.5 was yanked (reason: Break Change) and
      only the following versions of fish-audio-sdk are available:
          fish-audio-sdk<=2024.12.5
          fish-audio-sdk>2025
      we can conclude that fish-audio-sdk>=2024.12.5,<2025 cannot be used.
      And because camel-ai[all]==0.2.79 depends on
      fish-audio-sdk>=2024.12.5,<2025, we can conclude that
      camel-ai[all]==0.2.79 cannot be used.
      And because only camel-ai[all]==0.2.79 is available and you require
      camel-ai[all], we can conclude that your requirements are unsatisfiable.

Regarding this issue, I think this is unrelated to my pr and the reason I am getting this is because i pushed uv lock file due to which we tied creating new environment instead of one in cache

[Tanuj-Taneja1]

I have added the remaining tests and function
For now I have kept the reply_to_email and update_draft_message simple without attachment handling since their api dont support it directly.
I will be adding attachment post method in a seperate pr later and will update this code then

Also I choose update_draft_message instead of update_message as updating sent message only changes data on our side like changing cc recipients wont send them mail again, so it wouldnt be very helpful for a llm toolkit in my opinion or even in general other than faking a sent mail.

I have completed the mail part of actions for now but feel free to let me know if there are any suggestions or change either regarding this pr or in general that might help me.

Thanks
Will be working on calender part of toolkit in few days

[Tanuj-Taneja1]

Also to make reviewers life easier
Here is how you setup things at azure

1 Go to https://portal.azure.com/
2 Search for microsoft Entra ID and open it
3 Click on add button in horizontal menu and click app registeration
4 add name, supported account type and redirect uri (should be set to http://localhost:1000) and platform as web - this page should have client id
5 On the next page there should be a link for 'Add a certificate or secret' click on it then add a new secret , then copy the Value (Client secret) dont use Secret ID

Thats it
Now you just have to add the user that you want to sign in as
Go back to default directory overview page and add the user of your choice. For personal account select add external account

Now just add these creds to env (for personal account dont add tenant id or if you want add "common" ) and the code should work

Later we should also add all this to example file ( I didnt create it since i dont have open ai credits)

[waleedalzarooni]

Hi @Wendong-Fan Sorry to bother you but i was getting some error during setup env in the above tests Is this related to me pushing uv.lock or in general ( I didnt see anyone else getting this error)

Run source .venv/bin/activate
  × No solution found when resolving dependencies:
  ╰─▶ Because fish-audio-sdk==2024.12.5 was yanked (reason: Break Change) and
      only the following versions of fish-audio-sdk are available:
          fish-audio-sdk<=2024.12.5
          fish-audio-sdk>2025
      we can conclude that fish-audio-sdk>=2024.12.5,<2025 cannot be used.
      And because camel-ai[all]==0.2.79 depends on
      fish-audio-sdk>=2024.12.5,<2025, we can conclude that
      camel-ai[all]==0.2.79 cannot be used.
      And because only camel-ai[all]==0.2.79 is available and you require
      camel-ai[all], we can conclude that your requirements are unsatisfiable.

Regarding this issue, I think this is unrelated to my pr and the reason I am getting this is because i pushed uv lock file due to which we tied creating new environment instead of one in cache

Hey @Tanuj-Taneja1, I will have a look at this issue and see why this may be happening, also moving forward feel free to message me with any concerns or issues you may face!

[waleedalzarooni]

Hey @Tanuj-Taneja1,
I am currently working on reviewing your implementation, one thing to note for toolkits is that we typically have example files for each toolkit to demonstrate usage, would you mind creating one for the Microsoft toolkit (use the gmail_toolkit.py for reference)

Hi @waleedalzarooni I think I mentioned in a earlier conversation, but I dont currently have any openai credits, so wont be able to create a example file with default model.

I think I can give you a notebook file which runs all the functions and setup instructions , would that be helpful?

If yes heres the link: https://colab.research.google.com/drive/1jgJVfljz2HfdmgcmDZclhfdybCYNdi3u?usp=sharing It is mostly ai generated but everything works, I had to hide the cell ouputs though since they contain important message id and stuff. If it doesnt work for you feel free to let me know

No problem, I'll work on that thanks for the heads up and I'll make sure to let you know if I have any questions about the colab file!

[waleedalzarooni]

Hey @Tanuj-Taneja1,

Thank you so much for your patience, I have added an enhance PR (#3492) that fixes some of the async functionality as well as the way the port is assigned, since you must set a fixed port for use in the OAUTH process I allow the user to set the port in the env variable, would be great if you could review this when possible so we can merge this feature!

[Tanuj-Taneja1]

Hey @Tanuj-Taneja1,

Thank you so much for your patience, I have added an enhance PR (#3492) that fixes some of the async functionality as well as the way the port is assigned, since you must set a fixed port for use in the OAUTH process I allow the user to set the port in the env variable, would be great if you could review this when possible so we can merge this feature!

Hi @waleedalzarooni
I think we dont need to specify the port if we set redirect uri to "http://localhost/" in azure.

[waleedalzarooni]

Hey @Tanuj-Taneja1,
Thank you so much for your patience, I have added an enhance PR (#3492) that fixes some of the async functionality as well as the way the port is assigned, since you must set a fixed port for use in the OAUTH process I allow the user to set the port in the env variable, would be great if you could review this when possible so we can merge this feature!

Hi @waleedalzarooni I think we dont need to specify the port if we set redirect uri to "http://localhost/" in azure.

Yep, you're totally right I will reset to the original conventions in the enhanced PR

[fengju0213]

everything looks good！ some minor issues in this PR are around auth lifecycle and runtime robustness:

• In the browser OAuth fallback, the obtained refresh_token is not persisted to refresh_token_file_path, so
  the documented “reuse credentials without repeated browser prompts” doesn’t work and users get forced
  through browser login on every init.
• Scopes were provided in short form (e.g., Mail.Send), which can be unstable across SDK/adapters and lead
  to token/scope mismatch and intermittent 401s.
• The custom credential performed token refresh via synchronous HTTP, which can block the asyncio event
  loop in async Graph usage and cause latency spikes/timeouts.
• Tests were environment-sensitive (localhost port bind) and could fail in restricted CI/sandbox setups.

created an enhance pr, #3578 feel free to check it! @Tanuj-Taneja1
Changes i applied: implemented auth-code token exchange and persisted refresh_token to the configured file
with a regression test; normalized scopes to Graph fully-qualified form while keeping backward
compatibility; migrated the custom credential to an async implementation to avoid blocking; and adjusted
tests to avoid relying on local port binding.

[fengju0213]

hi @MuggleJinx if you have time，please take a look at this pr again！

[MuggleJinx]

Thanks! LGTM!

[Tanuj-Taneja1]

Hi @fengju0213

• In the browser OAuth fallback, the obtained refresh_token is not persisted to refresh_token_file_path, so
  the documented “reuse credentials without repeated browser prompts” doesn’t work and users get forced
  through browser login on every init.

I think that is still the case, I am still getting browser login with each init. Can you confirm that from your side?

[Tanuj-Taneja1]

Hi @fengju0213

• In the browser OAuth fallback, the obtained refresh_token is not persisted to refresh_token_file_path, so
  the documented “reuse credentials without repeated browser prompts” doesn’t work and users get forced
  through browser login on every init.

I think that is still the case, I am still getting browser login with each init. Can you confirm that from your side?

Hi figured out the reason of this and mentioned the reason in #3578
For the solution part we can set a default file path for refresh token and that should solve the issue, we can also remove the authorizationcredentialflow logic as it uses the exact logic you added of using getting refresh token from authentication code and use it to get tokens, it just saved the tokens in persistence cache instead of a common file which can be reused after first init

[fengju0213]

Hi @fengju0213

• In the browser OAuth fallback, the obtained refresh_token is not persisted to refresh_token_file_path, so
  the documented “reuse credentials without repeated browser prompts” doesn’t work and users get forced
  through browser login on every init.

I think that is still the case, I am still getting browser login with each init. Can you confirm that from your side?

Yes, that’s expected. It’s just that the previous persistence logic didn’t take effect. Now it can be saved properly, but to skip verification under the current implementation, a refresh token path needs to be provided. We could also change it to be passed automatically.

[Tanuj-Taneja1]

Hi @fengju0213

• In the browser OAuth fallback, the obtained refresh_token is not persisted to refresh_token_file_path, so
  the documented “reuse credentials without repeated browser prompts” doesn’t work and users get forced
  through browser login on every init.

I think that is still the case, I am still getting browser login with each init. Can you confirm that from your side?

Yes, that’s expected. It’s just that the previous persistence logic didn’t take effect. Now it can be saved properly, but to skip verification under the current implementation, a refresh token path needs to be provided. We could also change it to be passed automatically.

Actually previously even in browser auth persistance was there just not in refresh_token_file_path but using TokenCachePersistenceOptions. Browser auth is when either user didnt provide refresh_token_file_path or when authentication using it fails.

For Browser auth TokenCachePersistenceOptions stored refresh token and access token internally.

Your current implentation is exactly what AuthorizationCodeCredential and TokenCachePersistenceOptions did internally.

Feel free to refer this : https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/identity/Azure.Identity/samples/TokenCache.md

[fengju0213]

Hi @fengju0213

• In the browser OAuth fallback, the obtained refresh_token is not persisted to refresh_token_file_path, so
  the documented “reuse credentials without repeated browser prompts” doesn’t work and users get forced
  through browser login on every init.

I think that is still the case, I am still getting browser login with each init. Can you confirm that from your side?

Yes, that’s expected. It’s just that the previous persistence logic didn’t take effect. Now it can be saved properly, but to skip verification under the current implementation, a refresh token path needs to be provided. We could also change it to be passed automatically.

Actually previously even in browser auth persistance was there just not in refresh_token_file_path but using TokenCachePersistenceOptions. Browser auth is when either user didnt provide refresh_token_file_path or when authentication using it fails.

For Browser auth TokenCachePersistenceOptions stored refresh token and access token internally.

Your current implentation is exactly what AuthorizationCodeCredential and TokenCachePersistenceOptions did internally.

Feel free to refer this : https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/identity/Azure.Identity/samples/TokenCache.md

Thanks for the clear explanation. I also reviewed the Azure Identity docs you linked. Based on that, to achieve “reuse credentials without repeated browser prompts,” it seems the official way is to enable persistent caching via TokenCachePersistenceOptions. Are you suggesting we switch to using the official TokenCachePersistenceOptions for this?

[Tanuj-Taneja1]

Hi @fengju0213

• In the browser OAuth fallback, the obtained refresh_token is not persisted to refresh_token_file_path, so
  the documented “reuse credentials without repeated browser prompts” doesn’t work and users get forced
  through browser login on every init.

I think that is still the case, I am still getting browser login with each init. Can you confirm that from your side?

Yes, that’s expected. It’s just that the previous persistence logic didn’t take effect. Now it can be saved properly, but to skip verification under the current implementation, a refresh token path needs to be provided. We could also change it to be passed automatically.

Actually previously even in browser auth persistance was there just not in refresh_token_file_path but using TokenCachePersistenceOptions. Browser auth is when either user didnt provide refresh_token_file_path or when authentication using it fails.
For Browser auth TokenCachePersistenceOptions stored refresh token and access token internally.
Your current implentation is exactly what AuthorizationCodeCredential and TokenCachePersistenceOptions did internally.
Feel free to refer this : https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/identity/Azure.Identity/samples/TokenCache.md

Thanks for the clear explanation. I also reviewed the Azure Identity docs you linked. Based on that, to achieve “reuse credentials without repeated browser prompts,” it seems the official way is to enable persistent caching via TokenCachePersistenceOptions. Are you suggesting we switch to using the official TokenCachePersistenceOptions for this?

Yes we can just revert to our original logic, it used TokenCachePersistenceOptions.
Either way the current implementation is also correct though involves more loc. We can either fix it here or later since I was planning on removing auth logic from this code based on our discussion in #3553

[fengju0213]

Hi @fengju0213

• In the browser OAuth fallback, the obtained refresh_token is not persisted to refresh_token_file_path, so
  the documented “reuse credentials without repeated browser prompts” doesn’t work and users get forced
  through browser login on every init.

I think that is still the case, I am still getting browser login with each init. Can you confirm that from your side?

Yes, that’s expected. It’s just that the previous persistence logic didn’t take effect. Now it can be saved properly, but to skip verification under the current implementation, a refresh token path needs to be provided. We could also change it to be passed automatically.

Actually previously even in browser auth persistance was there just not in refresh_token_file_path but using TokenCachePersistenceOptions. Browser auth is when either user didnt provide refresh_token_file_path or when authentication using it fails.
For Browser auth TokenCachePersistenceOptions stored refresh token and access token internally.
Your current implentation is exactly what AuthorizationCodeCredential and TokenCachePersistenceOptions did internally.
Feel free to refer this : https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/identity/Azure.Identity/samples/TokenCache.md

Thanks for the clear explanation. I also reviewed the Azure Identity docs you linked. Based on that, to achieve “reuse credentials without repeated browser prompts,” it seems the official way is to enable persistent caching via TokenCachePersistenceOptions. Are you suggesting we switch to using the official TokenCachePersistenceOptions for this?

Yes we can just revert to our original logic, it used TokenCachePersistenceOptions. Either way the current implementation is also correct though involves more loc. We can either fix it here or later since I was planning on removing auth logic from this code based on our discussion in #3553

okay，we can merge it first，then we can improve the implement

[Wendong-Fan]

thanks @Tanuj-Taneja1 's contribution and @waleedalzarooni @fengju0213 @MuggleJinx help with reviewing, I added one enhance PR: #3608, feel free to review and let me know if there's anything we want to discuss further