Skip to content

Commit 3c89309

Browse files
Merge pull request #3164 from MicrosoftDocs/main639197503043063957sync_temp
For protected branch, push strategy should use PR and merge to target branch method to work around git push error
2 parents eacf84d + 655dbd0 commit 3c89309

19 files changed

Lines changed: 183 additions & 230 deletions
1.08 KB
Loading
4.47 KB
Loading
81.2 KB
Loading
1.51 KB
Loading
-16.8 KB
Loading

docs/data-engineering/using-python-experience-on-notebook.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: Learn how to work with pure Python notebooks for data exploration,
44
ms.reviewer: jingzh
55
ms.topic: how-to
66
ms.search.form: Create and use notebooks
7-
ms.date: 04/24/2026
7+
ms.date: 07/10/2026
88
ai-usage: ai-assisted
99
---
1010

@@ -15,7 +15,7 @@ The Python notebook is a new experience built on top of Fabric notebook. It is a
1515

1616
With a Python notebook, you can get:
1717

18-
- **Multiple built-in python kernels**: Python notebooks offer a pure Python coding environment without Spark, with two versions of Python kernel - Python 3.10 and 3.11 available by default, and the native ipython features supported such as iPyWidget, magic commands.
18+
- **Multiple built-in Python kernels**: Python notebooks offer a pure Python coding environment without Spark, with three built-in Python kernel versions: Python 3.10, Python 3.11, and Python 3.12. The environment supports native IPython features such as iPyWidget and magic commands.
1919

2020
- **Cost effective**: The new Python notebook offers cost-saving benefits by running on a single node cluster with 2vCores/16GB memory by default. This setup ensures efficient resource utilization for data exploration projects with smaller data size.
2121

@@ -97,7 +97,7 @@ The [Notebook resources](how-to-use-notebook.md#notebook-resources) built-in res
9797

9898
## Kernel operations
9999

100-
Python notebook support two built-in kernels right now, they are *Python 3.10* and *Python 3.11*, the default selected kernel is *Python 3.11*. you can easily switch between them.
100+
Python notebooks support three built-in kernels: *Python 3.10*, *Python 3.11*, and *Python 3.12*. The default selected kernel is *Python 3.12*. You can easily switch between them.
101101

102102
You can interrupt, restart, or switch kernel on the **Home** tab of the ribbon. Interrupting kernel in Python notebooks is same as canceling cell in Spark notebook.
103103

docs/iq/ontology/how-to-create-agent-foundry-iq.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: Create an Ontology Agent with Foundry IQ
33
description: Learn how to create a Foundry IQ agent that is grounded in an ontology (preview). The agent can answer natural-language questions using the ontology as a single source of truth.
4-
ms.date: 05/13/2026
4+
ms.date: 07/15/2026
55
ms.topic: how-to
66
---
77

@@ -28,6 +28,11 @@ Before you begin, make sure you have:
2828
* An active Azure subscription with permission to create Foundry resources.
2929
* A Microsoft Fabric workspace that contains at least one ontology (preview) item. For more information, see [Create ontology (preview) item](tutorial-1-create-ontology.md#create-ontology-preview-item).
3030
* Complete the [Foundry quickstart](/azure/foundry/quickstarts/get-started-code) to provision a Foundry project and a sample agent. The steps in this article assume that project already exists.
31+
* Ensure that the AI Search resource has the *Search Index Data Contributor* role for the Foundry Project. To assign this role, go to the [Azure portal](https://portal.azure.com) and open IAM for the AI Search resource. Follow the [Assign roles for development](/azure/search/search-security-rbac?tabs=roles-portal-admin%2Croles-portal%2Croles-portal-query%2Ctest-portal%2Ccustom-role-portal#assign-roles-for-development) instructions to assign the *Search Index Data Contributor* role to the managed identity of your Foundry project.
32+
33+
>[!IMPORTANT]
34+
>If you don't assign this role, the agent can't access the ontology and you see a 403 error in the Foundry agent.
35+
3136
* An Edge or Chrome browser, signed in with the same identity that has access to both the Foundry project and the Fabric workspace.
3237

3338
## Step 1: Create the knowledge source and knowledge base

docs/onelake/toc.yml

Lines changed: 55 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -15,54 +15,64 @@ items:
1515
href: onelake-open-access-quickstart.md
1616
- name: Understand medallion lakehouse architecture
1717
href: onelake-medallion-lakehouse-architecture.md
18-
- name: Shortcuts
18+
- name: Unify your data
1919
items:
20-
- name: What are shortcuts?
21-
href: onelake-shortcuts.md
22-
- name: Create shortcuts
20+
- name: Choose shortcuts or mirroring
21+
href: unify-data.md
22+
- name: Shortcuts
2323
items:
24-
- name: OneLake shortcut
25-
href: shortcuts/create-onelake-shortcut.md
26-
- name: ADLS Gen2 shortcut
27-
href: create-adls-shortcut.md
28-
- name: Blob Storage shortcut
29-
href: ./shortcuts/create-blob-shortcut.md
30-
- name: Dataverse shortcut
31-
href: create-dataverse-shortcut.md
32-
- name: OneDrive or SharePoint shortcut
33-
href: ./shortcuts/create-onedrive-sharepoint-shortcut.md
34-
- name: Amazon S3 shortcut
35-
href: create-s3-shortcut.md
36-
- name: Amazon S3 compatible shortcut
37-
href: create-s3-compatible-shortcut.md
38-
- name: Google Cloud Storage shortcut
39-
href: create-gcs-shortcut.md
40-
- name: Iceberg table shortcut
41-
href: onelake-iceberg-tables.md
42-
- name: On-premises data shortcut
43-
href: create-on-premises-shortcut.md
44-
- name: Manage shortcuts
24+
- name: What are shortcuts?
25+
href: onelake-shortcuts.md
26+
- name: Create shortcuts
27+
items:
28+
- name: OneLake shortcut
29+
href: shortcuts/create-onelake-shortcut.md
30+
- name: ADLS Gen2 shortcut
31+
href: create-adls-shortcut.md
32+
- name: Blob Storage shortcut
33+
href: shortcuts/create-blob-shortcut.md
34+
- name: Dataverse shortcut
35+
href: create-dataverse-shortcut.md
36+
- name: OneDrive or SharePoint shortcut
37+
href: shortcuts/create-onedrive-sharepoint-shortcut.md
38+
- name: Amazon S3 shortcut
39+
href: create-s3-shortcut.md
40+
- name: Amazon S3 compatible shortcut
41+
href: create-s3-compatible-shortcut.md
42+
- name: Google Cloud Storage shortcut
43+
href: create-gcs-shortcut.md
44+
- name: Iceberg table shortcut
45+
href: onelake-iceberg-tables.md
46+
- name: On-premises data shortcut
47+
href: create-on-premises-shortcut.md
48+
- name: Manage shortcuts
49+
items:
50+
- name: Edit or delete a shortcut
51+
href: shortcuts/edit-delete-shortcut.md
52+
- name: Shortcut security
53+
href: onelake-shortcut-security.md
54+
- name: S3 shortcut Entra integration
55+
href: amazon-storage-shortcut-entra-integration.md
56+
- name: Manage cloud connections
57+
href: manage-shortcut-connections.md
58+
- name: Access OneLake shortcuts in an Apache Spark notebook
59+
href: access-onelake-shortcuts.md
60+
- name: Assign variables to shortcuts
61+
href: assign-variables-to-shortcuts.md
62+
- name: Use shortcuts REST API
63+
href: /rest/api/fabric/core/onelake-shortcuts?toc=/fabric/onelake/toc.json&bc=/fabric/onelake/toc.json
64+
- name: Shortcut transformations
65+
items:
66+
- name: Transform structured data
67+
href: ./shortcuts/transformations.md
68+
- name: Transform unstructured data with AI
69+
href: ./shortcuts/transformations-ai.md
70+
- name: Mirroring
4571
items:
46-
- name: Edit or delete a shortcut
47-
href: ./shortcuts/edit-delete-shortcut.md
48-
- name: Shortcut security
49-
href: onelake-shortcut-security.md
50-
- name: S3 shortcut Entra integration
51-
href: amazon-storage-shortcut-entra-integration.md
52-
- name: Manage cloud connections
53-
href: manage-shortcut-connections.md
54-
- name: Access OneLake shortcuts in an Apache Spark notebook
55-
href: access-onelake-shortcuts.md
56-
- name: Assign variables to shortcuts
57-
href: assign-variables-to-shortcuts.md
58-
- name: Use shortcuts REST API
59-
href: /rest/api/fabric/core/onelake-shortcuts?toc=/fabric/onelake/toc.json&bc=/fabric/onelake/toc.json
60-
- name: Shortcut transformations
61-
items:
62-
- name: Transform structured data
63-
href: ./shortcuts/transformations.md
64-
- name: Transform unstructured data with AI
65-
href: ./shortcuts/transformations-ai.md
72+
- name: What is mirroring?
73+
href: ../mirroring/overview.md?toc=/fabric/onelake/toc.json&bc=/fabric/onelake/toc.json
74+
- name: Mirroring documentation
75+
href: ../mirroring/index.yml
6676
- name: Open access
6777
items:
6878
- name: How do I connect to OneLake?

docs/onelake/unify-data.md

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
---
2+
title: Unify data with OneLake shortcuts and mirroring
3+
description: Learn how shortcuts and mirroring make data across your estate available through OneLake, and how to choose between them.
4+
ms.reviewer: mideboer
5+
ms.topic: overview
6+
ms.date: 07/15/2026
7+
ai-usage: ai-assisted
8+
#customer intent: As a data professional, I want to understand how shortcuts and mirroring unify data in OneLake so that I can choose the right approach for my scenario.
9+
---
10+
11+
# Unify data with OneLake shortcuts and mirroring
12+
13+
[OneLake](onelake-overview.md) gives you one view of data across clouds, domains, and accounts. Unifying data in OneLake doesn't require storing all of that data in one place. The data can stay at its source or be replicated into OneLake, where Fabric workloads access it through a shared namespace.
14+
15+
Shortcuts and mirroring are complementary OneLake capabilities for creating this unified view without building and operating data movement pipelines. Shortcuts add selected data to the OneLake namespace. Mirroring adds an external database or catalog and determines whether its data can be accessed in place or must be replicated. This article explains how each capability works and when to use them. For the broader set of ways to ingest, transform, or move data in Fabric, see [Get data into Microsoft Fabric](../fundamentals/get-data.md) or [Choose a data movement strategy](../data-factory/decision-guide-data-movement.md).
16+
17+
## Shortcuts and mirroring at a glance
18+
19+
Shortcuts and mirroring operate at different levels of granularity and support different source formats.
20+
21+
| Capability | What it does | Source formats | Typical scenario |
22+
| --- | --- | --- | --- |
23+
| [Shortcuts](onelake-shortcuts.md) | Add a reference to selected tables, folders, or files in OneLake or external storage. The data stays at its source. | Open formats only | Bring a single table, folder, or schema into a lakehouse, or enable data mesh patterns across workspaces or tenants. |
24+
| [Mirroring](../mirroring/overview.md) | Add an external database or catalog to Fabric. Depending on the source, mirroring accesses the data in place or continuously replicates it into OneLake. | Open and proprietary formats | Bring in an entire external database or catalog, such as Snowflake, Azure Databricks, or Azure SQL Database. |
25+
26+
If your source stores data in a proprietary format, mirroring is your only option.
27+
28+
## When to use shortcuts
29+
30+
Use shortcuts when you want to include specific data in your unified OneLake view without copying it. Choose shortcuts when you:
31+
32+
- Share a single table, folder, or file across workspaces or tenants.
33+
- Combine data from multiple lakes or cloud accounts into one unified view.
34+
- Build data mesh patterns where each domain owns its data and other domains discover it through OneLake.
35+
- Make data in open formats, such as Delta or Iceberg, available from Azure Data Lake Storage, Amazon S3, Google Cloud Storage, or Dataverse.
36+
37+
Shortcuts work at the table, folder, or file level. You decide which data to make available and where it appears in OneLake. For more information, see [OneLake shortcuts](onelake-shortcuts.md).
38+
39+
### Transform data as you unify it
40+
41+
Shortcut transformations convert the data behind a shortcut into a queryable Delta table that stays in sync with the source, without building an ETL pipeline. Use them to make your data analytics-ready:
42+
43+
- **File transformations** convert structured files, such as CSV, Parquet, JSON, or Excel, into Delta tables. For more information, see [Shortcut transformations (file)](shortcuts/transformations.md).
44+
- **AI-powered transformations** apply language processing to `.txt` files to summarize content, detect sentiment, translate languages, redact personally identifiable information (PII), or extract named entities. For more information, see [Shortcut transformations (AI-powered)](shortcuts/transformations-ai.md).
45+
46+
## When to use mirroring
47+
48+
Use mirroring when you want to include an external database or catalog, and the data behind it, in Fabric as a unit. Mirroring is the right choice when you:
49+
50+
- Make an entire external database or catalog available in Fabric without designing a separate ingestion process.
51+
- Keep an analytics-ready copy of an operational database (such as Azure SQL Database, PostgreSQL, or Cosmos DB) in OneLake that stays in sync with the source.
52+
- Surface an external lakehouse or warehouse (such as Databricks Unity Catalog or Snowflake) in Fabric so you can query it alongside your Fabric data.
53+
54+
Mirroring always adds the catalog metadata, such as databases, schemas, and tables, to Fabric. How Fabric accesses the data depends on the source, as the next section describes.
55+
56+
### How mirroring makes data available in OneLake
57+
58+
Mirroring uses two underlying mechanisms to include external data in the unified OneLake view. The right combination depends on how the source stores its data.
59+
60+
- **Replication.** Mirroring copies data from the source into OneLake in an analytics-ready Delta format. Mirroring uses replication when the source stores data in a proprietary format.
61+
- **Shortcuts.** Mirroring references data in place through OneLake shortcuts. Mirroring uses shortcuts when the source stores data in an open format that OneLake can read directly.
62+
63+
Different sources use these mechanisms differently, as shown in the following table.
64+
65+
| Source pattern | Examples | What mirroring does |
66+
| --- | --- | --- |
67+
| Database mirroring (replication) | Azure Cosmos DB, Azure Database for PostgreSQL, Azure Database for MySQL (preview), Azure SQL Database, SQL Server, Oracle, SAP | Mirroring copies the catalog metadata and continuously replicates the data into OneLake as Delta tables. |
68+
| Metadata mirroring (shortcuts) | Azure Databricks, Snowflake, Dremio (preview) | Mirroring adds the catalog metadata to Fabric and uses shortcuts to reach the underlying data in place. |
69+
70+
In every case, mirroring adds the catalog metadata to Fabric. Whether it also copies the data depends on the source.
71+
72+
## Use shortcuts and mirroring together
73+
74+
Shortcuts and mirroring aren't mutually exclusive. You can use them together in many scenarios. For example, use mirroring to add a source system to Fabric once, and then create shortcuts to make selected data from that system available across workspaces, domains, or tenants. If mirroring replicates the source data into OneLake, shortcuts to that data don't create another copy or add storage cost.
75+
76+
Common patterns that combine the two capabilities include:
77+
78+
- **Mirror once, consume everywhere.** A central workspace mirrors an external database or catalog, such as an Azure SQL Database or a Snowflake account. Other workspaces in the same tenant have shortcuts to the mirrored tables instead of mirroring the same source again. One mirrored item represents the source, and every consumer reads the same up-to-date data.
79+
- **Domain mirroring with mesh consumption.** Domain teams mirror their own source systems into their domain workspaces. Downstream lakehouses and warehouses in consumer workspaces use shortcuts to access only the tables they need from the mirrored database. Each domain owns its connection to the source, and consumers stay decoupled from the source system.
80+
- **One lakehouse, multiple data locations.** A single lakehouse can contain shortcuts to mirrored tables alongside shortcuts to open-format data in Azure Data Lake Storage, Amazon S3, or another OneLake location. Reports and notebooks query the lakehouse as one unified store, whether the underlying data remains at its source or is replicated into OneLake.
81+
- **Cross-tenant or cross-cloud access.** One tenant mirrors a source into its own OneLake. A partner tenant creates a shortcut to the mirrored data instead of being granted direct access to the source system. The source system's credentials never leave the producing tenant.
82+
83+
A few things to know when you combine them:
84+
85+
- Shortcuts to mirrored tables inherit the security model of the mirrored item. Permissions on the mirrored database, schema, or table flow through to anyone reading through the shortcut. For more information, see [OneLake shortcut security](onelake-shortcut-security.md).
86+
- Shortcuts to mirrored tables are read-only. To change the data, change the source system and let mirroring propagate the update.
87+
- If the mirrored source is paused, deleted, or runs into a replication error, every shortcut that points to it shows the same state. Plan ownership and monitoring of the mirrored item accordingly.
88+
89+
## When to use data movement instead
90+
91+
Shortcuts and mirroring unify data by making it available through OneLake, and shortcut transformations can convert that data into analytics-ready Delta tables. They aren't replacements for Fabric's full set of ingestion and data movement tools. Choose a pipeline, dataflow, copy job, or eventstream when you need to:
92+
93+
- Apply complex or multi-source transformation logic (for example, joining sources, applying business rules, or reshaping schemas) that goes beyond what shortcut transformations do.
94+
- Schedule, orchestrate, or trigger movement on your own cadence.
95+
- Move data into a destination outside OneLake.
96+
- Stream events into Fabric for real-time processing.
97+
98+
These scenarios require Fabric's data movement and integration tools rather than OneLake unification alone.
99+
100+
- For an end-to-end view of every way to bring data into Fabric, see [Get data into Microsoft Fabric](../fundamentals/get-data.md).
101+
- For a direct comparison of pipelines, copy jobs, mirroring, and eventstreams, see the [Choose a data movement strategy](../data-factory/decision-guide-data-movement.md) decision guide.
102+
103+
## Related content
104+
105+
- [Shortcuts in OneLake](onelake-shortcuts.md)
106+
- [What is mirroring in Fabric?](../mirroring/overview.md)
107+
- [Get data into Microsoft Fabric](../fundamentals/get-data.md)
108+
- [Choose a data movement strategy](../data-factory/decision-guide-data-movement.md)
525 Bytes
Loading

0 commit comments

Comments
 (0)