Skip to content

Commit 0a9d287

Browse files
Merge pull request #1866 from MicrosoftDocs/main638845456695992383sync_temp
For protected branch, push strategy should use PR and merge to target branch method to work around git push error
2 parents ea11f7f + ca40269 commit 0a9d287

14 files changed

Lines changed: 328 additions & 123 deletions

docs/admin/region-availability.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,8 @@ This table lists regions where the only available Fabric workload is Power BI.
6161
| | Norway West | UAE Central | | Korea South |
6262
| | Spain Central | | | |
6363

64+
** Copilot is not supported for regions listed in this section.
65+
6466
### Public preview
6567

6668
This table lists regions where public preview features are available, according to workload.

docs/data-engineering/fabric-notebook-selection-guide.md

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ Use this high-level reference to quickly determine the most suitable notebook ty
3535

3636
Use this structured comparison table to understand the architectural and operational trade-offs between notebook types. Best suited for engineering evaluations or implementation planning.
3737

38-
**Execution & Performance**
38+
### **Execution & Performance**
3939

4040
| Scenario | Python Notebooks (2-core VM) | PySpark Notebooks (Spark Compute) |
4141
| --- | --- | --- |
@@ -46,27 +46,27 @@ Use this structured comparison table to understand the architectural and operati
4646
| High-Concurrency Execution | Manual FIFO-style parallelism per notebook | System-managed concurrency with support for parallel execution. |
4747
| Resource Customization & Scaling | Fixed compute (2-core VM); does not auto scale. Users can manually scale out using %%config within the notebook. | Flexible resource allocation; supports autoscaling and custom Spark configurations. |
4848

49-
**Workflow & Orchestration**
49+
### **Workflow & Orchestration**
5050

5151
| Scenario | Python Notebooks (2-core VM) | PySpark Notebooks (Spark Compute) |
5252
| --- | --- | --- |
5353
| API Orchestration | Effective for lightweight orchestration and control flows, especially REST/gRPC-based integrations | Less optimal for basic orchestration tasks due to longer start-up and distributed overhead. |
5454
| Complex ETL DAGs | Supports FIFO scheduling only | Supports both FAIR and FIFO scheduling with DAG customization. |
5555

56-
**Platform & Library Support**
56+
### **Platform & Library Support**
5757

5858
| Scenario | Python Notebooks (2-core VM) | PySpark Notebooks (Spark Compute) |
5959
| --- | --- | --- |
6060
| Library Access | Strong Python library support across multiple runtimes; however, limited access to Spark-native libraries may require manual integration. | Fully supports MLlib, Spark SQL, PySpark, and Spark Streaming. |
6161
| Delta Lake Handling | The Python Notebook runtime comes with pre-installed [deltas](https://delta-io.github.io/delta-rs/) and [duckdb](https://duckdb.org/) libraries, enabling both reading and writing of Delta Lake data. However, some Delta Lake features may still be unsupported. | Fully supported with native compatibility. |
6262

63-
**Production & Enterprise Readiness**
63+
### **Production & Enterprise Readiness**
6464

6565
| Scenario | Python Notebooks (2-core VM) | PySpark Notebooks (Spark Compute) |
6666
| --- | --- | --- |
6767
| Production Management | Limited production features; Does not support environment vars. | Support for production workflows with environment variables, library management through environment items, and item-based deployment. |
6868

69-
**Cost Considerations**
69+
### **Cost Considerations**
7070

7171
| Scenario | Python Notebooks (2-core VM) | PySpark Notebooks (Spark Compute) |
7272
| --- | --- | --- |
@@ -80,15 +80,15 @@ Use **Python Notebooks** for fast iteration, cost-effective analysis, and intera
8080

8181
Use **PySpark Notebooks** for distributed computing, production-grade ETL workflows, or scenarios where high concurrency and Spark-native APIs are essential.
8282

83-
**Choose Python Notebooks When:**
83+
### **Choose Python Notebooks When:**
8484

8585
- You need fast start-up (typically within seconds) on a lightweight 2-core container.
8686
- If minimizing compute cost is a priority - for interactive analysis or scheduled micro-jobs.
8787
- You want immediate access to pip-installable libraries and pre-installed DuckDB and Polars.
8888
- You need to test across different Python runtime versions.
8989
- Your data comfortably fits in the memory of a single node.
9090

91-
**Choose PySpark Notebooks When:**
91+
### **Choose PySpark Notebooks When:**
9292

9393
- Your workloads exceed the memory or compute limits of a single node.
9494
- You require high-concurrency pools to run parallel jobs across Notebooks.
@@ -109,8 +109,6 @@ This section provides a quick reference for the fundamental technical and archit
109109
| Cost Profile | Lower cost; ideal for short tasks and prototyping | Higher cost; suited for scalable, long-running workloads |
110110
| Python/Spark Versioning | Multiple Python versions available | Tied to specific Spark runtime version |
111111
| Custom Libraries | pip install + resource folders | pip install + resource folders + environments item |
112-
| ~~Environment Variables~~ | ~~Not supported~~ | ~~Supported~~ |
113-
| ~~Lakehouse Integration~~ | ~~Requires Lakehouse to be explicitly attached~~ | ~~Can read/write without attachment~~ |
114112
| Fabric Spark Capabilities | Limited access to Spark engine features | Full access: NEE, Autotune, VORDER, Vegas Cache |
115113
| Delta Lake Compatibility | Partially compatible; potential performance issues | Fully supported and optimized |
116114

docs/data-engineering/runtime.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ Below, you find a comprehensive comparison of key components, including Apache S
3232
3333
| | **[Runtime 1.1](./runtime-1-1.md)** | **[Runtime 1.2](./runtime-1-2.md)** | **[Runtime 1.3](./runtime-1-3.md)** |
3434
|-----------------------|-------------------------------------|-------------------------------------|------------------------------------|
35-
| **Release Stage** | EOSA | GA | GA |
35+
| **Release Stage** | EOS | EOSA| GA |
3636
| **Apache Spark** | 3.3.1 | 3.4.1 | 3.5.0 |
3737
| **Operating System** | Ubuntu 18.04 | Mariner 2.0 | Mariner 2.0 |
3838
| **Java** | 8 | 11 | 11 |

docs/data-factory/index.yml

Lines changed: 58 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -11,110 +11,108 @@ metadata:
1111
ms.topic: landing-page # Required
1212
author: whhender #Required; your GitHub user alias, with correct capitalization.
1313
ms.author: whhender #Required; microsoft alias of author; optional team alias.
14-
ms.date: 9/17/2024 #Required; mm/dd/yyyy format.
14+
ms.date: 6/02/2025 #Required; mm/dd/yyyy format.
1515
# linkListType: architecture | concept | deploy | download | get-started | how-to-guide | tutorial | overview | quickstart | reference | sample | tutorial | video | whats-new
1616

1717
landingContent:
1818
# Cards and links should be based on top customer tasks or top subjects
1919
# Start card title with a verb
2020
# Card
21-
- title: About Data Factory
21+
- title: What is Data Factory?
2222
linkLists:
2323
- linkListType: overview
2424
links:
2525
- text: What is Data Factory?
2626
url: data-factory-overview.md
27-
- text: Connector overview
28-
url: connector-overview.md
27+
- text: Data Factory demo
28+
url: https://aka.ms/DataFactoryLab
2929
- text: What's new in Data Factory?
3030
url: whats-new.md
31-
32-
# Card
33-
- title: Using Data Factory
34-
linkLists:
35-
- linkListType: get-started
36-
links:
37-
- text: Create your first dataflow
38-
url: create-first-dataflow-gen2.md
39-
- text: Create your first data pipeline
40-
url: create-first-pipeline-with-sample-data.md
41-
- text: Move and transform data
42-
url: transform-data.md
43-
- text: Migrate to Data Factory
44-
url: upgrade-paths.md
31+
- text: What can I connect to?
32+
url: connector-overview.md
4533
- text: Pricing in Data Factory
4634
url: pricing-overview.md
47-
- linkListType: tutorial
48-
links:
49-
- text: Create an end-to-end data integration scenario
50-
url: tutorial-end-to-end-introduction.md
35+
- text: Migrate to Data Factory in Fabric
36+
url: upgrade-paths.md
5137
- linkListType: learn
5238
links:
5339
- text: Data Factory training
5440
url: /training/browse/?terms=data%20factory%20fabric
55-
56-
# Card
57-
- title: Working with data pipelines
41+
# Card
42+
- title: Connect to your data
5843
linkLists:
59-
- linkListType: how-to-guide
44+
- linkListType: get-started
6045
links:
61-
- text: Data pipeline runs
62-
url: pipeline-runs.md
63-
- text: Copy data - copy activity
64-
url: copy-data-activity.md
65-
- text: Monitor data pipeline runs
66-
url: monitor-pipeline-runs.md
67-
- text: Activity Overview
68-
url: activity-overview.md
69-
- text: Move data with copy assistant
70-
url: tutorial-move-data-lakehouse-copy-assistant.md
71-
- text: Migrate from Dataflow Gen1 to Dataflow Gen2
72-
url: dataflow-gen2-migrate-from-dataflow-gen1.md
46+
- text: Dataflow Gen2 connectors
47+
url: dataflow-support.md
48+
- text: Data pipeline connectors
49+
url: pipeline-support.md
7350

7451
# Card
75-
- title: Data gateway
52+
- title: Connect on-premises to the cloud
7653
linkLists:
7754
- linkListType: how-to-guide
7855
links:
56+
- text: How to access on-premises data
57+
url: how-to-access-on-premises-data.md
7958
- text: What is an on-premises data gateway?
8059
url: /data-integration/gateway/service-gateway-onprem?toc=/fabric/data-factory/toc.json
81-
- text: On-premises data gateway FAQ
82-
url: /data-integration/gateway/service-gateway-onprem-faq?toc=/fabric/data-factory/toc.json
8360
- text: What is a virtual network data gateway?
8461
url: /data-integration/vnet/overview?toc=/fabric/data-factory/toc.json
85-
- text: Virtual network data gateway FAQ
86-
url: /data-integration/vnet/data-gateway-faqs?toc=/fabric/data-factory/toc.json
8762

8863
# Card
89-
- title: Working with dataflows
64+
- title: Transform data
9065
linkLists:
9166
- linkListType: how-to-guide
9267
links:
93-
- text: Data source management
68+
- text: Create your first dataflow
69+
url: create-first-dataflow-gen2.md
70+
- text: Connect your source to your dataflow
9471
url: data-source-management.md
95-
- text: Monitor your dataflows
96-
url: dataflows-gen2-monitor.md
9772
- text: Save a draft of your dataflow
9873
url: dataflows-gen2-save-draft.md
9974

100-
# Card
101-
- title: FAQs
75+
# Card
76+
- title: Move data
10277
linkLists:
10378
- linkListType: get-started
10479
links:
105-
- text: Frequently asked questions
106-
url: frequently-asked-questions.yml
80+
- text: Create your first data pipeline
81+
url: create-first-pipeline-with-sample-data.md
82+
- text: Move and transform data
83+
url: transform-data.md
84+
- linkListType: how-to-guide
85+
links:
86+
- text: Data pipeline runs
87+
url: pipeline-runs.md
88+
- text: Copy activity in data pipelines
89+
url: copy-data-activity.md
90+
- text: Copy job
91+
url: copy-job.md
92+
- linkListType: tutorial
93+
links:
94+
- text: Create an end-to-end data integration scenario
95+
url: tutorial-end-to-end-introduction.md
96+
- text: Move data with copy assistant
97+
url: tutorial-move-data-lakehouse-copy-assistant.md
10798

108-
# Card
109-
- title: Connectors
99+
# Card
100+
- title: Manage your pipelines
110101
linkLists:
111-
- linkListType: get-started
102+
- linkListType: concept
112103
links:
113-
- text: Dataflow Gen2 connectors
114-
url: dataflow-support.md
115-
- text: Data pipeline connectors
116-
url: pipeline-support.md
104+
- text: Schedule pipelines
105+
url: pipeline-runs.md#scheduled-data-pipeline-runs
106+
- text: Orchestrate with Apache Airflow
107+
url: apache-airflow-jobs-concepts.md
117108
- linkListType: how-to-guide
118109
links:
119-
- text: Connector certification
120-
url: connector-certification.md
110+
- text: Start a pipeline from a storage event
111+
url: pipeline-storage-event-triggers.md
112+
- text: Monitor your pipelines
113+
url: monitor-pipeline-runs.md
114+
- linkListType: tutorial
115+
links:
116+
- text: Run Fabric pipelines with Apache Airflow
117+
url: apache-airflow-jobs-run-fabric-item-job.md
118+

0 commit comments

Comments
 (0)