SYNAPSE 3 FILES SERIES

If it has been installed, updating (overwrite-installing) may fix problems, add new functions, or expand existing ones.

Speed Up Data Refresh Performance In Power BI Desktop Using Table.This package contains the files needed for installing the Razer Synapse 3.0 driver.

Keep The Existing Data In Your Power BI Dataset And Add New Data To It Using Incremental Refresh.

Understanding The 'The key didn't match any rows in the table' Error In Power Query In Power BI Or Excel.

So, based on these results it seems fair to draw the following conclusions:Д¬onclusion #1: if youвЂ™re importing all the data from files in ADLSgen2 then connecting direct is faster than going via Synapse ServerlessД¬onclusion #2: if youвЂ™re connecting direct to files in ADLSgen2 and importing all the data from them then CSV files are faster than Parquet filesД¬onclusion #3: if youвЂ™re transforming data then connecting to Parquet files via Synapse Serverless is a lot faster than any other method However I donвЂ™t have the time or expertise to test this properly so IвЂ™m going to declare it out of scope and concentrate on comparing Synapse Serverless performance with the performance of connecting to the same files direct. There is another variable here that IвЂ™m not considering: what if the number and size of files used affects performance? As other people have found, it certainly affects Synapse Serverless performance it may also affect Power Query performance too.

WhatвЂ™s more, Synapse Serverless on Parquet was substantially faster than Synapse Serverless on CSV: filtering by date via Serverless on Parquet took 7 seconds compared to 15 seconds via Serverless on CSV, and grouping by date via Serverless on Parquet also took 7 seconds compared to 15 seconds via Serverless on CSV.

The two transformation tests, filtering by date and grouping by date, were a lot faster than connecting direct to ADLSgen2.

Loading all the data via Synapse Serverless from Parquet files took 91 seconds whereas it only took 72 seconds via Synapse Serverless from CSV files.

Importing all the data from Parquet files via Synapse Serverless performed a lot worse than connecting direct to ADLSgen2 in fact it was the slowest method for loading all the data tested so far.

Filtering by date took 29 seconds for the Parquet files and 27 seconds for the CSV files grouping by date took 34 seconds for the Parquet files and 28 seconds for the CSV files.

The performance of the two subsequent tests for filtering by date and grouping by date were only slightly worse when connecting direct to the Parquet files in ADLSgen2 as when connecting to CSV files.

The performance of importing all the data by connecting direct to the files in ADLSgen2 was the slightly slower here for Parquet files (72 seconds) than in my first blog post with CSV files (65 seconds).

HereвЂ™s a table with all the average refresh times for each test:

Creating a view in Azure Synapse Analytics Serverless on top of the Parquet files and importing the data to Power BI from that using Power BIвЂ™s Synapse connector.

Connecting direct to the files in ADLSgen2 (using the AzureStorage.DataLake M function) from Power BI.

Doing a Group By on the date column and counting the rows for each date.

SYNAPSE 3 FILES SERIES

I then repeated some of the tests I ran in the first two posts in this series вЂ“ here and here. To test the performance of Parquet files I took the data that I have been using in this series and loaded it from the original CSV files into Parquet files using Azure Data Factory. Since I started this long and rambling series of posts on importing data from ADLSgen2 into Power BI a lot of people have asked me the same question: will using Parquet files instead of CSV files perform better? In this post youвЂ™ll find out.

Synapse 3 files