Free AI web copilot to create summaries, insights and extended knowledge, download it at here
3413
Abstract
zations-for-columnstore-compression">here</a>.</p><p id="fc40">An example of over-configured workload management is when resources are reserved for a workload group even when there are no active requests in it. When setting up a workload group, you can define the minimum % of resources that are always reserved for the group. This is very useful in cases when you have to ensure that SLAs are met, but it should always be done with caution. If such strict isolation of resources is not required, it’s better to use a shared resource pool in combination with different workload importances.</p><h2 id="3041">3. Using clustered columnstore index for staging</h2><p id="23d4">By default, dedicated SQL pool will set up tables using clustered columnstore indexes. These are highly compressed, column-based data structures optimized for analytical workloads on large tables, but they can be expensive to build.</p><p id="5dbe">When loading data into a clustered columnstore index, the rows are first split into row groups (batches) which then are separated into column segments before each of these segments get compressed.</p><figure id="c79e"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*NUSN2jBohgHVlSGhjbgToA.png"><figcaption><a href="https://learn.microsoft.com/en-us/sql/relational-databases/indexes/columnstore-indexes-data-loading-guidance?view=sql-server-ver16">Source</a></figcaption></figure><p id="0b0b">Because building a clustered columnstore index is a resource intensive process, using them for temporary staging tables creates an unnecessary overhead without providing any benefits. When comparing the throughput of bulk loading processes for clustered columnstore indexes and heap tables, a 2–3x difference can be observed (<a href="https://techcommunity.microsoft.com/t5/sql-server-blog/data-loading-performance-considerations-with-clustered/ba-p/305223">link</a>).</p><p id="1bcf">While avoiding the aforementioned 3 mistakes when working with a Synapse dedicated SQL pool can help you save cost by utilizing your resources more efficiently, finding the right combination of tools for your business needs can be challenging. We at Starschema can help you identify the technologies that will best serve your use cases and fine-tune them for optimal performance. <a href="https://starschema.com/contact">Reach out</a> — we’d love to talk.</p><p id="ed30">To learn how Synapse fares against Databricks, Snowflake, Redshift and BigQuery fare and scale in terms of query performance, cost per performance and differentiating feature value, see the results of our extensive testing in this white paper:</p><div id="2298" class="link-block">
<a href="https://starschema.com/kb/cloud-data-warehouse-benchmark-2023">
<div>
<div>
<h2>Cloud Data Warehouse Benchmark 2023</h2>
<div><h3>As cloud data warehouse providers race to improve the scalability, performance and cost-effectiveness of their…</h3></div>
<div><p>starschema.com</p></div>
</div>
<div>
<div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/0*QBnOLQvJ3CUAhQ_C)"></div>
</div>
</div>
</a>
</div><p id="28bd"><b>About the author</b></p><p id="a1e9"><i>Marton is a highly skilled data professional with experience in commerci
Options
al aviation and electronics manufacturing. He holds a Master’s degree in data science from Tilburg University and previously worked as a data scientist with the top management of a major European airline, where his transformational work helped save millions of euros for the company. In his current work as a data engineer at Starschema, he helps Fortune 500 companies build data platforms to unlock greater value from their data. Connect with Marton on <a href="http://www.linkedin.com/in/martonmesz">LinkedIn</a>.</i></p><p id="272c"><b>REACH OUT TO STARSCHEMA <a href="https://starschema.com/contact">HERE</a>:</b></p><div id="2f2e" class="link-block">
<a href="https://starschema.com/contact">
<div>
<div>
<h2>Let’s talk | Starschema</h2>
<div><h3>We help your organization become data-driven</h3></div>
<div><p>starschema.com</p></div>
</div>
<div>
<div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/0*JnBf22UiwDSON2-d)"></div>
</div>
</div>
</a>
</div><p id="742c"><b>READ MORE STORIES FROM STARSCHEMA:</b></p><div id="4934" class="link-block">
<a href="https://readmedium.com/find-the-balance-between-cloud-cost-and-efficiency-889958c39908">
<div>
<div>
<h2>Find the Balance between Cloud Cost and Efficiency</h2>
<div><h3>Learn how to measure the ROI of a cloud migration and get clarity on the opportunities and challenges inherent in…</h3></div>
<div><p>medium.com</p></div>
</div>
<div>
<div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/0*3onHo1F--Lp3oe9M)"></div>
</div>
</div>
</a>
</div><div id="a8c0" class="link-block">
<a href="https://readmedium.com/data-fabric-vs-data-mesh-find-the-right-fit-for-your-organization-40b37f4efba">
<div>
<div>
<h2>Data Fabric vs Data Mesh: Find the Right Fit for Your Organization</h2>
<div><h3>Learn the differences between data mesh and data fabric architectures and find the right one for your data governance…</h3></div>
<div><p>medium.com</p></div>
</div>
<div>
<div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/0*uR7nzrNVrfSbaH6x)"></div>
</div>
</div>
</a>
</div><div id="5263" class="link-block">
<a href="https://readmedium.com/from-guesswork-to-genius-how-to-get-maximum-value-from-marketing-data-and-automation-b7600346dee7">
<div>
<div>
<h2>From Guesswork to Genius: How to Get Maximum Value from Marketing Data and Automation</h2>
<div><h3>See how one company used marketing data to learn more about their audience and run more effective campaigns — and how…</h3></div>
<div><p>medium.com</p></div>
</div>
<div>
<div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/1*nJFWer2IZmGd2KJAGBR-3g.jpeg)"></div>
</div>
</div>
</a>
</div></article></body>
