Problems displaying this newsletter? View online.
Database Weekly
The Complete Weekly Roundup of SQL Server News
Hand-picked content to sharpen your professional edge
Editorial
 

Why Doesn't Database Software Help Us?

I remember when SQL Server 7.0 was released. One of the big marketing pushes was that the database would manage itself. I worked in a small startup at that time, and they kept joking that I would be out of a job soon. In fact, a family member in IT at the time, the late 90s, was going to get out of IT because he thought it was be a low paying, blue collar job in a decade.
 
He was wrong, and so were my co-workers. Since that time, I've had a number of jobs, and never much of a problem finding one that involved production work. I've also been well compensated, and I've had a nice career through the time I started working for Redgate. It's been great since then, but like Brent, I don't really do production work anymore.
 
Still, it does seem that database ought to do better managing themselves. Brent Ozar talked about that recently, including why he doesn't like production work. I think he makes good points, but I think there's another view of this.
 
The hassles he mentions, the lack of self-tuning, the need to tweak the new features, this means work. This means that someone needs to be paid to do this. And since many companies have lots of database, if you show you can do this stuff at scale, with things like PoSh and dbatools, you are worth being well paid.
 
I do think that databases ought to be more self-tuning and more self-contained. Settings like query items, client connections, jobs, backups, etc. ought to be stored inside the db. Query store ought to enhance slightly a track whether a db would benefit from more memory or CPUs. Not detailed history, but based on a short period, what would make this db run better?
 
Of course, as Brent mentioned, lots of this might not benefit vendors. After all, if we wrote better code, and knew where to focus, we might buy less copies of the database engine.

Steve Jones - SSC Editor

Join the debate, and respond to the editorial on the forums

 
The Weekly News
All the headlines and interesting SQL Server information that we've collected over the past week, and sometimes even a few repeats if we think they fit.
Vendors/3rd Party Products

Customizations in SQL Prompt

In this brand new course on the Redgate University Grant Fritchey walks you through the many ways in which SQL Prompt can be customized to suit your specific needs.

Using Flyway Output in PowerShell

How to send Flyway logging and error output to JSON and consume it in PowerShell to produce ad-hoc database migrations reports, including any errors that occurred, the version of the database, runtimes for each migration script and more.

Why is my SQL Server Query Suddenly Slow?

A SQL Server query is suddenly running slowly, for no obvious reason. Grant Fritchey shares a 5-point plan to help you track down the cause and fix the problem.

Building reusable table build scripts using SQL Prompt

You need a fast, general-purpose way to save the results of a query or batch or procedure into any sort of worktable, such as a temporary table or a table variable or table valued parameter. A simple SELECT…INTO isn't versatile enough for these requirements, and the alternative ways to handcraft the list of columns are slow and error prone. Phil Factor shows how to create a 'table-build generator' that will do all this, and save you a lot of time, especially if you use a lot of working tables in your code.

AI/Machine Learning/Cognitive Services

Messy Data Shouldn’t Stop Machine Learning in Its Tracks

Click to learn more about author Jon Reilly. Businesses are creating data at an incredible pace that will only accelerate. In fact, data storage company Seagate predicts it will...

Accessing text analytics from a chatbot: Sentiment

SAS Conversation Designer is available with every offering that also includes SAS Visual Analytics. Users can easily access Visual Text Analytics capabilities from SAS Conversation Designer with minimum additional...

Administration of SQL Server

Detecting Database Corruption

You don’t have to be Sherlock Holmes to detect Server Corruption. Steve Stedman and Derrick Bovenkamp share valuable information on how to spot server corruption before the damage is...

It’s Been Decades. Why Don’t Databases Manage Themselves Yet?

This month, to mark the 20th anniversary of BrentOzar.com, I’m stepping back and looking at the big picture rather than blogging about database problems & solutions like I usually...

T-SQL Tuesday Retrospective #012: Why DBA skills are necessary

This is my ongoing series of answering T-SQL Tuesday posts far too late to be of any use. Click here if you want to read previous entries. Paul Randal... The...

Trace Flag 8048 May Still Be Necessary After SQL Server 2016

Big Box Way back when, SQL Servers with lots of cores could hit some weird contention on CMEMTHREAD. To fix that and a lot of other issues associated with...

What Does a Database Administrator Actually Do?

BrentOzar.com is 20 years old, and I’m celebrating by being a little retrospective and future-looking this month. Today, I’m publishing a post that I could have published 10 years...

Your Database is Corrupt – Here’s Your “Now What?” Video

Do you know what to do when corruption hits? You should watch this quick 10 min video first if you plan on fixing it yourself? Steve Stedman and Derrick...

Azure CosmosDB

Azure Cosmos DB Continuous Backup

Azure Cosmos DB announced Continuous Backup in Cosmos DB on March 2021. This feature is currently in public preview mode and It is not recommended to use in production.

Azure Databricks, Spark and Snowflake

Spark – Reading Parquet – Why the Number of Tasks can be Much Larger than the Number of Row Groups

A row group is a unit of work for reading from Parquet that cannot be split into smaller parts, and you expect that the number of tasks created by...

Azure Synapse (SQL Data Warehouse and Data Lake)

Ingesting data into Azure Synapse Analytics using PolyBase

In this article, we will learn how to ingest data into Azure Synapse Analytics with a focus on using PolyBase to load the data.

Career Growth and Certifications

I’m Hiring: the Quest for an Intern

Catallaxy Services is looking for a paid intern. By which I mean that I am looking for an intern, but it sounds more official when I take on the...

Community Interests

Finding a SQL Saturday Board of Directors

I wrote recently about looking for candidates and the criteria that I initially considered. I am happy to say quite a few people expressed interest, and my list of... The...

Rebuilding SQL Saturday–Picking a Board of Directors

With Redgate planning to donating the SQL Saturda...

Computing in the Cloud (Azure, Google, AWS)

Apache Kafka MirrorMaker 2 (MM2) Part 1: Theory

Popular Team Technical Cassandra Elasticsearch Kafka Redis Spark Show All View by blogger Ben Bromhead Ben Slater Paul Brebner In this new two part blog series we’ll turn our gaze to the newest version of MirrorMaker 2 (MM2), the Apache Kafka cross-cluster mirroring, or replication, technology. MirrorMaker 2 is built on top of the Kafka Connect framework for increased reliability and scalability, and is suitable for more demanding geo-replication use cases including migration, backup, disaster recovery and fail-over.

Initial Thoughts on Dremio

I’ve been working on a project for the last few ...

Deploying Azure Data Services via Terraform Part 1: An Introduction

One of the most significant things to change the landscape for Azure data professionals will be general release of Azure Arc enabled Data Services. To provide an expedient means of experiencing all that Azure Arc has to offer, Microsoft has come up with Jumpstart – a collection of GitHub repos for deploying Arc in different scenarios.

Cloud Platforms: Google vs. AWS

Selecting both primary and secondary cloud services is now a common IT strategy for most enterprises. Recent research shows that about 90 percent of enterprises and non-profit organizations are...

AWS Fault Injection Simulator – Use Controlled Experiments to Boost Resilience

AWS gives you the components that you need to build systems that are highly reliable: multiple Regions (each with multiple Availability Zones), Amazon CloudWatch (metrics, monitoring, and alarms), Auto...

Conferences, Classes, Events, and Webinars

Webcast Recording – Building a Proper SQL Server DB Security Model

The recording for my presentation on Building a Pr...

DMO/SMO/Powershell

PowerShell: How to create Logging for your PowerShell Scripts

So you have already created your first PowrShell scripts? Now you want to enhance this scripts with error logging ? If your answer is yes, jump in this this article.

PowerShell editors and environments part 2

In this article, Greg Moore demonstrates two additional PowerShell editors: Azure Data Studio Notebooks and Visual Studio Code.… The post PowerShell editors and environments part 2 appeared first on Simple...

Data Privacy, Compliance, and GDPR

What is a data catalog – and why do you need one?

Learn why a data catalog is so valuable in helping you find and use big data at your business. The post What is a data catalog – and why do...

Data Visualisation

a personal touch

43 people tried their hand at drawing theiri data in our February 2021 storytelling with data challenge. Last month’s challenge...

Data Warehousing

Data Lakes and Modern Analytics for Education

Working in Institutional Research has allowed me to support various folks across college campuses. Among the compliance necessities, population surveys, and ad hoc data requests - the work I...

Database Design, Theory and Development

Where I See Databases Going in the Next 10 Years

This month, to mark the 20th anniversary of BrentOzar.com, I’m stepping back and looking at the big picture rather than blogging about database problems & solutions like I usually...

Foreign Keys – Are they worth it?

Last time, we learned about Primary Keys and some of the goodness that they give you. If you missed that, remember to go back and read that here. This time, though, we want to focus on the close kin of primary keys – the foreign key.

DevOps and Continuous Delivery (CI/CD)

Three Steps To Get Started With Database DevOps

Once you’ve committed to changing your culture in order to automate your database deployments, what’s next? You’ve already done the hard part, making the decision to shift the culture....

Code Review and DevOps

Almost every time I teach a class on DevOps, one of the questions that comes up is: How do we do code reviews in this automated process? There are...

DocumentDB/Key-Value/Graph/other NoSQL Databases

GRAPH ALGORITHMS AND INFERENCE

If you’ve used Google Maps for directions or “friended” someone on Facebook, you’ve experienced graph theory. Graph theory is all about connectedness, how things in the world relate to other things, and how those interactions affect the objects around them.

ETL/SSIS/Azure Data Factory/Biml

How I tested Parse data transformation in Azure Data Factory

(2021-Mar-15) While watching the first MiB movie with my family, it was quite clear that the character played by Will Smith at first didn’t have a clear idea about what... The...

Parquet Stream

I decided to stream about Parquet files. Enjoy! :{>

Hardware

The Western Digital WD Black SN850 Review: A Very Fast PCIe 4.0 SSD

Western Digital has rapidly risen to being a top-tier player in the market, and this is no more evident than with their newest high-end SSD, the WD Black SN850.

Intel’s New Adaptive Boost Technology: Floating Turbo Comes to Rocket Lake

A couple of days after Intel officially announced its 11th Generation Core Rocket Lake, the press received an email about a new feature coming to the platform that wasn’t...

My new Home Studio Setup

(Be sure to checkout the FREE SQLpassion Performance Tuning Training Plan - you get a weekly email packed with all the essential knowledge you need to know about performance... The...

AMD’s Epyc Milan offers double Intel Xeon’s datacenter performance

It's hard to overstate how badly AMD is shaming In...

AMD Ryzen Pro 5000 Mobile: Zen 3 comes to Commercial Notebooks

Alongside every launch of AMD’s consumer processors, the commercial offerings for business come along a few months after. Today we see the commercial launch of the Ryzen 5000 Mobile...

Intel Launches Rocket Lake 11th Gen Core i9, Core i7, and Core i5

In the myriad of news and early reviews, Intel is today officially launching its 11th Generation Core family of desktop processors, also known as Rocket Lake, built on Intel’s...

MDX/DAX

How to get multiple slicer items filtered using the DAX TREATAS function

Quite often I wanted to get the data a slicer quic...

DISTINCTCOUNT, DISTINCTCOUNTNOBLANK – DAX Guide

DISTINCTCOUNT: Counts the number of distinct values in a column. https://dax.guide/distinctcount/ DISTINCTCOUNTNOBLANK: Counts the number of distinct values in a column. https://dax.guide/distinctcountnoblank/

COUNTROWS – DAX Guide

COUNTROWS: Counts the number of rows in a table. https://dax.guide/countrows/

MIN, MINA, MINX – DAX Guide

MIN: Returns the smallest value in a column, or the smaller value between two scalar expressions. https://dax.guide/min/ MINA: Returns the smallest value in a column. https://dax.guide/mina/ MINX: Returns the...

Performance Tuning SQL Server

Load Testing with SqlQueryStress

Do you have a new SQL server that you need to load test against? What about a new stored procedure that needs tested with various parameters? Maybe you’re just trying to punish your CPU?

Reverse Engineering the Key Lookup Cost Formula

During a recent investigation into Key Lookup costing, I discovered something interesting. Perhaps a warning is due: I’m a nerd, and things I find “interesting” may cure insomnia in...

PowerPivot/PowerQuery/PowerBI

Improving The Performance Of Importing Data From ADLSgen2 In Power BI By Partitioning Tables

So far in this series of posts looking at the performance of importing data from files stored in ADLSgen2 into Power BI I have looked at trying to tune...

COMMON POWER BI ADMIN SCRIPTS PART II

Today’s post builds on top of Power BI Admin Scripts Part I by describing five additional PowerShell scripts that Power BI service administrators can utilize to address relatively common scenarios. Like Part I, the five new scripts have been added to my GitHub repository and I’ll only share context in the blog.

Using Microsoft Forms data with Power BI

Are you using Microsoft Forms to capture informati...

Comparing The Performance Of Importing Data Into Power BI From ADLSgen2 Direct And Via Azure Synapse Analytics Serverless, Part 3: Parquet Files

Since I started this long and rambling series of p...

Product Reviews and Articles

Creating Database Build Artifacts when Running Flyway Migrations

A PowerShell automation framework for running Flyway database migrations that will automatically generate all other required database build artifacts, in the version control system. The post Creating Database Build Artifacts...

Product Upgrades and Releases

Introducing Amazon S3 Object Lambda – Use Your Code to Process Data as It Is Being Retrieved from S3

When you store data in Amazon Simple Storage Service (S3), you can easily share it for use by multiple applications. However, each application has its own requirements and may...

Amazon S3 Glacier Price Reduction

The Amazon S3 Glacier storage class is ideal for data archiving and long-term backup of information that will be accessed at least once per quarter (Amazon S3 Glacier Deep...

Python

Local PySpark Development on Windows with WSL2, Docker Desktop, and VSCode

I’m not a frequent user of Windows, but I understand getting dependencies installed for local development can sometimes be a bit of a pain. I’m using an Azure VM1, but these instructions should work on a regular Windows 10 installation.

R Language

An introduction to k-gram language models in R

This post offers a brief introduction to k-gram language models, using the R package kgrams, which provides an interface for training, evaluating and computing with classical k-gram models.

Fuzzy joins in SQL Server Machine Learning using R scripts

In this article, we will discuss the Fuzzy joins in the SQL Server Machine Learning using R scripts.

Replication

Transactional Replication – is it keeping up?

Working with Transactional Replication with many o...

SQL Server on Linux

How to view SQL Server logging on Linux

How can I view SQL Server Logging on Linux ?

Security News and Issues

~4,300 publicly reachable servers are posing a new DDoS hazard to the Internet

DDoS-for-hire services adopt new technique that am...

Exploiting Spectre Over the Internet

Google has demonstrated exploiting the Spectre CPU...

“Expert” hackers used 11 0-days to infect Windows, iOS, and Android users

The breadth and abundance of exploits for unknown vulnerabilities sets group apart.

Easy SMS Hijacking

Vice is reporting on a cell phone vulnerability caused by commercial SMS services. One of the things these services permit is text message forwarding. It turns out that with...

On the Insecurity of ES&S Voting Machines’ Hash Code

Andrew Appel and Susan Greenhalgh have a blog post on the insecurity of ES&S’s software authentication system: It turns out that ES&S has bugs in their hash-code checker: if...

T-SQL

SQL INSERT INTO SELECT: 5 Easy Ways to Handle Duplicates

Total: 1 Average: 5You probably know how to insert...

When Should You Use Table Variables?

Screech Links: Table Variables Table Variables In SQL Server 2019 What’s Really Different About In Memory Table Variables? A Trick For Dealing With Table Variable Modification Performance Starting SQL:...

A Parameterization Puzzle With TOP PERCENT

Lawdy There was a three-part series of posts where I talked about a weird performance issue you can hit with parameterized top. While doing some query tuning for a...

T-SQL Merge Statements- Just Don’t

There are plenty of reasons not to use a MERGE statement. Many of those reasons have been covered in depth before. Instead of covering them myself, I will point you to another useful post on the subject.

Data dictionary script

I restarted speaking with New Stars of Data today ...

Deprecated features to take out of your toolbox – Part 2

With a real-world example, Aaron Bertrand discusses why you shouldn't use deprecated functionality like SQL Server Profiler. The post Deprecated features to take out of your toolbox – Part 2...

#TSQL2sday 136 Wrap-Up: Your Favorite (and Least Favorite) Data Types

For this month’s T-SQL Tuesday, I asked you to blog about your most-loved and least-loved data types. Crazy, right? How could people possibly love or hate data types? Well,...

Count Distinct and Window Functions

Or: How to make magic tricks with T-SQL Starting our magic show, let’s first set the stage: Count Distinct doesn’t work with Window Partition Preparing the example In order...

Lesser-known facts of Explicit Transactions

Transactions are of various kinds in SQL Server such as Autocommit, Implicit, Explicit and Batch-scoped. I personally found this article quite helpful if you wish to understand Implicit Transactions.... The...

Another Reason Why I Love Dynamic SQL: OUTPUT Parameters Can Be Input Parameters

Well-Treaded A lot has been written about dynamic SQL over the years, but I ran into a situation recently where I needed to rewrite some code that needed it...

What is interpolation?

In this article, Joe Celko explains interpolation and covers a bit about the history and what we all did before computers.

Tech News

AT&T lies about Calif. net neutrality law, claiming it bans “free data”

AT&T is angry that it must stop charging HBO Max rivals for data-cap exemptions.

One company wants to sell the feds location data from every car on Earth

Federal agencies already buy tons of user data; one vendor is now being sued.

I was a teenage Twitter hacker. Graham Ivan Clark gets 3-year sentence

Florida teen pleads guilty to attack that took over Twitter's internal systems.

The Lighter Side

BMW’s big electric car push: Selling 2 million new EVs by 2025

The iX arrives next year, but expect "mid $80s" pricing for the big BEV.

Virtualization and Containers/Kubernetes

Protected: Deploying Azure Data Services via Terraform Part 5: Deploying a Load Balancer to The Kubernetes Cluster

There is no excerpt because this is a protected post.

 
RSS FeedTwitter
This email has been sent to newsletter@newslettercollector.com. To be removed from this list, please click here. If you have any problems leaving the list, please contact the webmaster@sqlservercentral.com. This newsletter was sent to you because you signed up at SQLServerCentral.com. Note: This is not the SQLServerCentral.com daily newsletter list, and unsubscribing to this newsletter will not stop you receiving the SQL Server Central daily newsletters. If you want to be removed from that list, you can follow the instructions on the daily newsletter.
©2019 Redgate Software Ltd, Newnham House, Cambridge Business Park, Cambridge, CB4 0WZ, United Kingdom. All rights reserved.
webmaster@sqlservercentral.com

 

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -