Throwing Iron at the Cloud and AI

As organizations move to the cloud, the once essential role of the Database Administrator (DBA) as the guardian of system optimization, has been overshadowed, often viewed as a bottleneck to innovation. Yet, as technology evolves, the one thing I know is history repeats itself, and optimization skills are once again emerging as a critical necessity. Cloud vendors have astutely capitalized on a recurring pattern of technology to “throw iron” at problems, or scale by simply adding more resources. However, this approach inevitably reaches a breaking point, no matter how much “iron” is increased.

Cloud of Iron

In the cloud, scaling manifests as upgrading service tiers or increasing resource allocations, which is essentially a modern version of “throwing iron.” While the terminology has changed, the principle remains the same - without monitoring and optimizing systems, inefficiencies lead to greater consumption of CPU, memory, storage, and I/O. In on-premises environments, organizations were forced to optimize to avoid the significant expense of purchasing new hardware. In the cloud, however, scaling is deceptively easy - until the costs become unsustainable.

What many forget is that behind every cloud service, feature, or offering lies real hardware and code. Once systems migrate to the cloud, access to the underlying code, software, or hardware configurations often diminishes, making optimization even more challenging. To handle the demands of countless users, cloud platforms rely on throttling and tiered access. When consumption exceeds the limits of a service tier, the default solution is to scale, then adding more resources and incurring higher costs. With limited options for managing consumption within existing tiers, organizations often have no choice but to “throw iron” at the problem, even if the process is dressed up in new terminology.

Cloud Cost Consumption

This dynamic has spurred the growth of a niche yet vital field: cloud consumption optimization. These specialists are critical in environments centered on data, where growth is inevitable. Without proactive optimization, organizations risk spiraling consumption and skyrocketing costs.

Now, a new challenge has emerged: artificial intelligence. AI’s demands have shifted the focus from CPUs to GPUs, with resource costs that can overwhelm even large organizations. Cloud providers are grappling with the need to power AI workloads, with some envisioning nuclear-scale data centers to meet this escalating demand.

New Kid in the AI Pool

Enter Deepseek, a surprising disruptor offering efficient and optimized AI models designed to challenge the status quo. This marks a pivotal moment in the evolution of AI, as optimization once again takes center stage. Deepseek’s innovations demonstrate that organizations can achieve more with less, maybe even breaking the endless cycle of scaling to accomplish AI goals.

History shows us that what is complex will eventually be simplified, what is massive will be minimized, and what once required extraordinary resources will become accessible to all. As we navigate AI, one truth remains constant: optimization isn’t just an option – it’s an essential part of the evolution in technology.

Kellyn Gorman, aka DBAKevlar

Join the debate, and respond to the editorial on the forums

This article explores dynamic alerting, what it means, why it matters for DBAs, and how it works in Redgate Monitor. Learn how machine learning-driven thresholds increase alert relevance, helping teams focus on real performance issues while saving DBAs time.

While employees want to take advantage of the incr...

THE VIDEO THE SYNOPSIS In this video, I demonstrate how we can use PolyBase and MinIO to read files on a local machine in SQL Server. LINKS AND ADDITIONAL...

The data transformation tool dbt (data build tool) has become more and more popular over the past years. It focuses heavily on SQL, and it adds a bunch of interesting features into the mix, such as data lineage, automatic orchestration, reusable macros etc.

I would really love to have a better understanding of what is going on here!If you format a 32 MB disk using NTFS, you’ll get the following result:So about 10...

Vlad Drumea troubleshoots an issue: This is one of...

Haripriya Naidu points out that nothing comes for free: With every feature comes both advantages and disadvantages and it is important to be aware of…

It was announced in the most recent PASS Summit: The minimal Auto Pause delay was reduced from 1 hour to 15 minutes. This configuration defines the minimal time the database will be online when a user makes a query. If a user connects for one simple query and goes away, the database would be online for at least 1 hour. The new feature allows us to reduce this value to 15 minutes.

Organizations struggle with unpredictable cloud spending, but improved data visibility, cost management, and long-term forecasting can turn cloud financials into a strategic asset for optimizing IT investments.

When I look back at my career in technology, I’m amazed at the rate of change, but also by just how much has remained the same.

When I look back at my career in technology, I’m amazed at the rate of change, but also by just how much has remained the same. My first introductions...

Few things frustrate travelers more than lost baggage, and United Airlines certainly tested my patience last week. There were plenty Tags:  Development Del.icio.us Facebook TweetThis Digg StumbleUpon ...

Throughout this series, I’ve introduced you to different features in MongoDB and provided examples to help demonstrate how the database system works. The examples have all been based on...

In my previous post, I discussed how Linux will si...

Meagan Longoria shares some hard-earned experience: I have a few clients that incrementally load tables from a SQL Server source into their data warehouse or…

Reading Time: 6 minutesIn this post I want to share one way that you can authenticate as a service principal to run a Microsoft Fabric notebook from...

Eugene Meidinger looks out for the smaller fish in...

For several years, it has been a limiting factor that Azure Data Factory (ADF) and Azure Synapse Analytics, and more recently Fabric, did not allow dynamic content to be...

Part 2 in the series: A Comprehensive Beginners Gu...

Joe Fleming is not a fan of cursors: So what are cursors, and why are they so bad?  A cursor is a construct in SQL…

Indexing SQL Server Queries For Performance: Indexed View Maintenance Thanks for watching! Going Further If this is the kind of SQL Server stuff you love learning about, you’ll love...

Indexing SQL Server Queries For Performance: Fixing non-SARGable Predicates Thanks for watching! Going Further If this is the kind of SQL Server stuff you love learning about, you’ll love...

Indexing SQL Server Queries For Performance: Eager Index Spools Thanks for watching! Going Further If this is the kind of SQL Server stuff you love learning about, you’ll love...

On 6th of January 2025, Nathan Bossart committed p...

Hettie Dombrovskaya runs into an error: Here is a story. When anyone gives a talk about partitions, they always bring up an example of archiving:…

In this blog (the third in my series), I'd like to present yet another new feature in the PostgreSQL 17 release: enhancement to logical replication functionality in PostgreSQL. The...

You can’t just exec DROP ROLE your_role_name; if it’s granted perms or other roles are granted to it. I had to go fishing to find all the grants to... The...

Josephine Bush drops a role: You can’t just exec DROP ROLE your_role_name; if it’s granted perms or other roles are granted to it. I had to go…

Both Azure DevOps and GitHub are supported Git hos...

New research reveals that tech professionals prioritize innovation and continuous learning over salary. Companies must foster a culture of creativity and skill development to attract and retain the industry's...

Tom Collins takes a gander: I’m doing some sql server security privilege troubleshooting , because a customer has reported an incident  – they have privileges…

As Data Privacy Week sharpens the focus on protecting personal information, I’m reminded of a customer event a major North American bank hosted at SAS world headquarters. The bank’s chief...

A newly discovered VPN backdoor uses some interesting tactics to avoid detection: When threat actors use backdoor malware to gain access to a network, they want to make sure all...

Simon Frazer rolls back: If you’ve been working with T-SQL scripts for a while, you’ve likely encountered variables. These are essential for writing scripts that…

As well as the variables that you declare, SQL Ser...

In general, we try to avoid procedural logic when ...

The simplest way to join two tables is what's call...

Ronald Kraijesteijn builds a script: When testing a data warehouse, a common challenge is managing large datasets effectively. Often, you need to reset tables to a…

Sebastiao Pereira gets out the measuring tape: How do you calculate the distance between two different points on a sphere using TSQL in SQL Server?…

Hey data friends! This one comes from my personal vault (aka backlog of drafts I’ve been needing to write up) and is a really simple code that I always...

There is a feature of SQL I have not seen used much and probably with good reason. It’s the use of temporary tables. These are declared with the expected DDL. Since they were relatively easy to add to the language, they have been around for a while. The bad news is that in the early days SQL programmers tended to write code that looked more like a procedural language.