AWS Analytics Cost Optimization: A Negotiator's Pillar Guide

AWS Analytics is rarely the headline category in a CFO conversation about cloud cost, and that is part of the problem. Compute and storage attract attention; analytics quietly compounds. Across 500+ engagements we have run, the analytics line is the one most likely to double inside two years without anyone noticing. The same engagements show that 30 to 50 percent of that line is recoverable through scoping, query design, commitment shaping, and renewal negotiation. This pillar walks through how, service by service.

Why analytics spend behaves the way it does

Three structural features make analytics cost compound differently from compute:

1. Pricing dimensions are query-shaped, not capacity-shaped. Athena charges per byte scanned, Glue per DPU-hour, Kinesis per shard-hour and per PUT, Redshift per node-hour or per RPU-hour, OpenSearch per instance-hour. A bad query plan or a chatty producer can drive cost in ways that are invisible in the standard CloudWatch metrics.
2. Analytics workloads are bursty but rarely turned off. Many environments leave EMR clusters, Redshift clusters, and OpenSearch domains running 24/7 even though peak load is two hours a day. The idle cost dwarfs the active cost.
3. Data engineering and analytics teams own the cost lever, but rarely the cost report. The result is decisions that look optimal locally and inflate spend globally - duplicate datasets, redundant transformations, untimed concurrency scaling.

The analytics cost map

The pattern: the cost lever for analytics is almost always architectural or scoping-related, not negotiated discount. The negotiated discount applies on top, after the architecture is right.

Athena: the bytes-scanned game

Athena's pricing is the easiest in the analytics stack to understand and the hardest to bound at scale. $5 per TB scanned sounds modest until a single ad-hoc SELECT * query on a large unpartitioned table costs $50 by itself. Levers:

• Columnar formats: Convert raw JSON or CSV to Parquet or ORC. Typical reduction: 70 to 95 percent of bytes scanned.
• Partitioning: Partition by the predicate most queries filter on (typically time). Combined with Parquet, 99 percent reductions are routine.
• Filter pushdown: Encourage SELECT specific columns rather than SELECT *. Encourage WHERE on partition columns first.
• Workgroups with cost cap: Set a per-query and per-workgroup cap so a runaway query cannot bill four-figure dollars.
• Result reuse: Enable query result reuse for dashboards that re-run the same query thousands of times daily.

See our Athena query cost reduction deep dive for the full optimisation walkthrough.

EMR: transient and Spot or nothing

EMR was designed for transient compute and continues to bill best for that workload. Common cost-driving patterns:

• Always-on persistent clusters for workloads that run two hours per day. Switch to transient or autoscaling.
• On-Demand instance fleets when Spot would deliver the same SLA at 60 to 80 percent discount.
• Intel-only instance types when Graviton2 or Graviton3 instance types would deliver 20 percent better price-performance.
• EMR on EC2 when EMR Serverless or EMR on EKS would deliver lower idle cost.

The framework decision (Spark vs Presto vs Flink) matters less than the deployment model decision (transient vs persistent, EC2 vs Serverless). See the EMR cluster cost strategy piece for the deeper walkthrough.

Redshift: pause, resume, or commit

Redshift has three pricing modes: On-Demand, Reserved Nodes, and Serverless. The right answer depends on workload shape.

The most common waste pattern: a fully-loaded provisioned cluster that runs nights and weekends without any workload. Either pause it or move it to Serverless.

See Redshift pricing negotiation and Redshift Serverless pricing for service-specific deep dives.

Glue: DPUs and job design

Glue jobs bill $0.44 per DPU-hour (Spark) or $0.44 per DPU-hour (Python shell, lower default DPU count). Common cost drivers:

• Default 10 DPU jobs running 5 minutes when 2 DPUs at 20 minutes would deliver the same outcome at half the cost.
• Jobs without bookmark enabled re-processing the entire source every run.
• Crawler runs scheduled too frequently on slow-changing data.
• Worker type G.2X or G.4X selected when G.1X would meet performance needs.

Our Glue job cost optimization piece has the per-job tuning checklist.

Kinesis: shards or on-demand

Kinesis Data Streams has provisioned (shard-hour) and On-Demand (per-PUT) pricing. The crossover point is roughly 300 producers per shard. Levers:

• Right-size shard counts to producer mix; merge underused shards.
• Batch records with the Producer Library to reduce PUT cost.
• Use enhanced fan-out only when consumers actually need dedicated read throughput.
• For sporadic workloads, On-Demand removes the always-on shard tax.

See Kinesis pricing optimization for the full walkthrough.

OpenSearch: tiered storage is the unlock

OpenSearch domains are expensive because the default is "hot tier for everything." UltraWarm and Cold tier exist precisely to break that:

• Move read-mostly indices older than 7 days to UltraWarm; reduce hot data instance count.
• Move compliance-retained indices older than 30 days to Cold; pay storage-only.
• Right-size dedicated master nodes; over-provisioning here is common.
• Switch domain instance generation to current Graviton-based options for better price/performance.

The OpenSearch cost management piece covers tier transitions, ISM policies, and the data-stream pattern.

QuickSight: reader licence economics

QuickSight bills $24/author/month, $5/reader/month with annual commitment (or $0.30 per session up to $5), or capacity pricing. The optimisation rules:

• Switch heavy reader populations to per-session pricing if monthly usage falls below 17 sessions per user.
• Move very large reader populations to capacity pricing once total sessions exceed the break-even threshold.
• Audit author licences quarterly; many environments leave author seats assigned to readers who never author.

The QuickSight licensing strategy piece has the licensing arithmetic.

Kafka via MSK and the alternatives

MSK provisioned is straightforward EC2-style brokers; MSK Serverless removes the broker concept entirely. The decision rule:

• Steady-state Kafka with hundreds of MB/s throughput: provisioned MSK with reserved-style instance commits.
• Bursty workloads with idle periods: MSK Serverless almost always wins.
• Cross-AZ retention costs are non-trivial; tiered storage in MSK now bills lower-cost retention tier separately.

Lake Formation and governance

Lake Formation does not bill directly; it adds governance and policy enforcement to S3-resident data. Cost implications come through:

• Required Glue Catalog access, which can drive Glue read charges in high-fan-out architectures.
• S3 list-and-get patterns when permissions are checked at the table or column level.
• Operational overhead when permission models diverge from existing IAM policies.

See Lake Formation cost guide for the integration cost discussion.

Negotiation hooks across the analytics stack

Analytics services are some of the highest-margin lines in an AWS bill, which means they are also among the most negotiable inside an EDP commitment. Levers that work consistently:

• Reserved Node and Reserved Capacity discounts on Redshift and OpenSearch beyond the published list.
• EMR Serverless and Glue ramp credits for customers adopting new analytics services within an EDP renewal.
• Kinesis On-Demand surcharge waivers for customers committing to a Kinesis throughput baseline.
• QuickSight capacity pricing negotiated below list for large reader populations.
• Lake Formation early-adopter credits for governance roll-outs across multiple business units.
• Free Direct Connect or PrivateLink endpoints when an analytics architecture requires high-bandwidth ingest.

Build the cost model before the commit

The single biggest predictor of an over-committed analytics line in an EDP is signing before the cost model exists. Build it in this order:

1. Catalogue every analytics service in active use with its monthly spend trajectory.
2. Forecast 12 to 36 months at three growth scenarios.
3. Identify the optimisation actions that will land before the commit period starts.
4. Subtract those from the commit baseline.
5. Negotiate the commitment against the optimised baseline, not the as-is baseline.

Customers who commit at the as-is baseline routinely overpay 25 to 50 percent across the analytics line for the duration of the deal.

Implementation roadmap

1. Quarter 1: Athena bytes-scanned audit. Redshift utilisation audit. EMR persistent cluster audit. QuickSight licence audit.
2. Quarter 2: Implement Athena partitioning and columnar migration. Right-size Redshift. Switch EMR to transient or Serverless.
3. Quarter 3: OpenSearch tier migration. Glue DPU right-sizing. Kinesis shard rationalisation.
4. Quarter 4: Build EDP commitment model with optimised baseline. Negotiate analytics line as a bundle.

Typical 12-month outcome: 30 to 50 percent reduction in analytics spend, with no degradation in dashboard latency or batch SLA.

Action checklist

1. Run the analytics service inventory in every account.
2. Tag every analytics resource by owner and workload.
3. Implement the per-service optimisations linked above.
4. Build the 36-month forecast at three growth scenarios.
5. Negotiate the analytics line as a bundle in the next EDP cycle.
6. Contact us for an analytics audit benchmarked against 500+ engagements and $340M+ in documented savings.

For adjacent pillars see our database cost strategy pillar, data transfer cost guide, and serverless cost guide. Together they cover the data side of an enterprise AWS bill end-to-end.

Cost allocation across analytics services

The single largest blocker to optimising analytics spend is not knowing which team consumed which dollar. Tag every analytics resource by:

• Team: who owns the workload.
• Workload: what business process the resource supports.
• Environment: production, staging, dev.
• Cost-class: revenue-generating, compliance, internal analytics.

Cost and Usage Reports filtered by these tags become the input for chargeback. Without tags, optimisation lands at the platform team level, which is far less effective than landing it at the team that actually drives the cost.

The build-versus-buy decision in analytics

Many analytics teams default to building everything on managed AWS services because that is what the cloud architects know. The cost-aware question is whether a managed service tier is the right level of abstraction. Three patterns we see consistently:

• Snowflake on top of AWS: For analytics-heavy enterprises, Snowflake often runs cheaper than Redshift at equivalent performance once concurrency scaling is factored in. The trade-off is data egress to a non-AWS analytics layer.
• Databricks for Spark workloads: For Spark-heavy teams, Databricks often runs cheaper than EMR with better tooling. The trade-off is a separate vendor relationship.
• Open-source on EKS: Trino on EKS, ClickHouse on EKS, or Apache Druid on EKS can deliver lower TCO than the AWS-managed equivalent for teams with the operational capacity.

None of these is universally cheaper. The exercise is worth doing every two to three years for any workload above $500K annually in AWS analytics spend.

Concurrency scaling: useful, often abused

Redshift concurrency scaling and Athena workgroup capacity reservations are designed to handle bursty query patterns without provisioning peak capacity 24/7. The trap: leaving concurrency scaling on with no upper bound, paying for capacity that nobody actively requested. The pattern that works:

1. Set a per-day concurrency scaling cap.
2. Alert when usage approaches 80 percent of the cap.
3. Review monthly to confirm cap is right-sized for actual workload patterns.
4. If concurrency scaling consistently saturates the cap, the right answer is base capacity expansion with a Reserved commit, not unlimited scaling.

Data movement cost across analytics services

Analytics architectures move data between services constantly: Kinesis to S3 to Glue to Redshift, S3 to Athena to QuickSight, MSK to OpenSearch to Lambda. Each hop has cost implications:

• Cross-region transfer: $0.02/GB or more. Co-locate the entire analytics pipeline in one region unless DR demands otherwise.
• Cross-AZ transfer: $0.01/GB or more. Same-AZ placement of producer and consumer matters at high throughput.
• NAT Gateway: $0.045/GB processed plus hourly fee. Use VPC endpoints for AWS services to bypass.
• VPC endpoints: $0.01/hour per endpoint per AZ plus $0.01/GB processed. Cheaper than NAT for high-volume AWS-service traffic.

See the AWS data transfer cost guide for the full breakdown.

Analytics workload retirement

One of the highest-ROI activities in an analytics cost programme is retiring dashboards, datasets, and pipelines that nobody uses. The discovery process:

1. Pull QuickSight usage logs for the last 90 days; identify dashboards with zero views.
2. Identify the upstream Glue jobs, Redshift queries, and S3 datasets feeding those dashboards.
3. Notify the original requester with a 30-day deprecation notice.
4. Retire if no response; archive the pipeline definitions.

Typical enterprise saving: 5 to 15 percent of analytics spend lands on dead pipelines.

Predictive cost modelling for the next 36 months

EDP commitments are typically three years; the analytics line must be modelled across that horizon. The forecast inputs:

• Data volume growth rate per year (historical baseline).
• Query volume growth rate per year (driven by user adoption).
• New service adoption (e.g., Bedrock or SageMaker affecting analytics inputs).
• Architectural changes scheduled in the period (e.g., lakehouse migration, Snowflake adoption).

Build three scenarios - conservative, base, aggressive - and negotiate the commitment around the base scenario. Customers who commit to the aggressive scenario routinely overpay; customers who commit to the conservative scenario routinely face shortfall surcharge.

Vendor leverage during analytics renewals

The single biggest negotiation leverage in an analytics renewal is the credible threat of moving workloads off AWS. The credible threat requires:

• A documented architecture for the alternative (Snowflake, Databricks, GCP BigQuery, Azure Synapse).
• A migration cost estimate that matches the renewal savings ask.
• An implementation timeline that fits within the renewal window.
• Executive sponsorship to follow through if AWS does not concede.

Without all four, the lever is theatrical and AWS sales teams know it. With all four, AWS routinely concedes 20 to 40 percent additional discount on the analytics line.

Governance: the operating model that sustains savings

The biggest analytics cost reductions come from one-time optimisation projects. The biggest sustained savings come from governance. The operating model that holds:

1. A quarterly analytics cost review chaired by the head of data engineering.
2. Per-team chargeback driven by the tagging baseline.
3. A standing optimisation backlog that competes with feature work for engineering capacity.
4. Pre-commit cost models for every new analytics workload above a defined threshold.
5. Annual benchmarking against external advisory data.

Customers who implement this operating model maintain 30 to 50 percent lower analytics spend than peers indefinitely.

Engineering for cost observability

The teams that maintain low analytics spend over years all do the same thing: they build cost observability into the analytics platform itself. The capabilities that matter:

• Per-query cost attribution surfaced in the query result UI.
• Per-dashboard cost attribution surfaced in the dashboard owner view.
• Per-pipeline cost attribution surfaced in the pipeline orchestration tool (Airflow, Step Functions).
• Anomaly alerts wired to the engineering team that owns the cost spike, not to a central FinOps inbox.

This shifts cost optimisation from a quarterly central exercise to a continuous engineering practice. Customers who reach this maturity routinely sustain 30 to 50 percent lower analytics spend than peers indefinitely without dedicated FinOps headcount.

The role of independent advisory

AWS account teams have legitimate analytics expertise but are not buyer-side. Independent advisory complements internal cost owners by providing benchmark data, deal-shape templates, and negotiation muscle that cannot be developed in-house at the same speed. Across 500+ engagements we see the same pattern: analytics renewals run with independent advisory close at a 15 to 30 percent better discount than renewals run by internal teams alone.