Redshift Spectrum + Lake Formation — Enterprise Governance on NAS Data

TL;DR

In , Databricks hit boundaries. In , DuckDB Lambda was cheapest. In

How to Read This Article

This article is:

• A reproduction-focused validation report


• Evidence from one environment (Redshift Serverless 8 RPU, ap-northeast-1)


• A governance architecture guide for Lake Formation + FSx S3 AP

Read by role:

• 
  DWH engineer: Architecture → Setup → Benchmark Results


• 
  Security / governance reviewer: 4-Layer Authorization → Governance Impact


• 
  Data engineer: When to Use → Comparison with Athena


• 
  Partner / SA: Partner Decision Card → Discovery Questions

Prerequisite Concepts

Before reading this article, it helps to understand:

• 
  Redshift Spectrum — Redshift's ability to query data in S3 via external schemas (Glue Catalog)


• 
  Redshift Serverless — pay-per-query Redshift without cluster management (measured in RPU)


• 
  Lake Formation — AWS's centralized governance layer for data lakes (table/column/tag permissions)


• 
  Glue Catalog — AWS's metadata catalog (shared by Athena, Redshift Spectrum, EMR, Glue)


• 
  External Schema — a Redshift schema that maps to a Glue Catalog database

Architecture

┌─────────────────────────────────────────────────────────────────┐

│  Redshift Serverless (8 RPU)                                     │

│                                                                  │

│  ┌──────────────────────────────────────────────────────────┐   │

│  │  SQL Query                                                │   │

│  │  SELECT * FROM fsxn_spectrum.sensor_readings              │   │

│  │         JOIN local_table ON ...                            │   │

│  └──────────────────────────────────────────────────────────┘   │

│                          │                                       │

│              External Schema (Glue Catalog)                       │

└──────────────────────────┼───────────────────────────────────────┘

                           │

              ┌────────────┼────────────┐

              │            │            │

    Lake Formation    IAM Role    S3 Access Point

    (table/column     (API        (resource

     permissions)      access)     policy)

              │            │            │

              └────────────┼────────────┘

                           │

                           ▼

              FSx for ONTAP Volume (Parquet files)

4-Layer Authorization:

1. 
   Lake Formation — Who can access which tables/columns (fine-grained)


2. 
   IAM — Who can call which AWS APIs


3. 
   S3 Access Point Policy — Which principals can access this access point


4. 
   File System — UNIX permissions on the underlying files

Benchmark Results

Environment: Redshift Serverless 8 RPU, ap-northeast-1. FSx for ONTAP Single-AZ, 128 MB/s.

Performance note: Redshift Serverless has cold start overhead (~2-3s for first query). Warm queries on provisioned Redshift clusters would be faster. For simple scans, Athena is ~2x faster because it has no DWH initialization overhead.

Evidence Matrix

Setup

Step 1: Create External Schema (reuses Glue Catalog)

CREATE EXTERNAL SCHEMA fsxn_spectrum

FROM DATA CATALOG

DATABASE 'fsxn_athena_verification'

IAM_ROLE 'arn:aws:iam::<ACCOUNT_ID>:role/fsxn-redshift-spectrum-role'

REGION 'ap-northeast-1';

Key insight: This uses the same Glue Catalog database that Athena uses. No additional table registration needed — if Athena can query it, Redshift Spectrum can too.

Step 2: Query FSx for ONTAP Data

-- Simple count

SELECT COUNT(*) FROM fsxn_spectrum.sensor_readings;

-- Result: 10000 (3,231ms)



-- Aggregation

SELECT status, COUNT(*), AVG(temperature)

FROM fsxn_spectrum.sensor_readings

GROUP BY status;

-- Result: 3 groups (2,580ms)



-- JOIN with local Redshift table (DWH capability)

SELECT s.device_id, s.temperature, d.location

FROM fsxn_spectrum.sensor_readings s

JOIN device_master d ON s.device_id = d.device_id

WHERE s.temperature > 35;

Step 3: Add Lake Formation Governance

# Set Lake Formation admin

aws lakeformation put-data-lake-settings \

  --data-lake-settings '{"DataLakeAdmins": [{"DataLakePrincipalIdentifier": "arn:aws:iam::<ACCOUNT_ID>:user/<admin>"}]}'



# Grant table-level SELECT to a role

aws lakeformation grant-permissions \

  --principal '{"DataLakePrincipalIdentifier": "arn:aws:iam::<ACCOUNT_ID>:role/fsxn-analyst-role"}' \

  --resource '{"Table": {"DatabaseName": "fsxn_athena_verification", "Name": "sensor_readings"}}' \

  --permissions '["SELECT", "DESCRIBE"]'

References

• GitHub: fsxn-lakehouse-integrations

Key achievement: This validation established that Redshift Spectrum + Lake Formation provides the strongest enterprise governance path for FSx for ONTAP S3 AP data — 4-layer authorization (Lake Formation → IAM → S3 AP → File System), table/column-level access control, and seamless sharing of Glue Catalog with Athena. The same governance configuration applies to both Athena and Redshift Spectrum queries, enabling a unified governance model across query engines.

All benchmarks are from a specific test environment (Redshift Serverless 8 RPU, FSx for ONTAP Single-AZ 128 MB/s, ap-northeast-1). Performance improves with warm queries and provisioned clusters.

Disclaimer: This article is an independent validation report and does not represent AWS or NetApp official guidance. Product behavior and platform capabilities may change. Always validate in your own environment.