Obiz Solutions

What is AWS Athena?

Athena is an AWS service that lets you run SQL directly against files already sitting in S3 (CSV, JSON, Parquet, plain-text logs…) without standing up a database or ETL-ing the data anywhere first. It’s serverless “schema-on-read”: you declare a table’s structure (DDL) pointing at an S3 folder, and Athena reads the files against that schema at query time — it doesn’t store any data of its own.

How it works

  1. The Glue Data Catalog stores table definitions (column names, types, S3 location) — like a database’s metadata, but with no storage engine attached
  2. When a query runs, Athena reads the files in S3 directly at that moment, processed by the Presto/Trino engine underneath
  3. Results come back; nothing gets “inserted” or persisted — the next query reads whatever’s currently in S3

Creating a table for CloudFront logs

An example matching the CloudFront logs set up in the Setup and deploy guide (33 fields, plain-text, tab-delimited):

CREATE EXTERNAL TABLE cloudfront_logs (
  date STRING, time STRING, x_edge_location STRING, sc_bytes BIGINT,
  c_ip STRING, cs_method STRING, cs_host STRING, cs_uri_stem STRING,
  sc_status STRING, cs_referer STRING, cs_user_agent STRING,
  cs_uri_query STRING, cs_cookie STRING, x_edge_result_type STRING,
  x_edge_request_id STRING, x_host_header STRING, cs_protocol STRING,
  cs_bytes BIGINT, time_taken FLOAT, x_forwarded_for STRING,
  ssl_protocol STRING, ssl_cipher STRING,
  x_edge_response_result_type STRING, cs_protocol_version STRING,
  fle_status STRING, fle_encrypted_fields STRING, c_port INT,
  time_to_first_byte FLOAT, x_edge_detailed_result_type STRING,
  sc_content_type STRING, sc_content_len BIGINT, sc_range_start STRING,
  sc_range_end STRING
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
LOCATION 's3://YOUR_BUCKET-logs/cf-logs/'
TBLPROPERTIES ('skip.header.line.count'='2');

LOCATION points at the whole folder — Athena automatically reads every log file CloudFront writes there, including ones added later, with no need to recreate the table.

Example queries

-- Top 20 most-viewed pages
SELECT cs_uri_stem, COUNT(*) AS views
FROM cloudfront_logs
WHERE cs_method = 'GET' AND sc_status = '200'
GROUP BY cs_uri_stem
ORDER BY views DESC
LIMIT 20;

-- Traffic by day
SELECT date, COUNT(*) AS requests
FROM cloudfront_logs
GROUP BY date
ORDER BY date;

Cost

Athena charges by how much data a query scans ($5/TB), not by the hour like a database server. For a personal blog’s logs (usually a few MB to a few dozen MB), each query costs less than a cent. No querying means no charge at all — very different from paying for a database/server to run 24/7.

vs. GoAccess

GoAccess and Athena solve the same problem (reading CloudFront logs) in two different ways:

  • GoAccess: download the log files, run one command, get an HTML dashboard right away — fast and visual, but you have to re-download logs every time you want it refreshed
  • Athena: query SQL directly against S3, nothing to download, flexible enough to answer arbitrary questions (hourly traffic, filter by status code…) — but requires knowing SQL, and there’s no built-in visual dashboard (you’d build one or add QuickSight)

When to reach for Athena

  • Answering a custom question a pre-built dashboard doesn’t cover (e.g. filtering by a specific user-agent, breaking traffic down by hour)
  • Not wanting to download logs locally, preferring to query S3 directly
  • Already comfortable with SQL — no new tool to learn

Further reading

Official docs: docs.aws.amazon.com/athena