Adding columns to an existing JsonLines-backed Athena schema

If you use AWS Athena with JsonLines-backed files whose schema changes occasionally, you may think that the only way to change the corresponding Presto schema is to drop the table and create it again. Normally this is not a problem, except you have to spend hours reloading all the partitions…

Also, ALTER TABLE doesn’t work with Athena because it fails with “no viable alternative at input”. (sigh) [Update Early 2020: Now I know this has to do with Athena’s SQL parser! It just means that such a command simply does not exist.]

Not to worry, there is another way! If your Athena instance is upgraded to be backed by AWS Glue Catalog, then simply changing the schema there automatically reflects it on your AWS Athena instance.


The button you need to press

This is no different from having an AWS Glue Crawler periodically detect the schema of your data store.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s