Adding columns to an existing JsonLines-backed Athena schema

If you use AWS Athena with JsonLines-backed files whose schema changes occasionally, you may think that the only way to change the corresponding Presto schema is to drop the table and create it again. Normally this is not a problem, except you have to spend hours reloading all the partitions…

Also, ALTER TABLE doesn’t work with Athena because it fails with “no viable alternative at input”. (sigh)

Not to worry, there is another way! If your Athena instance is upgraded to be backed by AWS Glue Catalog, then simply changing the schema there automatically reflects it on your AWS Athena instance.

Screenshot_20191008_155722
The button you need to press

This is no different from having an AWS Glue Crawler periodically detect the schema of your data store.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s