Synchronize data from external sources

Prerequisite: Create a partitioned table

By using the new keyword PARTITIONED BY, you will be able to synchronize your external database with Indexima at minimum cost.

You can create a partitioned external table like this :

CODE

CREATE EXTERNAL TABLE nyc_yellow FROM my_source_table IN 'jdbc:...'() PARTITIONED BY (year, month);

Synchronize an external table

By using the SYNCHRONIZE keyword on the desired partition, Indexima will apply all retrieve and all updates from the external source.

You can use it like this :

CODE

SYNCHRONIZE nyc_yellow PARTITION (year = 2016, month = 11);

Automatically synchronize external tables

Only available on Snowflake

Instead of running the SYNCHRONIZE command, Indexima will periodically ask Snowflake what are the last changes. By enabling this feature, Indexima will always be up-to-date with your external database.

For this purpose, you have 3 parameters to play with :

external.synchronize.check.rate : Number of seconds between external table synchronization check.
external.synchronize.check.cron : Cron expression for table synchronization check, more precise than external.synchronize.check.rate.
external.synchronize.check.user : The user who will run the SYNCHRONIZE during the automatic update.