Data sources are often evolving with the introduction of new features and retiring old ones. This causes the schema (i.e. data fields) to change over time, so ensuring that your data pipelines are kept up to date Jepto automatically updates your pipelines to take advantage of these new fields.

These changes can happen quite often and are not a distraction that you should be concerned with, which is why Jepto will always conduct the maintenance behind the scenes. This article just highlights what happens when this occurs.

At Jepto, we automatically make schema changes from the source to your data warehouse and the update depends on the situation:

New Field

If your source adds a new field, we will detect the change and add the field in your warehouse, backfilling where applicable.

Removed Field

If a field is removed, your existing data is maintained and it's ignored in all future syncs so that the old data remains accessible.

Datatype Changed

If a fields datatype has changed, we attempt to accommodate both the old and new data by retaining the old field and creating a new field with descriptive names.

New Table

If your source adds a new table, we will determine if we can add it into an existing table (de-normalized structure) or add it as a new table.

Deleted Table

If a table is deleted in your source, no changes will be made to your Data Warehouse and it's ignored on all future syncs.