GCS Migration Guide

Upgrading to 1.0.0

We have revamped the implementation to use the File-Based CDK. The goal is to increase resiliency and reduce development time. Here are the breaking changes:

• [CSV] Mapping of type array and object: before, they were mapped as large_string and hence casted as strings. Given the new changes, if array or object is specified, the value will be casted as array and object respectively.
• [CSV] decimal_point option is deprecated: It is not possible anymore to use another character than . to separate the integer part from non-integer part. Given that the float is format with another character than this, it will be considered as a string.

Given that you are not affected by the above, your migration should proceed automatically once you run a sync with the new connector. To leverage this:

• Upgrade source-gcs to use v1.0.0
• Run at least one sync for all your source-gcs connectors
  • Migration will be performed and an AirbyteControlMessage will be emitted to the platform so that the migrated config is persisted

If a user tries to modify the config after source-gcs is upgraded to v1.0.0 and before there was a sync or a periodic discover check, they will have to update the already provided fields manually. To avoid this, a sync can be executed on any of the connections for this source.

Other than breaking changes, we have changed the UI from which the user configures the source:

• You can now configure multiple streams by clicking on Add under Streams.
• Output Stream Name has been renamed to Name when configuring a specific stream.
• Pattern of files to replicate field has been renamed Globs under the stream configuration.