Snowflake

Luden supports Snowflake as a destination. For more information about Snowflake see docs. Snowflake destinations can work in stream and batch modes. In stream mode Luden uses plain insert statements for storing data. In batch mode Luden writes incoming events in formatted file on the AWS S3 or Google Cloud Storage (intermediate Stage layer) and uses COPY command to store data from Stage files into Snowflake.

Configuration

Snowflake destination in batch mode can be configured via S3 or Google Cloud Storage. In the stream mode, it can be configured without any. The config consists of the following schema:

destinations:
  my_snowflake_via_aws_s3_or_gcp:
    type: snowflake
    snowflake:
      account: hha56552.us-east-1
      schema: MYSCHEMA
      warehouse: my_warehouse
      db: mydb
      username: user
      password: pass
      parameters:
        name: value
      stage: my_stage
# via s3
    s3:
      access_key_id: ...
      secret_access_key: ...
      bucket: ...
      region: ...
# or via gcp
    google:
      gcs_bucket: ...
      key_file: ...