PUT /platform/jobs/jdbc/{jobId}/sync-mode

Overview

Migrate a JDBC job's sync mode (load type) with validation and automatic state management. This endpoint allows changing between different load types while handling the incremental state appropriately.

Load Types

Load Type	Description	Required Fields
`full_load`	Extracts all data every run	None
`incremental`	Extracts only new/modified records based on a column	`incremental_column_name`, `incremental_column_type`
`incremental_with_qualify`	Incremental with deduplication using primary keys	`incremental_column_name`, `incremental_column_type`, `primary_keys`

State Management

The endpoint automatically handles state transitions:

From	To	State Action	Behavior
`full_load`	`incremental`	Initialize	Next run performs full extraction, then tracks incrementally
`full_load`	`incremental_with_qualify`	Initialize	Next run performs full extraction with deduplication
`incremental`	`full_load`	Reset	Clears incremental state
`incremental`	`incremental_with_qualify`	Preserve	Keeps existing incremental bookmark
`incremental_with_qualify`	`incremental`	Preserve	Keeps existing incremental bookmark
`incremental_with_qualify`	`full_load`	Reset	Clears incremental state

Request Body

Field	Type	Required	Description
`target_load_type`	string	Yes	Target load type: `full_load`, `incremental`, or `incremental_with_qualify`
`incremental_column_name`	string	Conditional	Column used for incremental tracking. Required for incremental types.
`incremental_column_type`	string	Conditional	Data type of the incremental column. Required for incremental types.
`primary_keys`	array	Conditional	List of primary key columns. Required for `incremental_with_qualify`.
`force`	boolean	No	Force migration even with warnings (default: `false`)

Allowed Incremental Column Types

Each database has specific allowed data types for incremental columns:

PostgreSQL

timestamp with time zone
timestamp without time zone
date
integer
bigint
smallint
serial
bigserial

MySQL

datetime
timestamp
date
int
bigint
smallint
mediumint
tinyint

Oracle

TIMESTAMP
TIMESTAMP WITH TIME ZONE
TIMESTAMP WITH LOCAL TIME ZONE
DATE
NUMBER

SQL Server

datetime
datetime2
smalldatetime
date
int
bigint
smallint
tinyint

Job ID Format

The jobId follows the format: {pipeline_id}_{job_index}

Example: 1b33ad2f_33d3_4837_9eeb_83c82c8b909d_0 (first job in pipeline 1b33ad2f_33d3_4837_9eeb_83c82c8b909d)

Examples

Migrate from full_load to incremental

PUT /platform/jobs/jdbc/1b33ad2f_33d3_4837_9eeb_83c82c8b909d_0/sync-mode

{
  "target_load_type": "incremental",
  "incremental_column_name": "updated_at",
  "incremental_column_type": "timestamp with time zone"
}

Response (200 OK):

{
  "status": "success",
  "message": "Successfully migrated sync mode to incremental",
  "migration_details": {
    "target_load_type": "incremental",
    "state_action": "initialize",
    "warnings": []
  }
}

Migrate from incremental to incremental_with_qualify

PUT /platform/jobs/jdbc/1b33ad2f_33d3_4837_9eeb_83c82c8b909d_0/sync-mode

{
  "target_load_type": "incremental_with_qualify",
  "incremental_column_name": "updated_at",
  "incremental_column_type": "timestamp with time zone",
  "primary_keys": ["id"]
}

Response (200 OK):

{
  "status": "success",
  "message": "Successfully migrated sync mode to incremental_with_qualify",
  "migration_details": {
    "target_load_type": "incremental_with_qualify",
    "state_action": "preserve",
    "warnings": []
  }
}

Migrate to full_load

PUT /platform/jobs/jdbc/1b33ad2f_33d3_4837_9eeb_83c82c8b909d_0/sync-mode

{
  "target_load_type": "full_load"
}

Response (200 OK):

{
  "status": "success",
  "message": "Successfully migrated sync mode to full_load",
  "migration_details": {
    "target_load_type": "full_load",
    "state_action": "reset",
    "warnings": []
  }
}

Migrate with multiple primary keys

PUT /platform/jobs/jdbc/a9f82c14_7e21_4b5a_8c9d_12345abcdef0_0/sync-mode

{
  "target_load_type": "incremental_with_qualify",
  "incremental_column_name": "modified_at",
  "incremental_column_type": "datetime",
  "primary_keys": ["order_id", "product_id"]
}

Response (200 OK):

{
  "status": "success",
  "message": "Successfully migrated sync mode to incremental_with_qualify",
  "migration_details": {
    "target_load_type": "incremental_with_qualify",
    "state_action": "initialize",
    "warnings": []
  }
}

Error Responses

400 Bad Request - Validation Error

Missing required fields for incremental load type:

{
  "detail": {
    "error": "ValidationError",
    "message": "Migration validation failed",
    "validation_errors": [
      {
        "field": "incremental_column_name",
        "message": "Incremental column name is required for incremental load types",
        "error_code": "MISSING_INCREMENTAL_COLUMN_NAME"
      },
      {
        "field": "incremental_column_type",
        "message": "Incremental column type is required for incremental load types",
        "error_code": "MISSING_INCREMENTAL_COLUMN_TYPE"
      }
    ],
    "required_fields": ["incremental_column_name", "incremental_column_type"]
  }
}

400 Bad Request - Invalid Column Type

{
  "detail": {
    "error": "ValidationError",
    "message": "Migration validation failed",
    "validation_errors": [
      {
        "field": "incremental_column_type",
        "message": "Invalid incremental column type 'varchar' for plugin 'postgresql'. Must be one of: ['timestamp with time zone', 'timestamp without time zone', 'date', 'integer', 'bigint', 'smallint', 'serial', 'bigserial']",
        "error_code": "INVALID_INCREMENTAL_COLUMN_TYPE"
      }
    ],
    "required_fields": []
  }
}

400 Bad Request - Missing Primary Keys

{
  "detail": {
    "error": "ValidationError",
    "message": "Migration validation failed",
    "validation_errors": [
      {
        "field": "primary_keys",
        "message": "Primary keys are required for incremental_with_qualify load type",
        "error_code": "MISSING_PRIMARY_KEYS"
      }
    ],
    "required_fields": ["primary_keys"]
  }
}

400 Bad Request - Not a JDBC Job

{
  "detail": {
    "error": "ValidationError",
    "message": "Migration validation failed",
    "validation_errors": [
      {
        "field": "plugin",
        "message": "This job is not a JDBC connector job",
        "error_code": "INVALID_CONNECTOR_TYPE"
      }
    ],
    "required_fields": []
  }
}

404 Not Found

{
  "detail": "Job with id invalid_job_id not found"
}

Notes

When migrating to incremental_with_qualify, the pipeline DAG is automatically updated to include the deduplication task
When migrating from incremental_with_qualify to other types, the DAG is updated to remove the deduplication task
The force parameter can be used to proceed with migration even when warnings are present
After migration, the next pipeline execution will use the new sync mode configuration