[EN] Objects

About

After the connection test, you will be able to view and select which objects - tables and views - from the connected source will be imported in the data collection and the columns that will be imported. Additionally, you can define the synchronization mode for each entity.

📘

What are objects?

Objects are representations of a set of information, following the pattern of how your data is organized in your database.

Select the Objects

For this, select the chosen entity(s) in the checkbox and click on "Select Columns". To import all objects from the connected source, select the first checkbox.

📘

When selecting more than one entity, a different dataset will be created in the catalog for each imported table or view.

🚧

When the collection is set as incremental and does not enable the reference field, it is because there is no column with a compatible data type in that table. In these cases, it is necessary to define it as full load or exclude it from the collection.

Select the Columns

Select the columns of each entity that will be collected and then choose the synchronization mode.

📘

We recommend that unnecessary columns for generating analyses be removed so that the collection is smaller and occurs in less time.

Additionally, do not extract columns that have unsupported data types at Dadosfera, as the execution will fail.

Synchronization Mode

Full Load

Extracts all data in each collection.

Incremental (Append)

Only new records added to the database will be collected. It is also necessary to define a column that will be the reference field for carrying out the increment of the collection based on the sequential order of this field, consult the page of each source for the types of data accepted to be reference fields in incremental collection.

Limitations:

• If records are updated in the source, they will not change in Dadosfera. The incremental collection will only bring new records.
• Records deleted in the source will not be deleted in Dadosfera.

📘

Commonly accepted types for Incremental (Append) load

• Dates
• Integer Numbers
• Timestamps

More details in the documentation of each connector.

📘

We recommend preferring incremental append synchronization, as:

• Access to the database is less intrusive, greatly reducing the possibility of causing momentary instability in the database.
• The amount of data extracted in each collection tends to be considerably smaller, therefore the quantity of events used will likely remain within the limit of the contracted plan.

Only in case of a need to update old data, we recommend carrying out the full load synchronization.

What is considered an event in Dadosfera?