Data Mappings

Mapping Configuration in Daana

Once you've pressed the "Configure Mapping" button, you are taken to the mapping configuration screen. Here, you will define how the data from your source systems maps to your information model in Daana. This screen offers a clear view of all entities, allowing you to manage mappings at both the table and attribute levels.

1. Overview of the Mapping Screen

The main screen shows your entities in a collapsed view. From here, you can expand each entity and configure how its data should be mapped. At the bottom, you will find four important buttons:

• Export: Save your mappings to a JSON file.

• Import: Load a previously saved or modified JSON structure. Daana will verify the integrity of the structure before applying changes.

• Reset: Revert the mapping screen back to the last saved configuration.

• Save: Save your current configuration to preserve your mappings.

2. Mapping Groups

When you expand an entity, you will see different sections, starting with Mapping Groups. A mapping group allows you to handle multiple systems that don’t overlap in instances (an instance is a representation of a unique identification of a data object) but contribute to the same entity. For example, if you run a retail business with two Order systems, one for online store and one for the physical store. The same order (instance) will never exist in both systems but we want to map the respective order system against the same entity - ORDER. This way we can then represent and store the data from a business perspective, instead of a source system perspective. The solution is Subject Oriented and Integrated.

3. Tables

Under each mapping group, you'll define the tables that contribute to this entity’s data. For example, a ORDER entity may have information coming from different tables: the ORDER DATE might come from one table, while the ORDER CATEGORY might be stored in another table in the source system.

It might even be that the data for certain attributes comes from a completly different system. As long as we can define the identity in each source and that they are overlapping in values - Lets say that a Order system sends certain data to a Shipping system with its Order Number and the ORDER entity that we modeled has information (attributes) about the Order that only exists in the Shipping System (ORDER_SHIPPING_DATE, ORDER_DELIVERY_DATE).

Daana can add multiple different "tables" to one mapping group. The important part here is that each table needs to be identified by the same values, that the Order Number is the same in both the Order System as well as the Shipping System. The name of the column that holds the value for the Order Number does not need to be the same in the different tables. Daana will then match and integrate the data on the identity of the instances of data.

Mapping groups are used when integrating multiple sources against the same entity, but where the sources do not contain the same instances.

Multiple tables in the same mapping group is used to integrate different sources to the same entity but containing the same instances. Used where specific sources are loading specific attributes for the same instance.

You can only have one source loading a specific attribute for a specific instance.

Example: The same Customer can't be loaded with First Name from two different source tables, since they will overwrite each other

Filter Symbol

• Clicking this icon allows you to configure additional settings for your tables, like the Ingestion Strategy or WHERE clause. The ingestion strategy defines how data from this table should be ingested into Daana (e.g., Full, Incremental, etc.). The WHERE clause allows you to filter the rows that are ingested.

4. Allow Multiple Identifiers

If an entity can be identified by multiple keys (for example, a customer might be identified by both a customer_id and an organization_number), you can enable the Allow multiple identifiers option. Daana will then manage surrogate keys to ensure the data stays consistent, even when different keys are used.

5. Attributes

Each entity has a list of attributes. In the Attributes section, you’ll map each attribute to its corresponding column in your source tables. This is where the real magic happens—defining how the raw data aligns with the structured information model.

• Source Expression: In many cases, this will be a direct column-to-attribute mapping. However, you can also use SQL expressions (like CASE statements) for more complex transformations.

• Filter Symbol: Here, you can override the Ingestion Strategy or the WHERE clause at the attribute level, providing more granular control over the data flow.

6. Ingestion Strategy and WHERE Clause at Attribute Level

In cases where a particular attribute needs a different ingestion strategy or filtering than the table level, you can override these settings. For example, you might want a specific attribute to use an incremental ingestion strategy while the rest of the table uses a full snapshot.

7. Effective Datetime

You can specify an Effective Datetime. This allows you to decide how Daana should track when changes to the data occurred. You can select a column from the table where you source the data for the specific attribute. The column that you choose need to contain a Date/Timestamp information, you can also, if needed, define a source expression if the column(s) needs some transformation to represent the Effective Datetime. If you do not specify an Effective Datetime, then Daana will use the timestamp of when the execution of the pipeline for the specific attribute started as Effective Datetime.

---

By configuring mappings in Daana, you create a clear, automated flow of data from raw sources to well-structured business information. This mapping process ensures that the data you care about is always ready and up-to-date, without the need for manual intervention.