One reliable foundation for all your data
Your sources come together in one lakehouse: a data lake for raw source data and a data warehouse for analysis-ready data. The platform checks your data on every load. On top of that, you define owners, definitions and quality rules. That way you know who is responsible for what, where your data comes from and the state of your data quality.
One source of truth
All your sources come together in one lakehouse, from raw source data to analysis-ready data. You do not have to manage a separate data lake or data warehouse environment.
Data that adds up
You define quality rules according to the data quality dimensions from DAMA-DMBOK, such as completeness, validity and uniqueness. The platform checks your data on every load. Duplicate and incorrect records stand out, and you fix an error and start again.
Control and provenance
You define owners, definitions and responsibilities. With data lineage you see where data comes from and how it changes.
What you connect
You connect sources to the platform. Also see all five steps of the platform.
The foundation
From source to catalogue
Retrieve
The platform retrieves your sources, at a fixed time or as soon as a file arrives.
Store
The data goes into the lakehouse.
Check
On every load, the platform checks the data against your quality rules. You see an error straight away.
Process
Transformations turn it into analysis-ready data.
Describe
The mapper turns tables and columns into assets in the catalogue. It fills in empty fields and leaves completed fields alone. A data steward completes the rest using the questionnaire.
Use
Dashboards, solutions and APIs work with the same data, and everyone can see where it comes from.
What you do with your data
Dashboards
For management steering and for teams’ day-to-day work.
Solutions
Workflow applications in which staff carry out their tasks, on the same data.
APIs
Other systems use the same up-to-date figures, without anyone retyping anything.
Export
Export to Excel or CSV, set up per solution.
The rules
What you manage
Catalogue
All datasets and tables in your organisation, with filters and a search function. For each asset you see the attributes, the origin and the quality.
Glossary
Definitions that everyone in your organisation uses.
Picklists
Fixed values with readable display names.
Storage locations
You decide which assets are visible in the catalogue.
Quality rules
With statements and assignments, and reporting on quality and use.
Automatic or with approval
Changes go through automatically, or you first set them up as a change request. A colleague approves or rejects the request before it goes ahead. This is how you apply the four-eyes principle to the changes you choose.
Step 1
A data custodian completes an asset’s details in a form.
Step 2
For a change, you create a change request.
Step 3
A reviewer approves or rejects the request. You choose who is allowed to.
Step 4
The approved change appears in the catalogue.
Access and logging
You manage access per solution, per page and per component. For each user and each role you decide who sees what and what they may do with it.
An audit trail records who did what. This supports the requirements of the GDPR.
Who it is for
Who works with it
For data managers, information managers and architects at municipalities, provinces and housing associations.
Core group
Monitors progress per programme, with a data custodian in the lead role, and reports to management.
Data steward or data owner
Describes assets, completes them and reviews changes.
BI or data engineer
Connects sources and builds the pipelines and transformations.
Data architect
Sets up the lakehouse and determines how data is stored.
Geodata administrator
Manages map data and geo-databases.
Functional administrator
Sets up the environment, with roles and permissions.
Customer cases
What is the impact on your organisation?
You work from one source instead of separate systems. You see what data you have, who looks after it and whether it is correct. You know which sources cover your information needs. And you talk about the figures and information you publish with more confidence.
Frequently asked questions
1
What does Data management and governance do?
Your sources come together in one lakehouse and the platform checks the data at every import. On top of that, you record owners, definitions, quality rules and lineage.
2
What is a lakehouse?
A data lake and a data warehouse in one. Raw source data and analysis-ready data sit in one place. At Clappform, the lakehouse is part of the platform itself.
3
How do I see duplicate or incorrect records?
The platform checks your data at every load, according to your quality rules. You see an error, fix it and restart.
4
How do I see where data comes from?
With data lineage you see where data comes from and how it changes.
5
Does the platform follow DAMA?
The quality rules follow the data quality dimensions from DAMA-DMBOK.
6
What do data owner, data steward and data lineage mean?
A data owner is responsible for a dataset or domain, and decides on use and access. A data steward looks after the description and quality of data on behalf of the owner. Data lineage is the origin of data, from source system to dashboard, with the processing steps along the way.
7
Where is my data stored?
On Azure or Scaleway. You choose where your data is stored: within the EU or in the Netherlands.
8
Does this support the GDPR?
Yes. Clappform applies the data minimisation principle and therefore complies with the GDPR.
9
Who is this solution made for?
The solution was originally made for the Province of North Brabant, which uses it to bring the metadata of its data assets together in one place. Read the customer case for the story.