High Level Design

System Context

Here we have the main actors of the solution:

TODO: insert a diagram

Main Components:

• API: This is the main pillar of Darkseal. Here we have defined how we can interact with the metadata Entities. It powers all the other components of the section.

• UI: Discovery-focused tool that helps users keep track of all the data assets in the organization. Its goal is to enable and fuel collaboration.

• Ingestion Framework: Based on the API specifications, this system is the foundation of all the Connectors, i.e., the components that define the interaction between Darkseal and external systems containing the metadata we want to integrate.

• Entity Store: MySQL storage that contains real-time information on the state of all the Entities and their Relationships.

• Search Engine: Powered by Elasticsearch/OpenElasticsearch, it is the indexing system for the UI to help users discover the metadata.