Welcome to MortalMesh!
What is MortalMesh?
MortalMesh is an open-source project that enables a lakehouse architecture on top of data lakes.
Main Features
Specifically, MortalMesh offers:
ACID transactions on Spark: Serializable isolation levels ensure that readers never see inconsistent data.
Scalable metadata handling: Leverages Spark distributed processing power to handle all the metadata for petabyte-scale tables with billions of files at ease.
Streaming and batch unification: A table in MortalMesh is a batch table as well as a streaming source and sink. Streaming data ingest, batch historic backfill, and interactive queries all just work out of the box.
Schema enforcement: Automatically handles schema variations to prevent insertion of bad records during ingestion.
Time Travel: Data versioning enables rollbacks, full historical audit trails, and reproducible machine-learning experiments.
Upsets and Deletes: Supports merge, update and delete operations to enable complex use cases like change-data-capture, slowly-changing-dimension (SCD) operations, streaming upserts, and so on.
Last updated
