Naming Conventions
This document outlines the standardized naming conventions for Microsoft Fabric resources across our data engineering ecosystem. Following these conventions ensures consistency, clarity, and ease of navigation across our Medallion Architecture implementation and Deployment process.
Workspaces serve as the primary containers for our data assets. We maintain three distinct workspace types:
Workspace Type | Naming Pattern | Example | Purpose |
---|---|---|---|
Source | src_[source_name]_[deployment stage] | src_dalux_dev | Contains raw ingested data from a single source. Only data available from the source |
Project | prj_[project name]_[deployment stage] | prj_mor_dev | Houses project-specific data transformations |
BI | bi_[application name]_[deployment stage] | bi_project_delivery_dev | Dedicated to semantic models and semantic models |
Lakehouses represent our primary storage and processing layer within each workspace:
Layer | Naming Pattern | Location | Example | Purpose |
---|---|---|---|---|
Bronze and Silver | src_lh_[source_name] | Source Workspace | src_lh_bimcollab | Raw data from source systems |
Gold | gold_lh_[project_name] | Project Workspace | gold_lh_permits | Business-ready, project specific logics, project specific data |
Schemas help organize tables within lakehouses based on their processing stage and data domain. Our schema strategy reflects both the medallion architecture layers and the source origins of data.
Schema Type | Naming Pattern | Location | Example | Purpose |
---|---|---|---|---|
Bronze | bronze | Source Lakehouse | bronze | Contains raw, unmodified data from source systems |
Silver | silver | Source Lakehouse | silver | Contains cleaned, validated, and standardized data |
Source Shortcut | src_[source_name] | Project Lakehouse | src_dalux | Contains shortcuts to silver tables from specific sources |
Gold | gold | Project Lakehouse | gold | Contains project-specific business logic and transformations |
Utility | util | Any Lakehouse | util | Contains utility tables, metadata, and audit information |
The schema structure helps maintain clear boundaries between medallion layers while providing context on data origins. For project lakehouses, the source-specific shortcut schemas (src_[source_name]) make it easy to identify which source a particular table originated from, especially important when a project combines data from multiple sources.
Tables follow consistent patterns based on their processing stage and business purpose:
Layer | Table Type | Naming Pattern | Example | Description |
---|---|---|---|---|
Bronze | Source | raw_[entity] | raw_projects | Unmodified source data |
Silver | Cleansed | cl_[entity] | cl_projects | Cleaned, typed, and validated data |
Silver | Conformed | cf_[entity] | cf_projects | Standardized business entities |
Gold | Dimension | dim_[dimension] | dim_project | Dimensional model dimension tables |
Gold | Fact | fact_[business process] | fact_project_changes | Dimensional model fact tables |
Gold | Aggregate | agg_[measure]_[grain] | agg_project_status_monthly | Pre-aggregated metrics |
Any | Reference | ref_[entity] | ref_status_codes | Reference or lookup data |
Any | Temporary | tmp_[entity]_[purpose] | tmp_projects_dedupe | Temporary processing tables |
Shortcut tables should maintain the same name as their source tables to preserve clarity and context.
Key Columns
Column Type | Naming Pattern | Example | Purpose |
---|---|---|---|
Primary Key | [entity]_id | project_id | Natural business key |
Surrogate Key | [entity]_key | project_key | Generated surrogate key |
Foreign Key | [referenced entity]_id | customer_id | Reference to another entity |
Foreign Surrogate Key | [referenced entity]_key | customer_key | Reference to surrogate key |
Common Data Elements
Column Type | Naming Pattern | Examples |
---|---|---|
Timestamps | [event]_timestamp | created_timestamp, modified_timestamp |
Dates | [event]_date | start_date, due_date, birth_date |
Flags | is_[state] | is_active, is_deleted, is_primary |
Amounts | [type]_amount | invoice_amount, tax_amount |
Counts | [item]_count | record_count, error_count |
Status | [entity]_status | project_status, order_status |
Technical Columns
Column Type | Naming Pattern | Examples |
---|---|---|
Partition Columns | partition_[type] | partition_date, partition_region |
Audit Columns | etl_[event]_[detail] | etl_insert_timestamp, etl_source_system |
Version Control | record_version | Records the version number of the record |
Deleted Flag | is_deleted | Soft delete indicator |
Validity Period | valid_from_date, valid_to_date | For SCD Type 2 dimensions |
JSON/Complex Types
Column Type | Naming Pattern | Examples |
---|---|---|
Raw JSON | [entity]_json | event_json, payload_json |
Struct Types | [domain]_details | address_details, contact_details |
Array Types | [item]_array | product_array, tag_array |
Deployment Pipeline Naming Conventions
(Add your pipeline naming conventions here, e.g., pl_[project]_[purpose]_[stage] etc.)
This is a living document. Update your naming conventions as your data platform evolves.