Data masking in Azure Databricks Delta Lake: Innovative Routines International has extended the IRI Voracity platform and the IRI FieldShield data masking tool to include native support for Azure Databricks Delta tables. This enables organisations to directly identify, classify, mask and transform sensitive data in modern lakehouse environments without having to export the data or convert it to other formats.
Delta Lake forms the standard storage layer of Azure Databricks and is the foundation of modern lakehouse architectures. As an open-source technology, Delta Lake extends the Parquet file format with a transaction protocol that enables ACID-compliant transactions, reliable metadata management and high scalability. Thanks to full integration with Apache Spark and Structured Streaming, the same data can be used for both batch and real-time processing. Features such as schema enforcement, versioning (‘Time Travel’) and powerful analytical capabilities ensure consistent and high-performance data processing.
With this new integration, IRI FieldShield can protect sensitive data – including personally identifiable information (PII), protected health information (PHI) and financial information – directly within Delta Lake tables. Processing takes place on data stored in Azure Data Lake Storage and managed via Azure Databricks. Performance, reliability and scalability are maintained even with large volumes of data.
Furthermore, IRI FieldShield supports Databricks’ three-tier Unity Catalog (catalogue, schema and table). This enables sensitive data to be identified and secured centrally across entire lakehouse environments – in both batch and streaming workloads. At the same time, Delta Lake’s governance, integrity and consistency features remain fully intact.
Key benefits of the new integration include:
With this extension, IRI strengthens the protection of cloud-native data platforms and helps organisations comply with regulatory requirements such as the GDPR, HIPAA, PCI DSS and CPRA. At the same time, data analytics, AI and data engineering projects benefit from a secure and high-performance foundation for processing sensitive data in modern lakehouse architectures.
Efficiency meets experience: For more than four decades, our software solutions have been supporting companies in data management and data protection – technologically leading, reliable in productive use and applicable across all industries.
In use since 1978: Numerous well-known companies, service providers, financial institutions and state and federal authorities are among our long-standing customers.
Maximum compatibility: Our software supports both classic mainframe platforms (Fujitsu BS2000/OSD, IBM z/OS, z/VSE, z/Linux) and modern open system environments such as Linux, UNIX derivatives and Windows.