Senior Databricks Developer

Job Description: This role leads the design, development, and optimization of large-scale data solutions using Azure Databricks, Apache Spark, and Delta Lake. The Senior Databricks Developer brings strong technical expertise and architectural thinking to build scalable and reliable cloud-based data engineering solutions. The position supports enterprise data platforms and works closely with engineering teams to integrate data services with application components when required. Responsibilities: rchitect, design, and build large-scale ETL and ELT data pipelines using Databricks with PySpark and Spark SQL. Optimize Spark code, cluster usage, and Delta Lake pipelines to improve performance and efficiency. Implement Delta Lake capabilities including transactional integrity, schema evolution, and data versioning. Lead the migration of legacy or on-premise data workloads to Azure Databricks. Design data models, ensure data integrity, and support governance using catalog and metadata frameworks. Build and orchestrate data workflows using orchestration tools and scheduling frameworks. Integrate Databricks solutions with cloud data storage, messaging, analytics, and security services. Conduct code reviews, mentor junior engineers, and enforce engineering standards and best practices. Troubleshoot and resolve production issues across Spark jobs, workflows, and pipelines. Collaborate with software engineering teams and contribute to service integration when needed. Skills: Strong hands-on expertise with PySpark, Spark SQL, Apache Spark internals, and performance tuning. Deep understanding of Delta Lake, data versioning, and lakehouse architecture. dvanced SQL skills including complex queries, analytical functions, and optimization techniques. Experience integrating Databricks with cloud orchestration, storage, analytics, and messaging services. Hands-on experience with CI and CD pipelines and source control tools. Strong understanding of data modeling, distributed computing, and data security best practices. bility to design scalable, reusable, and maintainable data engineering frameworks. Experience: Extensive experience in data engineering with multiple years of hands-on Azure Databricks development. Experience leading or contributing to enterprise-scale data platform implementations.

Created: 2026-03-10