✨ Our 2026 Hiring Report is out — see what 146,050 job postings say about getting hired.

Read the report →
JOB DETAILS

Databricks Tech Lead

CompanyEXL
LocationChennai
Work ModeOn Site
PostedOctober 21, 2026
About The Company
Choosing a digital partner is about more than capabilities — it’s about collaboration and character. Unrealistic overhauls and off-the-shelf products ignore what matters most — your unique needs, culture, goals, and your legacy data and technology environments. At EXL, our collaboration is built on ongoing listening and learning to adapt our methodologies. We’re your business evolution partner—tailoring solutions that make the most of data to make better business decisions and drive more intelligence into your increasingly digital operations. Whether your goals are scaling the use of AI and digital, redesign operating models, or driving better and faster decisions, we’re here to partner with you to help you gain—and maintain—competitive advantage with efficient, sustainable models at scale. Our expertise in transformation, data science, and change management helps make your business more efficient and effective, improve customer relationships and enhance revenue growth. Instead of focusing on multi-year, resource- and time-intensive platform designs or migrations, we look deeper at your entire value chain to integrate strategies with impact. We use our specialization in analytics, digital interventions, and operations management—alongside deep industry expertise — to deliver solutions that help you outperform the competition. At EXL, it’s all about outcomes—your outcomes—and delivering success on your terms. Share your goals with us and together, we’ll optimize how you leverage data to drive your business forward. For more information, visit www.exlservice.com.
About the Role

We are looking for a skilled and passionate Databricks Engineer to design, build, and optimize enterprise-scale data lakehouse solutions on the Databricks platform. The successful candidate will be responsible for creating Databricks pipeline delivering Financial Crime platforms covering Anti-Money Laundering (AML), Know Your Customer (KYC), Customer Risk Assessment (CRA), Sanctions Screening, Transaction Monitoring, Fraud Detection, and Regulatory Reporting 

Responsibilities

Databricks Platform Engineering 

  • Design, build, and maintain Databricks workspaces, clusters, and compute pools across dev/test/prod environments. 

  • Configure and manage Databricks Unity Catalog for data governance, access control, fine-grained permissions, and data lineage. 

  • Optimize cluster configurations — instance types, auto-scaling policies, spot/preemptible nodes — for cost and performance. 

  • Implement workspace-level best practices: folder structures, access controls, secret management (Databricks Secrets / Azure Key Vault / AWS Secrets Manager). 

  • Manage Databricks jobs, workflows, and multi-task job orchestration with dependency management. 

 

Delta Lake & Lakehouse Architecture 

  • Design and implement Delta Lake tables with appropriate partitioning, Z-ordering, and file compaction (OPTIMIZE / VACUUM). 

  • Build Medallion Architecture (Bronze / Silver / Gold) layers for structured data lake organization. 

  • Implement Delta Live Tables (DLT) pipelines for declarative, reliable ETL/ELT with built-in data quality expectations. 

  • Manage schema evolution, table versioning, time travel, and Change Data Feed (CDF) for incremental processing. 

  • Design data lakehouse patterns integrating Delta Lake with external systems (Kafka, ADLS, S3, GCS). 

Qualifications

Education 

  • Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or related field. 

 

Experience 

  • 8+ years of total experience in data engineering or software engineering. 

  • 3+ years of dedicated hands-on experience with the Databricks platform in production environments. 

  • Strong background in big data engineering, cloud data platforms, and distributed computing. 

 

Databricks Platform 

  • Deep expertise in Databricks Workspaces, Clusters, Jobs, Workflows, and Repos. 

  • Proficiency with Unity Catalog — metastore setup, catalog/schema/table management, access controls, and data lineage. 

  • Hands-on experience with Delta Live Tables (DLT) — pipeline development, expectations, and monitoring. 

  • Strong command of Delta Lake internals — transaction log, ACID guarantees, file layout, and optimization techniques. 

  • Experience with Databricks SQL Warehouses, SQL Analytics, and dashboard creation. 

  • Knowledge of Databricks Photon engine, serverless compute, and cost optimization strategies. 

 

Key Skills
DatabricksData EngineeringUnity CatalogDelta LakeDelta Live TablesLakehouse ArchitectureMedallion ArchitectureData GovernanceETL/ELTDistributed ComputingDatabricks WorkflowsDatabricks SQLCloud Data PlatformsCluster OptimizationData LineageFinancial Crime Platforms
Categories
Data & AnalyticsTechnologySoftwareFinance & AccountingManagement & Leadership
Job Information
📋Core Responsibilities
Design, build, and optimize enterprise-scale Databricks lakehouse solutions, including workspaces, compute, governance, and orchestrated data pipelines. Develop and maintain Delta Lake and Medallion Architecture pipelines supporting financial crime capabilities such as AML, KYC, sanctions screening, transaction monitoring, fraud detection, and regulatory reporting.
📋Job Type
full time
📊Experience Level
5-10
💼Company Size
68543
📊Visa Sponsorship
No
💼Language
English
🏢Working Hours
40 hours
Apply Now →

You'll be redirected to
the company's application page