Lead Data Platform Engineer - Enterprise, Data & AI

What you'll do

• Scale self-healing data pipelines with automated schema handling, anomaly detection, and recovery.
• Establish data governance, security, access controls, quality testing, and metadata standards.
• Optimize storage, compute, and query performance; build CI/CD templates and deployment testing for data assets.

What they ask for

• 10+ years of hands-on experience in Data Platform Engineering or Software Engineering.
• Expertise in scaling, optimization, query tuning, compute allocation, and compute-storage lifecycle policies.
• Experience with enterprise security, RBAC, Active Directory/IAM roles, data masking, and compliance requirements.
• Experience establishing data governance, schema evolution controls, data quality audits, observability, and automated CI/CD deployments.

Nice to have

• Experience incorporating LLMs or GenAI into pipeline operations.
• Experience with Terraform, EKS, GPU/CPU optimization, or autonomous vehicle, robotics, or high-tech manufacturing sectors.

Pay

USD 230,000–277,000 per-year-salary (base salary)

Tools

Databricks, EMR (AWS), Active Directory/IAM, LLMs, GenAI, Terraform, EKS

AI focus

Builds AI-ready data foundations for AI agents; bonus work includes using LLMs or GenAI in pipeline operations such as code generation, triage, root-cause analysis, reconciliation, backfills, and anomaly detection.