Databricks AI/ML Engineer
Databricks AI/ML Engineer Experience: Approximately 8 years Core Technologies: Azure Databricks, Apache Spark, PySpark, Delta Lake Role Summary We are seeking an experienced Databricks AI/ML Engineer to design, develop, and deploy scalable machine learning and data engineering solutions on Azure Databricks. The role combines end-to-end ML engineering, MLOps, and Lakehouse-based data engineering, with a focus on reliable, secure, production-ready solutions that address business and operational needs. Key Responsibilities Machine Learning Engineering Design, develop, and deploy machine learning solutions using Azure Databricks and PySpark. Build end-to-end ML pipelines covering data preparation, feature engineering, model training, validation, deployment, and ongoing monitoring. Develop predictive and prescriptive analytics solutions for business and operational use cases. Optimize model performance, scalability, processing efficiency, and overall solution reliability. MLOps and Model Lifecycle Management Implement MLOps practices for model versioning, monitoring, lifecycle management, and controlled releases. Automate model retraining and establish mechanisms to monitor model performance and operational health. Contribute to CI/CD automation and DevOps implementation for data and ML platforms. Data Engineering and Lakehouse Development Process large-scale structured and unstructured datasets using Apache Spark and Databricks. Build and maintain ETL/ELT frameworks to ingest data from multiple enterprise source systems. Develop and optimize Delta Lake-based Lakehouse solutions. Implement batch and real-time data-processing workflows. Create reusable notebooks, data-processing frameworks, ML accelerators, and engineering components. Collaboration, Quality, and Production Support Collaborate with business stakeholders, data engineers, data scientists, architects, and reporting teams to translate requirements into practical AI/ML and data solutions. Troubleshoot production issues, perform root-cause analysis, and implement corrective and preventive measures. Ensure data quality and adherence to enterprise data governance, security, privacy, and compliance standards. Participate in code reviews and promote sound engineering, testing, documentation, and development practices. Required Skills and Experience Approximately 8 years of relevant experience in AI/ML engineering, data engineering, or related roles. Hands-on experience with Azure Databricks, Apache Spark, and PySpark. Experience building and operationalizing end-to-end machine learning pipelines. Strong understanding of feature engineering, model training, validation, deployment, and model performance monitoring. Practical knowledge of MLOps principles, model versioning, and ML lifecycle management. Experience with ETL/ELT design, data transformation, and large-scale data processing. Experience with Delta Lake and Lakehouse architecture concepts. Understanding of batch and real-time data-processing patterns. Familiarity with CI/CD, DevOps practices, code reviews, and automated deployment workflows. Ability to troubleshoot complex issues and optimize solutions for performance, scalability, and reliability. Strong collaboration and communication skills, with the ability to work across technical and business teams. Preferred Qualifications Experience delivering production-grade AI/ML solutions on Azure. Exposure to automated retraining, model drift detection, and model monitoring approaches. Experience working with enterprise data governance, access controls, and compliance requirements. Experience creating reusable frameworks or accelerators that improve development consistency and productivity.