Hands-on Databricks, supporting production ML products within an established data team right here in Brisbane!
This contract role supports machine learning products already running in production. It’s a blend of BAU platform support, deployment governance, and technical enablement for Data Scientists working in Databricks.
This is not a pure Data Scientist role, it’s ideal for someone who enjoys being the technical backbone of a data platform.
What you will be working on:
- Supporting Databricks based ML products in production with in initial BAU focus
- Transitioning models from Databricks notebook prototypes into production utilising ML and pipeline best practices.
- Design, develop, deploy and maintain end to end machine learning and data pipelines
- Manage data and data pipeline operations including data migration, resource management, data pipeline configurations, and troubleshooting
- Collaborate with Data Scientists, Data Engineers, DevOps Engineers and Application developers to design, build, deploy, and maintain robust solutions to critical problems
- Write and review technical documents, including design, development, and collaborative code reviews.
- Monitor and enhance the performance, cost, stability, and operational efficiency of existing tools and services.
- Maintenance of the Databricks infrastructure (including UC catalogue, compute, users and updates).
Experience:
- 3+ years in a DevOps or platform role within AWS environments
- Understanding of cloud-native, Agile and DevOps operating models
- Experience coding, scripting in Python, Bash or PowerShell
- CI/CD pipeline automation using Azure DevOps
- Infrastructure as Code experience using Terraform
- Experience with cloud monitoring and logging – CloudWatch
- Strong Python and Spark experience, including Databricks
- Experience deploying and supporting ML models in production
- Solid understanding of the ML model lifecycle
- Experience with MLflow and Feature Stores
- Background in big data, scalable or shared system design
- Experience working with relational and non-relational data stores across varying data speeds
- Familiarity with unit testing, agile delivery, and change/incident management
- Ability to manage multiple priorities and projects effectively
Qualifications:
- Degree in Computer Science, or similar, or equivalent practical experience
- IT or quantitative undergraduate degree
- AWS Cloud Certifications (Cloud Practitioner)
- Databricks certifications (Data Engineering Associate, Machine Learning Associate)
