Data Platform Lead

Data Platform Lead – (Azure Databricks) – 12 Month Daily rate

contract.

Data Platform Lead (Azure Databricks)

Contract: 12-month daily rate | Sydney (Hybrid)

Position Overview

The Technology Hub enables the organisation’s businesses to achieve their technology-driven goals.

The work undertaken spans leading and supporting enterprise transformation, building or adopting

modern technology platforms, protecting against risk, and delivering data insights that transform

technology capabilities.

The Infrastructure team is responsible for designing, implementing, and maintaining the

organisation’s IT backbone, including networks, servers, data centres, and cloud services. The team

ensures robust security, manages data storage and backup solutions, and provides technical support

and troubleshooting. The team also manages vendor relationships, procurement, and compliance

with relevant regulations. By optimising performance and planning for future needs, the

Infrastructure team ensures the stability, security, and efficiency of the IT environment, enabling

seamless operations across the organisation.

As the Data Platform Lead, you will be responsible for overseeing the design, implementation, and

operation of a modern data platform. You will ensure that data systems are robust, scalable, and

secure, supporting business needs and future growth. You will lead a team of engineers, provide

technical direction, and foster a collaborative, delivery-focused environment.

the data platform will support multiple enterprise domains, including HR (HCM) data. This includes

enabling secure, scalable analytics and AI use cases across employee, workforce, and organisational

data sourced from platforms such as Workday, SAP SuccessFactors, and Salesforce, while ensuring

strong governance, privacy, and compliance controls for sensitive employee information.

Why join us?

You will have the opportunity to work on complex, high-impact challenges across multiple industries,

contributing to outcomes that matter. The organisation values curiosity, collaboration, and

continuous improvement, offering an environment where people are encouraged to grow their skills,

explore new technologies, and make a tangible impact.

You will be part of a large, diverse, and highly skilled community that values bold ideas, teamwork,

and long-term outcomes.

What you’ll do

• Lead the design and implementation of data platform and infrastructure solutions aligned

with business requirements and industry best practices

• Ensure the stability, security, scalability, and reliability of the data platform

• Lead and mentor a team of data and platform engineers, supporting professional

development and capability uplift

• Drive platform improvements, promoting innovation and implementing solutions that

improve efficiency, reliability, and cost effectiveness

• Collaborate with architecture, engineering, security, and business teams to align platform

initiatives with organisational goals

• Oversee ongoing operations, support, and optimisation of the platform to minimise

downtime and improve performance

• Define, implement, and enforce platform standards, policies, and operating procedures

What we’re looking for

Technology-specific experience

• Strong experience with Spark, SQL, Python/PySpark, and building batch and streaming

pipelines (Structured Streaming, Auto Loader, Kafka/Event Hubs/Kinesis)

• Define and implement platform guardrails and standards, including Unity Catalog, access and

privilege models, cluster and SQL warehouse policies, workspace strategies, naming

conventions, and environment separation

• Build scalable ingestion and transformation frameworks using Delta Lake, Auto Loader, and

Delta Live Tables, including medallion (bronze/silver/gold) patterns for batch and streaming

workloads

• Establish CI/CD pipelines and infrastructure-as-code practices for repeatable provisioning

(e.g. Terraform, Databricks Terraform provider, GitHub Actions, Azure DevOps, Jenkins,

Databricks Repos)

• Implement observability and reliability practices: logging, metrics, lineage integration, data

quality checks, SLAs/SLOs, incident response, and capacity planning

• Design, build, and operate a Databricks lakehouse platform (workspaces, Delta Lake, Unity

Catalog, jobs/workflows, Databricks SQL) aligned to enterprise architecture and data

operating models

Industry experience

• Strong security and networking experience on a major cloud platform (AWS, Azure, or GCP),

including identity federation/SCIM, secrets management, private networking, firewalling,

encryption, and compliance controls

• Knowledge of FinOps and capacity management, including cost modelling, tagging and

chargeback, cluster policy design, autoscaling strategies, and cost optimisation for compute

and storage

• Solid understanding of lakehouse architectures and modern data engineering patterns,

including CDC, schema evolution, data quality frameworks, and performance optimisation

(including Photon)

Other role requirements

• Strong analytical and structured problem-solving skills, with the ability to translate between

business and technical stakeholders

• Experience assessing and prioritising AI and advanced analytics opportunities based on value,

feasibility, and delivery risk

• Proven capability in process mapping, requirements definition, and senior stakeholder

engagement

• Clear and confident communication skills for both executive and delivery audiences

• Relevant tertiary qualifications in technology, business, or related disciplines, with

foundational AI or data training highly regarded

What you’ll gain

• The opportunity to work with complex organisations and modern technologies that stretch

and grow your capabilities

• Flexible working arrangements that support work-life balance while meeting team and

delivery needs

• Clear career development pathways supported by continuous learning and leadership

development

• Competitive leave entitlements and wellbeing benefits

Equal Opportunity Statement

The organisation is committed to providing a fair, inclusive, and respectful recruitment process.

Applicants are encouraged to request reasonable adjustments or workplace accommodations to

support them throughout the selection process and in the role.