Lead Data Engineer

Job Description

We are seeking an experienced Lead Data Engineer to join a major Federal Government program focused on delivering modern, enterprise-scale data capabilities.

This role will play a key part in designing, building and optimising data platforms that support analytics, reporting, artificial intelligence and business decision-making. Working within a complex and regulated environment, you'll help drive the organisation's data transformation journey through the use of modern cloud technologies including Databricks and Microsoft Fabric. 

Key Responsibilities:

• Design, develop and maintain enterprise-scale data pipelines and integration solutions
• Build and optimise modern Lakehouse and data warehouse architectures
• Develop data transformation processes using Databricks, Microsoft Fabric and cloud technologies
• Implement data governance, quality, security and monitoring capabilities
• Support analytics, reporting, AI and machine learning initiatives
• Collaborate with stakeholders to deliver scalable and sustainable data solutions
• Provide technical leadership, mentoring and continuous improvement initiatives

Technical Skills & Experience:

• Strong experience with Databricks and/or Microsoft Fabric environments
• Advanced skills in Python, SQL, PySpark and Spark SQL
• Experience building and supporting data pipelines, data lakes and Lakehouse solutions
• Knowledge of Delta Lake, Databricks Workflows, Unity Catalog or Microsoft Fabric services
• Experience with Azure technologies including Data Factory, ADLS Gen2 or Fabric
• Strong understanding of DevOps, CI/CD, automation and Infrastructure as Code practices
• Knowledge of data governance, security and data quality frameworks

What's on Offer:

• 12-month contract with 2 x 12-month extension options
• Hybrid working arrangements across multiple Australian locations
• Opportunity to work on large-scale data transformation initiatives
• Exposure to Databricks, Microsoft Fabric and modern cloud technologies
• Collaborative team environment focused on innovation and continuous improvement
• Opportunity to contribute to national data, analytics and AI capabilities

Due to the nature of this role, this opportunity is open to Australian Citizens only.
Candidates who hold a Baseline Security Clearance (or above) are prefered. 

Calleo is an equal opportunity employer and we encourage applications from all people including Aboriginal and Torres Strait Islander peoples.

Follow Calleo on LinkedIn and visit our website to keep up to date on all our current job vacancies:
www.linkedin.com/company/calleoresourcing
www.calleo.com.au