Senior Data Engineer

Senior Data Engineer

Location: Remote, USA (candidates in PST, MT, or CST preferred)

Duration: 4-month contract

Pay Range: $86–91/hr

Schedule: 9:00 AM – 6:00 PM with flexibility

Summary

Our client's Core Data team builds and scales the data platform and analytics capabilities that power decision-making across the organization. Within Core Data, the Vehicle Catalog team owns the vehicle domain — ingesting, curating, and organizing vehicle data for analytics, ML, and BI use cases relied on by hundreds of stakeholders across dozens of domain teams. The team is migrating the Vehicle Catalog to a new Databricks account while simultaneously building out a new vehicle catalog on that platform. As a Senior Data Engineer, you will help refactor and move existing ETL/ELT and analytics pipelines into the new environment, migrate the underlying data, and validate that everything lands correctly. This is hands-on migration work alongside the engineering team on well-defined pipelines with clear acceptance criteria, with related migration requests — including dashboard and reporting migrations — picked up as they arise.

Job Responsibilities

In this role, you will manage and support data migration projects within the Core Data department. Key responsibilities include:

• Working with a Staff Data Engineer to understand the migration plan, sequence work against it, and provide regular progress updates.
• Refactoring existing ETL/ELT code written in PySpark, DABs, and dbt from the current Databricks environment to meet the patterns and standards of the new account.
• Migrating data pipelines and their underlying datasets, including backfills, and validating parity between source and target.
• Building and running reconciliation checks — row counts, schema conformance, freshness, completeness — to confirm migrated data matches the legacy environment.
• Migrating dashboards and downstream reporting assets to point at the new datasets, coordinating with the analysts who own them.
• Updating orchestration, scheduling, alerting, and runbooks so migrated pipelines are supportable after cutover.
• Troubleshooting pipeline failures and data discrepancies during and after cutover.
• Documenting what was migrated, what changed, and anything the team needs to know to own it going forward.
• Taking on adjacent migration and data engineering requests as the program evolves.

Essential Job Duties and Job Functions

• 6+ years of experience in Data Engineering, Analytics Engineering, or Software Engineering working with production data systems.
• Strong expertise in SQL and Python for building scalable data pipelines and transformations.
• Hands-on experience building ELT pipelines using modern cloud data platforms (Databricks experience required).
• Experience building robust testing frameworks for data migration (completeness, quality, etc.).
• Deep experience with dbt, including model development, testing, documentation, and CI/CD integration.
• Experience designing and operating production-grade data pipelines with monitoring and observability.
• Prior data or platform migration experience.

Knowledge and Skills

• Strong familiarity with AI development tools such as Claude Code or Devin for the software development process.
• Strong collaboration skills and ability to partner with engineering teams, analysts, and operational stakeholders.
• Familiarity with Fivetran connector management and ingestion architecture (nice-to-have).
• Experience with data contracts and schema governance (nice-to-have).
• Knowledge of streaming or near-real-time data pipelines (nice-to-have).

Education and Experience

Bachelor's or Master's degree in Computer Science, Engineering, Mathematics, or a related field, or equivalent practical experience.

...