Data Engineer

We are seeking a mid-level Data Engineer to support the development and maintenance of data pipelines, integrations, and data processing solutions within an investment firm. This individual will work with technology and business teams to move, transform, and organize data across enterprise systems and cloud platforms.

The ideal candidate has hands-on experience with Python, PySpark, Databricks, Unity Catalog, Azure Data Factory, Azure DevOps, and Salesforce. Experience across most of these technologies is preferred; candidates who have strong experience in the core data engineering technologies but are missing one or two requirements will also be considered.

This is a hands-on role requiring the ability to work independently, collaborate with technical stakeholders, and contribute quickly in a fast-paced environment.

Key Responsibilities

• Design, develop, and maintain data pipelines and data processing solutions.
• Develop data engineering applications using Python and PySpark.
• Build and support data workloads within Databricks.
• Work with Unity Catalog to manage data access, governance, and organization.
• Develop and maintain data ingestion and transformation pipelines using Azure Data Factory.
• Integrate data from Salesforce and other enterprise applications into downstream data platforms.
• Implement data transformations, validation, and quality checks to ensure data accuracy and consistency.
• Participate in CI/CD processes and follow established development and deployment practices.
• Use Git and Azure DevOps for source control, work management, and deployment.
• Troubleshoot data pipelines, integrations, and processing issues.
• Collaborate with technology, data, and business teams to understand data requirements and deliver practical solutions.
• Document data pipelines, processes, and technical solutions.

Required Qualifications

• Mid-level professional experience in data engineering or a related technical discipline.
• Hands-on experience with:
  • Python
  • PySpark
  • Databricks
  • Unity Catalog
  • Azure Data Factory
  • Azure DevOps
  • Git and CI/CD toolchains
  • Salesforce
• Experience developing and supporting data pipelines and integrations.
• Ability to work independently and quickly become productive in an existing technical environment.
• Strong problem-solving and troubleshooting skills.

Nice to Have

• Experience with SQL, including DML and DDL.
• Experience working with financial services, investment management, private equity, asset management, or other financial data environments.
• Experience integrating data from multiple enterprise applications and systems.
• Familiarity with data governance, data quality, and master/golden-source data concepts.