Director of Data and System Engineering - Full Time/Direct Hire - Hybrid -

Please share your resume to suman.dabbeta@yoh.com
Suman Dabbeta | Sr. Recruiter | Yoh
https://www.linkedin.com/in/sumandabbeta/


Local candidates only
Hybrid work 2-3 days - onsite work is mandatory


Director - Data & Software Engineering, is responsible for leading the core Data Engineering and Software Engineering functions. This role owns the architecture, delivery, and operational excellence of enterprise data platforms, APIs, backend services, software applications, and engineering practices that support trading, risk management, asset management, analytics, operations, and internal business platforms.
The role is primarily focused on data platform leadership and software engineering execution. The successful candidate will guide engineering teams, establish technical standards, drive modernization, oversee delivery of business-critical systems, and ensure platforms are scalable, secure, reliable, maintainable, and cost-effective. This leader must be hands-on enough to review architecture, challenge design decisions, understand code-level tradeoffs, and coach engineers toward pragmatic, high-quality solutions.
While this is not intended to be a dedicated AI Architect role, the Director must have a strong working understanding of AI Engineering and collaborate closely with the AI Architect to ensure BETM's data, API, and software platforms are ready for AI-enabled use cases. This includes supporting RAG patterns, semantic layers, governed data access, vector/search integrations, model-facing APIs, observability, evaluation workflows, and responsible AI controls in partnership with the AI Architect and broader technology leadership.
Key Responsibilities

• Lead and develop engineers across Data Engineering, Software Engineering, API development, platform services, and production support.
• Own the strategic direction, architecture, delivery, and operational health of BETM's enterprise data platforms and major software systems.
• Architect and develop latest trading and asset management platforms for management of client’s assets and trade in the ISO markets. 
• Establish engineering standards, development practices, documentation expectations, delivery discipline, code quality, testing practices, and operational readiness expectations.
• Define architecture standards for data platforms, APIs, backend services, business applications, distributed systems, and cloud-native engineering solutions.
• Provide technical leadership across Python, SQL, FastAPI, REST APIs, PostgreSQL, Redis, Snowflake, Azure services, event-driven patterns, and distributed systems.
• Lead design and modernization of software applications, internal platforms, data-backed products, and legacy systems while balancing delivery urgency with long-term maintainability.
• Own data engineering strategy, including ingestion pipelines, curated datasets, analytical data models, semantic layers, metadata, lineage, governance, data quality, testing, monitoring, and documentation.
• Drive Snowflake architecture, performance optimization, workload management, cost governance, access patterns, and platform reliability.
• Oversee software engineering delivery across APIs, backend services, integrations, internal applications, automation workflows, and business-facing platforms.
• Partner with the AI Architect to ensure data, API, and software platforms support AI Engineering patterns such as RAG, semantic search, embeddings, vector databases, AI evaluation workflows, and model-facing service integration.
• Collaborate with the AI Architect on AI governance, data access controls, auditability, observability, hallucination-risk mitigation, and responsible AI practices where platform and data engineering decisions are involved.
• Partner with stakeholders across trading, risk, asset management, operations, analytics, IT, and executive leadership to translate business objectives into actionable engineering roadmaps and delivery plans.
• Manage prioritization, capacity planning, staffing plans, delivery commitments, stakeholder expectations, and tradeoff decisions across multiple engineering workstreams.
• Own production supports routines, incident management, monitoring, alerting, runbooks, operational readiness, reliability improvements, and continuous improvement through automation.
• Drive CI/CD, secure deployment practices, automated testing, observability, service-level expectations, engineering KPIs, and platform health metrics.

Required Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, Data Science, or equivalent practical experience. 
• 15+ years of experience in software engineering, data engineering, platform engineering, or architecture. 
• 7+ years of engineering leadership experience managing teams of 10+ engineers across data, software, API, platform, or application engineering functions. 
• Deep hands-on expertise in Python, Snowflake, Databricks, REST API design, FastAPI or comparable API frameworks, distributed systems, and modern software architecture principles. 
• Strong production experience with Snowflake, including data modeling, workload design, query performance, governance, cost optimization, operational monitoring, and platform reliability. 
• Strong experience with PostgreSQL, Redis, Azure cloud services, DevOps practices, CI/CD pipelines, automated testing, observability, and production operations. 
• Proven experience designing and operating scalable data platforms, data pipelines, curated datasets, semantic layers, data quality frameworks, metadata/lineage practices, and governed analytical systems. 
• Proven experience leading software engineering teams delivering APIs, backend services, integrations, internal applications, automation platforms, and business-critical systems. 
• Strong architectural judgment with the ability to review solution designs, evaluate technical tradeoffs, challenge assumptions, and guide teams toward maintainable and cost-effective implementation choices. 
• Working understanding of AI Engineering concepts, including LLMs, RAG, embeddings, vector databases, semantic search, model-facing APIs, AI evaluation, observability, and responsible AI controls. 
• Ability to collaborate effectively with an AI Architect and translate AI platform needs into data, API, security, integration, and software engineering requirements. 
• Experience supporting high-availability, business-critical platforms requiring reliability, scalability, security, auditability, monitoring, and disciplined production support. 
• Strong communication skills with the ability to engage executives, business stakeholders, architects, product owners, analysts, and engineering teams. 
• Preferred experience in energy trading, commodities, utilities, financial services, or other data-intensive industries. 
• Preferred experience with Snowflake Cortex, Databricks, Azure OpenAI, Azure AI Services, LangGraph, Semantic Kernel, MCP, or comparable AI/data platform ecosystems, with emphasis on integration and platform readiness rather than pure AI research. 

 

Estimated Min Rate: $168000.00
Estimated Max Rate: $240000.00


What’s In It for You?
We welcome you to be a part of the largest and legendary global staffing companies to meet your career aspirations. Yoh’s network of client companies has been employing professionals like you for over 65 years in the U.S., UK and Canada. Join Yoh’s extensive talent community that will provide you with access to Yoh’s vast network of opportunities and gain access to this exclusive opportunity available to you. Benefit eligibility is in accordance with applicable laws and client requirements. Benefits include:

• Medical, Prescription, Dental & Vision Benefits (for employees working 20+ hours per week)
• Health Savings Account (HSA) (for employees working 20+ hours per week)
• Life & Disability Insurance (for employees working 20+ hours per week)
• MetLife Voluntary Benefits
• Employee Assistance Program (EAP)
• 401K Retirement Savings Plan
• Direct Deposit & weekly epayroll
• Referral Bonus Programs
• Certification and training opportunities

Note: Any pay ranges displayed are estimations. Actual pay is determined by an applicant's experience, technical expertise, and other qualifications as listed in the job description. All qualified applicants are welcome to apply.

Yoh, a Day & Zimmermann company, is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.

Visit https://www.yoh.com/applicants-with-disabilities to contact us if you are an individual with a disability and require accommodation in the application process.

For California applicants, qualified applicants with arrest or conviction records will be considered for employment in accordance with the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act. All of the material job duties described in this posting are job duties for which a criminal history may have a direct, adverse, and negative relationship potentially resulting in the withdrawal of a conditional offer of employment.

It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.

By applying and submitting your resume, you authorize Yoh to review and reformat your resume to meet Yoh’s hiring clients’ preferences. To learn more about Yoh’s privacy practices, please see our Candidate Privacy Notice: https://www.yoh.com/privacy-notice

Stay Safe During Your Job Search
Fraudulent recruiting communications have become increasingly common. Emails from Yoh recruiters will only come from an @yoh.com email address. Yoh will never ask candidates to pay fees, purchase equipment, send gift cards, or transfer funds as part of the recruiting or hiring process. If you receive a communication that appears suspicious or requests payment, contact Yoh directly before responding or sharing personal information.