Website Eaglecliff
This is an exciting opportunity for a Data Engineer with Energy Trading experience to join a Global Consultancy working on-site with an Energy giant.
Primary Role:
We are seeking an experienced Senior Data Engineer with proven expertise in the Energy Trading domain to join our team.
This role is pivotal in designing, developing, and maintaining scalable data solutions using the Azure and Databricks ecosystem.
The ideal candidate will work closely with business stakeholders to ensure data is accessible, reliable, and optimized for performance across trading and analytics platforms.
Key Responsibilities:
•Architect and maintain scalable data pipelines leveraging Azure Data Factory and Databricks to support business-critical analytics.
•Build and optimize ETL/ELT workflows for seamless integration of structured and unstructured data from multiple sources.
•Develop modular and maintainable Python scripts for data transformation, automation, and orchestration.
•Write and tune SQL queries to support reporting, analytics, and operational data needs.
•Partner with cross-functional teams to translate business requirements into technical data solutions.
•Uphold high standards of data governance, ensuring accuracy, consistency, and security across platforms.
•Proactively monitor pipeline health, troubleshoot issues, and implement resilience strategies.
•Embed observability into data workflows through logging, alerting, and error-handling mechanisms.
•Leverage domain expertise in energy trading to enhance data models and support advanced analytics use cases.
Required Skills & Qualifications:
•Minimum 5 years of hands-on experience in data engineering, with a strong track record of delivering production-grade solutions.
•Deep proficiency in Azure Data Factory and Databricks, including pipeline orchestration and Spark-based processing.
•Advanced knowledge of Python, with experience in building reusable components for data workflows.
•Strong command of SQL, including performance tuning and working with large datasets.
•Familiarity with streaming technologies such as Spark Streaming or Kafka for real-time data processing.
•Demonstrated experience in supporting and scaling enterprise-grade data platforms.
•Solid understanding of energy trading concepts, including trade lifecycle, market data, and financial instruments.
•Experience with Git and CI/CD practices for version control and deployment automation.
•Analytical mindset with a keen eye for detail and a proactive approach to problem-solving.
•Excellent interpersonal and communication skills, with the ability to work effectively in a collaborative environment.
This needs combining with a positive attitude and an ability to work within a large, globally dispersed project team in a multi-cultural environment. You also need to be a self-starter, a logical thinker and a quick learner, with strong initiative and excellent communication, interpersonal and presentation skills, able to write clearly and concisely. We believe in equality of opportunity for all job applicants regardless of gender, marital status, race, colour, nationality, ethnic origin, creed or religion, disability, sexual orientation or age.
With a focus within Energy Trading, Oil & Gas, Financial Markets and Commodities, we offer a transparent Recruitment Service that has proven to be reliable and effective for over 40 years. We are ISO accredited and proud of our excellent TrustPilot Reviews. Your search for a New Contract Assignment or for a New Permanent Job will be in safe hands with Eaglecliff Recruitment. Please telephone for an immediate response or email your CV for a quick response. Eaglecliff Ltd is acting in the capacity of an employment agency for permanent recruitment and an employment business for contractor resourcing.