Back to jobs

Data Engineer

Job description

Key Responsibilities

  • Design, develop, and automate ETL/ELT pipelines to extract, transform, and load data from multiple sources into data warehouses or cloud environments.
  • Build and optimize Alteryx and Azure Data Factory (ADF) workflows for complex data integration and processing tasks.
  • Develop, enhance, and maintain SQL Server stored procedures, functions, and scripts for data manipulation and performance optimization.
  • Leverage Airflow for orchestration, scheduling, and monitoring of data pipelines.
  • Develop automation and system integration solutions using PowerShell scripts.
  • Use Python for data transformation, validation, automation, and API integration.
  • Design and maintain cloud-based data architectures, preferably on Microsoft Azure (e.g., Data Lake, Synapse Analytics, Azure SQL).
  • Ensure data quality, consistency, security, and reliability across systems.
  • Collaborate with data analysts, scientists, and business stakeholders to ensure timely delivery of high-quality data for analytics and reporting.
  • Monitor, troubleshoot, and optimize data workflows and pipelines for performance and fault tolerance.
  • Document data engineering processes, data flows, and architecture.

Required Qualifications

  • Bachelor's degree in Computer Science, Information Systems, or related field.
  • 3-5 years of experience in data engineering, data integration, or related roles.
  • Strong expertise with Microsoft SQL Server, including T-SQL, stored procedures, query optimization, and indexing.
  • Hands-on experience with ETL tools such as Alteryx and Azure Data Factory.
  • Practical knowledge of Airflow DAG development and monitoring.
  • Proficiency in Python for data processing, automation, and scripting.
  • Strong hands-on experience with PowerShell scripting for automation and orchestration.
  • Familiarity with cloud data architectures, preferably Azure (Data Lake, Blob Storage, Synapse, ADF).
  • Solid understanding of data warehousing principles, data modeling, and ETL design patterns.