Senior Data Engineer

İşin təsviri

Responsibilities:

• Data Modeling and Database Design: Developing and refining data models, and optimizing database designs for performance and scalabilit;
• Data Pipeline Monitoring and Troubleshooting: Ensuring that all data pipelines are operating correctly and addressing any issues promptly;
• Data Quality Assurance: Implementing and overseeing processes to ensure the accuracy and integrity of data;
• Query Optimization: Continuously optimizing queries and data retrieval methods for performance and efficiency;
• Integration of New Data Sources: Adding and integrating new data sources into the existing data architecture;
• Automated Script Execution and Monitoring: Running and monitoring automated scripts for data extraction, transformation, and loading (ETL);
• Collaboration with Stakeholders: Communicating with business analysts, data scientists, and other stakeholders to understand data needs and requirements;
• Performance Reviews of Data Systems: Evaluating the performance of databases, data lakes, and other storage systems;
• Data Cleaning and Transformation: Regularly cleaning and transforming data to maintain its usefulness and relevance;
• Code Reviews and Updates: Reviewing and updating ETL scripts and data pipeline code for improvements and efficiency;
• Documentation and Knowledge Sharing: Updating documentation for data pipelines and databases, and sharing knowledge with the team.

Requirements:

• Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field;
• Minimum 3-4 years of experience in operations;
• Experience with data integration (ETL/ELT) concepts, demonstratable skills of implementing data pipelines
• Experience with orchestration tools (Airflow, etc.);
• Deep understanding of SQL and experience with relational databases like PostgresSQL, MSSQL, Oracle , as well as NoSQL databases like MongoDB;
• Proficiency in languages like Python(or Scala);
• Knowledge of good engineering practices around data modelling (Star, Snowflake, Data Vault, etc), data access and code writing;
• Understanding of containers and Kubernetes;
• Ablility to clearly define data quality issues;
• At least Intermediate English is a must, spoken Russian is a plus;
• Strong problem solving/critical thinking skills.

İnnovasiya və Rəqəmsal İnkişaf Agentliyi tərəfindən digər vakansiyalar

• Layihə meneceri
• Hesabatlılıq üzrə mütəxəssis
• Social Media Content Specialist
• Product Owner
• Software Developer