Data Engineer

Mindbox is a tech-driven company connecting top IT talents with technology projects for leading enterprises across Europe. The company focuses on consulting, engineering, and talent to build meaningful digital solutions.

• 5+ years of experience in software design and development in an Agile/DevOps environment.
• Strong expertise in PySpark or Scala development and design.
• Experience with scheduling tools such as Airflow.
• Hands-on experience with Hadoop ecosystem components: Apache Spark, Hive, YARN, MapReduce.
• Proficiency in Python, SQL, and ETL frameworks.
• Experience with RESTful services.
• Strong knowledge of Unix/Linux platforms.
• Experience building data pipelines using Hadoop components (Hive, Spark SQL).
• Familiarity with version control tools (Git, GitHub), automated deployment tools (Ansible, Jenkins), and JIRA.
• Understanding of big data modeling techniques (relational and non-relational).
• Ability to debug code issues and provide actionable feedback.

Nice to have:

• Experience with Elasticsearch.
• Development experience with Java APIs.
• Experience with data ingestion processes.
• Understanding of cloud design patterns.
• Exposure to DevOps and Agile methodologies (Scrum, Kanban).

• Design, develop, and maintain data pipelines using PySpark and related big data technologies.
• Implement tools and processes to ensure performance, scalability, availability, and accuracy.
• Collaborate with Business Analysts to interpret and implement requirements.
• Participate in Agile ceremonies including sprint planning, reviews, and retrospectives.
• Contribute to system architecture and design discussions.
• Promote development standards, conduct code reviews, and share knowledge with the team.
• Provide production support and troubleshooting for data pipelines.
• Ensure automated testing and deployment processes are in place.