Senior Data Engineer

DCG sp. z o.o. is a recruitment company focused on providing experienced professionals and continuous support to meet business needs through a partnership approach.

• Minimum 5 years of hands-on work with Spark and Scala
• At least 7 years of professional Python development
• Minimum 5 years of experience working with Linux environments
• Solid background in distributed data processing engines (e.g., Spark)
• Practical knowledge of Hadoop ecosystem components (Hive, Oozie, MapReduce, etc.)
• Strong to advanced SQL proficiency
• Proven track record of designing and implementing data flows
• Expertise in Bitbucket and Git
• AWS certification or practical experience with AWS services
• Experience with unit testing frameworks and tools (JUnit 5, Mockito, Spark testing frameworks)
• Understanding of code versioning and branching strategies
• Advanced level of English (written and spoken)

• Building distributed and highly parallelized Big Data processing pipelines that process massive amounts of structured and unstructured data in near real-time
• Leveraging Spark to enrich and transform corporate data to enable searching, data visualization, and advanced analytics
• Working closely with analysts and business stakeholders to develop analytics models
• Continuous delivery on Hadoop and other Big Data platforms
• Automating repeatable and reliable processes
• Collaborating closely with the QA team