Data Engineer (Python + Pyspark)

Scalo is a company working on a global platform for secure data exchange, analysis, and services across various industries. They design, develop, and support solutions based on Azure analytics tools such as Azure Databricks, Azure SQL, Azure Data Factory, and Azure Functions.

• 3-4 years of professional experience in software development, system integration, data warehousing, or related areas
• Proficient in Python programming, especially data processing with pandas and PySpark
• Practical knowledge of SQL and database design
• Experience working with Apache Spark / PySpark
• Good command of English, both spoken and written
• Analytical skills and ability to identify common elements in diverse problems

Nice to have:

• Data modeling, especially in the context of data warehouses and Big Data
• Knowledge of query and data processing languages and frameworks such as R, Scala, DAX, XPath, Kusto Query Language
• Experience with Visual Studio Code
• Creating deployment automation scripts in YAML for Microsoft DevOps environment
• Creating and using RESTful API services
• Experience with Microsoft Power BI

• Design, develop, and support solutions using Azure analytics tools (Azure Databricks, Azure SQL, Azure Data Factory, Azure Functions)
• Create fully automated data processing pipelines using Microsoft Azure components
• Develop, maintain, and document data ingestion processes, visualization applications, data management, and data catalogs
• Tune performance and optimize Apache Spark / PySpark applications
• Explore and analyze datasets from external and internal providers
• Collaborate daily within an international team
• Work remotely