Lead Data Engineer
Brak informacji o wynagrodzeniu
SeniorFull-time
#326147·Dodano 19 dni temu·24
Źródło: nofluffjobs.comTech Stack / Keywords
DegreePhDData pipelinesJavaScalaPythonApache SparkKafkaHadoopHiveSQLRDBMSMySQLPostgreSQLApache AirflowAWSGCPApache IcebergDelta Tables DatabricksCI/CDTerraformPrometheusGrafanaDockerKubernetes
Firma i stanowisko
Visa is a world leader in payments technology, facilitating transactions between consumers, merchants, financial institutions and government entities across more than 200 countries and territories, dedicated to uplifting everyone, everywhere by being the best way to pay and be paid.
Wymagania
- 10+ years of work experience with a Bachelor’s Degree or 8+ years of work experience with an Advanced Degree (e.g. Masters/MBA) or 3+ years of work experience with a PhD.
- Extensive experience as a Data Engineer or in a similar role, with deep expertise in modelling, designing, implementing, deploying, and maintaining complex architectures for Big Data pipelines (ETL, batch, streaming), from inception to delivery.
- Expertise in at least one programming language (preferably Java or Scala), and one scripting language (preferably Python).
- Strong experience in large-scale distributed data processing systems using Apache Spark, Kafka, Hadoop, Hive.
- Experience working with SQL and RDBMS like MySQL or Postgres.
- Experience in orchestration tools like Apache Airflow.
- Experience in on-prem and/or cloud-based data platforms (AWS or GCP).
- Familiarity with Agile methodologies, and modern DataOps / DevSecOps practices.
- Excellent written and verbal communication skills, interpersonal and presentation skills, with the ability to articulate data concepts to both technical and non-technical audiences.
- Ability to translate complex business problems into scalable technical solutions.
- Experience or willingness in GenAI / Agentic engineering (e.g., LLMs, MCPs), and tools such as Cloud Code.
- Proven ability to lead technical teams, manage multiple projects, and work effectively across geographies and functions.
- Strong understanding of data quality frameworks, validation methods, and monitoring tools.
- Experience working with Apache Iceberg, or Delta Tables Databricks.
- Experience in deployment with automated and scalable CI/CD tools (e.g. Github Actions, Jenkins, Maven), and IaC tools such as Terraform.
- Experience developing proper metrics instrumentation in Data Platform components using Prometheus, Grafana for operational excellence and incident management.
- Experience with containerization technologies and orchestration tools, including Docker and Kubernetes.
Obowiązki
- Oversee the entire data lifecycle, from data acquisition and ingestion to transformation, storage, and governance for both streaming and batch data pipelines and assets.
- Lead the analysis, design, development, and implementation of various initiatives, with end-to-end ownership to deliver business capabilities.
- Ensure data security, privacy, governance, and compliance with all relevant regulations, and develop and implement auditable policies and procedures.
- Actively contribute with hands on development to critical projects by developing scalable and reusable modules, core frameworks and automation tools.
- Develop new products iteratively by building quick Proofs of Concept (POCs) and converting ideas into real products.
- Establish engineering best practices for application development, testing, deployment, monitoring, and alerting.
- Champion the adoption of GenAI and Agentic AI technologies and develop strategies to integrate them into existing data pipeline workflows or develop new ones.
- Provide tech leadership and motivate a high performing team of data engineers through coaching and mentoring and elevate team talent expertise fostering a culture of innovation and continuous learning.
- Adhere to Visa's Leadership Principles by promoting collaboration, encouraging constructive debate, and executing with excellence.
Oferta
- Sport subscription
- Private healthcare
- International projects
- Free coffee
- Playroom
- Free snacks
- In-house trainings
- Modern office
Karta sportowa
Opieka zdrowotna
Szkolenia wewnętrzne
Darmowe napoje
Darmowe przekąski
Inne informacje
Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.
VISA
60 aktywnych ofert