Responsibilities
- Design, build and maintain complex ETL jobs that deliver business value
- Translate high-level business requirements into technical specs
- Ingest data from disparate sources into the data lake and data warehouse
- Cleanse and enrich data and apply adequate data quality controls
- Provide insight and direction to guide the future development of data platform
- Develop re-usable tools to help streamline the delivery of new projects
- Collaborate closely with other developers and provide mentorship
- Evaluate and recommend tools, technologies, processes and reference architectures
- Work in an Agile development environment, attending daily stand-up meetings and delivering incremental improvements
Competencies
- Proficient in data management including data analytical capability
- Excellent verbal and written communications, high attention to detail
- Ability to manage diverse projects impacting multiple roles and processes
- Candidate should be able to troubleshoot problem areas and identify data gaps and issues
- Ability to adapt to fast changing environment
- Strong experience with database modeling and schema design for relational databases
- Experience Working with Relational Databases like Postgres, Vertica etc.
- Experience with Git, JIRA, Jenkins, Confluence
- Experience with ETL, data transformations & Datawarehouse Concepts
- Understanding of Linux, AWS (or other cloud platforms), Python
- Cloud: Experience working in a cloud environment (e.g. AWS)
- Experience dealing with data pipelines using Python and/ Pyspark
- Advanced Data Processing: Experience using data processing technologies such as Apache Spark or Kafka
- Workflow: Good knowledge of orchestration and scheduling tools (e.g. Apache Airflow)
Experience
- Data: 5+ years of experience with data analytics and warehousing
- SQL: Deep knowledge of SQL and query optimization
- ETL: Strong understanding of ETL methodologies and tools
- Hands-on experience with various cloud-based database technologies (Aurora, Vertica, Redshift, etc.)
- Reporting: Experience with data reporting (e.g. Microstrategy, Tableau, Looker) and data cataloging tools (e.g. Alation)
- Experience in data governance a plus.