Data Engineer (CPT/JHB Hybrid)

at  Datafin IT Recruitment

Data Engineer (CPT/JHB Hybrid) IT - Analyst, Data Management
Cape Town - Western Cape ~ Johannesburg - Gauteng

ENVIRONMENT:

CREATE and implement scalable data architectures, develop and optimize ETL processes while ensuring the availability, quality and integrity of data throughout the organization of a leading Group operating a portfolio of leading responsible travel companies seeking a highly skilled Data Engineer. You will be responsible for developing, maintaining, and optimizing the data warehouse, data pipeline, and data products. Applicants will require a Bachelors Degree in Informatics/Computer Science or similar field with at least 4 years work experience in a similar role including with Big Data technologies like Hadoop, Microsoft Azure, AWS, etc. You must also have knowledge of SQL, PostgreSQL, MongoDB, Cassandra, Python, Java/Scala & ETL tools such as Talend, Informatica, Apache NiFi, Microsoft SSIS, etc.

REQUIREMENTS:

Qualifications

• Bachelors Degree in Informatics/Information Systems, Computer Science or related field.

Experience/Skills

• Minimum of 4 years experience in a Data Analyst, Data Scientist or Data Engineering role or similar.
• Experience with Big Data technologies like Hadoop, Microsoft Azure, AWS, etc.
• Knowledge of database management systems, both relational (e.g., SQL, PostgreSQL) and non-relational (e.g., MongoDB, Cassandra).
• Proficiency in data integration and ETL tools, such as Talend, Informatica, Apache NiFi, Microsoft SSIS, etc.
• Proficiency in programming languages (Python, Java, or Scala).

• Data Architecture/Infrastructure Design: Contribute to the design of the architecture of the data platform/warehouse for use by both the Data and Analytics team and Business teams.
• Data Integration - Merge and consolidate data into a unified format promoting data accuracy and reliability for analysis.
• Data Cleansing - Remove inconsistencies, errors, and duplications from datasets and ensure the data is of high quality.
• Data Transformation - Transform data into a consistent format through standardising and structuring it for meaningful analysis.
• Data Modelling - Design data structures that optimise data value, storage and retrieval such as data cubes, star schemas, or snowflakes. These models enhance data organisation and accessibility, support effective analysis and reporting.
• Data Storage: Implement data storage solutions (databases and data lakes).
• Data Monitoring: Monitor data systems performance and implement optimization strategies.
• Data Governance: Leverage data controls to maintain data privacy, security, compliance, and quality for all data domains within the Data and Analytics area of accountability.
• Cross-functional Collaboration: Work together with cross-functional teams to identify and address data-related issues.