Data Engineer

at  Creditsafe

DATA ENGINEER

Job Title: Data Engineer
Location: Cardiff,
Department: Data Engineering
We are currently looking a Data Engineer, to join the Data Engineering team in Cardiff.
You will be expected to attend the office 50% of the working week, to align with our hybrid working policy.

SKILLS AND QUALIFICATIONS

• Experience designing and building autonomous fault tolerant data pipelines
• Solid development experience within a commercial environment creating production grade ETL and ELT pipelines in python
• Comfortable implementing data architectures in analytical data warehouses such as Snowflake, Redshift or BigQuery
• Hands on experience with data orchestrators such as Airflow
• Knowledge of Agile development methodologies
• Awareness of cloud technology particularly AWS.
• Knowledge of automated delivery processes
• Hands on experience of best engineering practices (handling and logging errors, system monitoring and building human-fault-tolerant applications)
• Ability to write efficient code and comfortable undertaking system optimisation and performance tuning tasks
• Comfortable working with relational databases such as Oracle, PostgreSQL, MySQL
• Has exposure to DBT and data quality test frameworks
• Has awareness of Infrastructure as Code such as Terraform and Ansible

• Perform a role as part of an Agile team to develop, test and maintain high quality data processing systems that fulfil business needs.
• Extracting data from various data sources for example relational databases, files and API’s)
• Help evolve our data platform with a view towards growth and high throughput.
• Execute practices such as continuous integration and test driven development to enable the rapid delivery of working code.
• Design and build metadata driven data pipeline using Python and SQL in accordance with guidelines set by the Data Architect
• Ship medium to large features independently using industry standard processing patterns
  The responsibilities detailed above are not exhaustive and you may be requested to take on additional responsibilities deemed as reasonable by their direct line manager.