Senior Data Engineer

at  Creditsafe

Cardiff, Wales, United Kingdom -

Start DateExpiry DateSalaryPosted OnExperienceSkillsTelecommuteSponsor Visa
Immediate28 Nov, 2024Not Specified29 Aug, 2024N/AGood communication skillsNoNo
Add to Wishlist Apply All Jobs
Required Visa Status:
CitizenGC
US CitizenStudent Visa
H1BCPT
OPTH4 Spouse of H1B
GC Green Card
Employment Type:
Full TimePart Time
PermanentIndependent - 1099
Contract – W2C2H Independent
C2H W2Contract – Corp 2 Corp
Contract to Hire – Corp 2 Corp

Description:

SENIOR DATA ENGINEER

Job Title: Senior Data Engineer
Location: Cardiff,
Department: Data Cloud
Creditsafe are on the hunt for a Data Engineer to join our Data Cloud team.
About Creditsafe: With the largest owned database supported by a partner network, Creditsafe provides data on 320 million companies in over 100 countries to support its customers in this regard. Creditsafe is scaling business and currently going through a significant growth stage. Being a data company, Creditsafe collects and processes large amounts of complex data, received from multiple channels with the only purpose of serving our clients with opportunities to gain actionable insights. Our holistic data aggregation helps our clients to fill the knowledge gaps, influence decision making, and find hidden patterns.
About Opportunity: This opportunity is to join a team of highly technically skilled engineers who are redesigning Creditsafe’s data platform with high throughput and scalability as the primary goal. The data delivery platform is being built upon AWS Redshift and S3 cloud storage. The platform is expected to manage over a billion objects along with daily increments of more than 10 million objects while handling addition, deletion, and correction of our data and indexes in an auditable manner. Our data processing application is entirely based on Python and designed to efficiently transform incoming raw data volume into API consumable schema. The team is also building highly available and low latency APIs to enable our clients with faster data delivery.

Responsibilities:

  • You will actively contribute to the codebase and participate in peer reviews.
  • Design and build metadata driven, event based distributed data processing platform using technologies such as Python, Airflow, Redshift, DynamoDB, AWS Glue, S3.
  • As an experienced Engineer, you will play a critical role in the design, development, and deployment of our business-critical system.
  • You will be building and scaling Creditsafe’s data pipeline primarily moving data between Redshift and S3.
  • Execute practices such as continuous integration and deployment using test-driven development where applicable to enable the rapid delivery of working pipelines.
  • Understanding company & domain data to make recommendations to improve the existing product.


REQUIREMENT SUMMARY

Min:N/AMax:5.0 year(s)

Information Technology/IT

IT Software - Other

Software Engineering

Graduate

Proficient

1

Cardiff, United Kingdom