Data Engineer

at  Shaw Scott Inc

ABOUT US

We’re a global digital consultancy that helps brands create engaging and effective customer experiences through technology. We’re expanding the team and looking for talented, committed, and knowledgeable industry experts to help us redefine what a consultancy not only does for its clients, but for the people who make it happen. We’re passionately committed to supporting diversity, flexibility, and ongoing opportunity for all.

REQUIREMENTS:

• Must be able to speak and write in fluent English
• Self starter with can do attitude
• 3+ years of hands-on experience in data processing focused projects
• Expertise in Apache Spark along with Spark streaming & Spark SQL preferable
• Experience with Databricks and delta-lake
• Ability to build Apache Airflow pipelines
• Proficiency with Python and SQL
• Experience using ETL/data tools such as Redpoint Data Management
• Hands on experience with cloud provider such as AWS, Azure or GCP
• Understanding of relational database management systems
• Comfortable working in sprints
• Relevant experience with version control and code review
• Knowledge of Agile methodologies
• Proficient in MS Office suite
• Committed to excellent customer service and teamwork
• Ability to anticipate client needs and challenges to propose appropriate resolutions
• Ability to perform under pressure working on multiple projects and meet deadlines while demonstrating enthusiasm and integrity towards colleagues and clients
• Strong written and communication skills, including confidence liaising between colleagues and client teams as well as presenting
• Ability to forge strong and trusted relationships with tenacity, integrity, and a sense of humour

As a Data Engineer, you will support the execution of marketing activities by partnering with our clients and a cross-functional team of experts as part of the Technical Services team. You will be responsible for integrating, transforming, and consolidating data from various structured and unstructured data systems into structures that are suitable for building data pipelines for marketing and customer analysis. You will also be expected to set best practices and provide advice and guidance for our customers.

• Help stakeholders understand the data through exploration, building and maintaining secure and compliant data processing pipelines by using different tools and techniques
• Consult and help to develop, implement, and maintain sustainable, high-performance, growth-ready data-processing and data integration systems
• Design, construct, install, test and maintain highly scalable and optimized data pipelines with current monitoring and logging practices
• Bring together large, complex, and sparse data sets to fulfill both functional and non-functional business requirements, utilizing a variety of languages, tools, and frameworks to integrate data
• Design and implement data tools for our clients to help them in building, optimizing, and tuning of use cases
• Leverage and improve a cloud-based tech stack that includes technologies such as AWS, Databricks, Kubernetes, Spark, Airflow, Python, Redpoint, GCP
• Educating other team members on tools and best practises
• Design and build visually appealing business reports in tools like Power BI
• Successful candidates will have exceptional interpersonal, analytical, and problem-solving skills. We are looking for someone who excels at building rapport across teams and who can both prioritise and manage multiple requests while paying strong attention to the details.

This is a technical role and has a career path to become a Senior Manager or Director in the long term (dependent on performance and dedication).