DATA ENGINEER

On behalf of our client, Procom Consultants Group is currently looking for a senior Data Engineer.

DATA ENGINEER MANDATORY SKILLS

• Applied knowledge in Big Data platforms, ideally with exposure to Hadoop ecosystem (HDFS, Pig, Hive, SPARK, Big SQL, NoSQL, YARN)
• Solid experience in Scala
• Experience in designing efficient and robust data pipelines using Spark
• Excellent skill in SQL development
• Knowledge of data modeling and understanding of different data structures and their benefits and limitations under particular use cases
• Hands-on experience with structured and unstructured data like JSON
• Experience with enterprise systems such as Guidewire Claim Center, Guidewire Policy Center would be an asset
• Hands-on experience Cloud, data ingestion using REST API.
• Knowledge on Contact Centre business.

Analyze the source data, prepare data model & mapping, and develop Scala/Spark programs and related components in the work areas of Information Management ("IM") relating to claims, insurance and finance data. Use data pipelines to extract data from data sources which are in various formats (viz - flat files, XML, relational tables, Oracle logs), and Use tools (such as Stream Sets, Scala and Spark programs) to transform and store the data in big data platform (Data Lake) after data validation. Ingest data from cloud to Big Data