Piper Companies is seeking a Senior Data Engineer to join a leading shipping and logistics organization. The Senior Data Engineer will handle operational aspects of the data ecosystem, including monitoring, automation, and data flow.

QUALIFICATIONS FOR THE SENIOR DATA ENGINEER INCLUDE:

• 7+ years of experience in data strategies, analysis, architecture, and management
• Strong skills in Azure Data Factory, Databricks, SQL, Python (PySpark) and/or Scala
• Experience in building ETL/data warehouse transformation processes
• Hands-on experience with Azure Data Analytics platform (Azure Storage, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics)
• Experience with Apache Kafka and event-based data

• Work with Azure Databricks to develop and deploy solutions for reporting and analytics.
• Create real-time streaming pipelines.
• Data movement and orchestration using Azure Data Factory.
• Work with internal customers and partners to gather requirements, build, and deploy robust models.
• Develop code to aggregate and transform data to ADLS curated zone.
• Provide ongoing support of Databricks code.