Software Engineer (Intermediate to Senior) - Vancouver BC, CAN

at  Treasure Data

TREASURE DATA:

At Treasure Data, we’re on a mission to radically simplify how companies use data to create connected customer experiences. Our sophisticated cloud-based customer data platform drives operational efficiency across the enterprise to deliver powerful business outcomes in a way that’s safe, flexible, and secure.
We are thrilled that Gartner Magic Quadrant has recognized Treasure Data as a Leader in Customer Data Platforms for 2024! It’s an honor to be acknowledged for our efforts in advancing the CDP industry with cutting-edge AI and real-time capabilities. View the report here.
Furthermore, Treasure Data employees are enthusiastic, data-driven, and customer-obsessed. We are a team of drivers—self-starters who take initiative, anticipate needs, and proactively jump in to solve problems. Our actions reflect our values of honesty, reliability, openness, and humility.

PHYSICAL REQUIREMENTS:

Must be located in the Greater Vancouver, BC CAN area.

TRAVEL REQUIREMENTS:

Travel requirements typically amount to approximately 5% of the year, including one week in Japan annually, another in Mountain View, CA, and possibly an additional week elsewhere.

YOUR ROLE:

The Plazma team at Treasure Data is one of the essential elements of our CDP solution and is part of the Core Services group, which supports customer data ingestion and availability at a rate of 70B records per day. We develop & run the storage and query engine components and enable customers to find and store their data by offering comprehensive solutions based on OSS and proprietary software. You are expected to help the team develop the future of our Hadoop/Hive & Trino query engines and expand from there into our in-house developed storage solution. This includes maintaining technical excellence to address challenges that currently lack industry-wide solutions and delivering the roadmap together with your team. Our team consists of Big Data experts across Japan, Korea and Canada who are passionate about OSS contribution, and we take pride in the quality of service we offer.

RESPONSIBILITIES & DUTIES:

• Work as a member of the team by designing and developing Trino & Hadoop/Hive solutions
• Be responsible for providing solution expertise around Trino & Hadoop/Hive technologies. This includes technology assessment, use case development, as well as solution outline and design for modern data architectures
• Establish standards and guidelines for the design & development, tuning, deployment, and maintenance of advanced data access frameworks and distributed systems
• Document architectural and technology advancements
• Work with your team to set up the roadmap for Trino & Hadoop/Hive related products based on operational needs and customer requested features
• Mentor and train new members in the team
• Version and release management of Trino and Hive products
• Evaluate, test, and set a base version
• Back port any needed patches from trunk, which contains the latest cutting-edge version of the project, but therefore may also be the most unstable version
• Deploy new customer-facing features for Trino & Hadoop/Hive
• Coordinate with support and product teams on product releases
• Make contributions to the open source community
• Contribute to Trino & Hadoop/Hive open source community with bug fixes and new features
• Work with the Site Reliability team to automate Trino & Hadoop/Hive cluster operations to reduce operational overhead
• Design, develop, and evaluate metrics to ensure system health and plan infrastructure capacity of clusters
• Design and develop scripts to automatically start and stop clusters and switch traffic between active clusters for load balancing of customers’ workloads
• Design and develop failure recovery tools to automatically detect the occurrence of faults and recover faulty clusters
• Provide in-depth support services to Trino & Hadoop/Hive customers
• Take responsibility for on-call to support Trino & Hadoop/Hive customers
• Deal with escalations on product defects and performance issues, lead and perform in depth troubleshooting of Trino & Hadoop/Hive related systems
• Design and develop custom user-defined functions (UDF)