EDUCATION & EXPERIENCE

• Bachelor’s degree and five years of relevant experience or a combination of education and relevant experience.

KNOWLEDGE, SKILLS AND ABILITIES:

• Thorough understanding and experience in Data Lake, Lake House, and Data Warehousing Architecture. Should be able to suggest, architect, and implement a Data Lake/Lake house/Data Warehouse solution with a set of available cloud tools and programming.
• Hands-on experience and expertise in Advanced SQL, Advanced Python programming, AWS Data Engineering related Services - S3, Redshift, Athena, DynamoDB, IAM, DMS, VPC, and other services, Fivetran, Airflow orchestration, and other open-source tools, CI/CD practices, and Terraform.
• AWS services are extensively used to build Data Lake and integration services. High expertise is required in this area to use existing services and to be able to learn and use new services available.
• Experience in writing reusable complex Python/PySpark scripts for ELT, Business Logic OR APIs. Hands-on development work on all aspects of data analysis, data provisioning, modeling, performance tuning, and optimization.
• Experience in designing and implementing tight network and data security at various levels
• Mastery of relational, NoSQL, or NewSQL database systems. Expertise in working with unstructured, structured, and semi-structured data.
• Build scalable data pipelines for both real-time and batch using best practices in data modeling, ETL/ELT processing using various technology stacks.
• Experience configuring and working on data sharing across AWS and other cloud services.
• Experience with DataOps and a related set of practices, processes, and technologies.
• Experienced in Data Migration and Data Integration. Know the pain points in Data integration across SaaS applications and implement the best solution that fits the organization.
• Constantly monitor operations, tune for better performance and utilization.

OTHER REQUIREMENTS:

• This is a hybrid/onsite position. Bay Area local candidates preferred.
  The expected pay range for this position is $167,306 to $172,000 per annum.
  Stanford University provides pay ranges representing its good faith estimate of what the university reasonably expects to pay for a position. The pay offered to a selected candidate will be determined based on factors such as (but not limited to) the scope and responsibilities of the position, the qualifications of the selected candidate, departmental budget availability, internal equity, geographic location and external market pay for comparable jobs.
  At Stanford University, base pay represents only one aspect of the comprehensive rewards package. The Cardinal at Work website (https://cardinalatwork.stanford.edu/benefits-rewards) provides detailed information on Stanford’s extensive range of benefits and rewards offered to employees. Specifics about the rewards package for this position may be discussed during the hiring process.

PHYSICAL REQUIREMENTS*:

• Constantly perform desk-based computer tasks.
• Frequently sit, grasp lightly/fine manipulation.
• Occasionally stand/walk, use a telephone.
• Rarely writing by hand, lift/carry/push/pull objects that weigh up to 10 pounds.
• Consistent with its obligations under the law, the University will provide reasonable accommodations to applicants and employees with disabilities. Applicants requiring a reasonable accommodation for any part of the application or hiring process should contact Stanford University Human Resources by submitting a contact form.

JOB PURPOSE

Responsible for performing difficult or complex analysis, design and programming involving multi-project leadership and broad responsibility in support of new and existing Data Engineering Projects and production support activities.

CORE DUTIES:

• Collaborate with IT and business partners to devise a data strategy that caters to Stanford requirements.
• Deep understanding and commitment to software engineering principles/processes (e.g. Lean, Agile, DevOps, CI/CD) and continuous improvement through measurement
• Thorough knowledge, expertise, and practice Data Management Framework to design world-class data stores. Best practices, Data Quality, and security are critical.
• Understand data endpoints, consumers, and develop strategy.
• Fluid end-to-end data vision, design pipelines for seamless data flow.
• Lead and perform the design, development, implementation, and maintenance of complex Data Store/ Data Lake/Lake house and Data warehousing systems and data-intensive solutions that are scalable, optimized, and fault-tolerant.
• Design and implement Data Migration and Data Integration across cloud and hybrid environments.
• Mastery and hands-on experience with Data Engineering technologies and scripting languages. Identify new technologies and provide recommendations to Management.
• Solid understanding and experience in Cloud technologies and applications. Data Migration, Integration, API’s development, Data Streaming (Batch and continuous), and scheduling.
• Data Modeling skills. Able to come up with a Canonical Data Model and simplify data flow and interaction between different applications. Should be able to integrate new sources smoothly.
• Ability to translate complex functional and technical requirements into detailed architecture, design, and high-performing software.
• Design, build, and optimize pipelines for data collection for storage, access, and analytics.
• Out-of-the-box thinking to overcome engineering challenges with innovative design principles.