We are seeking a talented, self-directed Data Engineer to design, develop & implement high-volume, high-performance data structures for our internal customers. Gather business and functional requirements and translate these requirements into robust, scalable, operable solutions that work well within the overall data architecture. The ideal candidate will have excellent analytical abilities, and the ability to synthesize data into crisp and clear recommendations for business and product leaders. Participate in the full development life cycle, end-to-end, from design, implementation and testing, to documentation, delivery, support, and maintenance.
- Develop the end-to-end automation of data pipelines, making datasets readily-consumable by visualization tools and notification systems.
- Build robust and scalable data integration (ETL) pipelines using SQL, Python and Spark
- Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using SQL
- Implement data structures using best practices in data modeling, ETL/ELT processes, and SQL, and Redshift
- Work with developers to bring the machine-readable data into Relational database.
- Build and deliver high quality datasets to support business analysis and customer reporting needs
- Design, implement, and support data pipeline infrastructure making datasets available for presentation via visualization tools.
- Create automated alarming and dash boarding to monitor data integrity.
Qualification & Experience:
- Experience leading large-scale data warehousing and analytics projects, including using AWS technologies – Redshift, S3, EC2, Data-pipeline and other big data technologies
- Bachelor’s degree in Computer Science, Computer Engineering, Mathematics or a related field
- Experience with data modeling, data warehousing, and building ETL pipelines
- Query performance tuning skills using Unix profiling tools and SQL
- Experience in SQL
- Experience leveraging Python or Java to manipulate data and set up automated processes as per business requirements
- Experience in Java/Python, SQL
- Experience with Big Data Technologies (Hadoop, Hive,Pig, Spark, etc.)
- 5+ years of Industry experience as a Data Engineer or Data Scientist or Software Development Engineer with a track record of manipulating, processing, and extracting value from large datasets.
- 3+ years of experience as a Data Engineer or in a similar role
Vacancy Type: Full Time
Job Location: Sunnyvale, CA, US
Application Deadline: N/A