Senior Big Data Solutions Designer (GCP)

As a GCP Architect at Tredence, your role will involve working on the collecting, storing, processing, and analyzing of huge sets of data. Your primary focus will be on choosing optimal solutions for these purposes, maintaining, implementing, and monitoring them, and integrating them with the architecture used across our clients. **Key Responsibilities:** - Select and integrate any Big Data tools and frameworks required to provide requested capabilities. - Develop and maintain data pipelines implementing ETL processes, monitor performance, and advise on necessary infrastructure changes. - Translate complex technical and functional requirements into detailed designs. - Investigate and analyze alternative solutions to data storing, processing, etc. to ensure the most streamlined approaches are implemented. - Serve as a mentor to junior staff by conducting technical training sessions and reviewing project outputs. **Qualifications Required:** - Solid understanding of data warehousing and data modeling techniques. - Proficient understanding of distributed computing principles such as Hadoop v2, MapReduce, and HDFS. - Strong data engineering skills on GCP cloud platforms like Airflow, Cloud Composer, Data Fusion, Data Flow, Data Proc, and Big Query. - Experience with building stream-processing systems using solutions such as Storm or Spark-Streaming. - Good knowledge of Big Data querying tools like Pig, Hive, and Impala. - Experience with Spark, SQL, and Linux. - Knowledge of various ETL techniques and frameworks such as Flume, Apache NiFi, or DBT. - Experience with various messaging systems like Kafka or RabbitMQ. - Good understanding of Lambda Architecture, along with its advantages and drawbacks. At Tredence, you will have the opportunity to work with some of the smartest, friendliest, and hardest-working people in the data analytics space. You will work with the latest technologies and interface directly with key decision-makers at our clients, some of the largest and most innovative businesses in the world. We offer a 401k match, full medical, dental, and vision benefits, a fun team atmosphere, and work-life balance. Our people are our greatest asset, and we value every one of them. Come see why we are so successful in one of the most competitive and fastest-growing industries in the world. *Tredence 5 core values:* - Think Impact - Be Entrepreneurial - Constantly curious - Do the right thing - Pursue Excellence Tredence is an equal opportunity employer, celebrating and supporting diversity and committed to creating an inclusive environment for all employees. Visit our Website, YouTube page, LinkedIn page for more details. As a GCP Architect at Tredence, your role will involve working on the collecting, storing, processing, and analyzing of huge sets of data. Your primary focus will be on choosing optimal solutions for these purposes, maintaining, implementing, and monitoring them, and integrating them with the architecture used across our clients. **Key Responsibilities:** - Select and integrate any Big Data tools and frameworks required to provide requested capabilities. - Develop and maintain data pipelines implementing ETL processes, monitor performance, and advise on necessary infrastructure changes. - Translate complex technical and functional requirements into detailed designs. - Investigate and analyze alternative solutions to data storing, processing, etc. to ensure the most streamlined approaches are implemented. - Serve as a mentor to junior staff by conducting technical training sessions and reviewing project outputs. **Qualifications Required:** - Solid understanding of data warehousing and data modeling techniques. - Proficient understanding of distributed computing principles such as Hadoop v2, MapReduce, and HDFS. - Strong data engineering skills on GCP cloud platforms like Airflow, Cloud Composer, Data Fusion, Data Flow, Data Proc, and Big Query. - Experience with building stream-processing systems using solutions such as Storm or Spark-Streaming. - Good knowledge of Big Data querying tools like Pig, Hive, and Impala. - Experience with Spark, SQL, and Linux. - Knowledge of various ETL techniques and frameworks such as Flume, Apache NiFi, or DBT. - Experience with various messaging systems like Kafka or RabbitMQ. - Good understanding of Lambda Architecture, along with its advantages and drawbacks. At Tredence, you will have the opportunity to work with some of the smartest, friendliest, and hardest-working people in the data analytics space. You will work with the latest technologies and interface directly with key decision-makers at our clients, some of the largest and most innovative businesses in the world. We offer a 401k match, full medical, dental, and vision benefits, a fun team atmosphere, and work-life balance. Our people are our greatest asset, and we value every one of them. Come see why we are so successful in one of the most competitive and fastest-growing industries in