Big Data Architect

As a Data Architect at Persistent, you will play a crucial role in guiding the company into the future by utilizing the latest technology and information management methodologies to meet requirements for effective logical data modeling, metadata management, and database warehouse domains. Your responsibilities will include defining data retention policies, monitoring performance and advising necessary infrastructure changes, mentoring junior engineers, working with other architects to deliver best-in-class solutions, implementing ETL / ELT processes, orchestrating data flows, and recommending and driving adoption of newer tools and techniques from the big data ecosystem. Key Responsibilities: - Define data retention policies - Monitor performance and advise necessary infrastructure changes - Mentor junior engineers and collaborate with other architects to deliver top-notch solutions - Implement ETL / ELT processes and orchestrate data flows - Recommend and drive adoption of newer tools and techniques from the big data ecosystem Qualifications Required: - 10+ years of industry experience in building and managing big data systems - Experience in building, monitoring, and optimizing reliable and cost-efficient pipelines for SaaS - Proficiency in building stream-processing systems using solutions such as Storm or Spark-Streaming - Familiarity with integrating and dealing with data storage systems like SQL and NoSQL databases, file systems, and object storage like S3 - Experience with reporting solutions like Pentaho, PowerBI, Looker, including customizations - Proficiency and expertise in Cloudera / Hortonworks, Spark, HDF and NiFi, RDBMS, NoSQL databases like Vertica and Redshift, data modeling with physical design and SQL performance optimization - Knowledge of messaging systems such as JMS, Active MQ, Rabbit MQ, Kafka - Skilled in big data technology like Hadoop, Spark, NoSQL-based data-warehousing solutions - Experience with data warehousing, reporting including customization, Hadoop, Spark, Kafka, Core Java, Spring/IOC, design patterns - Familiarity with big data querying tools like Pig, Hive, and Impala, open-source technologies and databases (SQL & NoSQL), distributed computing principles, solving ongoing cluster issues, scaling data pipelines using open-source components and AWS services - Proficiency in cloud (AWS), provisioning, capacity planning, and performance analysis at various levels - Experience in web-based SOA architecture implementation with design pattern experience About Persistent: Persistent is an AI-led, platform-driven Digital Engineering and Enterprise Modernization partner that combines deep technical expertise and industry experience to help clients anticipate whats next. With a disruptors mindset, commitment to client success, and agility to thrive in a dynamic environment, Persistent has sustained growth momentum and received accolades in various prestigious awards. The company focuses on diversity and inclusion to unlock everyone's potential and fosters an inclusive environment where employees can thrive and unleash their full potential. As a Data Architect at Persistent, you will play a crucial role in guiding the company into the future by utilizing the latest technology and information management methodologies to meet requirements for effective logical data modeling, metadata management, and database warehouse domains. Your responsibilities will include defining data retention policies, monitoring performance and advising necessary infrastructure changes, mentoring junior engineers, working with other architects to deliver best-in-class solutions, implementing ETL / ELT processes, orchestrating data flows, and recommending and driving adoption of newer tools and techniques from the big data ecosystem. Key Responsibilities: - Define data retention policies - Monitor performance and advise necessary infrastructure changes - Mentor junior engineers and collaborate with other architects to deliver top-notch solutions - Implement ETL / ELT processes and orchestrate data flows - Recommend and drive adoption of newer tools and techniques from the big data ecosystem Qualifications Required: - 10+ years of industry experience in building and managing big data systems - Experience in building, monitoring, and optimizing reliable and cost-efficient pipelines for SaaS - Proficiency in building stream-processing systems using solutions such as Storm or Spark-Streaming - Familiarity with integrating and dealing with data storage systems like SQL and NoSQL databases, file systems, and object storage like S3 - Experience with reporting solutions like Pentaho, PowerBI, Looker, including customizations - Proficiency and expertise in Cloudera / Hortonworks, Spark, HDF and NiFi, RDBMS, NoSQL databases like Vertica and Redshift, data modeling with physical design and SQL performance optimization - Knowledge of messaging systems such as JMS, Active MQ, Rabbit MQ, Kafka - Skilled in big data