Senior DevOps Engineer with Hadoop

Client: - World's Leading Web Scraping company.

 

About the Job -

 

As a DevOps engineer, you'll be responsible for managing our Hadoop/HBase clusters and our

storage backend application that runs on top of these clusters.

 

We are looking for an DevOps engineer to join our Infrastructure team to help, maintain and

improve our technology stack which includes Hadoop, HBase, ELK, Prometheus/Grafana and

many more.

 

Our Hadoop/HBase cluster stores hundreds of terabytes of data and it’s one of the primary

components that makes up our Scrap Cloud product meaning your work will have direct and

visible impact in one of our star products.

 

Being such a key component it’s crucial to ensure data availability and performance when our customers access their data.

 

Roles & Responsibilities:

 

• Deploying and monitoring of the Hadoop/HBase services.
• Storage backend application development with Java and Python.
• Plan and deploy software upgrades, both at application and Hadoop/HBase layers.
• Write tools and scripts to automate common tasks in managing the Hadoop clusters.
• Troubleshooting Hadoop/HBase service issues.
• Help improve monitoring and identify key performance metrics.
• Proactively ensure service runs with minimal disruption.

 

Requirements -

 

• 5+ years’ experience doing DevOps work (deploying, maintaining, monitoring, and application development) with Hadoop/HBase technology, in moderate to large scale clusters.
• Deep understanding of HDFS and HBase internals.
• Experience with Hadoop MapReduce development.
• Solid experience in Linux troubleshooting, tuning, profiling, and monitoring.
• Solid experience with Java and understanding of the JVM.
• Skills in the following programming languages: Python, Bash/Shell scripting.
• Experience with Docker.
• Understanding of load balancing and reverse proxying techniques.

 

Bonus points:

1. Open-source contributions related to HBase or other NoSQL databases.

 

Benefits -

 

By joining the team, you will:

• Become part of a self-motivated, progressive, multi-cultural team.
• Have the freedom & flexibility to work remotely.
• Get the chance to work with cutting-edge open-source technologies and tools.
• Allocated hours for Open-Source contributions.

 

NOTE- Candidate should be flexible to work in China Time Zone.