JD - Kafka Engineer

Download as pdf or txt
Download as pdf or txt
You are on page 1of 2

Job Title: Kafka Engineer

Location: UK (Hybrid)

Type of Contract: Inside IR35

Contract Duration: Long Term

Rate: Open to Discuss

Job Description:

Key Qualities

 Must be able to communicate on technical levels with Engineers and stakeholders.

 Strong problem solving an analytical skill.

 Experience operating in an infrastructure as code and automation first principles


environment.

 Understand the software development lifecycle process, specifically the standard


engineering toolsets available to the team within CTO to design and develop software.

Key Responsibilities

 Work alongside product owners, technical leads and customers to strategize 3–5-year plans
for streaming software.

 Produce and maintain product and platform architecture in line with the central architecture
governance functions.

 Implement patterns that provide customers with robust deployment patterns meeting HSBC
service levels tiers.

 Ensure that platform and product architectures remain cost effective.

 Standing up and administer On-Prem clusters.

 Provide expertise in Kafka brokers, zookeepers, Kafka connect, schema registry, KSQL, Rest
proxy and Kafka Control center.

 Provide expertise and hands on experience working on Kafka connect using schema registry
in a very high volume.

 Provide administration and operations of the Kafka platform - provisioning, access lists
Kerberos and SSL configurations. Provide expertise and hands on experience working on
Kafka connectors such as MQ connectors, Elastic Search connectors, JDBC connectors, File
stream connector, JMS source connectors, Tasks, Workers, converters, Transforms.
 Provide expertise and hands on experience on custom connectors using the Kafka core
concepts and API.

 Involve in design and capacity review meetings to provide suggestions in Kafka usage.

 Participate in work planning and estimation to ensure optimum performance, high


availability, and stability of solutions.

 Create topics, setup redundancy cluster, deploy monitoring tools, alerts and has good
knowledge of best practices. Create stubs for producers, consumers, and consumer groups
for helping onboard applications from different languages/platforms.

 Use automation tools like provisioning using Docker, Jenkins, and GitLab.

 Setting up security on Kafka, Monitor, prevent and troubleshoot security-related issues.

 Ability to perform data related benchmarking, performance analysis and tuning.

Key Technologies

Messaging technologies –Apache Kafka, Confluent Kafka

DevOps toolsets – GitHub, JIRA, Confluence, Jenkins

Automation – Ansible, Puppet

Monitoring –Observability tools such as DataDog, NewRelic, Prometheus, Grafana

You might also like