US Infrastructure Specialist - Cloud Data Platform ...

Job Title : KAFKA Admin Location : Conshohocken, PA Duration : 6-12 months Job Description: 5+ years of hands-on experience with Kafka/Confluent in production. Strong expertise with: SSL/TLS end-to-end configuration in Kafka ecosystems RBAC authorization configuration and operational administration Designing for HA/redundancy and scaling for growth Monitoring/alerting with Prometheus & Grafana, plus operational tooling such as New Relic Performance testing and tuning (producers/consumers, brokers, Connect, infrastructure) Demonstrated experience implementing: Confluent Oracle Premium CDC Connector Confluent sink to ADLS Gen2 (ADLS2) Proficiency with Azure DevOps, Git, and building CI/CD pipelines. Working knowledge of Apache Flink and hands-on experience writing Kafka Streams. Key Responsibilities:Security & Access Control: Configure end-to-end SSL/TLS across Kafka/Confluent components and client integrations. Implement and manage RBAC for authorizations, service accounts, and least-privilege access. High Availability, Redundancy & Failover: Configure core components for redundancy and failover resilience (brokers/controllers, Connect, Schema Registry, etc.). Design and implement a Kafka disaster recovery (DR) cluster, including replication strategy, failover testing, and runbooks aligned to RPO/RTO. Scale & Future Growth: Plan and implement platform scalability for future growth (topic/partition strategy, retention, throughput, capacity planning). Establish sustainable operational practices for multi-team usage and governance. Monitoring, Alerting & Operations: Set up monitoring and alerts for streaming messages and platform health using Prometheus & Grafana. Integrate New Relic dashboards/alerts to support operational visibility, incident response, and service health metrics. Performance Engineering: Perform performance testing and tune Kafka/Confluent components for optimal throughput, latency, and stability. Troubleshoot complex production issues across brokers, networking, storage, Connect, and client workloads. Connectors & Data Integration: Implement and support Confluent Oracle Premium CDC Connector (configuration, offsets, schema evolution, error handling, operations). Implement and support Confluent Sink Connector to ADLS2 (Azure Data Lake Storage Gen2) with reliable delivery and partitioning strategies. Streaming Development: Build and support stream processing using Apache Flink (job configuration, deployment patterns, operationalization). Develop Kafka Streams applications (topology design, state stores, exactly-once/processing guarantees as needed). DevOps & Automation: Use Azure DevOps with Git integration for version control, reviews, and change management. Deploy and manage cloud resources using Terraform and Ansible. Build and maintain CI/CD pipelines for platform configuration, connectors, and streaming jobs across environments. Cost Allocation: Support chargeback/showback calculations for Kafka usage (e.g., throughput, storage, partitions, connector/resource utilization) and related reporting. Preferred Qualifications: Experience implementing and testing Kafka DR cluster architectures and operational runbooks. Familiarity with enterprise governance patterns (multi-tenancy, naming standards, quotas, schema governance). Experience defining usage metering to enable reliable chargeback/showback. Additional Expectations: Proficiency with using Linux CLI (preferably RHEL/SLES). Ability to participate in an on-call rotation and provide timely incident support (if required). Strong documentation and stakeholder communication skills across engineering, operations, and product teams. Ability to create design patterns/templates, provide development support and conduct knowledge transfer sessions for onboarding new use cases and modernizing existing ones.

Created: 2026-03-06