Big Data Developer

Big Data Dev/Spark Scala EngineerO Fallon, MO OnsiteFull TimeMust Have Technical/Functional SkillsExperience with Apache Ozone and/or Ceph as storage backends for analytics workloadsExperience implementing exactly once / at least once streaming semanticsStrong background in Spark performance tuning (CPU, memory, I/O, shuffle)Experience supporting mission critical production systems with strict SLAsFamiliarity with CI/CD pipelines and automated testing for data applicationsExperience designing observability for streaming systems (lag, throughput, backpressure)Technical SkillsLanguages: Scala, Python (PySpark), SQLBig Data: Apache Spark (Core, SQL, Structured Streaming)Streaming: KafkaIngestion / Orchestration: Apache NiFiStorage: Apache Ozone, Ceph, object storage conceptsOS & Tooling: Linux, Git, CI/CD, monitoring and logging toolsRequired QualificationsBachelor's degree in Computer Sci ence, Engineering, or equivalent practical experienceStrong hands on experience with Apache Spark in production environmentsAdvanced proficiency in Scala and PySparkSolid understanding of distributed systems and data processing at scaleStrong experience with Kafka based streaming architecturesHands on experience with Spark Structured StreamingExperience building batch and real time pipelinesHands on experience with Apache NiFi for data ingestion and flow managementStrong SQL skills and experience working with structured and semi structured dataExperience working with object storage or distributed storage platformsProficiency with Linux, shell scripting, and Git based version controlRegards,Shweta Sharmashweta.sharma@MSR Technology Group LLC

Created: 2026-05-16