Director of Software Engineering (Node.js & Web ...

Director of Software Engineering (Node.js & Web Scraping Expert)Join to apply for the Director of Software Engineering (Node.js & Web Scraping Expert) role at PortProContinue with Google Continue with GoogleDirector of Software Engineering (Node.js & Web Scraping Expert)Join to apply for the Director of Software Engineering (Node.js & Web Scraping Expert) role at PortProWe are seeking a Director of Software Engineering with deep expertise in Node.js development and large-scale web scraping. This role will lead the engineering team, designing and optimizing high-performance, distributed web scraping systems. The ideal candidate has extensive experience in handling anti-bot measures, data pipeline optimization, and scalable cloud-based architectures.Key Responsibilities- Software Engineering & Web Scraping Leadership:Architect, develop, and maintain scalable and distributed web scraping systems using Node.jsDesign and implement data extraction pipelines to process large volumes of structured and unstructured dataDevelop solutions to bypass anti-bot mechanisms, including CAPTCHA handling, session management, fingerprinting, and IP rotationOptimize scraping processes for performance, reliability, and efficiency while managing proxy services(residential, datacenter, rotating).Oversee data storage and processing strategies, ensuring high availability and consistencyCollaborate with Product, DevOps, and Data Science teams to integrate extracted data into analytics and business applicationsImplement best practices for microservices, API integrations, and real-time data streamingKey Responsibilities- Scalability, Security & DevOps:Lead the transition to cloud-native, containerized, and serverless architectures for web scrapingEnsure compliance with legal and ethical standards (robots.txt, GDPR, CCPA, etc.).Optimize cloud resources (AWS, GCP, or Azure) to support high-throughput scrapingManage real-time monitoring and alerting systems to detect scraping failures, IP bans, or performance bottlenecksWork closely with DevOps teams to optimize CI/CD pipelines, automated deployments, and system scalabilityKey Repsonsibilities- Engineering Team Management & Strategy:Lead, mentor, and grow a high-performance engineering teamDefine and execute the technology roadmap, aligning with business objectivesFoster a culture of continuous learning, collaboration, and innovationImplement agile development methodologies (Scrum, Kanban) to optimize project executionEnsure code quality, security, and best practices across all engineering effortsQualifications & Experience- Technical Expertise:10+ years of experience in software engineering, with at least 5+ years in web scraping and large-scale data extractionStrong hands-on expertise in Node.js, Puppeteer, Playwright, Cheerio, Selenium, and headless browser automationExtensive experience in handling CAPTCHAs, IP rotation, session management, and anti-bot evasion techniquesDeep knowledge of proxy management (residential, datacenter, rotating, and VPNs).Experience with NoSQL/SQL databases (MongoDB, PostgreSQL, Redis, Elasticsearch, etc.)Familiarity with data processing frameworks (Kafka, RabbitMQ, Spark, Airflow, etc.).Strong experience with CI/CD, containerization (Docker, Kubernetes), and cloud deployment (AWS/GCP/Azure)Qualifications & Experience- Leadership & Soft Skills:Proven track record of scaling engineering teams and leading complex projectsStrong problem-solving and debugging skills, especially for scraping challenges and performance bottlenecksExcellent communication and stakeholder management skillsPassion for mentorship, team development, and continuous learningPreferred Qualifications:Experience with machine learning for data extraction and NLPKnowledge of browser fingerprinting and bot detection mechanismsFamiliarity with enterprise-scale web crawling frameworks (Scrapy, Colly, Apify, etc.)Prior leadership experience in data-driven businesses or web scraping startupsSeniority levelSeniority levelNot ApplicableEmployment typeEmployment typeFull-timeJob functionJob functionEngineering and Information TechnologyIndustriesTruck TransportationReferrals increase your chances of interviewing at PortPro by 2xSign in to set job alerts for “Director of Software Engineering” roles.Continue with Google Continue with GoogleContinue with Google Continue with GoogleDirector of Engineering, Operations Tools SuiteDirector of Engineering, Fleet Management Delivery PlatformLos Angeles Metropolitan Area $210,000 - $250,000 17 hours agoSr. Director , Security Software EngineeringBeverly Hills, CA $280,000 - $300,000 1 day agoLos Angeles Metropolitan Area 2 weeks agoLos Angeles, CA$119,000.00-$195,000.002 weeks agoSenior Software Engineering Manager - REMOTELos Angeles, CA $160,000 - $180,000 11 hours agoSr. Manager, IT Software Implementation - USA RemoteSr. Manager, IT Software Implementation - USA RemoteSr. Director , Security Software EngineeringBeverly Hills, CA $280,000 - $300,000 2 days agoWe’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr

Created: 2025-09-17