Python Developer – Web Scraping & Automation (WFH)
Experience: 3-5 Years
Location: Remote
Mode of Engagement: Full-time
No of Positions: 3
Educational Qualifications: Bachelor’s degree in Computer Science / Information Technology
Industry: IT / Software Development
Notice Period: Immediate Joiners Preferred
What We Are Looking For:
- Strong hands-on experience in Python for large-scale web scraping and automation.
- Expert in web scraping using Scrapy, BeautifulSoup, Selenium, Playwright, and Requests.
- Excellent understanding of HTML, CSS, DOM structure, and browser-rendered content.
- Experience in backend/API development using FastAPI, Django, or Flask.
- Ability to handle proxies, user-agent rotation, cookies, sessions, and anti-bot mechanisms.
- Strong debugging, error-handling, and optimization skills.
- Ability to independently deliver reliable and scalable data extraction solutions.
Key Responsibilities:
- Develop, optimize, and maintain web scraping scripts and automation pipelines in Python.
- Extract data from static, dynamic, and JavaScript-heavy websites using browser automation.
- Manage pagination, authentication of workflows, cookies, sessions, and rendering challenges.
- Build and maintain REST APIs to support data access and processing.
- Implement end-to-end pipelines for data cleaning, transformation, and storage (CSV, JSON, DB).
- Work with SQL/NoSQL databases to store and manage scraped datasets.
- Use Docker for containerization and deploy scraping tools to cloud environments.
- Troubleshoot scraping failures, automate recovery, and ensure high data accuracy.
- Maintain logs, monitoring, and documentation for scraping workflows.
Qualifications:
- Strong programming experience in Python.
- Proficiency with Scrapy, Beautiful Soup, Selenium, Playwright, and Requests.
- Understanding HTTP, APIs, headers, cookies, sessions, and web protocols.
- Experience with backend frameworks: FastAPI / Django / Flask.
- Experience working with Docker and basic CI/CD workflows is good to have.
- Knowledge of relational and non-relational databases.
- Strong debugging, optimization, and troubleshooting abilities.