Dibya Sarothi
Dibya Sarothi
Automation & BotsProduction ReadySeptember 2026

Data Entry Automation Bot

Real Estate Listing Scraper & Automated Google Forms/Sheets Pipeline

Combines BeautifulSoup HTML extraction, regex price normalization, and Selenium WebDriver form submission into Google Sheets.

Data Entry Automation Bot overview
Project Overview

An end-to-end web scraping and automated data entry pipeline built with Python, Requests, BeautifulSoup4, and Selenium WebDriver. Scrapes rental listings, sanitizes messy currency/bedroom strings with regular expressions, and populates Google Forms sequentially into a live Google Sheet.

Problem Statement

Real estate portals mix pricing formatting, frequency suffixes (/mo), and unit metadata inside raw text blocks unsuitable for spreadsheet sorting.

Engineering Rationale

Combining lightweight HTTP HTML parsing for extraction with Selenium solely for interactive Google Form submission optimizes speed and reliability.

Engineering Objectives

Fetch and parse rental cards with BeautifulSoup data-test attribute selectors.
Normalize property addresses, cleaned integer prices, and links via regex sanitization.
Automate multi-listing submission into Google Forms for live Google Sheets aggregation.
Architecture Blueprint

System Architecture

Two-stage ETL pipeline: fast Requests + BeautifulSoup extraction & regex cleaning followed by Selenium formsolving class submission.

01

Hybrid HTTP + Selenium ETL

Uses fast static Requests/BS4 for bulk card extraction before opening Chrome WebDriver only for Google Forms submission.

Core & Backend Implementation

  • BeautifulSoup4 card parser targeting structured Zillow-clone attributes.
  • Regex normalizer stripping currency symbols, '+ 1 bd', and '/mo' suffixes.
  • Selenium formsolving class filling text inputs and advancing 'Submit another response' loops.

Client & Interface Layer

  • Automated Google Forms submission feeding a live Google Sheets ledger.

Data & Schema Design

Engine / Tier: Google Sheets Response Table

Rows structured by Address, Normalized Monthly Rent, and Listing URL.

Core Models & Entities
PropertyListing

Authentication & Security Controls

Auth Model: Target URLs and Google Form endpoints configured via .env
  • Keeps form submission endpoints isolated outside version control.

Engineering Challenges & Technical Decisions

Key Challenges Solved

Challenge: Inconsistent rent strings (e.g., '$2,895+ 1 bd' vs '$2,450/mo').
Engineering Solution: Applied targeted regex pattern extraction and comma stripping to yield clean numeric rents.

Trade-Off Decisions

Decision: Requests + BS4 Extraction Before Selenium Entry
Architectural Rationale: Significantly faster and less brittle than scraping pagination solely inside Chrome.

Testing & Verification Methodology

Frameworks & Approach: Dataset Length Alignment Verification
  • Verified 1:1 count alignment across scraped addresses, prices, and URLs before form submission.

Results & Engineering Takeaways

Learned hybrid ETL design pairing fast HTML parsers with browser automation only where client JS requires it.