Personal Tool Web Scraping Full-Stack

EasierAvenue

A StreetEasy tracker that scrapes new NYC listings and scores them, so I'm not the one refreshing a search five times a day.

Role
Solo builder
Timeline
2025 — ongoing
Stack
Next.js, Prisma/Postgres, Python
Platform
Web, deployed on Railway

Why I built this

Apartment hunting in NYC is basically its own part-time job. I got tired of refreshing StreetEasy hoping something decent in my price range and neighborhood would show up before someone else grabbed it, so I built something to do that refreshing for me.

Starting simple

The first version was about as basic as it gets: a Python script that scraped StreetEasy on an interval and emailed me when something new matched my filters, plus a small Flask page so I could change the neighborhood, price range, and bedroom count without touching the code. It ran in a Docker container, and honestly that was enough for a while.

Rebuilding it properly

Eventually an email with a link in it wasn't enough. I wanted to actually browse listings, see how long they'd been up, and get a sense of which ones were worth a call. That meant rebuilding it as a real Next.js app backed by Postgres through Prisma, deployed on Railway.

Along the way I ran into the stuff you'd expect from scraping a site that doesn't want to be scraped — StreetEasy started throwing 403s, so I added proper browser fingerprint headers and eventually routed requests through ScraperAPI when that wasn't enough. I also had to move off SQLite once more than one instance was running, since a file on disk doesn't survive a redeploy and definitely doesn't handle two scrapes writing at the same time.

What it does now

Every listing gets a first-seen and last-seen timestamp, so I can tell what's actually new versus what's just still sitting there. There's also a scoring layer, one part compares the price and price-per-square-foot against the neighborhood's median, the other looks at distance and walk time to the nearest subway station. Between the two I get a rough 1 to 10 sense of whether a listing is worth clicking into, instead of reading all of them myself.

What's next

  • Show a "new" badge based on first_seen_at instead of me eyeballing dates
  • Tune the scoring model now that I have more real listings to test it against
  • Look into whether a Redis cache in front of Postgres is worth it as query volume picks up