Yelp Analysis

Big data stories behind every star, check-in, and storefront.

An end-to-end Spark analytics project on the Yelp Open Dataset - market structure, customer language, weather mood, crime context, and review-trust signals.

Browse result charts View analysis tracks

Executive overview

What this project does

Built by the SCUI Big Data team, this repository turns the Yelp Open Dataset into reproducible analytics: domain notebooks, PySpark scripts, and publication-ready CSV + chart artifacts under results/. Work spans two tracks - core data analysis and external data enrichment - so findings sit in real-world context, not just platform metrics.

2analysis tracks
10domain modules
90generated charts
4team roles

People

The team behind the pipeline

Cross-functional ownership across product direction, analytics engineering, NLP, and quality assurance.

  • Iftekhar Alam Fahim

    Project Manager, Data Analyst, and Front-End Engineer

  • Ridoy Kumar Adhikary

    Data Analyst and Back-End Engineer

  • Rafi Ahmed

    Data Analyst and NLP/AI Engineer

  • MD Mayenaz Rasul Faraz

    Data Analyst and Quality Assurance (QA) Engineer

Workstreams

Two tracks, one dataset story

Core modules answer what happens inside Yelp. Enrichment modules ask why - by joining weather, crime, location risk, and manipulation signals.

Track 01

Data Analysis

Business, check-in, rating, review, user, and comprehensive cross-entity studies that map market structure and customer behavior.

  • Business Analysis - 13 scripts
  • Check-in Analysis - 6 scripts
  • Comprehensive Analysis - 3 scripts
  • Rating Analysis - 6 scripts
  • Review Analysis - 12 scripts
  • User Analysis - 11 scripts

Track 02

Data Enrichment

External and open-world context layered onto Yelp outcomes - weather mood, cursed storefronts, crime zones, and review-trust forensics.

  • Weather-Mood Analysis - 4 scripts
  • Cursed Storefronts & Attribution - 6 scripts
  • Review Manipulation - 7 scripts
  • Open-World Crime Enrichment - 9 scripts

Modules

What each domain investigates

Module Focus
Business Chain dominance, city/state hubs, category synergies, satisfaction gaps, and turnaround candidates.
Check-in Foot-traffic timing, growth rates, and engagement rhythms across venues.
Rating Star distributions, weekly patterns, cuisine/city differentials, nightlife weekend effects.
Review Volume trends, useful/funny/cool votes, bigrams, pain points, length-vs-rating, menu mining.
User Growth cohorts, elite impact, silent users, dining diversity, and rating evolution.
Comprehensive Cross-entity merchant leaders, review conversion, and check-in drop-off diagnosis.
Weather-Mood Extreme weather effects on ratings, check-ins, and comfort-food preference shifts.
Cursed Storefronts Location potential, closure clusters, parking/noise attributes, NLP pain-point attribution.
Review Manipulation 5-star spike detection, ghost accounts, syndicate networks, and fraud vocabulary signatures.
Crime Enrichment Crime-zone survival, ratings, cuisine mix, check-ins, and crime-dining correlation.

Selected results

Highlights from the analyses

A sample of publication findings exported from module notebooks into results/.

Market structure

Starbucks (724) and McDonald’s (703) lead chain density. Philadelphia tops city merchant counts (~14.5k). Restaurants, Food, and Shopping dominate category volume.

Weather mood

Extreme heat posts the lowest average rating (~3.75) with elevated 1-star share. Heavy snow cuts foot traffic by roughly 80% versus normal conditions.

Trust & manipulation

260 businesses flagged for anomalous 5-star spikes. Among suspicious reviews, over half come from zero-fan accounts; persuasive tokens like “definitely” and “recommend” dominate fraud-linked text.

Review language

Useful votes exceed 8.2M platform-wide. Pain-point bigrams recur around wait times and wasted money. 1-star reviews run longer (~134 words) than 5-star reviews (~85).

Location risk

Walkability and closure patterns surface “potentially cursed” and “severely cursed” storefront cohorts, with pain points clustering around bad location and price issues.

Crime context

Restaurant survival and engagement differ across high/medium crime zones; zone-level rating, cuisine, and check-in tables support cross-metric comparison.

Generated from analysis outputs

Result charts

Charts are generated for every analysis from the module CSV results - not the original exported PNG files. Filter by track or module to explore each study.

Data

Dataset & enrichment sources

Primary source is the Yelp Open Dataset - business, review, user, check-in, and tip entities. Enrichment modules join weather series and public urban/crime context where relevant.

Runtime

Technology & workflow

Processing runs on Apache Spark (PySpark) with Apache Zeppelin notebooks (%pyspark). Queries use Spark SQL / DataFrame APIs against Hive-compatible catalogs such as yelp_db.*, yelp.*, and external_data.*.

  1. Explore Open domain notebooks under notebooks/.
  2. Execute Run matching PySpark scripts in src/ once catalog tables are available.
  3. Publish Export CSV and graph artifacts to results/ for reporting and QA review. 
scui-bigdata-03-YelpAnalysis/
├── src/
│   ├── data-analysis/
│   └── data-enrichment/
├── notebooks/
├── results/
└── docs/          ← this GitHub Pages site

Status & roadmap

Active development

The project is in progress. Near-term work continues deepening cross-module integration, standardizing runtime bootstrap docs, and adding a planned conversational Text-to-SQL application when ready.

Academic / educational use, aligned with Yelp Dataset Terms of Use.

Open on GitHub