Skip to content

Warn instead of crashing when scraper encounters unknown enum values#31

Open
woofer300 wants to merge 2 commits into
mainfrom
chris/scraper-crash-fix
Open

Warn instead of crashing when scraper encounters unknown enum values#31
woofer300 wants to merge 2 commits into
mainfrom
chris/scraper-crash-fix

Conversation

@woofer300
Copy link
Copy Markdown
Member

@woofer300 woofer300 commented May 20, 2026

Overview

The scraper was crashing when Cornell's API changed and returned an unrecognized value (which last time was a new payment method), leaving menus out of date. Unknown enum values now leave warnings instead of crashing the scraper, and fall back to a default value.

Changes Made

  • mappers.ts: replaced specific errors with console.warn and default values
    • mapPaymentMethod (default to CARD)
    • mapEventType (default to GENERAL)
    • mapEateryType (default to GENERAL)
    • mapCampusArea (default to CENTRAL)
  • scraper.ts: errors on individual eatery don't crash the whole thing, instead, they are skipped and the failure is logged

Summary by CodeRabbit

  • Bug Fixes
    • Enhanced data processing resilience. The system now gracefully handles invalid values and transformation errors by logging warnings and continuing with valid data, rather than failing the entire operation.

Review Change Stack

@woofer300 woofer300 requested review from stef-rivera and yufanhao May 20, 2026 06:44
@woofer300 woofer300 self-assigned this May 20, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 20, 2026

Warning

Rate limit exceeded

@woofer300 has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 42 minutes and 35 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 71ba90a0-41bb-4ef5-b008-d6adbd214ee3

📥 Commits

Reviewing files that changed from the base of the PR and between 6a6d0db and f20ff09.

📒 Files selected for processing (1)
  • prisma/scraper.ts
📝 Walkthrough

Walkthrough

This PR shifts the error-handling strategy in data transformation and scraping from fail-fast (throw on first unknown value) to graceful degradation (log warning, use sensible default, continue processing). Four enum mapper functions now return fallback values when encountering unknown inputs, and the scraper orchestration allows partial success by logging skipped eateries instead of aborting.

Changes

Error handling strategy: from fail-fast to graceful degradation

Layer / File(s) Summary
Mapper enum fallback handling
prisma/mappers.ts
mapCampusArea, mapPaymentMethod, mapEateryType, and mapEventType replace error throws with console.warn calls and return defaults (CENTRAL, CARD, GENERAL, GENERAL) on unknown enum values.
Scraper partial result handling
prisma/scraper.ts
transformEateriesConcurrently logs a warning when transformation errors occur and allows successfully transformed eateries to be returned instead of aborting.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

🐰 With warnings logged in place of screams,
The pipeline flows through broken dreams,
No crashes now when values hide,
Just sensible defaults, side by side.
Graceful fades where once stood walls,
Progress wins when something falls. 🌱

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: replacing error-throwing behavior with warnings when the scraper encounters unknown enum values.
Description check ✅ Passed The description includes Overview and Changes Made sections that explain the problem and specific modifications, though it lacks Test Coverage and Next Steps sections from the template.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chris/scraper-crash-fix

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@prisma/scraper.ts`:
- Around line 387-390: After the transform step that currently logs `Skipped
${errors.length} API eatery(ies) due to transform errors`, add a hard-stop guard
that computes the failure rate (errors.length divided by the total number of API
eateries attempted in the transform) and aborts the run when failures are total
or near-total (e.g., failureRate === 1.0 or failureRate >= 0.9). Implement this
by deriving totalAttempts from the array of API eateries passed into the
transform step, calculating failureRate = errors.length / totalAttempts, and
throwing an Error (or calling process.exit(1)) with a clear message if the
threshold is met to prevent proceeding to any destructive snapshot deletes;
otherwise keep the existing warning and continue.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f0bf18c0-c425-4ec4-9443-fa66cf581ba0

📥 Commits

Reviewing files that changed from the base of the PR and between 956569c and 6a6d0db.

📒 Files selected for processing (2)
  • prisma/mappers.ts
  • prisma/scraper.ts

Comment thread prisma/scraper.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant