This repository was archived by the owner on Jul 16, 2025. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 0
Create Repositories Table and Implement Background Syncing Job #48
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mauricevogel
reviewed
Sep 9, 2024
Contributor
mauricevogel
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just had a brief look over the campaigns controller and here is already some feedback related to it!
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Labels
database
implications for database - structure and otherwise
enhancement
New feature or request
testing
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue Summary:
This PR addresses the need to refactor our application by introducing a repositories table in the database to store GitHub repositories associated with users. This enhancement will provide a consistent and reliable dataset for repository information, enabling improved performance and more robust features. Additionally, we are implementing a background job to keep the data in this table synchronized with the latest updates from the GitHub API.
Reason for Change:
Currently, our application fetches repository data directly from the GitHub API whenever needed. While functional, this approach presents several challenges:
Changes:
Create Repositories Table:
Introduce a new repositories table to store essential data about each repository:
id: Unique identifier for the repository.
full_name: Full name of the repository (e.g., user/repo-name).
name: Name of the repository.
owner_login: GitHub username of the repository owner.
html_url: URL to the repository on GitHub.
description: Brief description of the repository.
private: Boolean indicating whether the repository is private.
fork: Boolean indicating whether the repository is a fork.
created_on github_at, updated_on_github_at, pushed_to_github_at: Timestamps related to the repository, adjusted column names as to not conflict with the automatically created columns
created_atandupdated_at.Modify Associations:
Update the User model to reflect the new has_many :repositories association.
Update the Campaign model to include a belongs_to :repository association.
Ensure consistency in the Contribution model's relationships, with careful handling of dependent associations.
Implement Background Job for Syncing Data:
Add the
sidekiqgem, in conjunction withredisIntroduce a background job with that periodically syncs the repository data in the database with the latest information from GitHub.
Sync Frequency: Setting the sync frequency to every day at midnight(up for iterative change, might later implement the option for users to manually trigger a sync for immediate updates).
The job will:
Fetch the latest repository data for each user.
Update existing records in the repositories table.
Add new repositories that the user may have created.
Handle deletions or modifications to repositories.
Refactor Existing Code:
Update controllers and helpers to utilize the new repositories table instead of fetching data directly from the GitHub API.
Files to be updated include:
HomeController: Modify how repository data is fetched and displayed.
GithubApiHelper: Refactor to support both direct API calls and database interactions.
User model: Update associations to reflect the new relationship with repositories.
Additional models, controllers, or helpers that interact with repository data.
Tasks:
Expected Outcome:
closes #46