Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use ghfs for AnalyzeGithubContent #581

Merged
merged 3 commits into from
May 6, 2024
Merged

Use ghfs for AnalyzeGithubContent #581

merged 3 commits into from
May 6, 2024

Conversation

kromiii
Copy link
Contributor

@kromiii kromiii commented May 6, 2024

Problem

Sometimes, fetching a schema file from GitHub fails because the file exceeds the file size limit in GitHub API.

https://docs.github.com/ja/rest/repos/contents?apiVersion=2022-11-28#get-contents

Failed to analyze schema: unsupported content encoding: none, this may occur when file size > 1 MB, if that is the case consider using DownloadContents

Solution

We can avoid this by using ghfs (https://github.com/k1LoW/ghfs). It uses Git Database API, which has no file size limit.

Concern

I would appriciate if you would judge whether we need to rewrite the entire code to accommodate some large files.

Copy link
Owner

@k1LoW k1LoW left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GREAT! Thank you!!

One minor correction, please.

datasource/datasource.go Outdated Show resolved Hide resolved
Co-authored-by: Ken’ichiro Oyama <[email protected]>
@kromiii
Copy link
Contributor Author

kromiii commented May 6, 2024

GREAT! Thank you!!

One minor correction, please.

Thanks!

Copy link
Owner

@k1LoW k1LoW left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@k1LoW k1LoW merged commit f724f6b into k1LoW:main May 6, 2024
3 checks passed
@github-actions github-actions bot mentioned this pull request May 6, 2024
@kromiii kromiii deleted the use-ghfs branch November 10, 2024 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants