Codebase git mirroring #718
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sgfost
force-pushed
the
git-mirror
branch
4 times, most recently
from
35de762 to
a12a8c1
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
sgfost
force-pushed
the
git-mirror
branch
2 times, most recently
from
ca4ade6 to
4be9693
Compare
sgfost
force-pushed
the
git-mirror
branch
2 times, most recently
from
6b4e25f to
5a356fb
Compare
* fix release ordering to sort by semantic version number rather than by string
this API is responsible for managing a local git repository mirror for a comses codebase. PUBLIC release archives are commits/tags in the history `build()` and `append_releases()` are the two main API methods which construct (or rebuild) a git repo and add new releases to the repo, respectively
* indicate that ordered releases method on codebase now returns a list and add public_releases() which returns a queryset
currently these are not retroactively inserted into archive packages since that would require rebuilding everything. Generating git repos, however, will add them if they are missing ** includes an experimental refactor of metadata transformations which is used to implement the citation file format generation resolves comses/planning#234
currently a synchronous process with 0 error handling * add file size checking to the git repo fs api
this allows for retries regardless of where the failure occurred the main points are: - only build the git repo if it doesn't exist - only create the github repo if it doesn't exist - pushing changes that exist remotely is already handled gracefully
https://huey.readthedocs.io/ The huey service is essentially a mirror of the server with a connection to the same db and redis service that runs the huey consumer process **setup is currently only for development**
* now only attempt to create github releases if they don't already exist * fixed bug where local_releases were being overwritten with only the last updated releases instead of adding to the set
this is considerably easier to manage than creating a clone service, not sure if there is any potential downside to not isolating the two processes
checks that repo names are valid and available TODO: add a /github page for more in depth information about the feature and a summary in the action modal
will need to really clean up metadata coversions, especially before adding syncing but the one-way transformers idea likely won't hold up ideas for a better approach: - pydantic for validation/structuring - codemeta as an intermediate format
adding these manually was an easily forgotten step that wouldn't be noticed in dev but would fail to build in prod
mirroring strategy is not currently planned to be used with user repositories * use absolute url in github description
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
part 1 (1-way mirror):
adds a button that allows model submitters to create an auto-updating, read-only git repository archive which is hosted on a central organization
additions
CITATION.cffandLICENSEfiles in archive packages (resolves comses/planning#234)CITATION.cffis translated from codemetaLICENSEfile is built from license text templates in theLicensemodellibrary.fs.CodebaseGitRepositoryApi: functionality for building/updating a git repository from aCodebaselibrary.github.GithubApi: provides an interface overPyGithubfor interacting with repositories on githublibrary.github.GithubRepoNameValidator: providesvalidate()to make sure a repo name is valid and unusedhueyfor async task processing which runs on the server container (resolves comses/planning#231)mirror_codebase()andupdate_mirrored_codebase()huey tasks which call theCodebaseGitRepositoryApito build the git repo on the file system and thenGithubApito create/push to the remote/github/configuration steps
comses-model-libraryorganization with the following permissions:.env.envsecrets/