There are lots of ways to get involved with yt, as a community and as a technical system -- not all of them just contributing code, but also participating in the community, helping us with designing the websites, adding documentation, and sharing your scripts with others.
Coding is only one way to be involved!
There are three main communication channels for yt:
- Many yt developers participate in the yt Slack community. Slack is a free chat service that many teams use to organize their work. You can get an invite to yt's Slack organization by clicking the "Join us @ Slack" button on this page: https://yt-project.org/community.html
- yt-users is a relatively high-traffic mailing list where people are encouraged to ask questions about the code, figure things out and so on.
- yt-dev is a much lower-traffic mailing list designed to focus on discussions of improvements to the code, ideas about planning, development issues, and so on.
The easiest way to get involved with yt is to read the mailing lists, hang out in IRC or slack chat, and participate. If someone asks a question you know the answer to (or have your own question about!) write back and answer it.
If you have an idea about something, suggest it! We not only welcome participation, we encourage it.
The yt documentation is constantly being updated, and it is a task we would very much appreciate assistance with. Whether that is adding a section, updating an outdated section, contributing typo or grammatical fixes, adding a FAQ, or increasing coverage of functionality, it would be very helpful if you wanted to help out.
The easiest way to help out is to fork the main yt repository and make changes
in it to contribute back to the yt-project
. A fork is a copy
of a repository; in this case the fork will live in the space under your
username on github, rather than the yt-project
. If you have never made a
fork of a repository on github, or are unfamiliar with this process, here is a
short article about how to do so:
https://help.github.com/en/github/getting-started-with-github/fork-a-repo .
The documentation for
yt
lives in the doc
directory in the root of the yt git
repository. To make a contribution to the yt documentation you will
make your changes in your own fork of yt
. When you are done,
issue a pull request through the website for your new fork, and we can comment
back and forth and eventually accept your changes. See :ref:`sharing-changes` for
more information about contributing your changes to yt on GitHub.
If you have an image or video you'd like to display in the image or video galleries, getting it included it easy! You can either fork the yt homepage repository and add it there, or email it to us and we'll add it to the Gallery.
We're eager to show off the images and movies you make with yt, so please feel free to drop us a line and let us know if you've got something great!
Contributing code is another excellent way to participate -- whether it's bug fixes, new features, analysis modules, or a new code frontend. See :ref:`creating_frontend` for more details.
The process is pretty simple: fork on GitHub, make changes, issue a pull request. We can then go back and forth with comments in the pull request, but usually we end up accepting.
For more information, see :ref:`contributing-code`, where we spell out how to get up and running with a development environment, how to commit, and how to use GitHub. When you're ready to share your changes with the community, refer to :ref:`sharing-changes` to see how to contribute them back upstream.
Some of these fall under the other items, but if you'd like to help out with the website or any of the other ways yt is presented online, please feel free! Almost everything is kept in git repositories on GitHub, and it is very easy to fork and contribute back changes.
Please feel free to dig in and contribute changes.
If you're using yt and it has increased your productivity, please feel encouraged to share that information. Cite our paper, tell your colleagues, and just spread word of mouth. By telling people about your successes, you'll help bring more eyes and hands to the table -- in this manner, by increasing participation, collaboration, and simply spreading the limits of what the code is asked to do, we hope to help scale the utility and capability of yt with the community size.
Feel free to blog about, tweet about and talk about what you are up to!
There are some out-there ideas that have been bandied about for the future directions of yt -- stuff like fun new types of visualization, remapping of coordinates, new ways of accessing data, and even new APIs to make life easier.
yt is an ambitious project. Let's be ambitious together!
The community of participants in open source Scientific projects is made up of members from around the globe with a diverse set of skills, personalities, and experiences. It is through these differences that our community experiences success and continued growth. We expect everyone in our community to follow these guidelines when interacting with others both inside and outside of our community. Our goal is to keep ours a positive, inclusive, successful, and growing community.
As members of the community,
- We pledge to treat all people with respect and provide a harassment- and bullying-free environment, regardless of sex, sexual orientation and/or gender identity, disability, physical appearance, body size, race, nationality, ethnicity, and religion. In particular, sexual language and imagery, sexist, racist, or otherwise exclusionary jokes are not appropriate.
- We pledge to respect the work of others by recognizing acknowledgment/citation requests of original authors. As authors, we pledge to be explicit about how we want our own work to be cited or acknowledged.
- We pledge to welcome those interested in joining the community, and realize that including people with a variety of opinions and backgrounds will only serve to enrich our community. In particular, discussions relating to pros/cons of various technologies, programming languages, and so on are welcome, but these should be done with respect, taking proactive measure to ensure that all participants are heard and feel confident that they can freely express their opinions.
- We pledge to welcome questions and answer them respectfully, paying particular attention to those new to the community. We pledge to provide respectful criticisms and feedback in forums, especially in discussion threads resulting from code contributions.
- We pledge to be conscientious of the perceptions of the wider community and to respond to criticism respectfully. We will strive to model behaviors that encourage productive debate and disagreement, both within our community and where we are criticized. We will treat those outside our community with the same respect as people within our community.
- We pledge to help the entire community follow the code of conduct, and to not remain silent when we see violations of the code of conduct. We will take action when members of our community violate this code such as contacting [email protected] (all emails sent to this address will be treated with the strictest confidence) or talking privately with the person.
This code of conduct applies to all community situations online and offline, including mailing lists, forums, social media, conferences, meetings, associated social events, and one-to-one interactions.
The yt Community Code of Conduct was adapted from the Astropy Community Code of Conduct, which was partially inspired by the PSF code of conduct.
yt is a community project!
We are very happy to accept patches, features, and bugfixes from any member of the community! yt is developed using git, primarily because it enables very easy and straightforward submission of revisions. We're eager to hear from you, and if you are developing yt, we encourage you to subscribe to the developer mailing list. Please feel free to hack around, commit changes, and send them upstream.
Note
If you already know how to use the git version control system and are comfortable with handling it yourself, the quickest way to contribute to yt is to fork us on GitHub, make your changes, push the changes to your fork and issue a pull request. The rest of this document is just an explanation of how to do that.
See :ref:`code-style-guide` for more information about coding style in yt and :ref:`docstrings` for an example docstring. Please read them before hacking on the codebase, and feel free to email any of the mailing lists for help with the codebase.
Keep in touch, and happy hacking!
If you're interested in participating in yt development, take a look at the issue tracker on GitHub. You can search by labels, indicating estimated level of difficulty or category, to find issues that you would like to contribute to. Good first issues are marked with a label of new contributor friendly. While we try to triage the issue tracker regularly to assign appropriate labels to every issue, it may be the case that issues not marked as new contributor friendly are actually suitable for new contributors.
Here are some predefined issue searches that might be useful:
- Unresolved issues marked "new contributor friendly".
- All unresolved issues.
We provide a brief introduction to submitting changes here. yt thrives on the strength of its communities (https://arxiv.org/abs/1301.7064 has further discussion) and we encourage contributions from any user. While we do not discuss version control, git, or the advanced usage of GitHub in detail here, we do provide an outline of how to submit changes and we are happy to provide further assistance or guidance.
yt is licensed under the BSD 3-clause license. Versions previous to yt-2.6 were released under the GPLv3.
All contributed code must be BSD-compatible. If you'd rather not license in this manner, but still want to contribute, please consider creating an external package, which we'll happily link to.
yt is hosted on GitHub, and you can see all of the yt repositories at https://github.com/yt-project/. To fetch and modify source code, make sure you have followed the steps above for bootstrapping your development (to assure you have a GitHub account, etc.).
In order to modify the source code for yt, we ask that you make a "fork" of the main yt repository on GitHub. A fork is simply an exact copy of the main repository (along with its history) that you will now own and can make modifications as you please. You can create a personal fork by visiting the yt GitHub webpage at https://github.com/yt-project/yt/ . After logging in, you should see an option near the top right labeled "fork". You now have a forked copy of the yt repository for your own personal modification.
This forked copy exists on the GitHub repository, so in order to access it locally you must clone it onto your machine from the command line:
$ git clone https://github.com/<USER>/yt ./yt-git
This downloads that new forked repository to your local machine, so that you can access it, read it, make modifications, etc. It will put the repository in a local directory of the same name as the repository in the current working directory.
$ cd yt-git
Verify that you are on the main
branch of yt by running:
$ git branch
You can see any past state of the code by using the git log command. For example, the following command would show you the last 5 revisions (modifications to the code) that were submitted to that repository.
$ git log -n 5
Using the revision specifier (the number or hash identifier next to each changeset), you can update the local repository to any past state of the code (a previous changeset or version) by executing the command:
$ git checkout revision_specifier
You can always return to the most recent version of the code by executing the
same command as above with the most recent revision specifier in the
repository. However, using git log
when you're checked out to an older
revision specifier will not show more recent changes to the repository. An
alternative option is to use checkout
on a branch. In yt the main
branch is our primary development branch, so checking out main
should
return you to the tip (or most up-to-date revision specifier) on the main
branch.
$ git checkout main
Lastly, if you want to use this new downloaded version of your yt repository as
the active version of yt on your computer (i.e. the one which is executed when
you run yt from the command line or the one that is loaded when you do import
yt
), then you must "activate" by building yt from source as described in
:ref:`install-from-source`.
The root directory of the yt git repository contains a number of subdirectories with different components of the code. Most of the yt source code is contained in the yt subdirectory. This directory itself contains the following subdirectories:
frontends
This is where interfaces to codes are created. Within each subdirectory of yt/frontends/ there must exist the following files, even if empty:
data_structures.py
, where subclasses of AMRGridPatch, Dataset and AMRHierarchy are defined.io.py
, where a subclass of IOHandler is defined.fields.py
, where fields we expect to find in datasets are definedmisc.py
, where any miscellaneous functions or classes are defined.definitions.py
, where any definitions specific to the frontend are defined. (i.e., header formats, etc.)
fields
- This is where all of the derived fields that ship with yt are defined.
geometry
- This is where geometric helpler routines are defined. Handlers for grid and oct data, as well as helpers for coordinate transformations can be found here.
visualization
- This is where all visualization modules are stored. This includes plot collections, the volume rendering interface, and pixelization frontends.
data_objects
- All objects that handle data, processed or unprocessed, not explicitly defined as visualization are located in here. This includes the base classes for data regions, covering grids, time series, and so on. This also includes derived fields and derived quantities.
units
- This used to be where all the unit-handling code resided, but as of now it's mostly just a thin wrapper around unyt.
utilities
- All broadly useful code that doesn't clearly fit in one of the other categories goes here.
If you're looking for a specific file or function in the yt source code, use the unix find command:
$ find <DIRECTORY_TREE_TO_SEARCH> -name '<FILENAME>'
The above command will find the FILENAME in any subdirectory in the DIRECTORY_TREE_TO_SEARCH. Alternatively, if you're looking for a function call or a keyword in an unknown file in a directory tree, try:
$ grep -R <KEYWORD_TO_FIND> <DIRECTORY_TREE_TO_SEARCH>
This can be very useful for tracking down functions in the yt source.
If you have made changes to any C or Cython (.pyx
) modules, you have to
rebuild yt before your changes are usable. See :ref:`install-from-source`.
Modifications to the code typically fall into one of three categories, each of which have different requirements for acceptance into the code base. These requirements are in place for a few reasons -- to make sure that the code is maintainable, testable, and that we can easily include information about changes in changelogs during the release procedure. (See YTEP-0008 for more detail.)
For all types of contributions, it is required that all tests pass, or that all non-passing tests are specifically accounted for.
- New Features
- New unit tests (possibly new answer tests) (See :ref:`testing`)
- Docstrings in the source code for the public API
- Addition of new feature to the narrative documentation (See :ref:`writing_documentation`)
- Addition of cookbook recipe (See :ref:`writing_documentation`)
- Extension or Breakage of API in Existing Features
- Update existing narrative docs and docstrings (See :ref:`writing_documentation`)
- Update existing cookbook recipes (See :ref:`writing_documentation`)
- Modify of create new unit tests (See :ref:`testing`)
- Bug fixes
- Unit test is encouraged, to ensure breakage does not happen again in the future. (See :ref:`testing`)
- At a minimum, a minimal, self-contained example demonstrating the bug should because included in the body of the Pull Request, or as part of an independent issue.
When submitting, you will be asked to make sure that your changes meet all of these requirements. They are pretty easy to meet, and we're also happy to help out with them. See :ref:`code-style-guide` for how to easily conform to our style guide.
If you're new to git, the following resource is pretty great for learning the ins and outs:
There also exist a number of step-by-step git tutorials to help you get used to version controlling files with git. Here are a few resources that you may find helpful:
- http://swcarpentry.github.io/git-novice/
- https://git-scm.com/docs/gittutorial
- https://try.github.io/
The commands that are essential for using git include:
git <command> --help
which provides help for any git command. For example, you can learn more about thelog
command by doinggit log --help
.git add <paths>
which stages changes to the specified paths for subsequent committing (see below).git commit
which commits staged changes (stage usinggit add
as above) in the working directory to the repository, creating a new "revision."git merge <branch>
which merges the revisions from the specified branch into the current branch, creating a union of their lines of development. This updates the working directory.git pull <remote> <branch>
which pulls revisions from the specified branch of the specified remote repository into the current local branch. Equivalent togit fetch <remote>
and thengit merge <remote>/<branch>
. This updates the working directory.git push <remote>
which sends revisions on local branches to matching branches on the specified remote.git push <remote> <branch>
will only push changes for the specified branch.git log
which shows a log of all revisions on the current branch. There are many options you can pass togit log
to get additional information. One example isgit log --oneline --decorate --graph --all
.
We are happy to answer questions about git use on our IRC, slack chat or on the mailing list to walk you through any troubles you might have. Here are some general suggestions for using git with yt:
- Although not necessary, a common development work flow is to create a local
named branch other than
main
to address a feature request or bugfix. If the dev work addresses a specific yt GitHub issue, you may include that issue number in the branch name. For example, if you want to work on issue number X regarding a cool new slice plot feature, you might name the branch:cool_new_plot_feature_X
. When you're ready to share your work, push your feature branch to your remote and create a pull request to themain
branch of the yt-project's repository. - When contributing changes, you might be asked to make a handful of modifications to your source code. We'll work through how to do this with you, and try to make it as painless as possible.
- Your test may fail automated style checks. See :ref:`code-style-guide` for more information about automatically verifying your code style.
- You should only need one fork. To keep it in sync, you can sync from the website. See :ref:`sharing-changes` for a description of the basic workflow and :ref:`multiple-PRs` for a discussion about what to do when you want to have multiple open pull requests at the same time.
- If you run into any troubles, stop by IRC (see :ref:`irc`), Slack, or the mailing list.
The simplest way to submit changes to yt is to do the following:
- Build yt from the git repository
- Navigate to the root of the yt repository
- Make some changes and commit them
- Fork the yt repository on GitHub
- Push the changesets to your fork
- Issue a pull request.
Here's a more detailed flowchart of how to submit changes.
Fork yt on GitHub. (This step only has to be done once.) You can do this at: https://github.com/yt-project/yt/fork.
Follow :ref:`install-from-source` for instructions on how to build yt from the git repository. (Below, in :ref:`reading-source`, we describe how to find items of interest.) If you have already forked the repository then you can clone your fork locally:
git clone https://github.com/<USER>/yt ./yt-git
This will create a local clone of your fork of yt in a folder named
yt-git
.Edit the source file you are interested in and test your changes. (See :ref:`testing` for more information.)
Create a uniquely named branch to track your work. For example:
git checkout -b my-first-pull-request
Stage your changes using
git add <paths>
. This command take an argument which is a series of filenames whose changes you want to commit. After staging, executegit commit -m "<Commit description>. Addresses Issue #X"
. Note that supplying an actual GitHub issue # in place ofX
will cause your commit to appear in the issue tracker after pushing to your remote. This can be very helpful for others who are interested in what work is being done in connection to that issue.Remember that this is a large development effort and to keep the code accessible to everyone, good documentation is a must. Add in source code comments for what you are doing. Add in docstrings if you are adding a new function or class or keyword to a function. Add documentation to the appropriate section of the online docs so that people other than yourself know how to use your new code.
If your changes include new functionality or cover an untested area of the code, add a test. (See :ref:`testing` for more information.) Commit these changes as well.
Add your remote repository with a unique name identifier. It can be anything but it is conventional to call it
origin
. You can see names and URLs of all the remotes you currently have configured with:git remote -v
If you already have an
origin
remote, you can set it to your fork with:git remote set-url origin https://github.com/<USER>/yt
If you do not have an
origin
remote you will need to add it:git remote add origin https://github.com/<USER>/yt
In addition, it is also useful to add a remote for the main yt repository. By convention we name this remote
upstream
:git remote add upstream https://github.com/yt-project/yt
Note that if you forked the yt repository on GitHub and then cloned from there you will not need to add the
origin
remote.Push your changes to your remote fork using the unique identifier you just created and the command:
git push origin my-first-pull-request
Where you should substitute the name of the feature branch you are working on for
my-first-pull-request
.Note
Note that the above approach uses HTTPS as the transfer protocol between your machine and GitHub. If you prefer to use SSH - or perhaps you're behind a proxy that doesn't play well with SSL via HTTPS - you may want to set up an SSH key on GitHub. Then, you use the syntax
ssh://[email protected]/<USER>/yt
, or equivalent, in place ofhttps://github.com/<USER>/yt
in git commands. For consistency, all commands we list in this document will use the HTTPS protocol.Issue a pull request at https://github.com/yt-project/yt/pull/new/main A pull request is essentially just asking people to review and accept the modifications you have made to your personal version of the code.
During the course of your pull request you may be asked to make changes. These changes may be related to style issues, correctness issues, or requesting tests. The process for responding to pull request code review is relatively straightforward.
Make requested changes, or leave a comment indicating why you don't think they should be made.
Commit those changes to your local repository.
Push the changes to your fork:
git push origin my-first-pull-request
Your pull request will be automatically updated.
Once your pull request is merged, sync up with the main yt repository by pulling
from the upstream
remote:
git checkout main git pull upstream main
You might also want to sync your fork of yt on GitHub:
# sync my fork of yt with upstream git push origin main
And delete the branch for the merged pull request:
# delete branch for merged pull request git branch -d my-first-pull-request git push origin --delete my-first-pull-request
These commands are optional but are nice for keeping your branch list manageable. You can also delete the branch on your fork of yt on GitHub by clicking the "delete branch" button on the page for the merged pull request on GitHub.
Dealing with multiple pull requests on GitHub is straightforward. Development on
one feature should be isolated in one named branch, say feature_1
while
development of another feature should be in another named branch, say
feature_2
. A push to remote feature_1
will automatically update any
active PR for which feature_1
is a pointer to the HEAD
commit. A push to
feature_1
will not update any pull requests involving feature_2
.
We use the pre-commit framework to validate and
automatically fix code styling.
It is recommended (though not required) that you install pre-commit
on your machine
(see their documentation) and, from the top level of the repo, run
$ pre-commit install
So that our hooks will run and update your changes on every commit.
If you do not want to/are unable to configure pre-commit
on your machine, note that
after opening a pull request, it will still be run as a static checker as part of our CI.
Some hooks also come with auto-fixing capabilities, which you can trigger manually in a
PR by commenting pre-commit.ci autofix
(see ` <https://pre-commit.ci/#features>`_).
We use a combination of black,
ruff and cython-lint. See .pre-commit-config.yaml
and pyproject.toml
for the complete configuration details.
Note that formatters should not be run directly on the command line as, for instance
$ black yt
But it can still be done as
$ pre-commit run black --all-files
The reason is that you may have a specific version of black
installed which can
produce different results, while the one that's installed with pre-commit is guaranteed
to be in sync with the rest of contributors.
Below are a list of additional guidelines for coding in yt, that are not automatically enforced.
- In general, follow PEP-8 guidelines. https://www.python.org/dev/peps/pep-0008/
- Classes are
ConjoinedCapitals
, methods and functions arelowercase_with_underscores
.- Do not use nested classes unless you have a very good reason to, such as requiring a namespace or class-definition modification. Classes should live at the top level.
__metaclass__
is exempt from this.- Avoid copying memory when possible. For example, don't do
a = a.reshape(3, 4)
whena.shape = (3, 4)
will do, anda = a * 3
should benp.multiply(a, 3, a)
.- In general, avoid all double-underscore method names:
__something
is usually unnecessary.- When writing a subclass, use the super built-in to access the super class, rather than explicitly. Ex:
super().__init__()
rather thanSpecialGrid.__init__()
.- Docstrings should describe input, output, behavior, and any state changes that occur on an object. See :ref:`docstrings` below for a fiducial example of a docstring.
- Unless there is a good reason not to (e.g., to avoid circular imports), imports should happen at the top of the file.
- If you are comparing with a numpy boolean array, just refer to the array. Ex: do
np.all(array)
instead ofnp.all(array == True)
.- Only declare local variables if they will be used later. If you do not use the return value of a function, do not store it in a variable.
Internally, only import from source files directly -- instead of:
from yt.visualization.api import ProjectionPlot
do:
from yt.visualization.plot_window import ProjectionPlot
Import symbols from the module where they are defined, avoid transitive imports.
Import standard library modules, functions, and classes from builtins, do not import them from other yt files.
Numpy is to be imported as
np
.Do not use too many keyword arguments. If you have a lot of keyword arguments, then you are doing too much in
__init__
and not enough via parameter setting.Don't create a new class to replicate the functionality of an old class -- replace the old class. Too many options makes for a confusing user experience.
Parameter files external to yt are a last resort.
The usage of the
**kwargs
construction should be avoided. If they cannot be avoided, they must be documented, even if they are only to be passed on to a nested function.
The following is an example docstring. You can use it as a template for docstrings in your code and as a guide for how we expect docstrings to look and the level of detail we are looking for. Note that we use NumPy style docstrings written in Sphinx restructured text format.
r"""A one-line summary that does not use variable names or the
function name.
Several sentences providing an extended description. Refer to
variables using back-ticks, e.g. ``var``.
Parameters
----------
var1 : array_like
Array_like means all those objects -- lists, nested lists, etc. --
that can be converted to an array. We can also refer to
variables like ``var1``.
var2 : int
The type above can either refer to an actual Python type
(e.g. ``int``), or describe the type of the variable in more
detail, e.g. ``(N,) ndarray`` or ``array_like``.
Long_variable_name : {'hi', 'ho'}, optional
Choices in brackets, default first when optional.
Returns
-------
describe : type
Explanation
output : type
Explanation
tuple : type
Explanation
items : type
even more explaining
Other Parameters
----------------
only_seldom_used_keywords : type
Explanation
common_parameters_listed_above : type
Explanation
Raises
------
BadException
Because you shouldn't have done that.
See Also
--------
otherfunc : relationship (optional)
newfunc : Relationship (optional), which could be fairly long, in which
case the line wraps here.
thirdfunc, fourthfunc, fifthfunc
Notes
-----
Notes about the implementation algorithm (if needed).
This can have multiple paragraphs.
You may include some math:
.. math:: X(e^{j\omega } ) = x(n)e^{ - j\omega n}
And even use a greek symbol like :math:`omega` inline.
References
----------
Cite the relevant literature, e.g. [1]_. You may also cite these
references in the notes section above.
.. [1] O. McNoleg, "The integration of GIS, remote sensing,
expert systems and adaptive co-kriging for environmental habitat
modelling of the Highland Haggis using object-oriented, fuzzy-logic
and neural-network techniques," Computers & Geosciences, vol. 22,
pp. 585-588, 1996.
Examples
--------
These are written in doctest format, and should illustrate how to
use the function. Use the variables 'ds' for the dataset, 'pc' for
a plot collection, 'c' for a center, and 'L' for a vector.
>>> a = [1, 2, 3]
>>> print([x + 3 for x in a])
[4, 5, 6]
>>> print("a\n\nb")
a
b
"""
Avoid Enzo-isms. This includes but is not limited to:
Hard-coding parameter names that are the same as those in Enzo. The following translation table should be of some help. Note that the parameters are now properties on a
Dataset
subclass: you access them like ds.refine_by .
RefineBy `` => `` refine_by
TopGridRank `` => `` dimensionality
TopGridDimensions `` => `` domain_dimensions
InitialTime `` => `` current_time
DomainLeftEdge `` => `` domain_left_edge
DomainRightEdge `` => `` domain_right_edge
CurrentTimeIdentifier `` => `` unique_identifier
CosmologyCurrentRedshift `` => `` current_redshift
ComovingCoordinates `` => `` cosmological_simulation
CosmologyOmegaMatterNow `` => `` omega_matter
CosmologyOmegaLambdaNow `` => `` omega_lambda
CosmologyHubbleConstantNow `` => `` hubble_constant
Do not assume that the domain runs from 0 .. 1. This is not true everywhere.
Variable names should be short but descriptive.
No globals!