Skip to content

Commit

Permalink
Merge pull request #1 from Decompollaborate/develop
Browse files Browse the repository at this point in the history
1.0.1
  • Loading branch information
AngheloAlf authored Sep 21, 2023
2 parents 58bf93f + a11ab68 commit fed205b
Show file tree
Hide file tree
Showing 18 changed files with 251 additions and 52 deletions.
17 changes: 17 additions & 0 deletions .github/workflows/md_lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: Lint markdown files

# Build on every branch push, tag push, and pull request change:
on: [push, pull_request]

jobs:
checks:
runs-on: ubuntu-latest
name: Lint md files
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Lint markdown files
uses: articulate/[email protected]
with:
config: .markdownlint.jsonc
3 changes: 2 additions & 1 deletion .github/workflows/mypy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ jobs:
runs-on: ubuntu-latest
name: mypy
steps:
- uses: actions/checkout@v4
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python 3.7
uses: actions/setup-python@v4
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- name: Checkout repository
uses: actions/checkout@v4

- name: Install local ipl3checksum
run: pip install .
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/tests_other_repo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ on: [push, pull_request_target]

jobs:
build_repo:
name: Build repo
name: Test other repo
runs-on: ubuntu-latest

strategy:
Expand Down
35 changes: 18 additions & 17 deletions .github/workflows/upload_pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,29 +9,30 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- name: Checkout repository
uses: actions/checkout@v4

- name: Install build module
run: pip install build
- name: Install build module
run: pip install build

- name: Build wheel and source
run: python -m build --sdist --wheel --outdir dist/ .
- name: Build wheel and source
run: python -m build --sdist --wheel --outdir dist/ .

- uses: actions/upload-artifact@v3
with:
path: dist/*
- uses: actions/upload-artifact@v3
with:
path: dist/*

upload_pypi:
needs: [build_wheel]
runs-on: ubuntu-latest
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
steps:
- uses: actions/download-artifact@v3
with:
name: artifact
path: dist

- uses: pypa/[email protected]
with:
user: __token__
password: ${{ secrets.pypi_password }}
- uses: actions/download-artifact@v3
with:
name: artifact
path: dist

- uses: pypa/[email protected]
with:
user: __token__
password: ${{ secrets.pypi_password }}
2 changes: 0 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -162,8 +162,6 @@ cython_debug/
#.idea/


#
.vscode/

asm/
*.z64
Expand Down
14 changes: 14 additions & 0 deletions .markdownlint.jsonc
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
// https://github.com/DavidAnson/markdownlint/blob/main/doc/md024.md
// MD024 - Multiple headings with the same content
"MD024": {
"siblings_only": true
},

// https://github.com/DavidAnson/markdownlint/blob/main/doc/md013.md
// MD013 - Line length
"MD013": {
"code_block_line_length": 120,
"headings": false
}
}
7 changes: 7 additions & 0 deletions .vscode/extensions.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
// See https://go.microsoft.com/fwlink/?LinkId=827846
// for the documentation about the extensions.json format
"recommendations": [
"davidanson.vscode-markdownlint"
]
}
33 changes: 33 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [1.0.1] - 2023-09-21

### Added

- Allow invoking `ipl3checksum` as a CLI program.
- Currently it only allows the `-V`/`--version` argument, which prints the
version of the library.
- A `CHANGELOG.md`
- Cleanup the `README.md`
- Reorder sections.
- Add more notes about installing and the develop version.
- Reference the changelog.
- List features.
- Add a `py.typed` file.

## [1.0.0] - 2023-09-20

### Added

- Initial relase

[unreleased]: https://github.com/olivierlacan/keep-a-changelog/compare/1.1.0...HEAD
[1.0.1]: https://github.com/olivierlacan/keep-a-changelog/compare/1.0.0...1.1.1
[1.0.0]: https://github.com/Decompollaborate/ipl3checksum/releases/tag/1.0.0
112 changes: 89 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,7 @@ A Python library to calculate the IPL3 checksum for N64 ROMs.

## How to use it?

First you need to install the library, one way of doing it is via `pip`.

```bash
python3 -m pip install -U ipl3checksum
```

Now you can invoke the library from your script.
To calculate the checksum of a ROM:

```py
romBytes = # A big endian bytes-like object
Expand All @@ -36,32 +30,104 @@ cickind = ipl3checksum.detectCIC(romBytes)
print(cickind) # Either a `ipl3checksum.CICKind` or None if was not able to detect the CIC
```

## Features

- Supports all 6 retail CIC variants.
- Can calculate the checksum of a ROM using the algorithm of any of the
supported CIC variants.
- Can detect any of the supported CIC variants.

### Restrictions/requirements

- The library assumes the passed ROM contains a ROM header at offset range
`[0x0, 0x40]` and a correct IPL3 is at `[0x40, 0x1000]`
- Since the checksum algorithm is calculated on the first MiB after IPL3 (from
`0x1000` to `0x101000`), then the library expects the passed ROM to be at least
`0x101000` bytes long, otherwise the library will reject the ROM.
- If it is not the case, then pad your ROM with zeroes to that size.

## Installing

First you need to install the library, one way of doing it is via `pip`.

```bash
python3 -m pip install -U ipl3checksum
```

If you use a `requirements.txt` file in your repository, then you can add
this library with the following line:

```txt
ipl3checksum>=1.0.0,<2.0.0
``````
Now you can invoke the library from your script.
### Development version
The unstable development version is located at the [develop](https://github.com/Decompollaborate/ipl3checksum/tree/develop)
branch. PRs should be made into that branch instead of the main one.
The recommended way to install a locally cloned repo is by passing the `-e`
(editable) flag to `pip`.
```bash
python3 -m pip install -e .
```

In case you want to mess with the latest development version without wanting to
clone the repository, then you could use the following commands:

```bash
python3 -m pip uninstall ipl3checksum
python3 -m pip install git+https://github.com/Decompollaborate/ipl3checksum.git@develop
```

NOTE: Installing the development version is not recommended unless you know what
you are doing. Proceed at your own risk.

## Versioning and changelog

This library follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
We try to always keep backwards compatibility, so no breaking changes should
happen until a major release (i.e. jumping from 1.X.X to 2.0.0).

To see what changed on each release check either the [CHANGELOG.md](CHANGELOG.md)
file or check the [releases page on Github](https://github.com/Decompollaborate/ipl3checksum/releases).
You can also use [this link](https://github.com/Decompollaborate/ipl3checksum/releases/latest)
to check the latest release.

## Where does this come from?

This algorithm comes directly from the IPL3, which each variant is part of the first 0x1000 bytes of the rom of every retail N64 ROM.
This algorithm comes directly from the IPL3, which each variant is part of the
first 0x1000 bytes of the rom of every retail N64 ROM.

There are various implementations floating around on the internet, but for this specific one was reverse-engineered by myself.
I made this because I couldn't find a library to calculate this checksum, so I decided to reverse-engineer it myself instead of
taking somebody else's work. It also was an interesting learning experience.
There are various implementations floating around on the internet, but for this
specific one was reverse-engineered by myself. I made this because I couldn't
find a library to calculate this checksum, so I decided to reverse-engineer it
myself instead of taking somebody else's work. It also was an interesting
learning experience.

## Note about licensing

Most of the repository is licensed under the [MIT license](LICENSE), but I also made a
[reference implementation](docs/reference_implementation.md) that is part of the public domain (licensed under CC0-1.0), feel free to
use it however you prefer (acknowledgment is always appreciated, but not required).
Most of the repository is licensed under the [MIT license](LICENSE), but I also
made a [reference implementation](docs/reference_implementation.md) that is part
of the public domain (licensed under CC0-1.0), feel free to use it however you
prefer (acknowledgment is always appreciated, but not required).

## I want to learn more! What is an IPL3? What is CIC?

I'm not really the guy that can answer all your hunger for knowledge, but here are a few links that may be helpful:
I'm not really the guy that can answer all your hunger for knowledge, but here
are a few links that may be helpful:

* CIC-NUS: <https://n64brew.dev/wiki/CIC-NUS>
* Initial Program Load 3 (IPL3) <https://n64brew.dev/wiki/Initial_Program_Load#IPL3>
* List of retail games, containing which CIC they use: <https://docs.google.com/spreadsheets/d/1WgZ7DZSzWwYIxwg03yoN9NK_0okuSx9dVL2u5MWPQ60/edit#gid=1247952340>
* Research about the CIC 6105: <https://github.com/Dragorn421/n64checksum>
* Disassembly of all the retail IPL3 binaries: <https://github.com/decompals/N64-IPL/blob/main/src/ipl3.s>
- CIC-NUS: <https://n64brew.dev/wiki/CIC-NUS>
- Initial Program Load 3 (IPL3) <https://n64brew.dev/wiki/Initial_Program_Load#IPL3>
- List of retail games, containing which CIC they use: <https://docs.google.com/spreadsheets/d/1WgZ7DZSzWwYIxwg03yoN9NK_0okuSx9dVL2u5MWPQ60/edit#gid=1247952340>
- Research about the CIC 6105: <https://github.com/Dragorn421/n64checksum>
- Disassembly of all the retail IPL3 binaries: <https://github.com/decompals/N64-IPL/blob/main/src/ipl3.s>

## References

* "IPL3 checksum algorithm" section of the "PIF-NUS" article on n64brew.dev: <https://n64brew.dev/wiki/PIF-NUS#IPL3_checksum_algorithm>
* Used for getting the "8-bit IPL3" seed value.
* List of retail games, containing which CIC they use: <https://docs.google.com/spreadsheets/d/1WgZ7DZSzWwYIxwg03yoN9NK_0okuSx9dVL2u5MWPQ60/edit#gid=1247952340>
- "IPL3 checksum algorithm" section of the "PIF-NUS" article on n64brew.dev: <https://n64brew.dev/wiki/PIF-NUS#IPL3_checksum_algorithm>
- Used for getting the "8-bit IPL3" seed value.
- List of retail games, containing which CIC they use: <https://docs.google.com/spreadsheets/d/1WgZ7DZSzWwYIxwg03yoN9NK_0okuSx9dVL2u5MWPQ60/edit#gid=1247952340>
9 changes: 6 additions & 3 deletions docs/reference_implementation.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@

This whole file is licensed under CC0-1.0. See [docs/LICENSE](./LICENSE).

This file contains a reference implementation of the IPL3 checksum algorithm, adapted to work with every known retail CIC/IPL3 variant.
This file contains a reference implementation of the IPL3 checksum algorithm,
adapted to work with every known retail CIC/IPL3 variant.

This implementation is a very crude direct translation from the original assembly and it could be greatly simplified in a reimplementation.
This implementation is a very crude direct translation from the original
assembly and it could be greatly simplified in a reimplementation.

```c
/* SPDX-License-Identifier: CC0-1.0 */
Expand Down Expand Up @@ -158,7 +160,8 @@ void calculateChecksum(const uint8_t *rom, uint32_t cic, uint32_t *dst1, uint32_
switch (cic) {
case 6105:
case 7105:
/* ipl3 6105 copies 0x330 bytes from the ROM's offset 0x000554 (or offset 0x000514 into IPL3) to vram 0xA0000004 */
/* ipl3 6105 copies 0x330 bytes from the ROM's offset 0x000554 (or offset 0x000514 into IPL3) */
/* to vram 0xA0000004 */
/* lw $t7, 0x0($s6) */
t7 = readWord(rom, s6 - 0xA0000004 + 0x000554);

Expand Down
6 changes: 4 additions & 2 deletions notes/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Notes

This folder contains the scripts I made while I was trying to understand the checksum algorithm of every IPL3 variant.
This folder contains the scripts I made while I was trying to understand the
checksum algorithm of every IPL3 variant.

All of the implementations here are direct and crude translations from the original asm.
All of the implementations here are direct and crude translations from the
original asm.
8 changes: 7 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

[project]
name = "ipl3checksum"
version = "1.0.0"
version = "1.0.1"
description = "Library to calculate the IPL3 checksum for N64 ROMs"
readme = "README.md"
requires-python = ">=3.7"
Expand All @@ -17,5 +17,11 @@ dynamic = ["dependencies"]
requires = ["hatchling", "hatch-requirements-txt"]
build-backend = "hatchling.build"

[project.scripts]
ipl3checksum = "ipl3checksum.frontends.climain:ipl3checksumMain"

[tool.cibuildwheel]
skip = ["cp36-*"]

[tool.setuptools.package-data]
ipl3checksum = ["py.typed"]
2 changes: 1 addition & 1 deletion src/ipl3checksum/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

from __future__ import annotations

__version_info__: tuple[int, int, int] = (1, 0, 0)
__version_info__: tuple[int, int, int] = (1, 0, 1)
__version__ = ".".join(map(str, __version_info__))
__author__ = "Decompollaborate"

Expand Down
14 changes: 14 additions & 0 deletions src/ipl3checksum/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/usr/bin/env python3

# SPDX-FileCopyrightText: © 2023 Decompollaborate
# SPDX-License-Identifier: MIT

from __future__ import annotations

import argparse

from .frontends import climain


if __name__ == "__main__":
climain.ipl3checksumMain()
9 changes: 9 additions & 0 deletions src/ipl3checksum/frontends/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/usr/bin/env python3

# SPDX-FileCopyrightText: © 2023 Decompollaborate
# SPDX-License-Identifier: MIT

from __future__ import annotations


from . import climain as climain
Loading

0 comments on commit fed205b

Please sign in to comment.