Merge pull request #2 from aangelopoulos/eff-ppi

Integrate code from PPI++ paper.
aangelopoulos · Nov 2, 2023 · 2172d39 · 2172d39
2 parents e3d5a24 + 3971fd4
commit 2172d39
Show file tree

Hide file tree

Showing 59 changed files with 5,113 additions and 399 deletions.
diff --git a/README.md b/README.md
@@ -62,9 +62,9 @@ print(f"theta={Y_total.mean():.3f}, CPP={ppi_ci}")
 
 The expected results look as below $^*$: 
 ```
-theta=0.259, CPP=(0.235677274705698, 0.26595223970754855)
+theta=0.259, CPP=(0.2322466630315982, 0.2626038799812829)
 ```
-($^*$ these results were produced with ```numpy=1.26.0```, and may differ slightly due to randomness in other environments.)
+($^*$ these results were produced with ```numpy=1.26.1```, and may differ slightly due to randomness in other environments.)
 
 If you have reached this stage, congratulations! You have constructed a prediction-powered confidence interval.
 See [the documentation](https://ppi-py.readthedocs.io/en/latest/) for more usages of prediction-powered inference.
@@ -128,3 +128,11 @@ The contents of this repository will be pushed to PyPI whenever there are substa
 The repository maintainers will approve pull requests at their discretion. Before working on one, it may be helpful to post a question on the issues page to verify if the contribution would be a good candidate for merging into the main branch.
 
 Accepted pull requests will be run through an automated [Black](https://black.readthedocs.io/en/stable/) formatter, so contributors may want to run Black locally first.
+
+# Papers 
+
+The repository currently implements the methods developed in the following papers:
+
+[Prediction-Powered Inference](https://arxiv.org/abs/2301.09633)
+
+[PPI++: Efficient Prediction-Powered Inference](https://arxiv.org/abs/2301.09633)
diff --git a/docs/source/README.rst b/docs/source/README.rst
@@ -49,7 +49,7 @@ The expected results look as below :math:`^*`:
 
 .. code-block::
 
-   theta=0.259, CPP=(0.235677274705698, 0.26595223970754855)
+   theta=0.259, CPP=(0.2322466630315982, 0.2626038799812829)
 
 (:math:`^*` these results were produced with ``numpy=1.26.0``\ , and may differ slightly due to randomness in other environments.)
 
@@ -90,3 +90,12 @@ You can replace ``[ESTIMAND]`` with the estimand of your choice. For certain est
 All the prediction-powered point estimates and confidence intervals implemented so far can be imported by running ``from ppi_py import ppi_[ESTIMAND]_pointestimate, ppi_[ESTIMAND]_ci``. For the case of the mean, one can also import the p-value as ``from ppi import ppi_mean_pval``.
 
 Full API documentation can be found by following the links on the left-hand sidebar of this page.
+
+Papers
+======
+
+The repository currently implements the methods developed in the following papers:
+
+`Prediction-Powered Inference <https://arxiv.org/abs/2106.06487>`_
+
+`PPI++: Efficient Prediction-Powered Inference <https://arxiv.org/abs/2106.06487>`_
diff --git a/docs/source/baselines.rst b/docs/source/baselines.rst
@@ -9,7 +9,6 @@ Documentation for functions implementing baseline inference strategies can be fo
 .. autofunction:: ppi_py.conformal_mean_ci
 .. autofunction:: ppi_py.classical_quantile_ci
 .. autofunction:: ppi_py.classical_ols_ci
-.. autofunction:: ppi_py.classical_ols_covshift_ci
 .. autofunction:: ppi_py.postprediction_ols_ci
 .. autofunction:: ppi_py.logistic
 .. autofunction:: ppi_py.classical_logistic_ci
diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -13,7 +13,7 @@
 project = "ppi_py"
 copyright = "2023, Anastasios N. Angelopoulos, Stephen Bates, Clara Fannjiang, Michael I. Jordan, Tijana Zrnic, and others"
 author = "Anastasios N. Angelopoulos, Stephen Bates, Clara Fannjiang, Michael I. Jordan, Tijana Zrnic, and others"
-release = "0.1"
+release = "0.2"
 
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
@@ -27,7 +27,14 @@
 
 templates_path = ["_templates"]
 exclude_patterns = []
-autodoc_mock_imports = ["numpy", "scipy", "statsmodels", "sklearn", "pandas"]
+autodoc_mock_imports = [
+    "numpy",
+    "numba",
+    "scipy",
+    "statsmodels",
+    "sklearn",
+    "pandas",
+]
 
 # -- Options for HTML output -------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output

diff --git a/docs/source/ppi.rst b/docs/source/ppi.rst
@@ -11,8 +11,6 @@ Documentation for functions implementing some form of prediction-powered inferen
 .. autofunction:: ppi_py.ppi_quantile_ci
 .. autofunction:: ppi_py.ppi_ols_pointestimate
 .. autofunction:: ppi_py.ppi_ols_ci
-.. autofunction:: ppi_py.ppi_ols_covshift_pointestimate
-.. autofunction:: ppi_py.ppi_ols_covshift_ci
 .. autofunction:: ppi_py.ppi_logistic_pointestimate
 .. autofunction:: ppi_py.ppi_logistic_ci
 .. autofunction:: ppi_py.ppi_distribution_label_shift_ci
diff --git a/examples/.gitignore b/examples/.gitignore
@@ -0,0 +1 @@
+data/
diff --git a/examples/alphafold.ipynb b/examples/alphafold.ipynb
diff --git a/examples/ballots.ipynb b/examples/ballots.ipynb
diff --git a/examples/baselines/alphafold_lowN.ipynb b/examples/baselines/alphafold_lowN.ipynb
@@ -26,7 +26,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "sys.path.append(os.path.abspath(os.path.join(os.getcwd(), os.pardir, os.pardir)))\n",
+    "sys.path.append(\n",
+    "    os.path.abspath(os.path.join(os.getcwd(), os.pardir, os.pardir))\n",
+    ")\n",
     "sys.path.append(os.path.abspath(os.path.join(os.getcwd(), os.pardir)))\n",
     "import numpy as np\n",
     "import pandas as pd\n",

diff --git a/examples/baselines/forest_badmodel.ipynb b/examples/baselines/forest_badmodel.ipynb
@@ -60,9 +60,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "data = np.load(\n",
-    "    \"forest_badmodel.npz\"\n",
-    ") \n",
+    "data = np.load(\"forest_badmodel.npz\")\n",
     "Y_total = data[\"Y\"]\n",
     "Yhat_total = data[\"Yhat\"]"
    ]

diff --git a/examples/baselines/semisupervised.ipynb b/examples/baselines/semisupervised.ipynb
@@ -377,7 +377,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.15"
+   "version": "3.9.12"
   }
  },
  "nbformat": 4,

diff --git a/examples/census_healthcare.ipynb b/examples/census_healthcare.ipynb
diff --git a/examples/census_income.ipynb b/examples/census_income.ipynb
diff --git a/examples/census_income_covshift.ipynb b/examples/census_income_covshift.ipynb
diff --git a/examples/eff-ppi/.gitignore b/examples/eff-ppi/.gitignore
@@ -0,0 +1,2 @@
+data/
+.cache/
diff --git a/examples/eff-ppi/eff_ppi_logistic_comparison.ipynb b/examples/eff-ppi/eff_ppi_logistic_comparison.ipynb