Installation & Usage

► Install

Manual:

    git clone https://github.com/meyersbs/uncertainty.git
    cd uncertainty/
    pip install -r requirements.txt

PyPi:

    pip install uncertainty

► NLTK Data Dependencies

    python3 -m nltk.downloader treebank conll2000 wordnet punkt averaged_perceptron_tagger

► Usage

★ Training

NOTE: Training functionality is currently broken.

This codebase has pre-trained classifiers included, but if you would like to retrain them, here's how to do it:

    # Download the training data.
    python uncertainty/data/download.py

    # Train the binary classifier.
    python main.py train merged_data
    # Train the multiclass classifier.
    python main.py train -m merged_data

★ Classification

To run the classifiers via the command line, specify one of the commands below:

    python main.py predict --granularity=word "I wonder if I am the walrus."
    # OUTPUT:
    #   I wonder if I am the walrus.: ['C', 'C', 'U', 'C', 'C', 'C', 'C', 'C']

    python main.py predict -m --granularity=word "I wonder if I am the walrus."
    # OUTPUT:
    #   I wonder if I am the walrus.: ['C', 'C', 'N', 'C', 'C', 'C', 'C', 'C']

    python main.py predict --granularity=sentence "I wonder if I am the walrus."
    # OUTPUT:
    #   I wonder if I am the walrus.: U

    python main.py predict -m --granularity=sentence "I wonder if I am the walrus."
    # OUTPUT:
    #   I wonder if I am the walrus.: N

To run the classifiers in your python code:

    from uncertainty.classifier import Classifier
    cls = Classifier()
    cls.predict("I wonder if I am the walrus.")
    # OUTPUT:
    #    ['C', 'C', 'N', 'C', 'C', 'C', 'C', 'C']

    from uncertainty.classifier import Classifier
    cls = Classifier(binary=True)
    cls.predict("I wonder if I am the walrus.")
    # OUTPUT:
    #    ['C', 'C', 'U', 'C', 'C', 'C', 'C', 'C']

    from uncertainty.classifier import Classifier
    cls = Classifier(granularity='sentence')
    cls.predict("I wonder if I am the walrus.")
    # OUTPUT:
    #    N

    from uncertainty.classifier import Classifier
    cls = Classifier(granularity='sentence', binary=True)
    cls.predict("I wonder if I am the walrus.")
    # OUTPUT:
    #    U

To run predictions in bulk, try a bash script, such as:

    while read sent; do
      echo "python main.py predict --granularity=sentence ${sent}"
    done <test_data.txt