Skip to content

Latest commit

 

History

History
 
 

letor

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 

TextNet Models for LETOR 4.0

Deep Text Matching model for LETOR 4.0.

Dataset

Dataset can be download from LETOR 4.0. The content of the documents can be extract from the GOV2 dataset which need permission from here.

Data Example

2 qid:10032 1:0.056537 2:0.000000 3:0.666667 4:1.000000 5:0.067138 … 45:0.000000 46:0.076923 #docid = GX029-35-5894638 inc = 0.0119881192468859 prob = 0.139842

Preprocess

In order to run TextNet models, we need prepare files below:

Word Dictionary File

(eg. word_dict.txt)

We map each word to a uniqe number, called wid, and save this mapping in the word dictionary file.

For example,

word   wid
machine 1232
learning 1156

Corpus File

(eg. qid_query.txt and docid_doc.txt)

We use a value of string identifier (qid/docid) to represent a sentence, such as a query or a document. The second number denotes the length of the sentence. The following numbers are the wids of the sentence.

For example,

docid  sentence_length  sentence_wid_sequence
GX000-00-0000000 42 2744 1043 377 2744 1043 377 187 117961 ...

For DeepRank models (such as config/letor.deeprank.pyramid.config and config/letor.deeprank.2dgru.config), we need add some special marks for query-centric contexts.

The sentence_wid_sequence in document corpus file is the concatenate of all query-centric contexts.

  • -1: denotes the padding word.
  • [-10, -inf]: denotes the query term in the query. -10 represents the first query term, -11 represents the second query term and so on.

For example,

docid  sentence_length  sentence_wid_sequence
GX245-00-1220850@0 8702 1421 311 -10 3703 221 2134 
GX245-00-1220860@0 3158 3260 229 -13 2814 -1 -1

Relation File

(eg. relation.train.fold1.txt, relation.test.fold1.txt ...)

The relation files are used to store the relation between two sentences, such as the relevance relation between query and document.

For example,

relevance   qid   docid
1 3571 GX245-00-1220850
0 3571 GX004-51-0504917
0 3571 GX006-36-4612449

Embedding File

(eg. embed_wiki-pdc_d50_norm)

We store the word embedding into the embedding file.

For example,

wid   embedding
13275 -0.050766 0.081548 -0.031107 0.131772 0.172194 ... 0.165506 0.002235

Feature File

(eg. docid_snippos_sort_none.c1w.feat)

We store the feature vectors into the feature file.

For example,

key feat_vec
GX245-00-1220850@0  0.1143 0.1107 0.1034 0.1006 0.0988 0.0820 0.0774 0.0769 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.1958 0.1599 0.1585 0.1489 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.1594 0.0036 0

Config Files

The example config files are in config directory.

Config Fields File Type
data1_file Corpus File
data2_file Corpus File
rel_file Relation File
embedding_file Embedding File
feature_file Feature File

DeepRank

this model is an implementation of DeepRank: A New Deep Architecture for Relevance Ranking in Information Retrieval

  • config file: config/letor.deeprank.pyramid.config
  • config file: config/letor.deeprank.2dgru.config

MatchPyramid

this model is an implementation of Text Matching as Image Recognition

  • config file: config/letor.pyramid.config

Match-SRNN

this model is an implementation of Match-SRNN: Modeling the Recursive Matching Structure with Spatial RNN

  • config file: config/letor.2dgru.config

MV-LSTM

this model is an implementation of A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations

  • config file: models/letor.mvlstm.config

ARC-I / ARC-II

this model is an implementation of Convolutional Neural Network Architectures for Matching Natural Language Sentences

  • config file: config/letor.arc1.config
  • config file: config/letor.arc2.config

DSSM

this model is an implementation of Learning Deep Structured Semantic Models for Web Search using Clickthrough Data

  • config file: models/letor.dssm.config

CDSSM

this model is an implementation of Learning Semantic Representations Using Convolutional Neural Networks for Web Search

  • config file: models/letor.cdssm.config