Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MultiSequence Issue #17

Open
dulaku opened this issue Nov 5, 2016 · 0 comments
Open

MultiSequence Issue #17

dulaku opened this issue Nov 5, 2016 · 0 comments

Comments

@dulaku
Copy link

dulaku commented Nov 5, 2016

I'm trying to use MultiSequence as part of a natural language processing thingamajig. I just reinstalled torch, cutorch, nn, cunn, rnn, tds, and dataloader, so most recent versions of all of them (tds might be unnecessary, but I figure it can't hurt, right?).

During calls to subiter(), I was getting erratic crashes at Line 88,

input:narrow(1,start,size):copy(sequence:sub(tracker.idx, stop))

due to argument 3 of narrow() being out of range. Whenever this happened, size was 0. This happens when there's a sequence of length 1, which sets stop to 0 at Line 85, and when tracker.idx is 1.

Interestingly, this happens despite my data set containing no such sequences. I don't know how we're getting here - I adjusted things to print the sequence tensors as I add them to the list I feed the MultiSequence constructor, and not one of them is shorter than length 2, even on a set that causes this error. Nevertheless, printing self.sequences[tracker.seqid] just before the crash shows a length 1 tensor. This is the point at which I've convinced myself that I do not, in fact, grok the code well enough to find the real source of the bug. In any case, the sequence in question always seems to consist entirely of the EOS token by the time the code gets to this point, so there may be something to do with sequences getting truncated from the left.

With the "fix" below, the model does seem to be learning, but it could just be doing that in spite of a formatting error. Let me know if you need any code, data, etc.

For what it's worth, I "fixed" it by wrapping lines 85 through 96 with

if sequence:size(1) > 1 then  
  <existing code> 
else  
  start = start + 1  
  tracker.seqid = nil  
end

But that's mostly because I don't know exactly what I'm doing in the guts of this thing and it was easier to just kludge out the error case.

Thanks for your time

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant