added files to create presidential_speech data #40
Conversation
|
We don't need the |
|
That sounds fine. I'll drop the pyc and raw speechs, and resubmit. |
|
We also don't need the |
| wrd.var <- apply(dtm.mat.log,2,var) | ||
| top.wrd.var <- names(sort(wrd.var,decreasing = TRUE)[1:75]) | ||
| dtm.mat.log <- dtm.mat.log[,colnames(dtm.mat.log) %in% top.wrd.var] | ||
| saveRDS(dtm.mat.log,"presidential_speech.rds") |
| library(SnowballC) | ||
| library(parallel) | ||
| library(Matrix) | ||
| library(tidyverse) |
There was a problem hiding this comment.
Are all of these dependencies used? I only see tm stringr and tidyverse below.
| @@ -0,0 +1,11 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
This file appears to be identical to move_inaug_results.sh. Is that intentional?
| class InaugTextSpider(scrapy.Spider): | ||
| name = "inaug_text" | ||
| allowed_domains = ["http://www.presidency.ucsb.edu/"] | ||
| start_urls = ( |
There was a problem hiding this comment.
Add a note explaining what these URLs are and how to keep this list up to date (if possible)
| name = "sou_text" | ||
| allowed_domains = ["http://www.presidency.ucsb.edu"] | ||
| start_urls = ( | ||
| 'http://www.presidency.ucsb.edu/ws/index.php?pid=123408', |
There was a problem hiding this comment.
Explanatory note needed.
|
@jjn13 Any updates on this PR? |
Created raw-data directory and with code for creating presidential_speech dataset.