Skip to content
This repository has been archived by the owner on Mar 1, 2024. It is now read-only.

Enhancement to Semantic Scholar Loader - full text reader #482

Merged
merged 9 commits into from
Sep 7, 2023

Conversation

shauryr
Copy link
Contributor

@shauryr shauryr commented Aug 25, 2023

Added document loading option for full text extracted from downloaded open access pdfs.

As shown below, just marking full_text flag as True will download all the relevant open access pdfs and extract full text from the pdf and load it.

documents = s2reader.load_data(query_space, total_papers, full_text=True)
  • Added demo notebook
  • Updated the README
  • Added pypdf and arxiv to requirements.txt

Attaching a screenshot of the loader in action!

image

@shauryr shauryr changed the title Semantic Scholar Loader - full text reader Enhancement to Semantic Scholar Loader - full text reader Aug 25, 2023
@jerryjliu jerryjliu merged commit eba88c1 into run-llama:main Sep 7, 2023
2 checks passed
@jerryjliu
Copy link
Collaborator

sorry for the delay! merged, thanks @shauryr

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants