main content extractor - custom script added based on readabiility, p…

…ostlight
vtempest · Aug 29, 2024 · ca5e4e5 · ca5e4e5
1 parent f05ed8b
commit ca5e4e5
Show file tree

Hide file tree

Showing 155 changed files with 2,369 additions and 10,061 deletions.
diff --git a/data/specific-domain-extractors.js b/data/specific-domain-extractors.js
diff --git a/data/wiki-phrases-model-240k.json b/data/wiki-phrases-model-240k.json
diff --git a/docs/assets/navigation.js b/docs/assets/navigation.js
diff --git a/docs/assets/search.js b/docs/assets/search.js
diff --git a/docs/functions/src_autocomplete_autocomplete.suggestNextWordCompletions.html b/docs/functions/src_autocomplete_autocomplete.suggestNextWordCompletions.html
diff --git a/docs/functions/src_chrome_extension_allow_cors.modifyChromeAPIAllowCORS.html b/docs/functions/src_chrome_extension_allow_cors.modifyChromeAPIAllowCORS.html
@@ -0,0 +1,21 @@
+<!DOCTYPE html><html class="default" lang="en"><head><meta charset="utf-8"/><meta http-equiv="x-ua-compatible" content="IE=edge"/><title>modifyChromeAPIAllowCORS | ai-research-agent</title><meta name="description" content="Documentation for ai-research-agent"/><meta name="viewport" content="width=device-width, initial-scale=1"/><link rel="stylesheet" href="../assets/style.css"/><link rel="stylesheet" href="../assets/highlight.css"/><script defer src="../assets/main.js"></script><script async src="../assets/icons.js" id="tsd-icons-script"></script><script async src="../assets/search.js" id="tsd-search-script"></script><script async src="../assets/navigation.js" id="tsd-nav-script"></script></head><body><script>console.log(`Loaded ${location.href}`)</script><script>document.documentElement.dataset.theme = localStorage.getItem("tsd-theme") || "os";document.body.style.display="none";setTimeout(() => app?app.showPage():document.body.style.removeProperty("display"),500)</script><header class="tsd-page-toolbar"><div class="tsd-toolbar-contents container"><div class="table-cell" id="tsd-search" data-base=".."><div class="field"><label for="tsd-search-field" class="tsd-widget tsd-toolbar-icon search no-caption"><svg width="16" height="16" viewBox="0 0 16 16" fill="none"><use href="../assets/icons.svg#icon-search"></use></svg></label><input type="text" id="tsd-search-field" aria-label="Search"/></div><div class="field"><div id="tsd-toolbar-links"><a href="https://github.com/vtempest/ai-research-agent">Source Code</a><a href="https://qwksearch.com">Live Demo</a><a href="https://discord.gg/SJdBqBz3tV">Discord Chat</a></div></div><ul class="results"><li class="state loading">Preparing search index...</li><li class="state failure">The search index is not available</li></ul><a href="../index.html" class="title">ai-research-agent</a></div><div class="table-cell" id="tsd-widgets"><a href="#" class="tsd-widget tsd-toolbar-icon menu no-caption" data-toggle="menu" aria-label="Menu"><svg width="16" height="16" viewBox="0 0 16 16" fill="none"><use href="../assets/icons.svg#icon-menu"></use></svg></a></div></div></header><div class="container container-main"><div class="col-content"><div class="tsd-page-title"><ul class="tsd-breadcrumb"><li><a href="../index.html">ai-research-agent</a></li><li><a href="../modules/src_chrome_extension_allow_cors.html">src/chrome-extension/allow-cors</a></li><li><a href="src_chrome_extension_allow_cors.modifyChromeAPIAllowCORS.html">modifyChromeAPIAllowCORS</a></li></ul><h1>Function modifyChromeAPIAllowCORS</h1></div><section class="tsd-panel"><ul class="tsd-signatures"><li class="tsd-signature tsd-anchor-link"><a id="modifyChromeAPIAllowCORS" class="tsd-anchor"></a><span class="tsd-kind-call-signature">modify<wbr/>ChromeAPIAllowCORS</span><span class="tsd-signature-symbol">(</span><span class="tsd-kind-parameter">initiatorDomains</span><span class="tsd-signature-symbol">?</span>, <span class="tsd-kind-parameter">receivingDomains</span><span class="tsd-signature-symbol">?</span><span class="tsd-signature-symbol">)</span><span class="tsd-signature-symbol">: </span><span class="tsd-signature-type">void</span><a href="#modifyChromeAPIAllowCORS" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></li><li class="tsd-description"><div class="tsd-comment tsd-typography"><p><b>Allow CORS - Chrome Extension API.</b> Add (Access-Control-Allow-Origin: *) rule to the response
+header of all requests to enable cross-domain requests to scrape data from other domains.
+CORS is voluntarily enforced by browser frontends, but not by curl or backend
+requests. Allowing CORS on the front-end means no proxy server is needed and web apps
+have Browser App functionality and can turn any data into an API.
+<a href="https://chromewebstore.google.com/detail/allow-cors-access-control/lhobafahddgcelffkeicbaginigeejlf?">There are several extension enabling CORS.</a>
+It is suggested to allow only on specific initiator domains to prevent errors
+commonly caused on secure sites like <a href="http://Claude.ai">Claude.ai</a>. Needs <a href="https://developer.chrome.com/docs/extensions/mv3/declarativeNetRequest/">Chrome API
+&quot;permissions&quot;</a>:[ &quot;declarativeNetRequest&quot;]</p>
+<ul>
+<li></li>
+</ul>
+</div><div class="tsd-parameters"><h4 class="tsd-parameters-title">Parameters</h4><ul class="tsd-parameter-list"><li><span><span class="tsd-kind-parameter">initiatorDomains</span>: <span class="tsd-signature-type">string</span><span class="tsd-signature-symbol"> = &quot;*://*/*&quot;</span></span><div class="tsd-comment tsd-typography"><p>The domains allowed to initiate CORS request</p>
+</div><div class="tsd-comment tsd-typography"></div></li><li><span><span class="tsd-kind-parameter">receivingDomains</span>: <span class="tsd-signature-type">string</span><span class="tsd-signature-symbol"> = &quot;*&quot;</span></span><div class="tsd-comment tsd-typography"><p>The domains allowed to receive the request</p>
+</div><div class="tsd-comment tsd-typography"></div></li></ul></div><h4 class="tsd-returns-title">Returns <span class="tsd-signature-type">void</span></h4><div class="tsd-comment tsd-typography"><h4 class="tsd-anchor-link"><a id="Example" class="tsd-anchor"></a>Example<a href="#Example" aria-label="Permalink" class="tsd-anchor-icon"><svg viewBox="0 0 24 24"><use href="../assets/icons.svg#icon-anchor"></use></svg></a></h4><pre><code class="ts"><span class="hl-7">chrome</span><span class="hl-2">.</span><span class="hl-7">runtime</span><span class="hl-2">.</span><span class="hl-7">onInstalled</span><span class="hl-2">.</span><span class="hl-5">addListener</span><span class="hl-2">(() </span><span class="hl-1">=&gt;</span><span class="hl-2"> {</span><br/><span class="hl-2">   </span><span class="hl-5">modifyChromeAPIAllowCORS</span><span class="hl-2">([</span><span class="hl-6">&quot;qwksearch.com&quot;</span><span class="hl-2">]);</span><br/><span class="hl-2"> });</span>
+</code><button type="button">Copy</button></pre>
+
+</div><aside class="tsd-sources"><ul><li>Defined in src/chrome-extension/allow-cors.js:20</li></ul></aside></li></ul></section></div><div class="col-sidebar"><div class="page-menu"><div class="tsd-navigation settings"><details class="tsd-accordion"><summary class="tsd-accordion-summary"><h3><svg width="20" height="20" viewBox="0 0 24 24" fill="none"><use href="../assets/icons.svg#icon-chevronDown"></use></svg>Settings</h3></summary><div class="tsd-accordion-details"><div class="tsd-filter-visibility"><span class="settings-label">Member Visibility</span><ul id="tsd-filter-options"><li class="tsd-filter-item"><label class="tsd-filter-input"><input type="checkbox" id="tsd-filter-protected" name="protected"/><svg width="32" height="32" viewBox="0 0 32 32" aria-hidden="true"><rect class="tsd-checkbox-background" width="30" height="30" x="1" y="1" rx="6" fill="none"></rect><path class="tsd-checkbox-checkmark" d="M8.35422 16.8214L13.2143 21.75L24.6458 10.25" stroke="none" stroke-width="3.5" stroke-linejoin="round" fill="none"></path></svg><span>Protected</span></label></li><li class="tsd-filter-item"><label class="tsd-filter-input"><input type="checkbox" id="tsd-filter-inherited" name="inherited" checked/><svg width="32" height="32" viewBox="0 0 32 32" aria-hidden="true"><rect class="tsd-checkbox-background" width="30" height="30" x="1" y="1" rx="6" fill="none"></rect><path class="tsd-checkbox-checkmark" d="M8.35422 16.8214L13.2143 21.75L24.6458 10.25" stroke="none" stroke-width="3.5" stroke-linejoin="round" fill="none"></path></svg><span>Inherited</span></label></li><li class="tsd-filter-item"><label class="tsd-filter-input"><input type="checkbox" id="tsd-filter-external" name="external"/><svg width="32" height="32" viewBox="0 0 32 32" aria-hidden="true"><rect class="tsd-checkbox-background" width="30" height="30" x="1" y="1" rx="6" fill="none"></rect><path class="tsd-checkbox-checkmark" d="M8.35422 16.8214L13.2143 21.75L24.6458 10.25" stroke="none" stroke-width="3.5" stroke-linejoin="round" fill="none"></path></svg><span>External</span></label></li></ul></div><div class="tsd-theme-toggle"><label class="settings-label" for="tsd-theme">Theme</label><select id="tsd-theme"><option value="os">OS</option><option value="light">Light</option><option value="dark">Dark</option></select></div></div></details></div></div><div class="site-menu"><nav id="tsd-sidebar-links" class="tsd-navigation"><a href="https://github.com/vtempest/ai-research-agent" class="tsd-nav-link">Source Code</a><a href="https://qwksearch.com" class="tsd-nav-link">Live Demo</a><a href="https://discord.gg/SJdBqBz3tV" class="tsd-nav-link">Discord Chat</a></nav><nav class="tsd-navigation"><a href="../index.html"><svg class="tsd-kind-icon" viewBox="0 0 24 24"><use href="../assets/icons.svg#icon-1"></use></svg><span>ai-research-agent</span></a><ul class="tsd-small-nested-navigation" id="tsd-nav-container" data-base=".."><li>Loading...</li></ul></nav></div></div></div><footer></footer><div class="overlay"></div><script async src="https://www.googletagmanager.com/gtag/js?id=G-E5TZ32BZDF"></script><script>window.dataLayer = window.dataLayer || [];
+function gtag(){dataLayer.push(arguments);}
+gtag('js', new Date());
+gtag('config', 'G-E5TZ32BZDF');</script></body></html>