Skip to content

Commit

Permalink
Site updated: 2020-04-08 00:26:24
Browse files Browse the repository at this point in the history
  • Loading branch information
@dragonskyhydra
dragonskyhydra committed Apr 7, 2020
1 parent 47b1566 commit a13c15a
Show file tree
Hide file tree
Showing 19 changed files with 88 additions and 88 deletions.
2 changes: 1 addition & 1 deletion 2019/12/22/hive-notes2/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,7 @@ <h2 id="Hive正则表达式数据抽取笔记"><a href="#Hive正则表达式数
<ul class="pager">

<li class="previous">
<a href="/2020/04/07/hadoop-notes1/" data-toggle="tooltip" data-placement="top" title="hadoop生态笔记（1）">&larr; Previous post</a>
<a href="/2020/04/07/hadoop-notes1/" data-toggle="tooltip" data-placement="top" title="Hadoop生态笔记（1）">&larr; Previous post</a>
</li>


Expand Down
6 changes: 3 additions & 3 deletions 2020/04/07/hadoop-notes1/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
<script async defer src="https://buttons.github.io/buttons.js"></script>
<title>

hadoop生态笔记（1） - null
Hadoop生态笔记（1） - null

</title>

Expand Down Expand Up @@ -105,7 +105,7 @@
<div class="tags">

</div>
<h1>hadoop生态笔记（1）</h1>
<h1>Hadoop生态笔记（1）</h1>
<h2 class="subheading"></h2>
<span class="meta">
Posted by hydra on
Expand Down Expand Up @@ -245,7 +245,7 @@ <h2 class="subheading"></h2>
col-md-10 col-md-offset-1
post-container">

<h2 id="hadoop生态，数据存储。"><a href="#hadoop生态，数据存储。" class="headerlink" title="hadoop生态，数据存储。"></a>hadoop生态，数据存储。</h2><h3 id="数据采集，转换（存储容器格式）"><a href="#数据采集，转换（存储容器格式）" class="headerlink" title="数据采集，转换（存储容器格式）"></a>数据采集，转换（存储容器格式）</h3><p>Flume、kafka、sqoop</p>
<h2 id="Hadoop生态，数据存储。"><a href="#Hadoop生态，数据存储。" class="headerlink" title="Hadoop生态，数据存储。"></a>Hadoop生态，数据存储。</h2><h3 id="数据采集，转换（存储容器格式）"><a href="#数据采集，转换（存储容器格式）" class="headerlink" title="数据采集，转换（存储容器格式）"></a>数据采集，转换（存储容器格式）</h3><p>Flume、kafka、sqoop</p>
<h3 id="文件系统"><a href="#文件系统" class="headerlink" title="文件系统"></a>文件系统</h3><p>HDFS</p>
<h3 id="数据格式"><a href="#数据格式" class="headerlink" title="数据格式"></a>数据格式</h3><p>1、文本数据CSV</p>
<p>2、结构化文本数据XML和JSON，这种文件很难分片，Hadoop没有为这类格式提供内置的InputFormat。<br>使用类似Avro的容器格式。将数据转换为Avro的内容，从而为数据存储与数据处理提供更紧密、有效的方法。<br>使用处理XML或JSON文件的专用库。比如，XML，Pig的PiggyBank库中的XMLLoader。JSON，Elephant Bird项目提供的LzoJsonInputFormat。</p>
Expand Down
10 changes: 5 additions & 5 deletions archives/2019/11/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ <h1>数据之道</h1>
<i class="fa fa-angle-double-right" aria-hidden="true"></i>
<a href="/2020/04/07/hadoop-notes1/" style="color: #0085a1">
<span>
hadoop生态笔记（1）
Hadoop生态笔记（1）
</span>
</a>
<!-- <p class="post-meta">{{ post.date | date:"%Y-%m-%d" }}</p> -->
Expand Down Expand Up @@ -485,15 +485,15 @@ <h5><a href="/tags/">Tags</a></h5>



<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>
<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>



<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>
<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>



<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>
<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>


</div>
Expand Down Expand Up @@ -550,7 +550,7 @@ <h5>Recent posts</h3>
<ul>

<li>
<a href="/2020/04/07/hadoop-notes1/">hadoop生态笔记（1）</a>
<a href="/2020/04/07/hadoop-notes1/">Hadoop生态笔记（1）</a>
</li>

<li>
Expand Down
10 changes: 5 additions & 5 deletions archives/2019/12/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ <h1>数据之道</h1>
<i class="fa fa-angle-double-right" aria-hidden="true"></i>
<a href="/2020/04/07/hadoop-notes1/" style="color: #0085a1">
<span>
hadoop生态笔记（1）
Hadoop生态笔记（1）
</span>
</a>
<!-- <p class="post-meta">{{ post.date | date:"%Y-%m-%d" }}</p> -->
Expand Down Expand Up @@ -485,15 +485,15 @@ <h5><a href="/tags/">Tags</a></h5>



<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>
<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>



<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>
<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>



<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>
<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>


</div>
Expand Down Expand Up @@ -550,7 +550,7 @@ <h5>Recent posts</h3>
<ul>

<li>
<a href="/2020/04/07/hadoop-notes1/">hadoop生态笔记（1）</a>
<a href="/2020/04/07/hadoop-notes1/">Hadoop生态笔记（1）</a>
</li>

<li>
Expand Down
10 changes: 5 additions & 5 deletions archives/2019/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ <h1>数据之道</h1>
<i class="fa fa-angle-double-right" aria-hidden="true"></i>
<a href="/2020/04/07/hadoop-notes1/" style="color: #0085a1">
<span>
hadoop生态笔记（1）
Hadoop生态笔记（1）
</span>
</a>
<!-- <p class="post-meta">{{ post.date | date:"%Y-%m-%d" }}</p> -->
Expand Down Expand Up @@ -485,15 +485,15 @@ <h5><a href="/tags/">Tags</a></h5>



<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>
<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>



<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>
<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>



<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>
<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>


</div>
Expand Down Expand Up @@ -550,7 +550,7 @@ <h5>Recent posts</h3>
<ul>

<li>
<a href="/2020/04/07/hadoop-notes1/">hadoop生态笔记（1）</a>
<a href="/2020/04/07/hadoop-notes1/">Hadoop生态笔记（1）</a>
</li>

<li>
Expand Down
10 changes: 5 additions & 5 deletions archives/2020/04/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ <h1>数据之道</h1>
<i class="fa fa-angle-double-right" aria-hidden="true"></i>
<a href="/2020/04/07/hadoop-notes1/" style="color: #0085a1">
<span>
hadoop生态笔记（1）
Hadoop生态笔记（1）
</span>
</a>
<!-- <p class="post-meta">{{ post.date | date:"%Y-%m-%d" }}</p> -->
Expand Down Expand Up @@ -485,15 +485,15 @@ <h5><a href="/tags/">Tags</a></h5>



<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>
<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>



<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>
<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>



<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>
<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>


</div>
Expand Down Expand Up @@ -550,7 +550,7 @@ <h5>Recent posts</h3>
<ul>

<li>
<a href="/2020/04/07/hadoop-notes1/">hadoop生态笔记（1）</a>
<a href="/2020/04/07/hadoop-notes1/">Hadoop生态笔记（1）</a>
</li>

<li>
Expand Down
10 changes: 5 additions & 5 deletions archives/2020/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ <h1>数据之道</h1>
<i class="fa fa-angle-double-right" aria-hidden="true"></i>
<a href="/2020/04/07/hadoop-notes1/" style="color: #0085a1">
<span>
hadoop生态笔记（1）
Hadoop生态笔记（1）
</span>
</a>
<!-- <p class="post-meta">{{ post.date | date:"%Y-%m-%d" }}</p> -->
Expand Down Expand Up @@ -485,15 +485,15 @@ <h5><a href="/tags/">Tags</a></h5>



<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>
<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>



<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>
<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>



<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>
<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>


</div>
Expand Down Expand Up @@ -550,7 +550,7 @@ <h5>Recent posts</h3>
<ul>

<li>
<a href="/2020/04/07/hadoop-notes1/">hadoop生态笔记（1）</a>
<a href="/2020/04/07/hadoop-notes1/">Hadoop生态笔记（1）</a>
</li>

<li>
Expand Down
10 changes: 5 additions & 5 deletions archives/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ <h1>数据之道</h1>
<i class="fa fa-angle-double-right" aria-hidden="true"></i>
<a href="/2020/04/07/hadoop-notes1/" style="color: #0085a1">
<span>
hadoop生态笔记（1）
Hadoop生态笔记（1）
</span>
</a>
<!-- <p class="post-meta">{{ post.date | date:"%Y-%m-%d" }}</p> -->
Expand Down Expand Up @@ -485,15 +485,15 @@ <h5><a href="/tags/">Tags</a></h5>



<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>
<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>



<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>
<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>



<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>
<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>


</div>
Expand Down Expand Up @@ -550,7 +550,7 @@ <h5>Recent posts</h3>
<ul>

<li>
<a href="/2020/04/07/hadoop-notes1/">hadoop生态笔记（1）</a>
<a href="/2020/04/07/hadoop-notes1/">Hadoop生态笔记（1）</a>
</li>

<li>
Expand Down
10 changes: 5 additions & 5 deletions archives/page/2/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ <h1>数据之道</h1>
<i class="fa fa-angle-double-right" aria-hidden="true"></i>
<a href="/2020/04/07/hadoop-notes1/" style="color: #0085a1">
<span>
hadoop生态笔记（1）
Hadoop生态笔记（1）
</span>
</a>
<!-- <p class="post-meta">{{ post.date | date:"%Y-%m-%d" }}</p> -->
Expand Down Expand Up @@ -485,15 +485,15 @@ <h5><a href="/tags/">Tags</a></h5>



<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>
<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>



<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>
<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>



<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>
<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>


</div>
Expand Down Expand Up @@ -550,7 +550,7 @@ <h5>Recent posts</h3>
<ul>

<li>
<a href="/2020/04/07/hadoop-notes1/">hadoop生态笔记（1）</a>
<a href="/2020/04/07/hadoop-notes1/">Hadoop生态笔记（1）</a>
</li>

<li>
Expand Down
12 changes: 6 additions & 6 deletions index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,7 @@ <h1>数据之道</h1>
<div class="w3-row-padding">
<a href="/2020/04/07/hadoop-notes1/">
<h2 class="post-title">
hadoop生态笔记（1）
Hadoop生态笔记（1）
</h2>
<h3 class="post-subtitle">

Expand All @@ -264,7 +264,7 @@ <h3 class="post-subtitle">
<a href="/2020/04/07/hadoop-notes1/">

<div class="post-content-preview">
hadoop生态，数据存储。数据采集，转换（存储容器格式）Flume、kafka、sqoop
Hadoop生态，数据存储。数据采集，转换（存储容器格式）Flume、kafka、sqoop
文件系统HDFS
数据格式1、文本数据CSV
2、结构化文本数据XML和JSON，这种文件很难分片，Hadoop没有为这类格式提供内置的InputFormat。使用类似Avro的容器格式。将数据转换为Avro的内容，从而为数据存储与数据处理提供更紧密、有效的方法。使用处理XML或JSON文件的.........
Expand Down Expand Up @@ -681,15 +681,15 @@ <h5><a href="/tags/">Tags</a></h5>



<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>
<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>



<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>
<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>



<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>
<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>


</div>
Expand Down Expand Up @@ -746,7 +746,7 @@ <h5>Recent posts</h3>
<ul>

<li>
<a href="/2020/04/07/hadoop-notes1/">hadoop生态笔记（1）</a>
<a href="/2020/04/07/hadoop-notes1/">Hadoop生态笔记（1）</a>
</li>

<li>
Expand Down
8 changes: 4 additions & 4 deletions page/2/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -333,15 +333,15 @@ <h5><a href="/tags/">Tags</a></h5>



<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>
<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>



<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>
<a href="/tags/#RDBMS" title="RDBMS" rel="1">RDBMS</a>



<a href="/tags/#Vertica" title="Vertica" rel="2">Vertica</a>
<a href="/tags/#NoSQL" title="NoSQL" rel="1">NoSQL</a>


</div>
Expand Down Expand Up @@ -398,7 +398,7 @@ <h5>Recent posts</h3>
<ul>

<li>
<a href="/2020/04/07/hadoop-notes1/">hadoop生态笔记（1）</a>
<a href="/2020/04/07/hadoop-notes1/">Hadoop生态笔记（1）</a>
</li>

<li>
Expand Down
Loading

0 comments on commit a13c15a

Please sign in to comment.