Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SAMOA-65 JSON serializer/deserializer dedicated for Kafka components #64

Open
wants to merge 18 commits into
base: master
Choose a base branch
from
Open

SAMOA-65 JSON serializer/deserializer dedicated for Kafka components #64

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
0e90fe8
Initial structure of Kafka components.
pwawrzyniak Mar 14, 2017
2168400
Code for KafkaEntranceProcessor (consuming messages from Kafka)
pwawrzyniak Mar 17, 2017
cf2ff36
Updated comments
pwawrzyniak Mar 17, 2017
26d86da
Sample serializer/deserializer for JSON and InstanceContentEvent
pwawrzyniak Mar 17, 2017
45b3546
KafkaDestinationProcessor implementation (sending msg to Kafka
pwawrzyniak Mar 17, 2017
2e6d502
Tests for kafkaUtils and KafkaEntranceProcessor, minor changes in cla…
pwawrzyniak Mar 24, 2017
208febc
Tests for KafkaDestinationProcessor, minor changes in classes
pwawrzyniak Apr 11, 2017
ff6b84d
Added Kafka Avro serializer with unit tests. Added Kafka task with un…
Apr 28, 2017
7997fd5
Merge branch 'SAMOA-65' of https://github.com/pwawrzyniak/incubator-s…
May 1, 2017
ae7889f
Fixing avro deserialization
May 10, 2017
107c4c5
Fixing avro deserialization
May 10, 2017
cde0642
Rebuild of Consumer-related classes (added separate thread to read fr…
pwawrzyniak May 12, 2017
f20301a
Minor change in tests
pwawrzyniak May 12, 2017
b4ee754
Fixing avro deserializer (using ReflectDatumReader) to be able to rea…
May 16, 2017
d636ba9
Changes in JSON mapper
pwawrzyniak May 16, 2017
57f80a5
repo cleanup, split code into 3 branches
pwawrzyniak Jun 7, 2017
04a28bb
JSON mapper with test
pwawrzyniak Jun 7, 2017
1a2d170
Minor changes, code cleanup etc.
pwawrzyniak Jun 19, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 16 additions & 15 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
#maven
target/

#eclipse
.classpath
.project
.settings/

#DS_Store
.DS_Store

#intellij
.idea/
*.iml
*.iws
#maven
target/

#eclipse
.classpath
.project
.settings/

#DS_Store
.DS_Store

#intellij
.idea/
*.iml
*.iws
/samoa-api/nbproject/
283 changes: 154 additions & 129 deletions samoa-api/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,129 +1,154 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
#%L
SAMOA
%%
Copyright (C) 2014 - 2015 Apache Software Foundation
%%
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
#L%
-->

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<name>samoa-api</name>
<description>API and algorithms for SAMOA</description>

<artifactId>samoa-api</artifactId>
<parent>
<groupId>org.apache.samoa</groupId>
<artifactId>samoa</artifactId>
<version>0.5.0-incubating-SNAPSHOT</version>
</parent>

<dependencies>
<dependency>
<groupId>com.yammer.metrics</groupId>
<artifactId>metrics-core</artifactId>
<version>${metrics-core.version}</version>
</dependency>

<dependency>
<groupId>net.jcip</groupId>
<artifactId>jcip-annotations</artifactId>
<version>${jcip-annotations.version}</version>
</dependency>

<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>${commons-lang3.version}</version>
</dependency>

<dependency>
<groupId>com.github.javacliparser</groupId>
<artifactId>javacliparser</artifactId>
<version>${javacliparser.version}</version>
</dependency>

<dependency>
<groupId>org.apache.samoa</groupId>
<artifactId>samoa-instances</artifactId>
<version>${project.version}</version>
</dependency>

<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>${guava.version}</version>
</dependency>

<dependency>
<groupId>com.esotericsoftware.kryo</groupId>
<artifactId>kryo</artifactId>
<version>${kryo.version}</version>
</dependency>

<dependency>
<groupId>com.dreizak</groupId>
<artifactId>miniball</artifactId>
<version>${miniball.version}</version>
</dependency>

<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>${hadoop.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<version>${hadoop.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-minicluster</artifactId>
<version>${hadoop.version}</version>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>${maven-dependency-plugin.version}</version>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
<overWriteReleases>false</overWriteReleases>
<overWriteSnapshots>false</overWriteSnapshots>
<overWriteIfNewer>true</overWriteIfNewer>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
<?xml version="1.0" encoding="UTF-8"?>
<!--
#%L
SAMOA
%%
Copyright (C) 2014 - 2015 Apache Software Foundation
%%
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
#L%
-->

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<name>samoa-api</name>
<description>API and algorithms for SAMOA</description>

<artifactId>samoa-api</artifactId>
<parent>
<groupId>org.apache.samoa</groupId>
<artifactId>samoa</artifactId>
<version>0.5.0-incubating-SNAPSHOT</version>
</parent>

<dependencies>
<dependency>
<groupId>com.yammer.metrics</groupId>
<artifactId>metrics-core</artifactId>
<version>${metrics-core.version}</version>
</dependency>

<dependency>
<groupId>net.jcip</groupId>
<artifactId>jcip-annotations</artifactId>
<version>${jcip-annotations.version}</version>
</dependency>

<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>${commons-lang3.version}</version>
</dependency>

<dependency>
<groupId>com.github.javacliparser</groupId>
<artifactId>javacliparser</artifactId>
<version>${javacliparser.version}</version>
</dependency>

<dependency>
<groupId>org.apache.samoa</groupId>
<artifactId>samoa-instances</artifactId>
<version>${project.version}</version>
</dependency>

<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>${guava.version}</version>
</dependency>

<dependency>
<groupId>com.esotericsoftware.kryo</groupId>
<artifactId>kryo</artifactId>
<version>${kryo.version}</version>
</dependency>

<dependency>
<groupId>com.dreizak</groupId>
<artifactId>miniball</artifactId>
<version>${miniball.version}</version>
</dependency>

<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>${hadoop.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<version>${hadoop.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-minicluster</artifactId>
<version>${hadoop.version}</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>0.10.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>0.10.2.0</version>
<classifier>test</classifier>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.11</artifactId>
<version>0.10.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.11</artifactId>
<version>0.10.2.0</version>
<classifier>test</classifier>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>${maven-dependency-plugin.version}</version>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
<overWriteReleases>false</overWriteReleases>
<overWriteSnapshots>false</overWriteSnapshots>
<overWriteIfNewer>true</overWriteIfNewer>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Loading