energy-data

Java (Spring Boot) application for fetching, parsing, and saving hourly energy consumption data by type.

Description

The data is fetched from www.agora-energiewende.de as some mix of JS, JSON, and HTML elements, parsed and saved in database. The project is configured to be used with MySQL databases but can be easily switched to any other SQL database supported by Hibernate (see application.properties).

(This application fetches energy data for Germany, some or all names for energy types will be only in german)

GUI:

Example of data:

Installation

The project can be simply launched from your IDE by running EnergyDataApplication. All dependencies should be downloaded automatically by Maven.

Properties

This application needs a database to function, connection properties can be configured in application.properties, or you can create application-local.properties and configure it there (when using ....local.properties don't forget to change active profile in run configuration).

Example of application-local.properties:

# Database connection and driver
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.username=root
spring.datasource.password=password
spring.datasource.url=jdbc:mysql://localhost:3309/test-energy?rewriteBatchedStatements=true

# Use this to generate tables automatically
spring.jpa.hibernate.ddl-auto=update

Database type

This application is configured to be used with MySQL database, but can be changed (not tested) to another database. For that, you need to add JBDC Driver dependency for your database.

For example for PostgreSQL:

<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <version>42.5.4</version>
</dependency>

You also need to update your application properties (see Properties section), specifically spring.datasource.driver-class-name.