Merge branch 'master' of https://github.com/codemkim/Hackaton

README.md

@@ -0,0 +1,13 @@
+# GIS_Hackathon
+
+Team Header : [곽수연](https://github.com/suyeon-K)
+
+Collaborated with [곽희원](https://github.com/HeewonKwak), [김현우](https://github.com/codemkim), [정현경](https://github.com/hyeonkyeong31)
+
+WEB: 김현우(backend), 정현경(frontend)
+
+Model: 곽수연, 곽희원
+
+***
+
+Reference: [AliaksandrSiarohin](https://github.com/AliaksandrSiarohin/first-order-model), [LeadsheetArrangement](https://github.com/liuhaumin/LeadsheetArrangement)

Web/musegan_model/Pipfile

@@ -0,0 +1,29 @@
+[[source]]
+name = "pypi"
+verify_ssl = true
+url = "https://pypi.org/simple"
+
+[requires]
+python_version = "3.6"
+
+[dev-packages]
+
+[packages]
+imageio = "==2.1.2"
+madmom = "==0.16.dev0"
+Markdown = "==2.2.0"
+midi2audio = "==0.1.1"
+mido = "==1.2.8"
+music21 = "==5.1.0"
+numpy = "==1.14.3"
+Pillow = "==5.0.0"
+pretty-midi = "==0.2.8"
+protobuf = "==3.5.1"
+pypianoroll = "==0.1.3"
+PyYAML = "==3.12"
+scipy = "==1.0.0"
+SharedArray = "==2.0.4"
+six = "==1.11.0"
+tensorflow-gpu = "==1.2.0"
+tensorflow-tensorboard = "==0.4.0rc3"
+Werkzeug = "==0.14.1"

Web/musegan_model/README.md

@@ -0,0 +1,108 @@
+# LeadsheetArrangement 自動簡譜編曲 :musical_note:
+[Lead Sheet Arrangement](https://liuhaumin.github.io/LeadsheetArrangement/) is a task to automatically accompany the existed or generated lead sheets with multiple instruments. The rhythm, pitch notes, onsets and even playing style of different instruments are all learned by the model. 
+
+We train the model with Lakh Pianoroll Dataset (LPD) to generate pop song style arrangement consisting of bass, guitar, piano, strings and drums.
+
+Sample results are available
+[here](https://liuhaumin.github.io/LeadsheetArrangement/results).
+
+## Papers
+
+__Lead sheet generation and arrangement by conditional generative adversarial network__<br>
+Hao-Min Liu and Yi-Hsuan Yang,
+to appear in *International Conference on Machine Learning and Applications* (ICMLA), 2018.
+[[arxiv](https://arxiv.org/abs/1807.11161)]
+
+__Lead Sheet Generation and Arrangement via a Hybrid Generative Model__<br>
+Hao-Min Liu\*, Meng-Hsuan Wu\*, and Yi-Hsuan Yang
+(\*equal contribution)<br>
+in _ISMIR Late-Breaking Demos Session_, 2018.
+(non-refereed two-page extended abstract)<br>
+[[paper](https://liuhaumin.github.io/LeadsheetArrangement/pdf/ismir2018leadsheetarrangement.pdf)]
+[[poster](https://liuhaumin.github.io/LeadsheetArrangement/pdf/ismir-lbd-poster_A0_final.pdf)]
+
+__Lead sheet and Multi-track Piano-roll generation using MuseGAN__<br>
+Hao-Min Liu, Hao-Wen Dong, Wen-Yi Hsiao and Yi-Hsuan Yang,
+in *GPU Technology Conference* (GTC), 2018.
+[[poster](https://liuhaumin.github.io/LeadsheetArrangement/pdf/GTC_poster_HaoMin.pdf)]
+
+## Usage
+### Approach 1: Run pretrained model using Scripts
+1. Go to folder (./data/preprocessing/)
+2. Put your file in folder (./data/preprocessing/mysong_mid_C/) with name specified as __"test.mid"__
+    * Note that test.mid should be in C key and two tracks(melody and chord)
+3. Back to root directory and run __scripts.py__
+4. Find the outputs in folder (./output/)
+
+### Approach 2: Train or test the model step by step by your own
+#### Step 0: preprocessing the midi data
+1. Go to folder (./data/preprocessing/)
+2. Put your input song_name.midi file in folder (./data/preprocessing/mysong_mid_C/)
+    * Note that song_name.midi should be in C key and two tracks(melody and chord)
+3. Back to folder (./preprocessing/)
+4. Run pianoroll_mysong.ipynb
+    * Remember to change filenames in code lines into "song_name"
+    * Output files will be stored in folder(./data/chord_roll/val/)with name as "x_bar_chroma_song_name.npy" and "y_bar_chroma_song_name.npy"
+
+#### Step 1: Loading the data
+1. Open file store_sa.py
+2. Turn filename in code line # 36 into your filename, i.e. ('y_bar_chroma_song_name.npy')
+3. Load the data by running store_sa.py
+
+#### Step 2: adjust training or testing modes in main.py
+```python
+import tensorflow as tf
+from musegan.core import MuseGAN
+from musegan.components import NowbarHybrid
+from config import *
+
+# Initialize a tensorflow session
+
+""" Create TensorFlow Session """
+with tf.Session() as sess:
+
+    # === Prerequisites ===
+    # Step 1 - Initialize the training configuration        
+    t_config = TrainingConfig
+    t_config.exp_name = 'exps/nowbar_hybrid'        
+
+    # Step 2 - Select the desired model
+    model = NowbarHybrid(NowBarHybridConfig)
+
+    # Step 3 - Initialize the input data object
+    input_data = InputDataNowBarHybrid(model)
+
+    # Step 4 - Load training data
+    path_x_train_bar = 'tra_X_bars'
+    path_y_train_bar = 'tra_y_bars'
+    input_data.add_data_sa(path_x_train_bar, path_y_train_bar, 'train') # x: input, y: conditional feature
+
+    # Step 5 - Initialize a museGAN object
+    musegan = MuseGAN(sess, t_config, model)
+
+    # === Training ===
+    musegan.train(input_data)
+
+    # === Load a Pretrained Model ===
+    musegan.load(musegan.dir_ckpt)
+
+    # === Generate Samples ===
+    path_x_test_bar = 'val_X_bars'
+    path_y_test_bar = 'val_y_bars'
+    input_data.add_data_sa(path_x_test_bar, path_y_test_bar, key='test')
+    musegan.gen_test(input_data, is_eval=True)
+
+```
+#### Step 3: run main.py
+1. Training mode
+   * Checkpoints are stored in folder (./exps/nowbar_hybrid/checkpoint/)
+2. Testing mode
+   * the output files are stored in folder (./exps/nowbar_hybrid/gen/)
+
+#### Step 4: postprocessing the output midi
+1. Go to folder (./postprocessing/)
+2. Run file npy2mid.ipynb
+   * Remember to change filenames in code lines into "song_name"
+   * Output files are stored in folder (./output/)
+
+

Web/musegan_model/config.py

@@ -0,0 +1,229 @@
+'''
+Model Configuration
+'''
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import numpy as np
+from shutil import copyfile
+import os
+import SharedArray as sa
+import tensorflow as tf
+import glob
+
+print('[*] config...')
+
+# class Dataset:
+##TRACK_NAMES = ['mel', 'acc']
+TRACK_NAMES = ['bass', 'drums', 'guitar', 'piano', 'strings']
+
+
+def get_colormap():
+    colormap = np.array([[1., 0., 0.],
+                         [0., 1., 0.],
+                         [0., 0., 1.],
+                         [1., .5, 0.],
+                         [0., .5, 1.]])
+
+    ##colormap = np.array([[1., 0., 0.],
+    ##                     [0., 1., 0.]])
+    return tf.constant(colormap, dtype=tf.float32, name='colormap')
+
+###########################################################################
+# Training
+###########################################################################
+
+class TrainingConfig:
+    is_eval = True
+    batch_size = 64
+    #batch_size = 32
+    epoch = 3
+    iter_to_save = 100
+    sample_size = 64
+    print_batch = True
+    ##drum_filter = np.tile([1,0.3,0,0,0,0.3], 16) ## for 96 timesteps per bar
+    drum_filter = np.tile([1,0.3,0,0,0,0.3], 8)  ## for 48 timesteps per bar
+
+    scale_mask = [1., 0., 1., 0., 1., 1., 0., 1., 0., 1., 0., 1.]
+    inter_pair = [(0,2), (0,3), (0,4), (2,3), (2,4), (3,4)]
+    ##inter_pair = [(0,1)]
+    track_names = TRACK_NAMES
+    track_dim = len(track_names)
+    eval_map = np.array([
+                    [1, 1, 1, 1, 1],  # metric_is_empty_bar
+                    [1, 1, 1, 1, 1],  # metric_num_pitch_used
+                    [1, 0, 1, 1, 1],  # metric_too_short_note_ratio
+                    [1, 0, 1, 1, 1],  # metric_polyphonic_ratio
+                    [1, 0, 1, 1, 1],  # metric_in_scale
+                    [0, 1, 0, 0, 0],  # metric_drum_pattern
+                    [1, 0, 1, 1, 1]   # metric_num_chroma_used
+                ])
+
+
+    ##eval_map = np.array([
+    ##                [1,1],  # metric_is_empty_bar
+    ##                [1,1],  # metric_num_pitch_used
+    ##                [1,1],  # metric_too_short_note_ratio
+    ##                [1,1],  # metric_polyphonic_ratio
+    ##                [1,1],  # metric_in_scale
+    ##                [0,0],  # metric_drum_pattern
+    ##                [1,1]   # metric_num_chroma_used
+    ##            ])
+
+    exp_name = 'exp'
+    gpu_num = '1'
+
+
+###########################################################################
+# Model Config
+###########################################################################
+
+class ModelConfig:
+    output_w = 48
+    output_h = 84
+    lamda = 10
+    batch_size = 64
+    #batch_size = 32
+    beta1 = 0.5
+    beta2 = 0.9
+    lr = 2e-4
+    is_bn = True
+    colormap = get_colormap()
+
+# image
+class MNISTConfig(ModelConfig):
+    output_w = 28
+    output_h = 28
+    z_dim = 74
+    output_dim = 1
+
+# RNN
+class RNNConfig(ModelConfig):
+    track_names = ['All']
+    track_dim = 1
+    output_bar = 4
+    #output_bar = 8
+    z_inter_dim = 128
+    output_dim = 1
+    #output_dim = 5
+    acc_idx = None
+    state_size = 128
+
+class RNNHybridConfig(ModelConfig):
+    track_names = TRACK_NAMES
+    track_dim = 2
+    output_bar = 4
+    z_inter_dim = 32
+    z_intra_dim = 32
+    output_dim = 1
+    acc_idx = None
+    state_size = 32
+
+# condi 2 track TT
+class NowBarRNNHybridConfig(ModelConfig):
+    track_names = TRACK_NAMES
+    acc_idx = 0
+    track_dim = 2
+    output_bar = 4
+    z_inter_dim = 32
+    z_intra_dim = 32
+    output_dim = 1
+    state_size = 32
+
+# condi 6 track LPD
+##class NowBarRNNHybridConfig(ModelConfig):
+##    track_names = TRACK_NAMES
+##    acc_idx = 5
+##    track_dim = 6
+##    output_bar = 4
+##    z_inter_dim = 32
+##    z_intra_dim = 32
+##    output_dim = 1
+##    state_size = 32
+
+
+# onebar
+class OneBarHybridConfig(ModelConfig):
+    track_names = TRACK_NAMES
+    track_dim = 5
+    acc_idx = None
+    z_inter_dim = 64
+    z_intra_dim = 64
+    output_dim = 1
+
+class OneBarJammingConfig(ModelConfig):
+    track_names = TRACK_NAMES
+    track_dim = 5
+    acc_idx = None
+    z_intra_dim = 128
+    output_dim = 1
+
+class OneBarComposerConfig(ModelConfig):
+    track_names = ['All']
+    track_dim = 1
+    acc_idx = None
+    z_inter_dim = 128
+    output_dim = 5
+
+# nowbar
+class NowBarHybridConfig(ModelConfig):
+    track_names = TRACK_NAMES
+    track_dim = 5
+    ##acc_idx = 4
+    z_inter_dim = 64
+    z_intra_dim = 64
+    output_dim = 1
+    type_ = 0 # 0. chord sequence 1. chroma sequence 2. chroma vector  3. chord label
+    acc_output_w = 48 # chord sequence: 48, chroma sequence: 48, chroma vector: 4, chord vector: 4
+    acc_output_h = 84 # chord sequence: 84, chroma sequence: 12, chroma vector:12, chord vector: 84
+
+class NowBarJammingConfig(ModelConfig):
+    track_names = TRACK_NAMES
+    track_dim = 5
+    acc_idx = 4
+    z_intra_dim = 128
+    output_dim = 1
+
+class NowBarComposerConfig(ModelConfig):
+    track_names = ['All']
+    track_dim = 1
+    acc_idx = 4
+    z_inter_dim = 128
+    output_dim = 5
+
+# Temporal
+class TemporalHybridConfig(ModelConfig):
+    track_names = TRACK_NAMES
+    # track_dim = 5
+    track_dim = 1
+    output_bar = 4
+    z_inter_dim = 32
+    z_intra_dim = 32
+    acc_idx = None
+    output_dim = 1
+
+class TemporalJammingConfig(ModelConfig):
+    track_names = TRACK_NAMES
+    track_dim = 5
+    output_bar = 4
+    z_intra_dim = 64
+    output_dim = 1
+
+class TemporalComposerConfig(ModelConfig):
+    track_names = ['All']
+    track_dim = 1
+    output_bar = 4
+    z_inter_dim = 64
+    acc_idx = None
+    output_dim = 5
+
+class NowBarTemporalHybridConfig(ModelConfig):
+    track_names = TRACK_NAMES
+    acc_idx = 4
+    track_dim = 5
+    output_bar = 4
+    z_inter_dim = 32
+    z_intra_dim = 32
+    acc_idx = 4
+    output_dim = 1

Web/musegan_model/data/Readme.md

@@ -0,0 +1,5 @@
+# Training and Testing Data
+Because the size of the dataset is too large, it could not be uploaded on Github.
+Please find the attached link and get the traing and testing data from my Google Drive.
+
+- [data.zip](https://drive.google.com/open?id=1teFnkVZndIAuD97GQfddDWnu51CJvaqU)