diff --git a/code/plugins/plugins.json b/code/plugins/plugins.json index 8478b5e..f716e38 100644 --- a/code/plugins/plugins.json +++ b/code/plugins/plugins.json @@ -29,7 +29,7 @@ }, { "name": "AMICA", - "type": "wiki", + "type": "readme", "link": "https://github.com/sccn/amica", "desc": "Computes Adaptive Mixture Independent Component Analysis", "cat": "processing" diff --git a/plugins/amica/AMICA-Compilation-instructions.md b/plugins/amica/AMICA-Compilation-instructions.md deleted file mode 100644 index 69f8332..0000000 --- a/plugins/amica/AMICA-Compilation-instructions.md +++ /dev/null @@ -1,105 +0,0 @@ ---- -layout: default -parent: AMICA -grand_parent: Plugins -render_with_liquid: false - -title: AMICA-Compilation-instructions -long_title: AMICA-Compilation-instructions ---- -# How to compile with Intel Fortran on Windows - -1. Install Intel OneAPI Base Toolkit for Windows. -2. Install Intel OneAPI HPC Toolkit for Windows. -3. Open Intel OneAPI command window (in Start menu, 32 or 64 bit as appropriate). Change to directory with Amica code, and compile Amica with the command (/F sets the stack size): - - > mpif90 /Qopenmp /Qmkl /F2147483648 /DMKL /fpp /O3 /exe:amica15mkl.exe funmod2.f90 amica15.f90 - -4. Test: - - > .\amica15mkl.exe .\amicadefs.param - -5. The files impi.dll and libfabric.dll should be copied to executable folder when running outside OneAPI command window. Search OneAPI mpi directory for locations. - - -# How to compile with Intel Fortran on Mac - -0. These are old instructions. Try using Intel OneAPI modifiying the commands similar to the instructions for Windows above. -1. Install Intel Fortran compiler for Mac/Linux (free demo). - See https://software.intel.com/en-us/intel-parallel-studio-xe - -2. Compile MPICH2 setting environmental vars CC, CXX, FC, and F77 to icc and ifort. Set $FBIN to Intel Fortran bin directory. - - i) Download the mpich-3.2 code from: http://www.mpich.org/static/downloads/3.2/mpich-3.2.tar.gz - - ii) Compile mpich-3.2: - - $ cp /Users/$USER/downloads/mpich-3.2.tar.gz . - $ setenv CC $FBIN/icc - $ setenv CXX $FBIN/icc - $ setenv F77 $FBIN/ifort - $ setenv FC $FBIN/ifort - $ tar xvf mpich-3.2.tar.gz - $ cd mpich-3.2 - $ ./configure --prefix=/Users/$USER/mpich-3.2-install - $ make - $ make install - -3. Compile Amica with the command: - - $ ~/mpich-3.2-install/bin/mpif90 -L/Users/$USER/mpich-3.2-install/lib/ -I/Users/$USER/mpich-3.2-install/include/ -qopenmp -mkl -static-intel -O3 -fpp -DMKL amica15.f90 funmod2.f90 -o amica15mac - -4. Test: - - i) Download Sample EEG Data (Memorize.fdt and amicadefs.param) from: https://sccn.ucsd.edu/~jason/amica_web.html - - ii) Test binary: - - $ ./amica15mac ./amicadefs.param - - -# How to compile with Intel Fortran on Ubuntu - -0. These are old instructions. Try using Intel OneAPI modifiying the commands similar to the instructions for Windows above. -1. Install Intel Fortran compiler for Linux. -2. Compile MPICH2 setting environmental vars CC, CXX, FC, and F77 to icc and ifort. -3. Compile Amica with the command: - - $ /home/jason/mpich2-3.2-install/bin/mpif90 -I/opt/intel/mkl/include/ -fpp -qopenmp -O3 -mkl -static -static-intel -DMKL funmod2.f90 amica15.f90 -o amica15ub - -4. Test: - - $ ./amica15ub ./amicadefs.param - - -# How to compile with Intel Fortran on Expanse HPC - -1. load appropriate modules: - -``` - module purge - module load cpu/0.15.4 intel intel-mkl mvapich2 -``` - -2. Compile Amica with the command: - -``` - mpif90 -static-intel -fpp -O3 -march=core-avx2 -heap-arrays \ - -qopenmp -mkl -DMKL -o amica15ex funmod2.f90 amica17.f90 -``` - -3. Test: -``` - $ ./amica17nsg ./amicadefs.param -``` - -4. Run on compute partition: - -``` - - $ module load cpu/0.15.4 slurm intel intel-mkl mvapich2 - $ export OMP_NUM_THREADS=4 ; export SRUN_CPUS_PER_TASK=4 ; export MV2_ENABLE_AFFINITY=0 - $ srun --partition=compute --nodes= --tasks-per-node=32 --cpus-per-task=4 \ - --mem=249208M --account= --export=ALL -t 04:00:00 ./amica17nsg ./amicadefs.param - -``` \ No newline at end of file diff --git a/plugins/amica/AMICA-introduction.md b/plugins/amica/AMICA-introduction.md deleted file mode 100644 index a28ad17..0000000 --- a/plugins/amica/AMICA-introduction.md +++ /dev/null @@ -1,425 +0,0 @@ ---- -layout: default -parent: AMICA -grand_parent: Plugins -render_with_liquid: false - -title: AMICA-Introduction -long_title: AMICA-Introduction ---- - -![1000px](Graphic_abstract.png) - - -AMICA is developed by Jason Palmer ([Palmer, ICASSP, -2008](https://sccn.ucsd.edu/~jason/icassp08.pdf), [Palmer, Tech Report, -2012](https://sccn.ucsd.edu/~jason/amica_a.pdf)). Its applications to -EEG data are further validated in [Delorme, Plos One, -2012](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0030135), -and [Hsu, NeuroImage, -2018](https://sccn.ucsd.edu/~scott/pdf/Hsu18_AMICA.pdf). This tutorial -is mainly written and maintained by Sheng-Hsiou (Shawn) Hsu with links -to Jason's original sources, hoping to provide up-to-date information -and detailed instructions on AMICA. - -\(c\) Jason Palmer, University of California San Diego, 2015. - -Introduction ------------- - -Adaptive Mixture Independent Component Analysis (AMICA) is a binary -program (for Linux, Mac, and Windows) that performs an independent -component analysis (ICA) decomposition on input data, potentially with -multiple ICA models. It can be run standalone, or from MATLAB. Key -features of AMICA include: - -1. **Adaptive Source Densities**: the source density models are adapted - using a mixture of Generalized Gaussian density model, resulting in - extremely good fit between the density model and the actual density - of the source being estimated -2. **Multiple / Mixture Models**: AMICA allows multiple ICA models to - be learned simultaneously, automatically segmenting the data into - regions of local stationarity, and returning a set of components for - each model. AMICA can also be set to share components between models - to increase estimation efficiency -3. **Data Likelihood (Model Probability)**: likelihood of each model - for given and new data is available, allowing rejection of unlikely - data, as well as classification of new data -4. **Parallel Implementation**: program can use multiple cores in - single workstations (using portable OpenMP code), as well as - multiple nodes in a cluster (using portable MPI code). All binaries - allow multi-core (SMP) execution. Only the Linux version currently - supports clusters (we use the freely available Rocks / Sun Grid - Engine) - -#### AMICA Mathematical Explanation - -For an explanation of how AMICA works, see this -[page](Amica-Mathematical-Explanation). - -#### AMICA software releases - -- [AMICA on Github](https://github.com/japalmer29/amica) (contains the - latest version) -- [AMICA plugin for EEGLAB: direct - download](http://sccn.ucsd.edu/eeglab/plugins/amica1.5.1.zip) - (contains the latest stable version) - -#### AMICA tutorials - -- [AMICA tutorial slides for EEGLAB - workshop](https://sccn.ucsd.edu/mediawiki/images/3/3f/EEG_Nonstationarity_and_AMICA.pdf) -- [A tutorial introduction to AMICA applied to generated example - data.](https://sccn.ucsd.edu/wiki/Amica) - -Why AMICA? ----------- - -### First reason: AMICA achieves better ICA decomposition than other ICA approaches - -With a complex model that has more parameters to learn, AMICA can -achieve an (empirically) more accurate ICA decomposition. [Arnauld -Delorme et al. in -2012](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0030135) -compared several ICA algorithms and showed that AMICA outperformed all -other ICA approaches in terms of obtaining higher mutual information -reduction and more near-dipolar ICs. - - -![400px](Delorme_MIR.png) - - -This reason itself already makes AMICA attractive. But AMICA has a -capability that has been long-time overlooked – learning a mixture of -ICA models. - -### Second reason: Multi-model AMICA for modeling EEG dynamics and brain state changes - -[Hsu and Jung, -2017](https://www.researchgate.net/profile/Sheng-Hsiou_Hsu/publication/317671281_Monitoring_alert_and_drowsy_states_by_modeling_EEG_source_nonstationarity/links/59c52eeda6fdccc71914d395/Monitoring-alert-and-drowsy-states-by-modeling-EEG-source-nonstationarity.pdf) -hypothesizes that transitions of brain states involve changes in -cortical networks activities which can be identified by distinct ICA -models. This hypothesis motivates the application of AMICA – with -multiple ICA models – as a data-driven approach to modeling -nonstationary dynamics of continuous and unlabeled data. - -Recent studies ([Hsu et al., NeuroImage, -2018](https://www.sciencedirect.com/science/article/pii/S1053811918306888) -and [Hsu et al., BioCAS, -2018](https://ieeexplore.ieee.org/abstract/document/8584715)) apply -multi-model AMICA to modeling EEG dynamics that are associated with -brain state changes. We demonstrate that multi-model AMICA can -characterize EEG dynamics during sleep for automatic staging, assess -transitions between alert and drowsy states in a simulated driving -experiment, and uncover mental state changes during a guided-imagery -meditation / hypnotherapy. Some empirical results are presented in this -section. - -We encourage researchers to test the -capability of multi-model AMICA in detecting and modeling different -cognitive or mental states. For more details, we recommend -reading the paper ([Hsu, NeuroImage, -2018](https://www.sciencedirect.com/science/article/pii/S1053811918306888), -[PDF](https://sccn.ucsd.edu/~scott/pdf/Hsu18_AMICA.pdf)) - - -![600px](Graphic_abstract.png) - - -### Third reason: Powered by parallel processing and supercomputer - -It is important to point out that AMICA achieves better performance by -requesting more computational resources to learn a more complex model -and thus requiring longer runtime. Fortunately, AMICA’s inventor, Jason -Palmer, derived a Newton-based learning rule and devised a -parallel-processing-enabled, pre-compiled program that makes AMICA -learning process feasible (e.g. can finish in a few hours instead of a -few days). - -In 2018, the NIH-funded Neuroscience Gateway (NSG) project aims to -provide researchers (free) access to computing power on supercomputer -clusters to speed up computation-heavy process, including AMICA. With -access to 24 cores per node and potentially to multiple nodes on the -supercomputer, the runtime of AMICA can be significantly reduced. More -details on how to get access to NSG and how to run AMICA on NSG, please -see -[below](https://github.com/japalmer29/amica/wiki/AMICA#how-to-run-amica-option-2-neuroscience-gateway-nsg). - -What is AMICA? --------------- - -### AMICA in a nutshell: - -The figure below gives a schematic overview of the architecture of AMICA -and its models. - - -![800px](AMICA_diagram.png) - - -AMICA consists of three layers of mixing: - -- *First layer*: mixture of ICA models \\mathbf{A}_1, ... , - \\mathbf{A}_H that learn the underlying data clusters, as - shown in the illustration using simulated data from Laplace and - uniform distribution. -- *Second layer*: mixture of independent components (IC) - \\mathbf{A}_{h1}, ..., \\mathbf{A}_{hN} that - decompose the data cluster into statistically independent sources' - activations \\mathbf{s}_{h1}, ..., \\mathbf{s}_{hN}. -- *Third layer*: mixture of generalized Gaussian distribution - q(s;\\rho,\\mu,\\beta) that approximate the probability - distribution of the source activation p(\\mathbf{s}) - - - -- [A tutorial introduction to AMICA applied to generated example - data.](https://sccn.ucsd.edu/wiki/Amica) - -How to run AMICA? Option 1: AMICA plugin for EEGLAB ---------------------------------------------------- - - -![300px\|thumb\|right\|upright=3\|Manage EEGLAB extensions --\> Data -processing extensions](Plugin_manager_highlight.png) - - -### Download AMICA - -You can download the latest AMICA plugin via the EEGLAB extensions -manager from your EEGLAB. - -1. Open EEGLAB GUI -2. Open File --\> Manage EEGLAB extensions --\> Data processing - extensions -3. Install Plugin *AMICA* (Current version 1.5.1, last updated - Oct. 2018) - -Alternatively, you can [download the AMICA plugin -folder](http://sccn.ucsd.edu/eeglab/plugins/amica1.5.1.zip) or [fork the -AMICA repository on Github](https://github.com/japalmer29/amica) -directly and put it under the */eeglab/plugins/* folder. More -instructions on how to download and install EEGLAB plugins can be found -in the [EEGLAB Extensions Wiki -page](https://sccn.ucsd.edu/wiki/EEGLAB_Extensions). - -Note: the following sources may not be up-to-date - -- Amica Download on wiki: [Amica Download](Amica-Download) - -### Install AMICA - -The downloaded *amica1.5.1* package (not *amica1.5*) should contain all -of the compiled binary files for different OS. - -- **Linux/Unix Users**: Plug-and-play! May need to recompile only if - runs with multiple nodes or threads. -- **Windows Users**: Install *MPICH2* (32-bit: *mpich2-1.4-win-ia32* - or 64-bit: *mpich2-1.4-win-x86-64*), which is included in the - *amica1.5.1* folder. Otherwise they can be downloaded from - -- **Mac Users**: Change the permissions of the binary file - *amica15mac* to executable. Run "Terminal" from the Go-\>Utilities - menu, then at the prompt run: - -`   cd ``/eeglab/plugins/amica1.5.1` -`   chmod 777 amica15mac` - -For further information, follow [the instruction written by Jason -Palmer](https://sccn.ucsd.edu/~jason/amica_web.html). - -### Run AMICA - -- **From EEGLAB GUI** - -1. Tool --\> Run AMICA -2. Follow the [Amica 1.5 EEGLAB GUI Interface - Help](https://sccn.ucsd.edu/~jason/amica_help.html) for setting the - input parameters - - -![600px](AMICA_plugin_setting.png) - -- **Using matlab script**: sample script below: - -
- - EEG = pop_loadset(‘test_data.set'); - - % define parameters - numprocs = 1; % # of nodes (default = 1) - max_threads = 1; % # of threads per node - num_models = 1; % # of models of mixture ICA - max_iter = 2000; % max number of learning steps - - % run amica - outdir = [ pwd filesep 'amicaouttmp' filesep ]; - [weights,sphere,mods] = runamica15(EEG.data, 'num_models',num_models, 'outdir',outdir, ... - 'numprocs', numprocs, 'max_threads', max_threads, 'max_iter',max_iter); - - % type “help runamica15()” for a full list and explanation of the parameters - -
- -How to run AMICA? Option 2: Neuroscience Gateway (NSG) ------------------------------------------------------- - -EEGLAB scripts may now be run on high-performance computing resources -via the freely available [Neuroscience Gateway -Portal](https://www.nsgportal.org/) to the NSF-sponsored [Comet -supercomputer](https://ucsdnews.ucsd.edu/pressrelease/sdsc_to_double_comet_supercomputers_graphic_processor_count/) -of the [San Diego Supercomputer Center](https://sdsc.edu/). NSG accounts -are free and are not limited to US users, but the portal may only be -used for non-commercial purposes. You can visit [EEGLAB on -NSG](https://github.com/sccn/nsgportal/wiki) page for more -information. - -Computationally expensive operations such as AMICA analysis will be -directly benefited with the access to multiple nodes, each with 24 -threads / cores, on the Comet supercomputer, which significantly reduces -the runtime of AMICA and the expense of local computing resources. - -### Through NSG website - -1\. **Create an NSG account and login:** Follow the [instructions - -EEGLAB on -NSG](https://github.com/sccn/nsgportal/wiki) - -2\. **Prepare a MATLAB script that runs AMICA:** You can also include -EEGLAB preprocessing steps and post-analyses. A sample code below: - -
- - %% test_script_amica.m - % add eeglab to path - eeglab; close; - - % load dataset - filepath = [ pwd filesep ]; - filename = 'test_data.set'; - EEG = pop_loadset(filename, filepath); - - % define parameters - numprocs = 1; % # of nodes (1-4: default 1) - max_threads = 24; % # of threads (1-24: default = 24) - num_models = 1; % # of models of mixture ICA - max_iter = 2000; % max number of learning steps - - % run amica on NSG - outdir = [ pwd filesep 'amicaouttmp' filesep ]; - runamica15(EEG.data, 'num_models',num_models, 'outdir',outdir, ... - 'numprocs', numprocs, 'max_threads', max_threads, 'max_iter',max_iter); - -
- -The *runamica15* function will call the binary compiled on Expanse supercomputer that allows for -multi-threads and multi-nodes parallel processing. Comprehensive parameters evaluation haven't been completed yet for AMICA on Expanse. But the parameters recommended for the old Comet system still works on Expanse. Recommend set -*max_threads* to 24 since each submitted job will be assigned a node -which contains 24 cores on Comet. Requesting more than 1 node is made -available by setting *numproc* larger than 1 (max 4 nodes). Be sure to -make the number consistent with the input parameters set while -submitting the job on NSG (see step 4). - -![300px\|thumb\|right\|Sample data folder for -NSG](NSG_data_folder.png) 3. **Prepare a data folder:** -make sure you have the following files in the folder: (1) a Matlab -script, which runs AMICA analysis, and (2) EEG data file(s) called in -the script. - -4\. **Upload the zipped folder to NSG:** - - -![800px](NSG_upload_data.png) - - -5\. **Create new task and set input parameters:** - - -![400px](Amica_nsg_task.png) - - -When you click the *set parameters* button, you will see the tab below: - - -![600px](Amica_nsg_par.png) - - -For running AMICA, make sure the number of -nodes matches the *numprocs* in the matlab script because in the -NSG back-end different commands are called for single- vs. -multiple-nodes tasks. It is also advised to set the max hours to be 48 -hours to ensure you get the max available computation time. Click the -hyperlink on any of the parameter setting description will lead you to -the below advanced help message: - - -![700px](Amica_nsg_par_help.png) - - -6\. **Save and run task, view submitted job status, and download the -output file:** See the [EEGLAB on NSG -page](https://github.com/sccn/nsgportal/wiki) -for detailed instructions. - - -![600px\|Sample output files](Amica_output.png) - - -### Through EEGLAB plugin - nsgportal - -(Updated 07/15/2019) Now EEGLAB plugin - nsgportal - supports access to -NSG. EEGLAB users can now submit and manage jobs to NSG using EEGLAB GUI -or command line by installing the nsgportal. For more information and -detailed instruction, please visit its [Github -page](https://github.com/sccn/nsgportal/wiki). - -Parallel processing capability ------------------------------- - -The AMICA program is capable of scheduling and running on multiple -threads. In 2018, the NIH-funded Neuroscience Gateway (NSG) project aims -to provide researchers (free) access to computing power on supercomputer -clusters to speed up computation-heavy process, including AMICA. With -access to 24 cores per node and potentially to multiple nodes on the -supercomputer, the runtime of AMICA can be significantly reduced. - -### Runtime analysis - -AMICA runtime is proportional to the number of data samples (T) and the -total number of parameters to be estimated (M x N^2 + M x N x (4k + 1) + -M), where M is the number of models, N is the number of channels, and K -is the number of generalized Gaussians for estimating the PDFs. - -Analysis for the new Expanse supercomputer is still pending. Below is the runtime analysis of AMICA with multiple threads on [Comet, -San Diego Supercomputer Center -(SDSC)](https://www.sdsc.edu/support/user_guides/comet.html): - -- Benchmark data: 30-channel EEG, 1.7 million data samples. -- AMICA parameters: default -- Comet computing resources: 24 cores per node. NSG user can access 4 - nodes at max for running AMICA. -- Comet CPU spec: Intel Xeon E5-2680v3 processors, 2.5 GHz. - - -![300px](Runtime_cores.png) -![300px](Runtime_nodes.png) - - -Below is the runtime analysis of AMICA on different datasets with -various sizes using 1 node, 24 cores. - - -![500px](Runtime_dataset.png) - - -How to analyze AMICA results? ------------------------------ - -EEGLAB users can use postAMICAutility toolbox (download -[here](http://sccn.ucsd.edu/eeglab/plugins/postAmicaUtility2.01.zip)) -for reading and analyzing AMICA output. Please see the [AMICA tutorial -slides for EEGLAB -workshop](https://sccn.ucsd.edu/mediawiki/images/3/3f/EEG_Nonstationarity_and_AMICA.pdf) -pp. 22-25 for a quick overview. Detailed description can be found in -[Hsu, NeuroImage, -2018](https://sccn.ucsd.edu/~scott/pdf/Hsu18_AMICA.pdf). \ No newline at end of file diff --git a/plugins/amica/AMICA_diagram.png b/plugins/amica/AMICA_diagram.png deleted file mode 100644 index 4b1ed61..0000000 Binary files a/plugins/amica/AMICA_diagram.png and /dev/null differ diff --git a/plugins/amica/AMICA_plugin_setting.png b/plugins/amica/AMICA_plugin_setting.png deleted file mode 100644 index 4451a9a..0000000 Binary files a/plugins/amica/AMICA_plugin_setting.png and /dev/null differ diff --git a/plugins/amica/Amica-Download.md b/plugins/amica/Amica-Download.md deleted file mode 100644 index 7a8d17f..0000000 --- a/plugins/amica/Amica-Download.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -layout: default -parent: AMICA -grand_parent: Plugins -render_with_liquid: false - -title: AMICA-Download -long_title: AMICA-Download ---- -The information on this page may not be up-to-date, please [visit this -page](https://github.com/japalmer29/amica/wiki/AMICA#download-amica) for detailed -instructions on downloading, installing, and running AMICA. - - - -**Files** ---------- - -### Linux/Unix - -32 bit - -[64 bit](Amica12lnx64.zip) - -### Windows - -32 bit - -[64 bit](https://linkify.me/HxsKynA) - -### Mac - -[32 bit](Amica12mac32.zip) - -[64 bit](Amica12mac64.zip) - -To run the mac binary: - -1\. Copy the files amica12mac64 and libiomp5.dylib to a directory, like -/users/hans/amica/. - -2\. Open a terminal window and type: - - - -$ export DYLD_LIBRARY_PATH=/home/hans/amica (or whatever directory you -used in step 1.) - -3\. Copy runamica12.m and loadmodout12.m to a directory in your matlab -path. - -4\. In the file runamica12.m, set AMDIR to the directory -(/users/hans/amica) and set AMBIN to amica12mac64. - -5\. Test by running matlab and typing: - - - -\>\> runamica12(randn(2,1000),'/users/hans/test/'); - - - - - -\>\> mods = loadmodout12('/users/hans/test/') - -### Sample Data - -[Sample datafile (32 channels)](Eeglab_testdat.zip) - -**Compiling** -------------- - -### Mac - -```mv libiomp5dylib /home/me/lib/libiomp5.dylib -export DYLD_LIBRARY_PATH=/home/me/lib - -../mpich2/bin/mpif90 -fpp -openmp /Developer/opt/intel/mkl/lib/libmkl_core.a \ -/Developer/opt/intel/mkl/lib/libmkl_intel_thread.a /Developer/opt/intel/mkl/lib/libmkl_intel_lp64.a \ -amica12.f90 funmod2.f90 -mssse3 -o amica12mac64 -O3 - -../mpich2/bin/mpif90 -fpp -openmp /Developer/opt/intel/mkl/lib/libmkl_core.a \ -/Developer/opt/intel/mkl/lib/libmkl_intel_thread.a /Developer/opt/intel/mkl/lib/libmkl_intel.a \ - amica12.f90 funmod2.f90 -msse3 -o amica12mac32 -O3 -``` diff --git a/plugins/amica/Amica-Mathematical-Explanation.md b/plugins/amica/Amica-Mathematical-Explanation.md deleted file mode 100644 index 84bf192..0000000 --- a/plugins/amica/Amica-Mathematical-Explanation.md +++ /dev/null @@ -1,262 +0,0 @@ ---- -layout: default -parent: AMICA -grand_parent: Plugins -render_with_liquid: false - -title: AMICA-Mathematical-Explanation -long_title: AMICA-Mathematical-Explanation ---- -This page is about the Amica algorithm itself. For installation and -application of Amica, refer to this [page](AMICA) instead. - -This page provides a tutorial introduction to ICA using the Amica -algorithm and Matlab. For general tutorial -and information on Amica, including how to run Amica and how to analyze -Amica results, [visit this -page](AMICA). - -### **Introduction** - - We will first illustrate the tools using generated example -data. For this purpose, we provide a matlab function to generate data -from the Generalized Gaussian distribution. A description, as well as -the notation we will use for the Generalized Gaussian density can be -found here: - -[Generalized Gaussian Density](Generalized-Gaussian-Probability-Density-Function) - -Further mathematical background and detail can be found at the following -pages: - -[Linear Representations and Basis -Vectors](Linear-Representations-and-Basis-Vectors) - -[Random Variables and Probability Density -Functions](Random-Variables-and-Probability-Density-Functions) - -Let us consider first the case of two independent Generalized Gaussian -sources with shape parameter . - - -In Matlab, using the function gg6.m (available in the Matlab code -section of the above Generalized Gaussian link), we can generate - of these sources with the command: -`s = gg6(2,2000,[0 0],[1 3],[1.5 1.5]);` - -Suppose our basis vectors and source vectors are given by: - - - - - - -In Matlab: - -``` -A = [1 4 ; 4 1]; -x = A*s; -``` - - -![900px](Ica24.png) - - -Matlab code used to generate this figure is here: -[LT_FIG.m](ICA_FIG.m) - -PCA ---- - -PCA can be used to determine the "principle axes" of a dataset, but it -does not uniquely determine the basis associated with the linear model -of the data, - - - - -PCA is only concerned with the second order (variance) characteristics -of the data, specifically the mean, , andcovariance, - - - - -For independent sources combined linearly, the covariance matrix is just -(assuming zero mean), - - - - -where is the diagonal matrix of source variances.Thus any basis, , satisfying, - - - -will lead to a covariance matrix of . And sincethe decorrelating matrices are all determined solely based on the -covariance matrix, it is impossible to determine a unique linear basis -from the covariance alone. Matrices, unlike scalar real numbers, -gnerally have an infinite number of "square roots". See -[Linear Representations and Basis Vectors](Linear-Representations-and-Basis-Vectors) -for more information. As shown in that section, PCA can be used to -determine the subspace in which the data resides if it is not full rank. - -To further illustrate the non-uniqueness of the PCA basis, consider the -following figure in which (i.e. Laplacian)sources are combined with the linear matrix, - -``` -N = 2000; -A = [1 2 ; 4 0.1]; -A(:,1)=A(:,1)/norm(A(:,1)); -A(:,2)=A(:,2)/norm(A(:,2)); -s = gg6(2,N,[0 0],[1 3],[1 1]); -x = A*s; -``` - -We produce two sets of decorrelated sources, one that simply projects -onto the eigenvectors, and the other which uses the unique symmetric -square root "sphering" matrix (see [Linear Representations and Basis -Vectors](Linear-Representations-and-Basis-Vectors) for more -detail.) - -``` -[V,D] = eig(x*x'/N); -Di = diag(sqrt(1./diag(D))); -u = Di * V' * x; -y = V * Di * V' * x; -``` - - -![1000px](Pca_p1.png) - - -Similarly for sources, -``` -s = gg6(2,2000,[0 0],[1 3],[10 10]); -x = A*s; -``` - - -![1000px](Pca_p10.png) - - -Gaussian sources, unlike non-Gaussian sources, are completely determined -by second order statistics, i.e. mean and covariance. If Gaussian random -variables are uncorrelated, then they are independent as well. -Futhermore, linear combinations or Gaussians are Gaussian. So linearly -mixed Gaussian data is Gaussian with covariance matrix, - -``` -s = gg6(2,2000,[0 0],[1 3],[2 2]); -x = A*s; -``` - - - - -And any of the infinite number of root matrices satisfying this equation will yield Gaussian data with identical -statistics to the observed data. - - -![1000px](Pca_p2.png) - - -As seen in the bottom row, the joint source density is radially -symmetric, i.e. it "looks" exactly the same in all directions. There is -no additional structure in the data beyond the source variances and the -covariance. To fix a particular solution, constraints must be placed on -the variance of the sources, or the norm of the basis vectors. However, -this generally cannot be done since the ICA model assumes only -independence of the sources. The source or basis vector norm is usually -fixed, which determines the magnitude of the non-fixed variance or norm. - -Fortunately, real sources are usually non-Gaussian, so if the ICA model -holds and the sources are independent, then there is a unique basis -representation of the data such that the sources are independent. - -Basic ICA ---------- - -Let's take some two dimensional data following the ICA model. - -``` -s = gg6(2,2000,[0 0],[1 3],[1 1]); -A = [1 2 ; 4 0.1]; -x = A*s; -``` - -The program amica_ex.m will be used to illustrate ICA using the Amica -algorithm. The output is shown below: - - -![800px](Ica1.png) - - -We can also experiment with multimodal sources using the function -ggrandmix0.m: - -``` -s = ggrandmix0(2,20000,3); % random 2 x 20000 source matrix, with 3 mixtures in each source density -x = A*s; -``` - - -![800px](Ica2.png) - - -Multiple Models ---------------- - -In the same way we created complex mixture source densities by combining -a number of Generalized Gaussian constituent densities, we can create a -complex data model by combining multiple ICA models in an ICA mixture -model. The likelihood of the data from a single time point, then is -given by, - - - - -where are the mixing matrixand density parameters associated with model . - -![500px](Gg_clb.png) - - -We can use the program ggmixdat.m to generate random ICA mixture model -data with random Generalized Gaussian sources: - -`[x,As,cs] = ggmixdata(2,2000,3,[1 1; 1 1],ones(2,2),2);` - -We use the program amica_ex.m to illustate: - -`[A,c,LL,Lt,gm,alpha,mu,beta,rho] = amica_ex(x,3,1,200,1,1e-6,1,0,1,As,cs);` - - -![500px](Ica_mix_dat.png) - - -The output is shown below: - - -![1000px](Ica_mix.png) - - -Given a mixture model, we can use Bayes' Rule to determine the posterior -probability that model generated the data at time point be the index of the model active at time point given, we have, - - - -A plot of the posterior log likelihood given each model is shown below. -
-![1000px](LLggmix.png) - -
- - diff --git a/plugins/amica/Amica_nsg_par.png b/plugins/amica/Amica_nsg_par.png deleted file mode 100644 index 332e747..0000000 Binary files a/plugins/amica/Amica_nsg_par.png and /dev/null differ diff --git a/plugins/amica/Amica_nsg_par_help.png b/plugins/amica/Amica_nsg_par_help.png deleted file mode 100644 index 480dd12..0000000 Binary files a/plugins/amica/Amica_nsg_par_help.png and /dev/null differ diff --git a/plugins/amica/Amica_nsg_task.png b/plugins/amica/Amica_nsg_task.png deleted file mode 100644 index cfcc6e7..0000000 Binary files a/plugins/amica/Amica_nsg_task.png and /dev/null differ diff --git a/plugins/amica/Amica_output.png b/plugins/amica/Amica_output.png deleted file mode 100644 index 18f7c96..0000000 Binary files a/plugins/amica/Amica_output.png and /dev/null differ diff --git a/plugins/amica/Delorme_MIR.png b/plugins/amica/Delorme_MIR.png deleted file mode 100644 index e058d3a..0000000 Binary files a/plugins/amica/Delorme_MIR.png and /dev/null differ diff --git a/plugins/amica/Entropy-and-Mutual-Information.md b/plugins/amica/Entropy-and-Mutual-Information.md deleted file mode 100644 index a438080..0000000 --- a/plugins/amica/Entropy-and-Mutual-Information.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -layout: default -parent: AMICA -grand_parent: Plugins -render_with_liquid: false - -title: Entropy-and-Mutual-Information -long_title: Entropy-and-Mutual-Information ---- -A multivariate pdf is generally a function of a vector space. If the individual variables, or components of the random vector, are independent, then the pdf factorizes into the product of univariate pdfs associated with the components. - -# Mutual Information and Pairwise Mutual Information (PMI) -Mutual Information can be used as a measure of dependence. The MI between two random variables and is always non-negative. Let and have probability density functions and . Then: - -![](https://latex.codecogs.com/svg.latex?I(X;Y)%20=%20D\big(p(x,y)\,\big\|\,p(x)p(y)\big)%20=%20\iint%20p(x,y)%20\log%20\frac{p(x,y)}{p(x)p(y)}%20dx\hspace{1pt}%20dy%20\ge%200) - -Furthermore, unlike correlation, the mutual information is equal to zero if and only if and are independent. - -In terms of entropy, the mutual information can be written, - -![](https://latex.codecogs.com/svg.latex?I(X;Y)%20=%20H(X)%20+%20H(Y)%20-%20H(X,Y)) - -With a reasonable number of samples, we can estimate the bivariate entropy. diff --git a/plugins/amica/Generalized-Gaussian-Probability-Density-Function.md b/plugins/amica/Generalized-Gaussian-Probability-Density-Function.md deleted file mode 100644 index 67c2c83..0000000 --- a/plugins/amica/Generalized-Gaussian-Probability-Density-Function.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -layout: default -parent: AMICA -grand_parent: Plugins -render_with_liquid: false - -title: Generalized-Gaussian-Probability-Density-Function -long_title: Generalized-Gaussian-Probability-Density-Function ---- - - -Density Function ----------------- - -The Generalized Gaussian density has the following form: - - - - -where (rho) is the "shape parameter". The density isplotted in the following figure: - - -![Image:Ggpdf.png](Ggpdf.png) - - -Matlab code used to generate this figure is available here: -[ggplot.m](ggcode). - -Adding an arbitrary location parameter, , and inverse scaleparameter, , the density has the form, - - - -![Image:Ggpdf2.png](Ggpdf2.png) - - -Matlab code used to generate this figure is available here: -[ggplot2.m](ggcode2). - -Generating Random Samples -------------------------- - -Samples from the Generalized Gaussian can be generated by a -transformation of Gamma random samples, using the fact that if is a distributed random variable, and is an independent random variable taking the value -1 or +1with equal probability, then, - - - -is distributed . That is, - - - -where the density of is written in a non-standard butsuggestive form. - -Matlab Code ------------ - -Matlab code to generate random variates from the Generalized Gaussian -density with parameters as described here is here: - -[gg6.m](gg6.m) - -As an example, we generate random samples from the example Generalized -Gaussian densities shown above. - - -![Image:Ggpdf3.png](Ggpdf3.png) - - -Matlab code used to generate this figure is available here: -[ggplot3.m](ggcode3). - -Mixture Densities ------------------ - - A more general family of densities can be constructed -from mixtures of Generalized Gaussians. A mixture density, -, is made up of constituent densities together with probabilities associated with each constituent density. - - - -The densities have different forms, or parameter values. Arandom variable with a mixture density can be thought of as being -generated by a two-part process: first a decision is made as to which -constituent density to draw from, where the densityis chosen with probability , then the value of therandom variable is drawn from the chosen density. Independent -repetitions of this process result in a sample having the mixture -density . -As an example consider the density, - - - - -!![500px](Ggmix1.png)!![500px](Ggmix2.png) - - -Matlab code used to generate these figures is available here: -[ggplot4.m](ggcode4). diff --git a/plugins/amica/Gg_clb.png b/plugins/amica/Gg_clb.png deleted file mode 100644 index ac6d8c7..0000000 Binary files a/plugins/amica/Gg_clb.png and /dev/null differ diff --git a/plugins/amica/Ggmix1.png b/plugins/amica/Ggmix1.png deleted file mode 100644 index 40d24b7..0000000 Binary files a/plugins/amica/Ggmix1.png and /dev/null differ diff --git a/plugins/amica/Ggmix2.png b/plugins/amica/Ggmix2.png deleted file mode 100644 index 4c9b7ab..0000000 Binary files a/plugins/amica/Ggmix2.png and /dev/null differ diff --git a/plugins/amica/Ggpdf.png b/plugins/amica/Ggpdf.png deleted file mode 100644 index cb61ba2..0000000 Binary files a/plugins/amica/Ggpdf.png and /dev/null differ diff --git a/plugins/amica/Ggpdf2.png b/plugins/amica/Ggpdf2.png deleted file mode 100644 index 3670923..0000000 Binary files a/plugins/amica/Ggpdf2.png and /dev/null differ diff --git a/plugins/amica/Ggpdf3.png b/plugins/amica/Ggpdf3.png deleted file mode 100644 index 354bde1..0000000 Binary files a/plugins/amica/Ggpdf3.png and /dev/null differ diff --git a/plugins/amica/Graphic_abstract.png b/plugins/amica/Graphic_abstract.png deleted file mode 100644 index 609fcde..0000000 Binary files a/plugins/amica/Graphic_abstract.png and /dev/null differ diff --git a/plugins/amica/Ica1.png b/plugins/amica/Ica1.png deleted file mode 100644 index 21c3379..0000000 Binary files a/plugins/amica/Ica1.png and /dev/null differ diff --git a/plugins/amica/Ica2.png b/plugins/amica/Ica2.png deleted file mode 100644 index a0b9f3a..0000000 Binary files a/plugins/amica/Ica2.png and /dev/null differ diff --git a/plugins/amica/Ica24.png b/plugins/amica/Ica24.png deleted file mode 100644 index 058587b..0000000 Binary files a/plugins/amica/Ica24.png and /dev/null differ diff --git a/plugins/amica/Ica_mix.png b/plugins/amica/Ica_mix.png deleted file mode 100644 index 6101637..0000000 Binary files a/plugins/amica/Ica_mix.png and /dev/null differ diff --git a/plugins/amica/Ica_mix_dat.png b/plugins/amica/Ica_mix_dat.png deleted file mode 100644 index c2e3415..0000000 Binary files a/plugins/amica/Ica_mix_dat.png and /dev/null differ diff --git a/plugins/amica/LLggmix.png b/plugins/amica/LLggmix.png deleted file mode 100644 index e1e4f71..0000000 Binary files a/plugins/amica/LLggmix.png and /dev/null differ diff --git a/plugins/amica/Linear-Representations-and-Basis-Vectors.md b/plugins/amica/Linear-Representations-and-Basis-Vectors.md deleted file mode 100644 index b0ca0de..0000000 --- a/plugins/amica/Linear-Representations-and-Basis-Vectors.md +++ /dev/null @@ -1,374 +0,0 @@ ---- -layout: default -parent: AMICA -grand_parent: Plugins -render_with_liquid: false - -title: Linear-Representations-and-Basis-Vectors -long_title: Linear-Representations-and-Basis-Vectors ---- - This page describes basic linear algebra concepts related -to linear representations in vector spaces. - -**Basis Vectors** ------------------ - - - -Originally we are given the recorded data in the channel space, say with - channels, and samples (i.e. time points, frames). Thedata can be thought of as a collection of vectors in-dimensional space, each of which in the case of EEG is asnapshot of the electric potential at the electrodes (relative to a -given reference) at a particular time point. - -The data can also be thought of as a collection of time series,or channel vectors, in -dimensional space; or as a collection ofspatiotemporal data segments (each e.g. an matrix) in-dimensional space. As we are concerned here withinstantaneous ICA, we'll primarily think of the data as a set of - vectors in -dimensional space, disregarding thetemporal order of the vectors. - -ICA is a type of linear representation of data in terms of a set of -"basis" vectors. Since we're working here in channel space, thevectors we're interested in will be in . Toillustrate in the following we'll use a three dimensional example, say -recorded using three channels. The data then is given to us in three -dimensional vector space. - - -![700px](R3_2.png) - - -Each of these data points is a vector in three dimensional space. - - - - -In general, any point in -dimensional space can be representedas a linear combination of any vectors that are linearlyindependent. For example let's take the vectors, - - - - -Linear independence means that no vector in the set can be formed as a -linear combination of the others, i.e. each vector branches out into a -new dimension, and they do not all lie in a zero volume subspace of -. Equivalently, there is no vector that can mulitply to produce thezero vector: - - - - -Mathematically, this is true if and only if: - - - - -So for example any data vector, ,can be represented in terms of three linearly independent basis vectors, -(unique) coefficient vector, : - - - -A linear representation of the data is a fixed basis set, -, that is used to represent each data point: - - - -\\triangleq \[\\mathbf{c}_1\\cdots \\mathbf{c}_T\], the we can -write, - - - - -where is the data matrix, is the matrix of basis vectors,and is the coefficient (orloading, or weight) matrix, with giving the"coordinates" of the point in the coordinate spacerepresented by the basis . -We have assumed thus far that the data itself is "full rank", i.e. that -there exists a set of data vectors that are linearlyindependent. It may happen, however, that the data do not lie in the -"full volume" of , but rather occupy a subspace ofsmaller dimension. - -In three dimensions, for example, all of the data might exist in a -two-dimensional subspace. - - -!![400px](R3_3.png)!![400px](R3_4.png) - - -The data is still represented as points or vectors in three dimensional -space, with three coordinates, but in fact only two coordinates are -required (once a "center" point has been fixed in the subspace). - -Even if the data does not lie *exactly* in a subspace, it may be the -case that one of dimensions (directions) is just numerical noise. -Eliminating such extraneous dimensions can lead to more efficient and -stable subsequent processing of the data. - -To understand how the data occupies the space volumetrically, and in the -case of data that is not full rank, how to determine which subspace the -data lies in, we will use Principle Component Analysis, described in the -next section. - - - -**Principle Component Analysis (PCA)** --------------------------------------- - - Let the data be represented by an vectors contained in the columns. Let us also assume that thedata is "zero mean", i.e. that the mean of each channel (row of -) has been removed (subtracted from the row), so that: - - - -Now, one way to determine the rank of the data is to examine the -covariance matrix, or matrix of channel correlations, which is defined -by, - - - - -The matrix has the same rank, or intrinsicdimensionality, as the matrix . If we perform aneigen-decomposition of , we get, - - - -where and are the eigenvalues and eigenvectors respectively. - -Since is symmetric and "positive semidefinite",all the eigenvalues are real and non-negative. (and thus ) is full rank if and only if all -If some of the eigenvalues are zero, then the data is not full rank, and -the rank is equal to the number of nonzero eigenvalues. In this case, -the data lies entirely in the -dimensional subspace spanned bythe eigenvectors corresponding to the nonzero eigenvalues. - - - - -and, - - - - -where is the data matrix, is the matrix of basis vectors, and is the coefficient matrix, with giving the "coordinates" of the point in the-dimensional space of the nonzero eigenvectors. -The data is reduced in dimension from to by "projecting" onto the -dimensionalspace, - - - -Analysis may be conducted on the reduced data , e.g.ICA may be performed, giving results in dimensional space. Thecoordinates in the original dimensional data space are thengiven by simply multiplying the dimensional vectors by. The , rank , matrix, - - - -in this case is called a "projection matrix", projecting the data in the -full space onto the subspace spanned by the first eigenvectors. - - -**Singular Value Decomposition (SVD)** --------------------------------------- - - - -A related decomposition, called the Singular Value Decomposition (SVD), -can be performed directly on the data matrix itself to produce a linear -representation (of possibly reduced rank). The SVD decomposes the data -matrix into, - - - - -where is the data matrix, is the matrix of ortho-normal (orthogonal and unit norm) "left -eigenvectors", is the diagonalmatrix of strictly positive "singular values", and isthe matrix of orthonormal "right eigenvectors". -From the SVD, we see that, - - - -so that and . The SVD directly gives the linear -representation: - - - - -. The vectors in orthonormal (orthogonal and unit norm), and the rows of - are orthogonal (since is diagonal,and is orthonormal.) -The SVD gives the unique linear representation (assuming singular values -are distinct) of the data matrix - such that the columns of are orthonormal, and the rows of values are all distinct; a subspace determined by equal singular values -does not have a unique orthonormal basis in this subspace, allowing for -arbitrary cancelling rotations of the left and right eigenvectors in -this subspace.) - -Having the rows of be orthogonal, i.e. uncorrelated,is a desirable feature of the representation, but having the basis -vectors be orthonormal is overly restrictive in many cases of interest, -like EEG. However, if we only require the rows of tobe orthogonal, then we lose the uniqueness of the representation, since -for any orthonormal matrix , and any full rankdiagonal matrix , we have, - - - -where the rows of the new coefficient matrix - are stillorthogonal, but the new matrix of basis vectors in the columns of, -, are nolonger orthogonal. - -A linear representation of the data, - - - - -implies that the coefficients can be recovered from the data using the -inverse of (or in the case of rank deficient, any left inverse, like the pseudoinverse): - - - - - -**PCA and Sphering** --------------------- - - We have seen that the SVD representation is one linear -representation of the data matrix. The SVD puts, - - - - -where is the identity matrix. -Another representation, which we call "sphering", puts, - - - - -This latter representation has certain advantages. We can show, e.g., -that the sphering transformation leaves the data changed as little as -possible among all "whitening" transformations, i.e. those that leave -the resulting rows of the coefficient matrix uncorrelated with unit -average power. - - - - -This is equivalent to taking . Let thegeneral form of a "whitening" decorrelating transformation, then, be: - - - - -for arbitrary orthonormal matrix . We measure thedistance of the transformed data from the original data by the sum of -the squared errors: - - - -Writing in the general form of the decorrelatingtransformation, we get, - - - - -Equality is achieved in the last inequality if and only if -. The resulting minimal squared error isthe same squared error that would be result from simply normalizing the -variance of each channel, which is equivalent to the transformation -. -We shall refer to this particular whitening transformation, -as the inverse of the "square root" of the covariance matrix -. It is the unique symmetric matrix - - -Remarks: - -We can view this result as saying that the whitening matrix -either as a collection of channel vectors, or as a collection of channel -. -We have found in practice, performing ICA on EEG data, that using the -(symmetric) sphering matrix as an initialization of for ICA generally -yields the best results and the quickest convergence, especially in -whitening transformation produces more independent components than the -latter. This is confirmed empirically in our mutual information -computations. - -Why should the sphering matrix - produce moreindependent time series and a better starting point for ICA than the -whitening matrix ? In the case ofEEG, this is likely due to the fact that the EEG sensor electrodes are -spread out at distances of the same order as the distance between the -EEG sources. Thus the sources tend to have a much larger effect on a -relatively small number of sensors, rather than a moderate effect on all -of the sensors. - -The whitening matrix , inprojecting the data onto the eigenvectors of the covariance matrix, -produces time series that are each mixtures of all of the channels, and -in this sense more mixed than the original data, in which the sources -distribute over a relatively small number of channels. - -The sphering matrix onthe other hand, rotates the transformed data back into its original -coordinates, and produces time series that are closest to the original -data, which was relatively independent at the start. - -By leaving the data in the eigenvector coordinate system, the whitening -matrix forces the ICA algorithm to“undo” a great deal of mixing in the time series, and as a starting -point for iterative algorithms, makes it more difficult (in terms of -potential local optima) and more time consuming (since the starting -point is farther from the ICA optimum). - -**EEG Data Reference and Re-referencing** ------------------------------------------ - - EEG data is recorded as a potential difference between the -electrode location and the reference. Biosemi active recordings use a -reference that is separate from the scalp electrodes. If data is -recorded with a specific electrode reference, then the data essentially -includes a "zero" channel corresponding to the signal at the reference -location relative to itself. - -A commonly used reference is the "average reference", which consists -essentially of subtracting the mean scalp potential at each time point -from the recorded channel potential. Let the vector of all ones be -denoted, . If the datais denoted , then average referenced data isequivalent to, - - - - -The average reference reduces the rank of the data because the -referencing matrix is rank (note that if you include theoriginal reference when computing average reference, average reference -does not reduce the rank of the data). In particular, the vector - is in the "null space" of the referencing matrix: - - - -The left-hand side is transformed as - - - - - - -Here, the (1/**n**) is key since -(**e***T*  \* **e**)/**n** = 1. Therefore, - - - - - - -Re-referencing to a specific channel or channels can be represented -similarly. Let the vector with one in the *j*th position be denoted - - - - -Suppose e.g. that the mastoid electrode numbers are and. Then the linked mastoid re-reference is equivalent to: - - - -Again, however, is in the null space of thisreferencing matrix, showing that the rank is . Any referencingmatrix will be rank deficient, and will thus leave the data rank -deficient by one dimension. - -In addition to referencing, EEG pre-processing usually includes -high-pass filtering (to reduce non-stationarity caused by slow drifts). -Linear filtering (such as high, low, band-pass, FIR, IIR, etc.) can be -represented as a matrix multiplication of the data on the right by a -large matrix whose columns are time shifted versionsof each other. The combined referencing and filtering operations can be -represented as: - - - - -The resulting referenced and filtered matrix should remain rank -deficient by one. However when referencing is done first, reducing the -rank by one, and then filtering is performed, it may happen that the -rank of the data increases so that it becomes essentially full rank -again. This is apparently due to numerical effects of multiplying (in -effect) by a matrix . -To summarize, re-referencing should reduce the rank of the data, -relegating it to an dimensional subspace of the-dimensional channel space. However, subsequent filtering of therank-reduced referenced data *may* increase the rank of the data again -(so that the minimum singular value is significantly larger than zero.) -In this case, numerical noise in the vector (direction) - is essentially added back into the data as anindependent component. - - \ No newline at end of file diff --git a/plugins/amica/NSG_data_folder.png b/plugins/amica/NSG_data_folder.png deleted file mode 100644 index 1e7281d..0000000 Binary files a/plugins/amica/NSG_data_folder.png and /dev/null differ diff --git a/plugins/amica/NSG_upload_data.png b/plugins/amica/NSG_upload_data.png deleted file mode 100644 index 5d80024..0000000 Binary files a/plugins/amica/NSG_upload_data.png and /dev/null differ diff --git a/plugins/amica/Npdf.png b/plugins/amica/Npdf.png deleted file mode 100644 index 5f149be..0000000 Binary files a/plugins/amica/Npdf.png and /dev/null differ diff --git a/plugins/amica/Pca_p1.png b/plugins/amica/Pca_p1.png deleted file mode 100644 index 2471298..0000000 Binary files a/plugins/amica/Pca_p1.png and /dev/null differ diff --git a/plugins/amica/Pca_p10.png b/plugins/amica/Pca_p10.png deleted file mode 100644 index d7d831c..0000000 Binary files a/plugins/amica/Pca_p10.png and /dev/null differ diff --git a/plugins/amica/Pca_p2.png b/plugins/amica/Pca_p2.png deleted file mode 100644 index 25aa7f8..0000000 Binary files a/plugins/amica/Pca_p2.png and /dev/null differ diff --git a/plugins/amica/Plugin_manager_highlight.png b/plugins/amica/Plugin_manager_highlight.png deleted file mode 100644 index cd0964c..0000000 Binary files a/plugins/amica/Plugin_manager_highlight.png and /dev/null differ diff --git a/plugins/amica/R3_2.png b/plugins/amica/R3_2.png deleted file mode 100644 index d161be8..0000000 Binary files a/plugins/amica/R3_2.png and /dev/null differ diff --git a/plugins/amica/R3_3.png b/plugins/amica/R3_3.png deleted file mode 100644 index 106bdbe..0000000 Binary files a/plugins/amica/R3_3.png and /dev/null differ diff --git a/plugins/amica/R3_4.png b/plugins/amica/R3_4.png deleted file mode 100644 index a5da832..0000000 Binary files a/plugins/amica/R3_4.png and /dev/null differ diff --git a/plugins/amica/Random-Variables-and-Probability-Density-Functions.md b/plugins/amica/Random-Variables-and-Probability-Density-Functions.md deleted file mode 100644 index 05b9787..0000000 --- a/plugins/amica/Random-Variables-and-Probability-Density-Functions.md +++ /dev/null @@ -1,194 +0,0 @@ ---- -layout: default -parent: AMICA -grand_parent: Plugins -render_with_liquid: false - -title: Random-Variables-and-Probability-Density-Functions -long_title: Random-Variables-and-Probability-Density-Functions ---- - This page reviews the concepts of random variables -(rv's) and probability density functions (pdfs). It describes -Kullback-Leibler (KL) Divergence and Maximum Likelihood (ML) estimation, -as well as multivariate probability densities and the effect of linear -transformations on multivariate probability density functions. - -**RVs and PDFs** ----------------- - - A random variable can be thought of as anordinary variable , together with a rule for assigning to everyset a probability that the variable takes a value inthat set, , which in our case will bedefined in terms of the probability density function: - - - - -That is, the probability that is given by theintegral of the probability density function over .So a (continuous) random variable can be thought of as a variable and a -pdf. When the values taken by a random variable are discrete, e.g. 0 or -1, then the distribution associated with the random variable is referred -to as a probability *mass* function, or pmf. Here we will be concerned -primarily with signals taking values in a continuous range. - -Continuous random variables are often taken to be Gaussian, in which -case the associated probability density function is the Gaussian, or -Normal, distribution, - - - - -![500px](Npdf.png) - - -The Gaussian density is defined by two parameters: the location, or -mean, , and the scale, or variance, . - - -### Confidence intervals and *p* values - - An example of using the density function to calculate -probabilities is the computation of confidence intervals and that, - - - - -Similarly, a (one-sided) value or score for an observation, given a probability density function isgiven by, - - - - -This gives the probability that the random variable takes a value in the -tail region, defined (after the observation) as the set of values with -positive magnitude at least as great as the observed value, given that -the probability density is . (A two-sided valueconcerning the magnitude would include the integral from to as well.) A low value can be used as evidencethat the probability density function is not the trueprobability density function , i.e. to reject the nullhypothesis that is the probability density function, ormodel, associated with , on the grounds that if it were thecorrect model, then an event of very low probability would have -occurred. - -Note that the value of a pdf at any point is not a probability value. -Probabilities for continuous random variables are only associated with -*regions*, and are only determined by integrating the pdf. - - - -### Model Comparison and Posterior Likelihood Evaluation - - Related to the idea of values is testing the"goodness of fit" of a model. The model is defined in terms of a -probability distribution, and the fit of the model is defined in terms -of the fit of the model probability distribution to the actual -probability distribution. - -Bayes' Rule is often used to calculate the probability that a certain -model, say from a set of models,, generated an observation : - - - - - -**Model Fitting -- KL Divergence and ML Estimation** ----------------------------------------------------- - - A probability density with a set of parameters can be -thought of as a class or set of probability density functions, for example the set of all Gaussian densities with . - -Fitting a model to an observed data set can be thought of as looking for -the particular density in the class of densities defined by the model, -that "best fits" the distribution of the data. One way of defining the -distance between two densities, as a measure of fit, is the -Kullback-Leibler Divergence: - - - -where is a model density, and is the truedensity. The KL divergence is non-negative and zero if and only if -densities are the same. However note that it is non-symmetric in the -densities. If we write out the KL divergence as stated, we get, - - - -where is the entropy of . This shows that we the KLdivergence can be viewed as the excess entropy, or minimal coding rate, -imposed by assuming that the distribution of is . -Writing the KL divergence in this way also shows its relationship to -Maximum Likelihood (ML) estimation with independent samples. In this -case, the ML problem, assuming a model with parameters , for the random variable , is to maximize: - - - -But by the law of large numbers, we have, - - - -So in fact, - - - -and we see that as , ML estimation is equivalent todetermining the density in the class of densities defined by the -variation of the parameter . - - -**Multivariate Probability Densities and Independence** -------------------------------------------------------- - - As in the univariate case, multivariate RVs are defined -by rules for assigning probabilities to the events that the multivariate -random random variable (i.e. random vector) takes a value in some -multidimensional set. - - - - -A set of random variables is defined to be independent if it's joint -probability density function factorizes into the product of the -"marginal" densities: - - - - -In the case of a random vector with independent components, the -probability that the vector takes a value in a hypercubic set is simply -the product of the probabilities that the individual components lie in -the region defining the respective side of the hypercube: - - - - - - -**Probability Densities of Linear Transformations of RVs** ----------------------------------------------------------- - - - -If is a fixed real number, and is a randomvariable with pdf , then a random variable definedby - - - - -has pdf, - - - - -If is an invertible matrix, and is a random vector with pdf p_{\mathbf{s}}(\\mathbf{s}), then the probability density of the random vector -, produced by the linear transformation, - - - -is given by the formula, - - - - -If is not square, but rather is "undercomplete", then PCA analysis can readily identifyan orthonormal basis for the -dimensional subspace in which thedata resides, and subsequent processing, e.g. ICA, can generally be -carried out in the reduced -dimensional space and a square r\\times r linear transformation. - -If there is additional non-negligible noise in the undercomplete or -complete (square) case, - - - - -with ,then the problem essentially becomes an "overcomplete" one with - -If the matrix is "overcomplete" with ,then the pdf of cannot generally be determined inclosed form unless is Gaussian. We will consider theovercomplete in another section. - - \ No newline at end of file diff --git a/plugins/amica/Runtime_cores.png b/plugins/amica/Runtime_cores.png deleted file mode 100644 index 534a3e9..0000000 Binary files a/plugins/amica/Runtime_cores.png and /dev/null differ diff --git a/plugins/amica/Runtime_dataset.png b/plugins/amica/Runtime_dataset.png deleted file mode 100644 index 3163fff..0000000 Binary files a/plugins/amica/Runtime_dataset.png and /dev/null differ diff --git a/plugins/amica/Supplementary-functions.md b/plugins/amica/Supplementary-functions.md deleted file mode 100644 index f22ade7..0000000 --- a/plugins/amica/Supplementary-functions.md +++ /dev/null @@ -1,229 +0,0 @@ ---- -layout: default -parent: AMICA -grand_parent: Plugins -render_with_liquid: false - -title: Supplementary-functions -long_title: Supplementary-functions ---- -The functions below are not included in AMICA but they might still be useful. - -**gg6** -```matlab -function x = gg6(n,N,mu,beta,rho) -% Generate N Generalized Gaussian m-dimensional vectors with means mu(:), -% inverse scales beta(:), and parameters rho(:). -if length(mu) == 1 - mu = mu*ones(1,n); - beta = beta*ones(1,n); - rho = rho*ones(1,n); -end -x = zeros(n,N); -for i = 1:n - x(i,:) = mu(i) + (1/sqrt(beta(i))) * ( mygamrnd(1/rho(i),1,1,N)).^(1/rho(i) ) .* ((rand(1,N)<0.5)*2-1) ; -end - -function r = mygamrnd(a,b,rows,columns) -% This is essentially copied from the gamrnd function in the Matlab Stats -% Toolbox. If you have that toolbox, you can just use the code above and -% replace mygamrnd() with gamrnd(). -% Initialize r to zero. -lth = rows*columns; -r = zeros(lth,1); -a = a(:); b = b(:); -scalara = (length(a) == 1); -if scalara - a = a*ones(lth,1); -end -scalarb = (length(b) == 1); -if scalarb - b = b*ones(lth,1); -end -% If a == 1, then gamma is exponential. (Devroye, page 405). -k = find(a == 1); -if any(k) - r(k) = -b(k) .* log(rand(size(k))); -end -k = find(a < 1 & a > 0); -% (Devroye, page 418 Johnk's generator) -if any(k) - c = zeros(lth,1); - d = zeros(lth,1); - c(k) = 1 ./ a(k); - d(k) = 1 ./ (1 - a(k)); - accept = k; - while ~isempty(accept) - u = rand(size(accept)); - v = rand(size(accept)); - x = u .^ c(accept); - y = v .^ d(accept); - k1 = find((x + y) <= 1); - if ~isempty(k1) - e = -log(rand(size(k1))); - r(accept(k1)) = e .* x(k1) ./ (x(k1) + y(k1)); - accept(k1) = []; - end - end - r(k) = r(k) .* b(k); -end -% Use a rejection method for a > 1. -k = find(a > 1); -% (Devroye, page 410 Best's algorithm) -bb = zeros(size(a)); -c = bb; -if any(k) - bb(k) = a(k) - 1; - c(k) = 3 * a(k) - 3/4; - accept = k; - count = 1; - while ~isempty(accept) - m = length(accept); - u = rand(m,1); - v = rand(m,1); - w = u .* (1 - u); - y = sqrt(c(accept) ./ w) .* (u - 0.5); - x = bb(accept) + y; - k1 = find(x >= 0); - if ~isempty(k1) - z = 64 * (w .^ 3) .* (v .^ 2); - k2 = (z(k1) <= (1 - 2 * (y(k1) .^2) ./ x(k1))); - k3 = k1(find(k2)); - r(accept(k3)) = x(k3); - k4 = k1(find(~k2)); - k5 = k4(find(log(z(k4)) <= (2*(bb(accept(k4)).*log(x(k4)./bb(accept(k4)))-y(k4))))); - r(accept(k5)) = x(k5); - omit = [k3; k5]; - accept(omit) = []; - end - end - r(k) = r(k) .* b(k); -end -r = reshape(r,rows,columns); -``` - -**ggcode** -```matlab -x = -6:0.01:6; -rho = [1 1.5 2 4 10]; -p = []; -for k = 1:length(rho) - p = [p exp(-abs(x').^rho(k)) / (2*gamma(1+1/rho(k)))]; -end -figure, hold on; set(gca,'fontsize',14); -plot(x,p,'linewidth',2); -str = num2str(rho'); -clear str2; -for k = 1:length(rho) - str2(k,:) = ['\it\rho =' str(k,:)]; -end -legend(str2); xlabel('\itx'); ylabel('\itGG(x\rm;\it\rho)'); xlim([-6 6]) -``` - -**ggcode2** -```matlab -x = -10:0.01:10; -mu=[-3 -1 2]; -beta= [1 5 2]; sbeta = sqrt(beta); -rho=[1 2 10]; -clear p2; -for k = 1:length(mu) - p2(:,k) = exp(-abs(sbeta(k)*(x-mu(k))').^rho(k)) * sbeta(k) / (2*gamma(1+1/rho(k))); -end -figure, hold on; set(gca,'fontsize',14); -plot(x,p2,'linewidth',2); -rstr = num2str(rho'); mstr = num2str(mu'); bstr = num2str(beta'); -clear str2; -for k = 1:length(rho) - str2(k,:) = ['\it\mu = ' mstr(k,:) ', \it\beta = ' bstr(k,:) ', \it\rho = ' rstr(k,:) ]; -end -legend(str2,'Location','NE'); xlabel('\itx'); ylabel('\itGG(x\rm;\it\mu,\it\beta,\it\rho)'); -set(gca,'XTick',-10:10);xlim([-8 8]); -``` - -**ggcode3** -```matlab -clear x h b; -figure, hold on; -mu = [-3 -1 2]; -beta = [1 5 2]; -rho = [1 2 10]; -delt = 0.15; -cents = -12:delt:12; -for k = 1:length(mu) - x(k,:) = gg6(1,100000,mu(k),beta(k),rho(k)); - h(k,:) = histc(x(k,:),cents); -end -bar(cents,h(1,:)/(100000*delt),'b'); -bar(cents,h(2,:)/(100000*delt),'g'); -bar(cents,h(3,:)/(100000*delt),'r'); -xlim([-8 8]); set(gca,'XTick',-10:10); -set(gca,'fontsize',14); -xlabel('\itx \rm(bin)'); ylabel('normalized histogram'); -``` - -**ggcode4** -```matlab -clear x h b; -x = -10:0.01:10; -mu=[-2 0 2]; -beta= [1 1 1]; sbeta = sqrt(beta); -rho=[1 2 10]; -alpha = [0.5 0.2 0.3]; -p = zeros(size(x)); -for j = 1:length(mu) - d(j,:) = alpha(j) * exp(-abs(sbeta(j)*(x-mu(j))).^rho(j)) * sbeta(j) / (2*gamma(1+1/rho(j))); - p = p + d(j,:); -end -figure, hold on; set(gca,'fontsize',14); -plot(x,d','g'); -plot(x,p,'linewidth',2); -set(gca,'XTick',-10:10); xlim([-8 6]); ylim([0 0.3]); -xlabel('\itx');ylabel('\itp_M(x)'); - -N = 100000; -clear x h b; -figure, hold on;set(gca,'fontsize',14); -mu=[-2 0 2]; -beta= [1 1 1]; sbeta = sqrt(beta); -rho=[1 2 10]; -alpha = [0.5 0.2 0.3]; -for k = 1:length(mu) - x(k,:) = gg6(1,N,mu(k),beta(k),rho(k)); -end -nrnd = ceil(10*rand(1,N)); -ind = 1 * (nrnd <= 5) + 2 * (nrnd > 5).*(nrnd <= 7) + 3 * (nrnd > 7); -[h,b] = hist([x(1,ind==1) x(2,ind==2) x(3,ind==3)],150); -delt = b(2)-b(1); -bar(b,h/(N*delt),'b'); -set(gca,'XTick',-10:10); xlim([-8 6]); ylim([0 0.3]); -xlabel('\itx \rm(bin)'); ylabel('normalized histogram'); -``` - -**ICA_fig.m** -```matlab -N=2000; -A = [1 4;4 1]; -s = gg6(2,N,[0 0],[1 3],[1.5 1.5]); -x = A*s; -[h1,b1] = hist(s(1,:),50); -[h2,b2] = hist(s(2,:),50); -figure, hold on; set(gca,'fontsize',14); -subplot(2,4,1); set(gca,'fontsize',14); -plot(s(1,:),s(2,:),'b.'); axis(4*[-1 1 -1 1]); xlabel('\its_{\rm1}'); ylabel('\its_{\rm2}'); -subplot(2,4,2); set(gca,'fontsize',14); -bar(b2,h2/(N*(b2(2)-b2(1)))); axis(4*[-1 1 0 1/4]); xlabel('\its_{\rm2}'); ylabel('norm. histogram'); -subplot(2,4,5); set(gca,'fontsize',14); -bar(b1,h1/(N*(b1(2)-b1(1)))); axis(4*[-1 1 0 1/4]); xlabel('\its_{\rm1}'); ylabel('norm. histogram'); -subplot(2,4,3); hold on; set(gca,'fontsize',14); -subplot(2,4,3); plot(x(1,:),x(2,:),'g.'); axis(15*[-1 1 -1 1]); -subplot(2,4,3); plot([0 A(1,1)],[0 A(2,1)],'r-','LineWidth',2); -subplot(2,4,3); plot([0 A(1,2)],[0 A(2,2)],'r-','LineWidth',2); -xlabel('\itx_{\rm1}'); ylabel('\itx_{\rm2}'); -[h1,b1] = hist(x(1,:),50); -[h2,b2] = hist(x(2,:),50); -subplot(2,4,4); set(gca,'fontsize',14); -bar(b2,h2/(N*(b2(2)-b2(1))),'g'); axis([-15 15 0 0.25]); xlabel('\itx_{\rm2}'); ylabel('norm. histogram'); -subplot(2,4,7); set(gca,'fontsize',14); -bar(b1,h1/(N*(b1(2)-b1(1))),'g'); axis([-15 15 0 0.25]); xlabel('\itx_{\rm1}'); ylabel('norm. histogram'); -``` \ No newline at end of file diff --git a/plugins/amica/index.md b/plugins/amica/index.md index a342b4b..61eb46f 100644 --- a/plugins/amica/index.md +++ b/plugins/amica/index.md @@ -4,7 +4,6 @@ title: AMICA long_title: AMICA parent: Plugins render_with_liquid: false -has_children: true nav_order: 4 --- To view the plugin source code, please visit the plugin's [GitHub repository](https://github.com/sccn/amica).