Code to select the aod motion computation method based on file versio… #16

saumilpatel · 2017-02-20T23:48:08Z

…n. Supports file version in which motion data are 3D

…ected spikes data

…n. Supports file version in which motion data are 3D

dimitri-yatsenko · 2017-02-21T13:18:13Z

startup.m

+setPath ;
+setenv('DJ_HOST', 'at-database.ad.bcm.edu');
+setenv('DJ_USER','spatel');
+setenv('DJ_PASS','spatel');


argh! checked in password. Please change your password and exclude your startup.m from the path or set the environment variables in a separate script such as your ~/.profile (on Mac OS)

argh!, removed the startup from the location where it was stored, moved the credentials to matlab's default startup on the machine, also changed password

it is probably worth doing a squash of these two commits together and then doing a force push to your repository to overwrite them, so when the code is merged that never shows up in the history

dimitri-yatsenko · 2017-02-21T13:20:04Z

schemas/+aod/trackMotion3D.m

+analyzenthframe = 1 ; % skip frames if > 1
+numframestoaverageforref = 10 ; % frames to average to determine the reference frame against which motion is computed
+usemulticore = true ;
+postrefstartframetime = 0 ; % N secs to skip before analyzing motion signals, in many cases, right after starting the point scan, there is a slight drift


for better readability, use camelCase or underscore_separated compound names.

dimitri-yatsenko · 2017-02-21T13:24:59Z

schemas/+aod/trackMotion3D.m

+    count = 0 ;
+    for ii=numframestoaverageforref+analyzenthframe:analyzenthframe:size(dat2,1)
+        count = count + 1 ;
+        for jj=ii-analyzenthframe+1:ii


you can replace this loop with array operations:

adat(count,:) = adat(count, :) + mean(dat2(ii-analyzenthframe+1:ii,:))

dimitri-yatsenko · 2017-02-21T13:28:47Z

schemas/+aod/trackMotion3D.m

+
+% take into account frames to skip, i.e. sampling rate of motion,
+% downsample with averaging
+adat = zeros(size(dat2)) ;


replace these loops with matlab's implementation of downsampling.

did not change, cant find downsampling code that will average samples, dont want to deal with filters

dimitri-yatsenko · 2017-02-21T13:32:42Z

schemas/+aod/trackMotion3D.m

+needreference=false ; % no need for a reference, start computing positions relative to the reference of spheres in subsequent motion frames
+if (usemulticore)
+    parfor ii=1:count
+        [shifts(ii+numframestoaverageforref,:)] = computeshifts(adat(ii,:)',coordinates,gridsize,calibparams,refparams,needreference) ;  


don't need brackets [ ... ] around shifts(...)

dimitri-yatsenko · 2017-02-21T13:40:01Z

schemas/+aod/trackMotion3D.m

+adat = zeros(size(dat2)) ;
+if (analyzenthframe>1)
+    count = 0 ;
+    for ii=numframestoaverageforref+analyzenthframe:analyzenthframe:size(dat2,1)


This loop may ignore the tail of the dat2 when analyzenthframe>1. Is that intended?

dont follow, but my best guess is what you mean is that samples at the end may not be processed, ?

dimitri-yatsenko · 2017-02-21T13:41:22Z

schemas/+aod/trackMotion3D.m

+numsamples = br1.sz ;
+duration = numsamples(:,1)/br1.Fs ; % sec
+
+%numframes = size(dat,1) ;


remove commented code.

dimitri-yatsenko · 2017-02-21T13:42:45Z

schemas/+aod/trackMotion3D.m

+% requires z:\libraries\matlab in path
+run(getLocalPath('/lab/libraries/hdf5matlab/setPath.m')) ;
+
+analyzenthframe = 1 ; % skip frames if > 1


Rename analyzenthframe to something like downsampleFactor?

2. replaced downsampling inner loop with vector operation 3. removed commented code 4. matlab style changes Nothing done here affects functionality

… rebase

peabody124 · 2017-03-23T15:06:20Z

schemas/+aod/ScanMotion.m

-            coordinates = br.motionCoordinates;
-
+            fn = char(fetchn(acq.AodScan & key, 'aod_scan_filename')) ; % get the filename from db
+            oldversion = aod.checkFileversion(fn) ; % check if motion data is in old format, i.e. 2 planes or in new format, i.e. a volume


this is a typo, or more likely the checkFileVersion should have been checked into the aod directory instead of the acq directory.

oops... yes a typo, should be acq.checkFileVersion

it's probably best to keep with the aod schema since it doesn't apply to anything else in acq. Also, if we can refactor it into the AodMotionReader class so that this isn't exported in general that would be better (see comment below)

agree... will move it

peabody124 · 2017-03-23T15:07:55Z

schemas/+aod/trackMotion3D.m

+
+%% test function computeshifts
+% requires z:\libraries\matlab in path
+run(getLocalPath('/lab/libraries/hdf5matlab/setPath.m')) ;


this shouldn't be here. the user should have set up the path correctly before hand. this would likely have unintended side effects.

agree, will remove

peabody124 · 2017-03-23T15:10:30Z

schemas/+acq/checkFileversion.m

+function oldversion=checkFileVersion(fn)
+
+oldversion = true ;
+try


this is failing for me when analyzing

M/Mouse/2016-03-15_13-56-05/AODAcq/2016-03-15_14-38-45_%d.h5

specifically the h5readatt fails so we fall through to the catch statement. i believe this experiment was recorded with the new format, though.

Huh... works fine for the file you mention on at-ssp, Matlab R2016b. Is h5readatt working for other files ?

Can you check it from a mac? i'm using this from my local machine (R2016a but that command was introduced in 2011a)

Yeah, h5readatt does not work on my Mac 2016a... let me check why

it works if the file name is /Volumes/M....
Is this what you meant in the comment getFile... To translate the file name for the platform used ?

peabody124 · 2017-03-23T15:20:52Z

schemas/+aod/trackMotion3D.m

+useMultiCore = true ;
+postRefStartFrameTime = 0 ; % N secs to skip before analyzing motion signals, in many cases, right after starting the point scan, there is a slight drift
+
+br = aodReader(fn, 'Motion') ;


here I think this should use getFile(fn, 'Motion') to convert the path from /raw/blah to the real location

this I am not following...

I ended up replacing this with

fn2 = findFile(RawPathMap, fn) br = aodReader(fn2, 'Motion'); br1 = AodMotionCorrectionReader(fn2) ; % if motion was corrected by using aod offsets, this is where they would be read

sounds good

peabody124 · 2017-03-23T15:36:10Z

schemas/+aod/trackMotion3D.m

+calibparams = [0 1; 0 1; 0 1] ; % dont apply any calibration
+shifts = zeros(size(dat2,1),3); % store motion vectors here
+dat2(1,:) = mean(dat2(1:numFramesToAverageForRef,:)) ; % average frames for computing reference
+[shifts(1,:), refparams] = computeshifts(dat2(1,:)',coordinates,gridSize,calibparams,refparams,needReference) ; % this position of the fitted sphere is treated as the reference


this function is missing. above it references it being in lab\libraries\matlab but i don't see it there. also, please let's not spread these dependencies across multiple repositories.

This is in z:\libraries\TwoPhoton\AOD_Control\MotionCorrection. Note that we are using the same motion detection code that is used from the Labview program during real-time acquisition.

I don't really like duplicating code in two places, but at the same time in this case I think it would be better than spanning dependencies between two pretty separate projects. It also avoids unintended side effects if one of the formats changes the other would break.

OK, would make the sessions folder self contained but we will have to remember to keep the two folders synced.

peabody124 · 2017-03-23T15:37:31Z

schemas/+aod/trackMotion3D.m

+postRefStartFrameTime = 0 ; % N secs to skip before analyzing motion signals, in many cases, right after starting the point scan, there is a slight drift
+
+br = aodReader(fn, 'Motion') ;
+br1 = AodMotionCorrectionReader(fn) ; % if motion was corrected by using aod offsets, this is where they would be read


do we need a separate class here just to access the sampling rate? can we not expand the original AodMotionReader to handle both formats correctly? I only see br1 used below to access the number of samples and sampling rate, which AodMotionReader should just be taught to do correctly for this new format.

Probably not but we are just reusing code that we have.

peabody124 · 2017-03-23T15:38:43Z

schemas/+aod/trackMotion3D.m

@@ -0,0 +1,86 @@
+
+function [x,y,z,t] = trackMotion3D(fn, analysischan) 


could we refactor this a bit so move all the code that deals with the raw file access into ScanMotion (like it is for the original code) and then make this method just take the data set and process it? that would keep the two consistent.

We have to run motion detection during realtime acq as well as during offline analysis. Because realtime implementation came first, the code is residing in folder structure that we have for realtime stuff. We can rearrange if it is absolutely necessary, for sure we dont want to duplicate the same code in different folders

Removed path setup from within trackMotion3D

Conflicts: james_setPath.m processing/cleanup.m processing/spikesUtah.m

Incorporate dangling changes and updates to AOD schema

Fix issue atlab#13

Conflicts: schemas/+ephys/SpikesAlignedTrial.m

updated session cleanup code store results of saccadeflashexperiment to database

James Cotton and others added 11 commits June 17, 2013 22:49

Volume ignore improvement

8486b32

AOD: switch some objects to dj.AutoPopulate

f250860

AOD: check in some changes laying around

ee7d0f7

Merge branch 'master' of github.com:peabody124/sessions

c1307c1

Merge branch 'master' of github.com:peabody124/sessions

18c46c0

Whatever James had not commited...

a9aa2f1

Fix bug where running new spike detection method deletes previous det…

7e0835e

…ected spikes data

SyncNetworkFewTrials: check in missing file

45a1f9e

AOD: switch some objects to dj.AutoPopulate

a61d775

AOD: check in some changes laying around

7e2dc49

Code to select the aod motion computation method based on file versio…

4ac02b1

…n. Supports file version in which motion data are 3D

dimitri-yatsenko reviewed Feb 21, 2017

View reviewed changes

dimitri-yatsenko suggested changes Feb 21, 2017

View reviewed changes

unknown and others added 3 commits February 21, 2017 10:25

1. changed variables names

452df84

2. replaced downsampling inner loop with vector operation 3. removed commented code 4. matlab style changes Nothing done here affects functionality

Work remotely

0a854db

Merge branch 'master' of https://github.com/saumilpatel/sessions into…

418afa1

… rebase

peabody124 reviewed Mar 23, 2017

View reviewed changes

unknown and others added 5 commits March 23, 2017 13:03

Changed location of checkFilename from +acq to +aod

1d741aa

Removed path setup from within trackMotion3D

Check in code for Saumil's motion tracking

817e259

detectLcdPhotodiodeFlips: get rid of external dependecy

ee19c89

Small fixes

aa2dad5

AOD: improve robustness of syuncing

a667013

eywalker and others added 11 commits September 24, 2017 20:00

Merge remote-tracking branch 'origin/james'

f1c2a70

Conflicts: james_setPath.m processing/cleanup.m processing/spikesUtah.m

Make RawPathMap updatable

d7620db

Adjust diode offset accepted range

205cdc4

Add special handling for NatImExperiment end time

5c681f8

Merge pull request atlab#17 from peabody124/james

1302179

Incorporate dangling changes and updates to AOD schema

Merge pull request atlab#14 from atlab/bugfix

666ddae

Fix issue atlab#13

Merge branch 'master' of https://github.com/atlab/sessions

e2d236e

Conflicts: schemas/+ephys/SpikesAlignedTrial.m

code to record notification sent to slack in database

33b7752

updated session cleanup code store results of saccadeflashexperiment to database

Merge branch 'master' of https://github.com/saumilpatel/sessions

0a7ed69

added /Volumes/M to search path

f876949

NotificationSent table moved from acq to stimulation

a40ff29

		@@ -0,0 +1,86 @@

		function [x,y,z,t] = trackMotion3D(fn, analysischan)

Code to select the aod motion computation method based on file versio… #16

Are you sure you want to change the base?

Code to select the aod motion computation method based on file versio… #16

Conversation

saumilpatel commented Feb 20, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dimitri-yatsenko Feb 21, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dimitri-yatsenko Feb 21, 2017 •

edited

Loading