This is the educational project with main purpose is to develop and test the multidimensional adaptive grid which can be used in Supervised Learning and Reinforcement Learning areas. Additional purpose of this project is to learn and try CUDA programming with Alea GPU.
In typical reinforcement learning scenario an agent takes actions corresponding to current policy and state of an environment. Adaptive grid is needed to represent a manifold of possible actions of an agent and manifold of states of an environment with ability to adjust grid resolution in regions of interest. The concept of adaptive refinement of the manifolds of possible states and actions is close to common learning practices (for example, in learning of car driving at a very first stage there is no need for any distinction of severity of bends, instead, the only distinction between left and right turns could be suitable. The more experience and average speeds, the more details about bends and the more precise steering are needed for further improvement of driving skills)
The class AdaptiveGrid is implemented as voxel tree. Each voxel exposes its properties and methods through the IVoxel interface. The maximum number of voxels and shape of the grid are set at creation time of the AdaptiveGrid instance and can't be changed during the liftime of the instance. The voxels are stored in the internal single-dimensional array, which entirely initialized at creation time of the AdaptiveGrid instance. So, during different manipulations with the grid (voxel tessellation or merging), voxels are only changed their properties and no voxel instances are created or deleted. The grid can be processed in multithreaded manner. The methods RefineGrid_CGParallel, RefineGrid_FGParallel and others represent templates of multithreaded processing and also show how the voxel iterator NextAtLevel works.
The TestWpf project is intended for testing of two-dimensional AdaptiveGrid<double> instances, for estimation of performance
of the grid processing methods and for trying some CUDA programming, which in this case was used for bitmaps calculation.
The TestWpf has graphical output for visual presentation of the grid and voxels. Several simple abstract
models
were created for the
testing of the grid refinement processes and for fun in some ways.
