Merge pull request #346 from clowder-framework/release/1.20.1

Release/1.20.1

CONTRIBUTORS.md

@@ -10,6 +10,7 @@ Following is a list of contributors in alphabetical order:
 - Chris Navarro
 - Chrysovalantis Constantinou
 - Constantinos Sophocleous
+- Dipannita Dey
 - Gene Roeder
 - Gregory Jansen
 - Indira Gutierrez

app/controllers/Datasets.scala

@@ -681,8 +681,8 @@ class Datasets @Inject() (
             (SortingUtils.sortFolders(filteredFolders, sortOrder).slice(limit * filepageUpdate, limit * (filepageUpdate + 1)),
               SortingUtils.sortFiles(filteredFiles, sortOrder).slice(limit * filepageUpdate - filteredFolders.length, limit * (filepageUpdate + 1) - filteredFolders.length))
           } else {
-            (folders.get(childFolders.reverse.slice(limit * filepageUpdate, limit * (filepageUpdate + 1))).found,
-              files.get(childFiles.reverse.slice(limit * filepageUpdate - childFolders.length, limit * (filepageUpdate + 1) - childFolders.length)).found)
+            (folders.get(childFolders.reverse.slice(limit * filepageUpdate, limit * (filepageUpdate + 1))).found.reverse,
+              files.get(childFiles.reverse.slice(limit * filepageUpdate - childFolders.length, limit * (filepageUpdate + 1) - childFolders.length)).found.reverse)
           }
 
         // Get comment counts per file

app/views/collections/parentAllocation.scala.html

@@ -76,7 +76,7 @@ <h4>Parent collections</h4>
         allowClear: true,
         ajax: {
             url: function(params) {
-                return jsRoutes.api.Collections.listPossibleParents( '@resourceId', params.term,null, 5).url;
+                return jsRoutes.api.Collections.listPossibleParents(null, '@resourceId', params.term, null, 5).url;
             },
             data: function(params) {
                 return { title: params.term };

app/views/curations/publishedGrid.scala.html

@@ -66,7 +66,8 @@
 
       <script src="@routes.Assets.at("javascripts/filtrify.js")"  type="text/javascript"></script>
 	  <link rel="stylesheet" href="@routes.Assets.at("stylesheets/filtrify.css")">
-      <script src="/assets/javascripts/lib/masonry.pkgd.min.js" type="text/javascript"></script>
+
+      <script src="@routes.Assets.at("javascripts/lib/masonry.pkgd.min.js")" type="text/javascript"></script>
       <script>
 
 		$(document).ready(function(){

doc/src/sphinx/admin/customizing.rst

@@ -1,33 +1,73 @@
 .. _customization:
 
-******
-Customization
-******
+********************************
+Customizing
+********************************
+
+The default configuration
+==========================
+
+.. warning::
+  Do **not** make changes to the original files in ``/conf``. Instead, create a ``/custom`` folder shown below.
+
+The default configuration is fine for simple testing, but if you would like to modify any of the settings, you can find
+all the configuration files under the ``/conf`` directory. The following files are of particular importance:
+
+- ``/conf/application.conf`` includes all the basic configuration entries. For example the MongoDB credentials for
+  deployments where MongoDB has non default configuration.
+- ``/conf/play.plugins`` is used to turn on and off specific functionality in the system. Plugins specific to Clowder are
+  available under ``/app/services``.
+- ``/conf/securesocial.conf`` includes configuration settings for email functionality when signup as well as ways to
+  configure the different identity providers (for example Twitter or Facebook). More information can be found on the
+  `securesocial <http://securesocial.ws/>`_ website.
+
+
+How to customize Clowder
+============================
+
+To customize Clowder, create a folder called ``custom`` inside the Clowder folder (``clowder/custom``).
+Add the following. Modifications included in these files will overwrite defaults in ``/conf/application.conf`` and ``/conf/play.plugins``.
+
+.. code:: bash
+  
+      cd clowder
+      mkdir custom
+      touch custom/application.conf custom/play.plugins
+
+
+If you are working on the source code this folder is excluded from git so you can use that also to customize your development environment, and not accidentally commit changes to either ``play.plugins`` or ``application.conf``. If you make any changes to the files in the custom folder you will need to `restart the application` (both in production and development).
 
-To customize Clowder you can put all configuration changes in a folder called custom inside the Clowder folder. If you are working on the source code this folder is excluded from git so you can use that also to customize your development environment, and not accidentally commit changes to either play.plugins or application.conf. If you make any changes to the files in the custom folder you will need to restart the application (both in production and development).
 
 play.plugins
-============
+--------------
 
-The play.plugins file is used to enable plugins. You can only enable plugins, you can not disable plugins. This is one of the reasons why we minimize the number plugins that are enabled by default. For example most instances at NCSA will have the following plugins enabled.
+The ``/custom/play.plugins`` file describes all the `additional` plugins that should be enabled. **This file can only add additional plugins,
+and is not capable of turning off any of the default ones enabled in** ``/conf/play.plugins``.
+For example the following ``play.plugins`` file will enable some additional plugins:
 
 .. code-block:: properties
   :caption: play.plugins
 
   9992:services.RabbitmqPlugin
+  10002:securesocial.core.providers.GoogleProvider
   11002:services.ElasticsearchPlugin
 
+
 custom.conf
-===========
+--------------
+
+``/custom/custom.conf`` is used to override any of the defaults in the ``application.conf`` or any included conf files (such as ``securesocial.conf``). Common changes we do is to modify Clowder to use a directory on disk to store all blobs instead of storing them in mongo. Following is an example that we use for some of the instances we have at NCSA.
+
+- **One change every instance of Clowder should do is to modify the commKey and application.secret.** 
+
 
-The custom.conf file is used to override any of the changes in the application.conf or any included conf files (such as securesocial.conf). One change every instance of Clowder should do is to modify the commKey and application.secret. Common changes we do is to modify Clowder to use a directory on disk to store all blobs instead of storing them in mongo. Following is an example that we use for some of the instances we have at NCSA.
 
 .. code-block:: properties
   :caption: custom.conf
 
-  # security options
-  application.secret="1234567890123456789012345678901234567890"
-  commKey=notreallyit
+  # security options -- should be changed!
+  application.secret="some magic string"
+  commKey=magickey
 
   # email when new user tries to sign up
   smtp.from="[email protected]"
@@ -36,7 +76,7 @@ The custom.conf file is used to override any of the changes in the application.c
   # URL to mongo
   mongodbURI = "mongodb://mongo1:27017,mongo2:27017,mongo3:27017/server1?replicaSet=CLOWDER"
 
-  # where to store the blobs
+  # where to store the blobs (highly recommended)
   service.byteStorage=services.filesystem.DiskByteStorageService
   medici2.diskStorage.path="/home/clowder/data"
 
@@ -52,7 +92,9 @@ The custom.conf file is used to override any of the changes in the application.c
   elasticsearchSettings.serverPort=9300
 
   # securesocial customization
+  # set this to true if using https
   securesocial.ssl=true
+  # this will make the default timeout be 8 hours
   securesocial.cookie.idleTimeoutInMinutes=480
 
   # twitter setup
@@ -62,17 +104,24 @@ The custom.conf file is used to override any of the changes in the application.c
   securesocial.twitter.consumerKey="key"
   securesocial.twitter.consumerSecret="secret"
 
+  # google setup
+  securesocial.google.authorizationUrl="https://accounts.google.com/o/oauth2/auth"
+  securesocial.google.accessTokenUrl="https://accounts.google.com/o/oauth2/token"
+  securesocial.google.clientId="magic"
+  securesocial.google.clientSecret="magic"
+  securesocial.google.scope="https://www.googleapis.com/auth/userinfo.profile https://www.googleapis.com/auth/userinfo.email"
+
   # enable cache
   ehcacheplugin = enabled
 
 
 messages.XY
-===========
+---------------------
 
 This allows to translate or customize certain aspects of Clowder. All messages in Clowder are in english and are as messages.default. Unfortunately it is not possible to use messages.default to use for translations since it falls back to those embedded in the Clowder jar files. To update the messages in english, you can use messages.en. The default is for Clowder to only know about english, this can be changed in your custom.conf with ``application.langs="nl"``.
 
 Customizing Web UI
-==================
+---------------------
 
 The ``public`` folder is place where you can place customizations for previews, as well as new stylesheets. To add a new stylesheet you should place it in the public/stylesheets/themes/ folder. The name should be <something>.min.css or <something>.css. The user will at this point see in their customization settings the option to select <something> as their new theme to be used.
 

doc/src/sphinx/changelog_md_to_rst.rst

@@ -0,0 +1,6 @@
+Changelog
+***********
+
+.. To read our changelog.md file, this plugin converts markdown to .rst formatting (`pip install m2r2`)
+
+.. mdinclude:: changelog.md

doc/src/sphinx/conf.py

@@ -22,7 +22,7 @@
 author = 'Luigi Marini'
 
 # The full version, including alpha/beta/rc tags
-release = '1.20.0'
+release = '1.20.1'
 
 
 # -- General configuration ---------------------------------------------------
@@ -31,6 +31,7 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
+    "sphinx_rtd_theme", "sphinx_design", "m2r2"
 ]
 
 # Add any paths that contain templates here, relative to this directory.

doc/src/sphinx/develop/contribute_docs.rst

@@ -0,0 +1,89 @@
+.. index:: How to Contribute Documentation
+
+How to Contribute Documentation
+=====================================
+
+Documentation is stored in ``doc/src/sphinx``. 
+
+Dependencies are stored in ``doc/src/sphinx/requirements.txt``.
+
+
+
+
+.. tab-set::
+
+  .. tab-item:: conda
+
+    Create a virtual environment for documentation: 
+
+    .. code:: bash
+
+      conda create -n clowder_docs python=3.8 -y
+      conda activate clowder_docs
+
+    Now we must edit the `requirements.txt` file to be compatible with Conda. These packages are not available on conda-forge. 
+
+    Comment out the top three lines like so:
+
+    .. code:: properties
+
+      # -i https://pypi.org/simple/
+      # sphinx-rtd-theme==0.5.0
+      # sphinx_design==0.0.13
+      ... 
+
+    Install the dependencies. It's always better to run all conda commands before installing pip packages.
+
+    .. code:: bash
+
+      conda install --file requirements.txt -y
+      pip install sphinx-rtd-theme==0.5.0 sphinx_design==0.0.13
+
+  .. tab-item:: pyenv
+
+    Create a virtual environment for documentation:
+
+    .. code:: bash
+
+      pyenv install 3.7.12 # or any 3.{7,8,9}
+      pyenv virtualenv 3.7.12 clowder_docs
+
+      # make virtual environemnt auto-activate
+      cd doc/src/sphinx
+      pyenv local clowder_docs
+
+    Install doc dependencies: 
+
+    .. code:: bash
+
+      pip install -r requirements.txt
+
+Now, build HTML docs for viewing: 
+
+    .. code:: bash
+
+      # run from doc/src/sphinx
+      sphinx-autobuild . _build/html
+
+Open http://127.0.0.1:8000 in your browser. Saved changes will be auto-updated in the browser.
+
+
+.. dropdown:: (Optional alternative) Static builds
+
+    If you do not want dynamic builds, you can statically generate the HTML this way.
+
+    .. code:: bash
+
+      cd doc/src/sphinx
+      make html
+    
+    View docs by opening ``index.html`` in the browser
+    ``clowder/doc/src/sphinx/_build/html/index.html``
+
+
+
+⭐ If you experience *any* trouble, come ask us on `Slack here <https://join.slack.com/t/clowder-software/shared_invite/enQtMzQzOTg0Nzk3OTUzLTYwZDlkZDI0NGI4YmI0ZjE5MTZiYmZhZTIyNWE1YzM0NWMwMzIxODNhZTA1Y2E3MTQzOTg1YThiNzkwOWQwYWE>`_! ⭐
+
+.. note::
+
+  To see how to install Clowder, please see :ref:`installing_clowder`.

doc/src/sphinx/index.rst

@@ -36,7 +36,7 @@ Contents
     :caption: General
 
     overview
-    changelog
+    changelog_md_to_rst
     license
 
 .. toctree::
@@ -59,9 +59,10 @@ Contents
     :caption: Development
 
     develop/architecture
-    develop/setup_dev
     develop/extractors
     develop/previewers
+    develop/contribute_docs
+    .. not included:  publishing_data
 
 .. toctree::
     :maxdepth: 1