User manual & docs

resolves #47

README.md

@@ -3,17 +3,21 @@
 
 A free open-source toolbox for advanced processing of VLE (vapor-liquid equilibrium) data, which comprises tests of thermodynamic consistency & model fitting using non-linear regression.
 
-App is currently WIP 🚧, buildable but not released yet,  
+➡ App is currently WIP 🚧, buildable but not released yet,  
 though if you'd like to see what it looks like, [here's a 3 minute](https://drive.google.com/file/d/1BY5ER9d8al4iYIa0OSvPQxuiorSwq9Jj/view?usp=sharing) demo showcasing it from tech perspective 👀
 
+🎓 **User manual** can be found [here](docs/user/manual.md).
+
+🐛 **Encountered a bug?** Please, do report it [here](docs/user/bug_tracking.md).
+
+## Development
+
 The project is a monorepo consisting of:
 - [Python Core+Backend+CLI](docs/appPy.md)
 - [Typescript Frontend](docs/appUI.md)
 
-In the initial phase, VLizard is intended as a toolbox for VLE data, but in future its scope may be expanded to include tools for LLE or reaction kinetics data.
-
 Conventions upheld across this repo are summarized [here](docs/conventions.md).  
 Credits for literature & software resources [here](docs/references.md).
 
-## Setup, run, build
+### Setup, run, build
 See respective docs for [Py backend](docs/appPy.md) and [TS frontend](docs/appUI.md)

appUI/src/pages/Systems/ChooseTDTestDialog.tsx

@@ -22,7 +22,7 @@ export const ChooseTDTestDialog: FC<DialogProps & DatasetIdentifier> = ({ open,
                 <VLEAnalysisButton {...props} />
                 <Button
                     startIcon={<QuestionMark />}
-                    href="https://github.com/Lemonexe/VLizard/blob/master/docs/user/VLE_analyses.md"
+                    href="https://github.com/Lemonexe/VLizard/blob/master/docs/user/tests.md"
                     variant="outlined"
                     sx={{ my: 2 }}
                 >

docs/user/bug_tracking.md

@@ -5,4 +5,12 @@ In case you encounter a bug in VLizard app, please report it to the author:
 - write an email to [[email protected]](mailto:[email protected])
 - or create an issue at [github](https://github.com/Lemonexe/VLizard/issues)
 
+Please include following information in you report:
+- VLizard version _(can be found in "About" page at top bar)_
+- Operating system
+- Description of the bug, ideally with some of the following:
+  - steps to reproduce
+  - data files which cause the bug _("Open fata folder" at top bar)_
+  - screenshots or videos
+
 Thank you for helping improve the app and make it more stable by reporting bugs!

docs/user/manual.md

@@ -1,3 +1,114 @@
 ## VLizard user manual
 
-TODO
+Welcome to the user manual for VLizard, a VLE wizard. 🧙‍♂️⚗🦎
+
+This document will quickly guide you through the process of using VLizard app for processing VLE data.  
+It does not cover all advanced features, but should be enough to get you started.  
+
+🎓 Tests of thermodynamic consistency are covered in separate documents: [Overview of tests](tests.md), [Theoretical background](test_theory.md).
+
+#### Content
+1. [First navigation](#first-navigation)
+2. [Adding Pure compounds](#adding-pure-compounds)
+3. [Adding VLE dataset](#adding-vle-dataset)
+4. [Fitting thermodynamic models](#fitting-thermodynamic-models)
+5. [Other features](#other-features)
+6. [Troubleshooting](#troubleshooting)
+
+### First navigation
+
+After opening the app, you should see the welcome page, and most importantly, the navigation top bar:
+
+![top bar](images/top_bar.png)
+
+The "About" icon will lead to this manual.
+As for the other icons, we will get to them later.  
+Let's visit the **Pure compounds** page.
+
+### Adding Pure compounds
+
+For any kind of VLE data processing you need to have a vapor pressure model for each pure compound.
+
+I think an example is worth a thousand words, so if you have VLizard freshly installed, there is example data from literature for ethanol & water.  
+So click the "Perform analysis" icon:
+![analysis icon](images/analysis_icon.png)
+
+You should see a dialog with behavior of the vapor pressure model, which should be self-explanatory.  
+💡 Any dialog can be closed by clicking on X, the blurred background, or by pressing Escape.  
+💡 Any plot can be saved as vectorized `svg` image to your computer.
+
+Try editing a model via the pencil icon.  
+💡 This is fullscreen dialog, it can be closed only via buttons, not Escape key, so you don't accidentally lose progress. 
+
+You can now Add a new model at top of page, and fill the dialog similarly.
+In case you have obtained the model parameters elsewhere, enter them.
+Or, if you have experimental data for vapor pressure, you can "Perform fitting" – paste your data into the table and Optimize parameters.
+
+Don't worry about temperature range, it's only informative – calculations will warn you when you extrapolate too much _(can be tweaked in settings)_.  
+👉 After adding a model, it's recommended to do an analysis to sanity check it.
+
+Of course, any model can be deleted, but let's avoid for now, and proceed by visiting the **Binary systems** page.
+
+### Adding VLE dataset
+
+All binary systems are listed here.
+Each system can have multiple datasets, and you can fold/expand them using arrows.
+
+Look at one of the example datasets:  
+Icon ![edit table icon](images/edit_table_icon.png) lets you edit the VLE data.  
+Icon ![visualize icon](images/visualize_icon.png) visualizes your data.   
+Icon ![analysis icon](images/analysis_icon.png) **is most important** – it opens a menu with a wide range of analyses. The help button will help you get familiar with them _(leads to [overview of tests](tests.md))_.
+
+In order to do the same with your data, you can enter it similarly to editing.
+Either "Add new" system, or save some clicks by adding dataset for an existing system via ![new table icon](images/new_table_icon.png)  
+💡 If you have your table differently arranged, no problem, just rearrange the column headers and pick your units of measure.
+
+✅ Congratulations, you now have mastered the most important feature of VLizard!  
+You can visit the **Fitting** page if you're interested.
+It's also needed for the van Ness test.
+
+### Fitting thermodynamic models
+
+Last but not least, VLizard can fit your VLE data with a thermodynamic model.  
+Here you can see your systems, but with no fitted models, so add one via ![new table icon](images/new_table_icon.png)
+
+Select the datasets to be fitted and your model.  
+Initial parameters are provided, which you can tweak.  
+You may specify which parameters should not be optimized = left as initial estimate.  
+The "Optimize & Save" button will proceed with fitting, and results will be displayed and persisted.   
+💡 Mark all parameters as constant if you'd like to just input fitted parameters from elsewhere.
+
+### Other features
+
+There are some icons in the top bar we have not talked about yet:
+
+- Home icon will get you to the welcome page. Not particularly useful.
+- Refresh will refresh all vapor pressure models, VLE data and fitted models
+  - Use if you have manipulated the data manually _outside_ VLizard. 
+- Open data folder – opens the VLizard data, located in your Documents directory.
+  - Data is stored in `.tsv` files, which may be edited manually with Excel, but:
+    - ⚠ Do only at your own risk, the app expects a very specific format, and may crash otherwise. Doing a backup is advised.
+    - Though moving or deleting files is safe.
+- Settings – so you can tweak VLizard to your liking!
+
+### Troubleshooting
+
+Oops, sometimes trouble happens!
+Here are some general steps you can try.
+
+If nothing helps, please report the bug 🐛 [using this contact](bug_tracking.md).
+
+- Sometimes you may get an error notification at left bottom.
+  - This means that a specific action failed, but you can still do other things. 
+  - Please check if the error appears only for particular data.
+    - If so, check the data for mistakes.
+    - If there is no visible mistake, send the data along with bug report.
+  - Please note that for fitting, some crashes are normal! Optimization simply does not always converge.
+    - Try different initial params, or select different portion of data.
+- If you encounter an error screen, the app is unusable 😱
+  - `ctrl+R` refreshes the UI without having to restart whole app. 
+  - But restart will be necessary for this error screen: `Lost connection to the Core server!`
+- `ctrl+shift+I` will display the UI console. Please copy any error you see there, and send it along with a bug report. 
+
+In any case, keep in mind this a free open-source project.
+I am doing my best to make it functional and user-friendly, but do not expect miracles 🙂

docs/user/test_theory.md

@@ -0,0 +1,169 @@
+## Theoretical background for thermodynamic consistency tests
+
+This is just quick reference for the theoretical background of testing procedures.
+See [Overview](tests.md) document for practical aspects of the tests.
+For more detailed theoretical explanation, please refer to literature, such as a comprehensive
+[2017 review by J. Wisniak et al.](https://doi.org/10.1016/j.jct.2016.10.038)
+Alternatively, see [all literary sources](../references.md) for VLizard.
+
+### Basis
+
+The Gibbs-Duhem equation _(G-D)_ is used for most of the tests.
+This is the most general form:
+```math
+    \sum_i x_i \mathrm{d} \ln \gamma_i = \frac{V_m^\mathrm{E}}{R T} \mathrm{d} p - \frac{H_m^\mathrm{E}}{R T^2} \mathrm{d} T
+```
+
+We can use definition of excess Gibbs energy in dimensionless form (divided by $RT$), and enumerate for binary system:
+```math
+    \mathrm{d} g^\mathrm{E} = \frac{V_m^\mathrm{E}}{R T} \mathrm{d} p - \frac{H_m^\mathrm{E}}{R T^2} \mathrm{d} T + \ln \frac{\gamma_1}{\gamma_2} \mathrm{d} x_1
+```
+
+### Fredenslund test
+
+The idea here is that VLE data ($p$, $T$, $x$, $y$) together with vapor pressure models is an overdetermined system in its nature.
+In Fredenslund test, the data is fitted with a model, but not directly – the data is first reduced using $g^\mathrm{E}$.
+The model is then following:
+
+```math
+    g^\mathrm{E} = x_1 (1 - x_1) \sum_{k=0}^n a_k L_k(x_1)
+```
+Where $L_k$ is the Legendre polynomial of order $k$ (see [wikipedia](https://en.wikipedia.org/wiki/Legendre_polynomials\#Shifted_Legendre_polynomials)).
+
+You may choose $n$, the number of polynomials (model params):
+- $n=4$ **most usually**
+- $n=3$ if you have very few data points
+- $n=5$ if you expect highly non-ideal system and have many data points.
+
+This model has no thermodynamic meaning, but suitable mathematical properties for the analysis _(orthogonality)_.
+
+Using the $g^\mathrm{E}$ calculated from model, the activity coefficients are recalculated as:
+```math
+    \ln \gamma_1^\mathrm{cal} = g^\mathrm{E}_\mathrm{cal} + x_2 \tfrac{\mathrm{d} g^\mathrm{E}_\mathrm{cal}}{\mathrm{d} x_1}
+```
+```math
+    \ln \gamma_2^\mathrm{cal} = g^\mathrm{E}_\mathrm{cal} - x_1 \tfrac{\mathrm{d} g^\mathrm{E}_\mathrm{cal}}{\mathrm{d} x_1}
+```
+
+$p$ and $y_1$ are then calculated using Raoult-Dalton's Law.
+
+Then, the residuals between calculated and experimental values of $p$ and $y_1$ are evaluated.
+
+Finally, following metrics are calculated _(average relative, resp. absolute error)_: 
+
+```math
+    \overline{\delta p} = \sum_{j=1}^N \frac{1}{N} \frac{|p_j^\mathrm{exp} - p_j^\mathrm{cal}|}{p_j^\mathrm{exp}}
+```
+
+```math
+    \overline{\Delta y_1} = \sum_{j=1}^N \frac{1}{N} |y_j^\mathrm{exp} - y_j^\mathrm{cal}|
+```
+
+The same is done for $y_2$ as for $y_1$.
+
+All must be within acceptable limits, which is 1 %.
+
+### van Ness test
+
+By deriving $g^\mathrm{E}$ and substituting G-D we get:
+
+```math
+    \frac{\mathrm{d} g^\mathrm{E}}{\mathrm{d} x_1} = \ln \frac{\gamma_1}{\gamma_2}
+```
+
+The idea of van Ness test is to subtract that equation for `calculated - experimental`, like this:
+
+```math
+    \frac{\mathrm{d} g^\mathrm{E}_\mathrm{cal}}{\mathrm{d} x_1} - \frac{\mathrm{d} g^\mathrm{E}_\mathrm{exp}}{\mathrm{d} x_1} = \ln \frac{\gamma_1^\mathrm{cal}}{\gamma_2^\mathrm{cal}} - \ln \frac{\gamma_1^\mathrm{exp}}{\gamma_2^\mathrm{exp}}
+```
+
+The left-side is zero – but only if the model was fitted for $\gamma$ or $g^\mathrm{E}$.
+That means when the residual function, which was optimized to zero, is calculated using residuals of those.
+You may use a model fitted in VLizard, which uses $\gamma$ for optimization.
+
+⚠ You may use model params obtained elsewhere, but **do not** use such model for van Ness test, if it was optimized for $T$, $x$ or $y$!  
+It will make the test mathematically invalid.
+
+The left-side should be zero, so a residual from zero can be calculated:
+
+```math
+    \Delta_j = \ln \frac{\gamma_{1j}^\mathrm{cal}}{\gamma_{2j}^\mathrm{cal}} - \ln \frac{\gamma_{1j}^\mathrm{exp}}{\gamma_{2j}^\mathrm{exp}}
+```
+
+Finally, the root mean square of the residuals is calculated:
+
+```math
+    RMS = \sqrt{ \tfrac{1}{n} \textstyle\sum_j^n \Delta_j^2 }
+```
+
+Instead of a single threshold to formally accept or reject the data, the van Ness test assigns a conventional index from 1 to 10, where 1 is perfect and 10 is unacceptable, for a more nuanced evaluation.
+
+### Gamma offset test
+
+The gamma offset test does examine thermodynamic consistency, though not using G-D, but only the _definition of activity_.  
+The activity coefficients must always be 1 for pure components, so it is just a simple check to see if $\gamma_i(x_i = 1) = 1$.  
+If that's not the case, then the VLE data does not align with the vapor pressure models for pure components.
+
+A modified van Laar equation is used, which includes the error, or "offset" parameter $E_i$:
+
+```math
+    \ln \gamma_1 = A_{12} \left(\frac{A_{21} x_2}{A_{12} x_1 + A_{21} x_2}\right)^2 + E_1
+```
+```math
+    \ln \gamma_2 = A_{21} \left(\frac{A_{12} x_1}{A_{12} x_1 + A_{21} x_2}\right)^2 + E_2
+```
+
+Both residuals are respectively weighted by $x_1$, $x_2$, so that both $E_1$ and $E_2$ are biased towards the pure regions, and accurately reflect them.
+
+Conventional criterion is then used to formally accept or reject the data: $E_i < 1.5 %$.
+
+### Redlich-Kister test
+
+This test uses G-D in integral form, while disregarding $V_m^\mathrm{E}$ and $H_m^\mathrm{E}$ (the pressure and temperature dependence):
+
+```math
+    \int_{x_1=0}^{x_1=1} \ln \frac{\gamma_1}{\gamma_2} \mathrm{d} x_1 = 0
+```
+
+The curve forms two areas $A$ and $B$, one above and one below the x-axis.
+
+Conventional criterion is used to formally accept or reject the data: $D < 10$.
+
+```math
+    D = 100 \frac{|A - B|}{A+B}
+```
+
+Usually, $V_m^\mathrm{E}$ is very small, so for isothermal data, the test is reliable.
+
+But $H_m^\mathrm{E}$ is usually not negligible, so for isobaric data, the test is a gross approximation.
+
+That's why it should be used only to reject the data when result is negative, but it has no meaning when positive, as per J. Wisniak.
+
+### Herington test
+
+A variation of Redlich-Kister test for isobaric data, which aims to replace the temperature dependence with an empirical parameter: 
+
+```math
+    J = 150 \frac{T_\mathrm{max} - T_\mathrm{min}}{T_\mathrm{min}}
+```
+
+The conventional criterion is then $|D-J| < 10$.
+
+Empirical parameter $J$ is not theoretically founded, and according to J. Wisniak, many systems do not conform to it.
+The test is therefore deprecated. 
+
+### Slope test
+
+At first glance, this is the most mathematically straightforward test.
+Just like the Redlich-Kister test, the pressure and temperature dependence is disregarded.
+G-D is used in _derived_ form:
+
+```math
+    x_1 \frac{\mathrm{d} \ln \gamma_1}{\mathrm{d} x_1} + x_2 \frac{\mathrm{d} \ln \gamma_2}{\mathrm{d} x_1} = 0
+```
+
+For each point the residual from zero is calculated.
+Outlying points with particularly high residuals are then considered suspicious, but no formal criterion has been established.
+
+However, the apparent simplicity is deceptive – problem lies in numerical calculation of the derivatives, which is generally unreliable, and makes the test seldom usable.
+VLizard uses a derivation formula using _all_ points from the non-equidistant grid, which helps smooth out the sensitivity to random error, but it is often still too noisy to give meaningful information on thermodynamic consistency.