From 50084d8139abd062d0be413bedade983801a9505 Mon Sep 17 00:00:00 2001 From: Luka van der Plas Date: Thu, 1 Feb 2024 18:19:25 +0100 Subject: [PATCH] add citation, licence, update readme --- CITATION.cff | 23 ++++++ CONTRIBUTING.md | 183 ++++++++++++++++++++++++++++++++++++++++++++++ LICENSE | 3 +- README.md | 187 ++++-------------------------------------------- 4 files changed, 220 insertions(+), 176 deletions(-) create mode 100644 CITATION.cff create mode 100644 CONTRIBUTING.md diff --git a/CITATION.cff b/CITATION.cff new file mode 100644 index 0000000..1c3f28a --- /dev/null +++ b/CITATION.cff @@ -0,0 +1,23 @@ +cff-version: 1.2.0 +title: I-Analyzer +message: >- + If you use this software, please cite it using the + metadata from this file. +type: software +authors: + - name: 'Research Software Lab, Centre for Digital Humanities, Utrecht University' + website: 'https://cdh.uu.nl/centre-for-digital-humanities/research-software-lab/' + city: Utrecht + country: NL + - family-names: Ponzanesi + given-names: Sandra + affiliation: Utrecht University + - family-names: Lange + name-particle: de + given-names: Julia + affiliation: Utrecht University +repository-code: 'https://github.com/UUDigitalHumanitieslab/digital-atlas' +url: 'https://atlas.hum.uu.nl/' +license: BSD-3 +version: 0.0.0 +date-released: '2024-02-01' diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..9094b4f --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,183 @@ +This file contains instructions for developers. + +Note that this application is based on our [cookiecutter webapp deluxe template](https://github.com/UUDigitalHumanitieslab/cookiecutter-webapp-deluxe); the instructions below come from the general repository. + +However, unlike most of our applications, digital ATLAS works without a backend or database: all necessary data is included in the frontend. The Django backend is still included because that allows the application to run in our normal deployment configuration. Keep this in mind when consulting the documentation below. + +## Before you start + +You need to install the following software: + + - Python >= 3.8, <= 3.10 + - virtualenv + - [Visual C++ for Python][1] (Windows only) + - Node.js >= 8 + - Yarn + - [WebDriver][2] for at least one browser (only for functional testing) + - WSGI-compatible webserver (deployment only) + - PostgreSQL >= 10, client, server and C libraries (not used yet) + + +[1]: https://wiki.python.org/moin/WindowsCompilers +[2]: https://pypi.org/project/selenium/#drivers + + +## How it works + +This project integrates four isolated subprojects, each inside its own subdirectory with its own code, package dependencies and tests: + +- **backend**: the server side web application based on [Django][3] and [DRF][4]. Within this project this only serves the Angular frontend + +- **data**: reads the Excel sheet with all the collected data and exports it to a format which can be used by the frontend + + - **frontend**: the client side web application based on [Angular](https://angular.io) + + - **functional-tests**: the functional test suite based on [Selenium][6] and [pytest][7] + +[3]: https://www.djangoproject.com +[4]: https://www.django-rest-framework.org +[6]: https://www.selenium.dev/documentation/webdriver/ +[7]: https://docs.pytest.org/en/latest/ + +Each subproject is configurable from the outside. Integration is achieved using "magic configuration" which is contained inside the root directory together with this README. In this way, the subprojects can stay truly isolated from each other. + +If you are reading this README, you'll likely be working with the integrated project as a whole rather than with one of the subprojects in isolation. In this case, this README should be your primary source of information on how to develop or deploy the project. However, we recommend that you also read the "How it works" section in the README of each subproject. + + +## Development + +### Quickstart + +First time after cloning this project: + +```console +$ cd frontend +$ yarn +$ yarn build +``` + +Running the application in [development mode][8] (hit ctrl-C to stop): + +```console +$ yarn start +``` + +This will run the frontend and watch all source files for changes. You can visit the frontend on http://localhost:4200/. + + +### Commands for common tasks + +The `package.json` next to this README defines several shortcut commands to help streamline development. In total, there are over 30 commands. Most may be regarded as implementation details of other commands, although each command could be used directly. Below, we discuss the commands that are most likely to be useful to you. For full details, consult the `package.json`. + +Install the pinned versions of all package dependencies in all subprojects: + +```console +$ yarn +``` + +Run backend and frontend in [production mode][8]: + +```console +$ yarn start-p +``` + +Run the functional test suite: + +```console +$ yarn test-func [FUNCTIONAL TEST OPTIONS] +``` + +The functional test suite by default assumes that you have the application running locally in production mode (i.e., on port `4200`). See [Configuring the browsers][10] and [Configuring the base address][11] in `functional-tests/README` for options. + +[10]: functional-tests/README.md#configuring-the-browsers +[11]: functional-tests/README.md#configuring-the-base-address + +Run *all* tests (mostly useful for continuous integration): + +```console +$ yarn test [FUNCTIONAL TEST OPTIONS] +``` + +Run an arbitrary command from within the root of a subproject: + +```console +$ yarn back [ARBITRARY BACKEND COMMAND HERE] +$ yarn front [ARBITRARY FRONTEND COMMAND HERE] +$ yarn func [ARBITRARY FUNCTIONAL TESTS COMMAND HERE] +``` + +For example, + +```console +$ yarn back less README.md +``` + +is equivalent to + +```console +$ cd backend +$ less README.md +$ cd .. +``` + +Run `python manage.py` within the `backend` directory: + +```console +$ yarn django [SUBCOMMAND] [OPTIONS] +``` + +`yarn django` is a shorthand for `yarn back python manage.py`. This command is useful for managing database migrations, among other things. + +Manage the frontend package dependencies: + +```console +$ yarn fyarn (add|remove|upgrade|...) (PACKAGE ...) [OPTIONS] +``` + + + +### Notes on Python package dependencies + +Both the backend and the functional test suite are Python-based and package versions are pinned using [pip-tools][13] in both subprojects. For ease of development, you most likely want to use the same virtualenv for both and this is also what the `bootstrap.py` assumes. + +[13]: https://pypi.org/project/pip-tools/ + +This comes with a small catch: the subprojects each have their own separate `requirements.txt`. If you run `pip-sync` in one subproject, the dependencies of the other will be uninstalled. In order to avoid this, you run `pip install -r requirements.txt` instead. The `yarn` command does this correctly by default. + +Another thing to be aware of, is that `pip-compile` takes the old contents of your `requirements.txt` into account when building the new version based on your `requirements.in`. You can use the following trick to keep the requirements in both projects aligned so the versions of common packages don't conflict: + +```console +$ yarn back pip-compile +# append contents of backend/requirements.txt to functional-tests/requirements.txt +$ yarn func pip-compile +``` + + +### Development mode vs production mode + +The purpose of development mode is to facilitate live development, as the name implies. The purpose of production mode is to simulate deployment conditions as closely as possible, in order to check whether everything still works under such conditions. A complete overview of the differences is given below. + +dimension | Development mode | Production mode +-----------|--------------------|----------------- +command | `yarn start` | `yarn start-p` +base address | http://localhost:8000 | http://localhost:4200 +backend server (Django) | in charge of everything | serves backend only +frontend server (angular-cli) | serves | watch and build +static files | served directly by Django's staticfiles app | collected by Django, served by gulp-connect +backend `DEBUG` setting | `True` | `False` +backend `ALLOWED_HOSTS` | - | restricted to `localhost` +frontend sourcemaps | yes | no +frontend optimization | no | yes + + +## Deployment + +Both the backend and frontend applications have a section dedicated to deployment in their own READMEs. You should read these sections entirely before proceeding. All instructions in these sections still apply, though it is good to know that you can use the following shorthand commands from the integrated project root: + +```console + +# collect static files of both backend and frontend, with overridden settings +$ yarn django collectstatic --settings SETTINGS --pythonpath path/to/SETTINGS.py +``` + +You should build the frontend before collecting all static files. diff --git a/LICENSE b/LICENSE index 9533a45..86e23ad 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,7 @@ BSD 3-Clause License -Copyright (c) 2018, UU Digital Humanities Lab +Copyright (c) 2018-2022, UU Digital Humanities Lab +Copyright (c) 2023-2024, Research Software Lab (Centre for Digital Humanities, Utrecht University) All rights reserved. Redistribution and use in source and binary forms, with or without diff --git a/README.md b/README.md index 6d5ee84..97e662c 100644 --- a/README.md +++ b/README.md @@ -2,186 +2,23 @@ [![Actions Status](https://github.com/UUDigitalHumanitieslab/digital-atlas/workflows/Unit%20tests/badge.svg)](https://github.com/UUDigitalHumanitieslab/digital-atlas/actions) -The Digital ATLAS of postcolonial Europe visualizes the sites, archives, galleries, museums, monuments, organizations and events, collecting information on postcolonial intellectuals and the main migrant organizations/manifestations in Europe. +The [Digital ATLAS of postcolonial Europe](https://atlast.hum.uu.nl) visualizes the sites, archives, galleries, museums, monuments, organizations and events, collecting information on postcolonial intellectuals and the main migrant organizations/manifestations in Europe. -Presented at [Postcolonial Publics: Art and Citizen Media in Europe -Ca' Foscari University of Venice, VIU (May 26th, 2022)](https://www.unive.it/data/33113/1/60248). +Presented at [Postcolonial Publics: Art and Citizen Media in Europe, Ca' Foscari University of Venice, VIU (May 26th, 2022)](https://www.unive.it/data/33113/1/60248). +## Contents -## Before you start +The repository consist of three modules: +- `backend`: a [Django](https://www.djangoproject.com/) backend that is included so the application can be run in our normal deployment environment. It's not actively used. +- `data`: [python](https://python.org) scripts to convert excel data provided by researchers into JSON format. +- `frontend`: the web interface, implemented as an [Angular](https://angular.io/) application -You need to install the following software: +See [CONTRIBUTING](/CONTRIBUTING.md) for details on the setup of the application and common actions when working on the source code. - - Python >= 3.8, <= 3.10 - - virtualenv - - [Visual C++ for Python][1] (Windows only) - - Node.js >= 8 - - Yarn - - [WebDriver][2] for at least one browser (only for functional testing) - - WSGI-compatible webserver (deployment only) - - PostgreSQL >= 10, client, server and C libraries (not used yet) - +## Usage -[1]: https://wiki.python.org/moin/WindowsCompilers -[2]: https://pypi.org/project/selenium/#drivers +Digital ATLAS is a web application. You can view the deployed version as [atlast.hum.uu.nl](https://atlas.hum.uu.nl). +## Licence -## How it works - -This project integrates four isolated subprojects, each inside its own subdirectory with its own code, package dependencies and tests: - -- **backend**: the server side web application based on [Django][3] and [DRF][4]. Within this project this only serves the Angular frontend - -- **data**: reads the Excel sheet with all the collected data and exports it to a format which can be used by the frontend - - - **frontend**: the client side web application based on [Angular](https://angular.io) - - - **functional-tests**: the functional test suite based on [Selenium][6] and [pytest][7] - -[3]: https://www.djangoproject.com -[4]: https://www.django-rest-framework.org -[6]: https://www.selenium.dev/documentation/webdriver/ -[7]: https://docs.pytest.org/en/latest/ - -Each subproject is configurable from the outside. Integration is achieved using "magic configuration" which is contained inside the root directory together with this README. In this way, the subprojects can stay truly isolated from each other. - -If you are reading this README, you'll likely be working with the integrated project as a whole rather than with one of the subprojects in isolation. In this case, this README should be your primary source of information on how to develop or deploy the project. However, we recommend that you also read the "How it works" section in the README of each subproject. - - -## Development - -### Quickstart - -First time after cloning this project: - -```console -$ cd frontend -$ yarn -$ yarn build -``` - -Running the application in [development mode][8] (hit ctrl-C to stop): - -```console -$ yarn start -``` - -This will run the frontend and watch all source files for changes. You can visit the frontend on http://localhost:4200/. - - -### Commands for common tasks - -The `package.json` next to this README defines several shortcut commands to help streamline development. In total, there are over 30 commands. Most may be regarded as implementation details of other commands, although each command could be used directly. Below, we discuss the commands that are most likely to be useful to you. For full details, consult the `package.json`. - -Install the pinned versions of all package dependencies in all subprojects: - -```console -$ yarn -``` - -Run backend and frontend in [production mode][8]: - -```console -$ yarn start-p -``` - -Run the functional test suite: - -```console -$ yarn test-func [FUNCTIONAL TEST OPTIONS] -``` - -The functional test suite by default assumes that you have the application running locally in production mode (i.e., on port `4200`). See [Configuring the browsers][10] and [Configuring the base address][11] in `functional-tests/README` for options. - -[10]: functional-tests/README.md#configuring-the-browsers -[11]: functional-tests/README.md#configuring-the-base-address - -Run *all* tests (mostly useful for continuous integration): - -```console -$ yarn test [FUNCTIONAL TEST OPTIONS] -``` - -Run an arbitrary command from within the root of a subproject: - -```console -$ yarn back [ARBITRARY BACKEND COMMAND HERE] -$ yarn front [ARBITRARY FRONTEND COMMAND HERE] -$ yarn func [ARBITRARY FUNCTIONAL TESTS COMMAND HERE] -``` - -For example, - -```console -$ yarn back less README.md -``` - -is equivalent to - -```console -$ cd backend -$ less README.md -$ cd .. -``` - -Run `python manage.py` within the `backend` directory: - -```console -$ yarn django [SUBCOMMAND] [OPTIONS] -``` - -`yarn django` is a shorthand for `yarn back python manage.py`. This command is useful for managing database migrations, among other things. - -Manage the frontend package dependencies: - -```console -$ yarn fyarn (add|remove|upgrade|...) (PACKAGE ...) [OPTIONS] -``` - - - -### Notes on Python package dependencies - -Both the backend and the functional test suite are Python-based and package versions are pinned using [pip-tools][13] in both subprojects. For ease of development, you most likely want to use the same virtualenv for both and this is also what the `bootstrap.py` assumes. - -[13]: https://pypi.org/project/pip-tools/ - -This comes with a small catch: the subprojects each have their own separate `requirements.txt`. If you run `pip-sync` in one subproject, the dependencies of the other will be uninstalled. In order to avoid this, you run `pip install -r requirements.txt` instead. The `yarn` command does this correctly by default. - -Another thing to be aware of, is that `pip-compile` takes the old contents of your `requirements.txt` into account when building the new version based on your `requirements.in`. You can use the following trick to keep the requirements in both projects aligned so the versions of common packages don't conflict: - -```console -$ yarn back pip-compile -# append contents of backend/requirements.txt to functional-tests/requirements.txt -$ yarn func pip-compile -``` - - -### Development mode vs production mode - -The purpose of development mode is to facilitate live development, as the name implies. The purpose of production mode is to simulate deployment conditions as closely as possible, in order to check whether everything still works under such conditions. A complete overview of the differences is given below. - -dimension | Development mode | Production mode ------------|--------------------|----------------- -command | `yarn start` | `yarn start-p` -base address | http://localhost:8000 | http://localhost:4200 -backend server (Django) | in charge of everything | serves backend only -frontend server (angular-cli) | serves | watch and build -static files | served directly by Django's staticfiles app | collected by Django, served by gulp-connect -backend `DEBUG` setting | `True` | `False` -backend `ALLOWED_HOSTS` | - | restricted to `localhost` -frontend sourcemaps | yes | no -frontend optimization | no | yes - - -## Deployment - -Both the backend and frontend applications have a section dedicated to deployment in their own READMEs. You should read these sections entirely before proceeding. All instructions in these sections still apply, though it is good to know that you can use the following shorthand commands from the integrated project root: - -```console - -# collect static files of both backend and frontend, with overridden settings -$ yarn django collectstatic --settings SETTINGS --pythonpath path/to/SETTINGS.py -``` - -You should build the frontend before collecting all static files. +This repository is shared under a [BSD 3-Clause licence](/LICENSE).