wq.app 1.2 alpha

wq.app 1.2 alpha is a preview of the next version of wq.app. The primary change is the integration of Redux and related libraries as part of a comprehensive overhaul of the data model layer (#105).

This release achieves the second goal in the 2019 roadmap for wq.app.

Overview

• @wq/store is now a wrapper around Redux and Redux Persist.
• @wq/router is now a wrapper around Redux-First Router.
• @wq/model is now a wrapper around Redux-ORM.
• @wq/outbox is now a wrapper around Redux Offline.
• @wq/outbox now supports batch submissions and optimistic updates.
• @wq/model now supports automatic normalization of server-nested model APIs.

New Features

Redux Integration (#105, #113)

• @wq/store now provides dispatch(), getState(), and subscribe() methods, which directly wrap the corresponding Redux store methods.
• The following @wq/app plugin hooks are now available:
  • reducer() and actions for plugin-specific Redux state (@wq/store)
  • thunks and render() for more complex Redux actions and rendering (@wq/router)
• When debug is active, @wq/store will log all Redux events and state changes (including page navigation and rendering events) to the console via Redux Logger.
• @wq/router routes are now named, with Redux action types that generally match the template name.
• @wq/model's [model].filter() is now a wrapper for Redux-ORM's filter(), which provides additional features such as predicate function filters and better indexing.
• Direct access to the Redux-ORM queryset is also available via [model].objects.all().
• @wq/outbox supports several new methods to control syncing: waitForItem(), waitForAll(), retryItem(), retryAll(), pause(), resume(), and empty().

Other Improvements

• The first loaded page is now always re-rendered by the client during application startup, as well as after authentication changes (#79)
• New applyState configuration option to control when updates are applied to the local model (#85, #86). A different set of Redux actions will be applied for form submission/success/error depending on this setting. The available options are:
  • "ON_SUCCESS" (default): form submissions will not be reflected in the local model state until after the form is successfully synced to the server.
  • "IMMEDIATE": form submissions are optimistically reflected in the local model state before they are sent to the server.
  • "LOCAL_ONLY": form submissions are not synced to the server at all. In this case, nothing is stored in the outbox, so the only reason to use outbox.save() is to maintain API consistency with other forms.
• Support syncing multiple items in a single fetch() (#110), by leveraging @tomaszn's fork of Django Batch Requests.
• Support registering a top-level model and also having it as an attachment (nested record) for another model (#114). The records will be normalized for local storage and denormalized for the API.
• Add removeattachment action to @wq/app:patterns plugin (d5372ff)
• Generate sourcemaps for wq/* AMD modules (43222ff)

Bug Fixes

• The integration of Redux Offline addresses a number of issues with offline state and syncing, particulary #33, #94/#95, and #102.
• @wq/map:
  • Copy layer config in case it is modified by function (440dc9b)
  • Don't update sticky bounds when map is off-screen (5952fe6)

Breaking Changes

This release necessarily changes a lot of the underlying data model, while keeping the UI layer mostly the same. If you are only using the high-level API, only the changes to @wq/app should affect you. If you are using app.models or the lower level modules directly, you may want to review the other changes below.

@wq/app

• app.go() has been removed. You can generally use app.nav() instead, with a context() plugin for cases where you need a custom context.
• app.sync(true) has been renamed to app.retryAll()
• app.sync() and config.backgroundSync are no longer available. The timing of syncing (other than manual retries) is now handled by @wq/outbox and Redux Offline.
• Login/logout events are now dispatched as Redux actions rather than as jQuery events. Pages are re-rendered after auth changes, so any updates to the UI should happen in the template and context rather than in a jQuery event handler.
• The first page is always re-rendered by the client during application startup.

@wq/store

• The jsonp and parseData configuration options no longer exist. To customize how data is retrieved and parsed, use an ajax() plugin hook instead.
• The promise returned by ds.set() no longer waits until the data has been fully persisted to offline storage before resolving.
• ds.storageUsage() has been removed.
• ds.reset() no longer has the capability of clearing out persisted storage for stores other than the current one.

@wq/router

• Route info is now provided only through the template context (and not as router.info).
• With the exeption of path, the arguments to router.register() have completely changed, though they serve a similar function. With wq.app 1.1 and earlier, a callback function was required, and needed to explicitly call router.go() with the generated context. Starting in 1.2, the callback function is optional, and only needs to return a new context object.
• router.go() is now called automatically whenever the Redux state changes. So, it is not necessary (or possible) to call it directly.
• The arguments to router.addRoute() have also changed somewhat, and the function is deprecated in favor of run() plugin hooks anyway.

@wq/model

• A name is now required for all model instances. (@wq/app provides this automatically)
• find(), update(), fetchUpdate() and remove() no longer accept a custom id column as the second argument. If the primary key is not "id", specify idCol when defining the model.
• filter() is now based on Redux-ORM's filter(), which uses strict equality when comparing object attributes. In wq.app 1.1 and earlier, {'type_id': '3'} and {'type_id': 3} returned the same result, whereas in wq.app 1.2 they are different.

@wq/outbox

• The Promise returned by outbox.save() is now resolved before the record is synced. If you need to wait for the sync result, call outbox.waitForItem() after calling outbox.save(). Relatedly, outbox.save() no longer accepts a third noSend argument. If you would like to save an item to the outbox without triggering an immediate sync attempt, call outbox.pause() before outbox.save().
• outbox.sendItem() and outbox.sendAll() have been removed. Similar functionality is provided by outbox.waitForItem(), outbox.waitForAll(), and outbox.retryAll().
• @wq/outbox no longer uses an internal @wq/model instance for managing unsynced records. Instead, the underlying Redux Offline outbox state is wrapped with a model-like API. If you have code relying on outbox.model.load(), change it to use outbox.loadItems() instead which provides an equivalent structure.
• batchService should be specified as a URL path relative to the service root (rather than relative to the domain)
• parseBatchResult(), applyResult() and updateModels() are no longer available as configuration options. The first two can be replaced with a custom ajax() plugin hook, while updateModels() is now implemented via Redux actions.

wq.app 1.1.2

wq.app 1.1.2 brings a significant overhaul to the internal source structure, while (mostly) retaining full backwards compatibility. Specifically, wq.app is now organized as a monorepo containing several npm packages (see #1, #84, #66, and #109). The new packages are written as ES6 modules, but compiled back into AMD modules for compatibility with the existing wq.app PyPI package.

This release achieves the first goal in the 2019 roadmap for wq.app.

Using fetch()

While wq.app 1.1.2 is intended to be fully backwards compatible with 1.1.1, there is one important exception: the use of fetch() to handle AJAX requests. If you need to support older browsers without a built-in fetch() (such as IE 11), you will need to include a polyfill before loading wq.app:

<script src="https://cdnjs.cloudflare.com/ajax/libs/fetch/3.0.0/fetch.min.js"></script>
<script src="/js/lib/require.js" data-main="/js/myapp"></script>

In addition, if you are using a custom ajax() plugin hook (added in 1.1.1), note that the arguments have changed slightly:

• For POST requests, the url is a URL object, and data is a FormData object.
• For GET requests, the url a string, and data is an object containing URL parameters.
• If the request fails with a server error, the plugin should throw an Error with a json attribute if the error is an object or a text attribute otherwise. (See the default implementation for an example). Unlike $.ajax(), fetch() does not automatically throw in the case of 400 and 500 errors.

Deprecated Modules

wq/autocomplete.js, wq/console.js, wq/json.js, and wq/progress.js are deprecated and have been removed from the source repository. For backwards compatibility, the Python package includes copies of these modules from 1.1.1.

Other Changes

• Ensure base URL is used when falling back to server rendering (#112 via @tubaman)
• Various outbox improvements (7f548bc)

wq.app 1.1.1

wq.app 1.1.1 brings a couple of bug fixes, a new ajax() plugin hook, and an improved wq/chartapp.js API.

Note: there are a number of exciting changes in the works, and wq.app 1.1.1 will be the last release before the big upgrades begin. See #111 for more information.

Bug Fixes

• Prevent users accidentally double-clicking the submit button (#100)
• wq phonegap command compatibility with 8-bit icons (#108 via @tomaszn)

New Features

ajax() plugin hook

You can now completely override the behavior of GET and POST requests made by wq/model.js and wq/outbox.js. To do this, define a wq.app plugin with an ajax() method that takes four arguments: url, data, method, and headers.

define({
    "ajax": function(url, data, method, headers) {
        if (method == "POST") {
            return somePostMethod(url, data, headers)
        } else {
            return someGetMethod(url, headers);
        }
    }
});

Here are a few things to keep in mind:

• The data object for post requests may be either a plain object or a FormData instance.
• You are free to rewrite or completely ignore the passed URL, for example to integrate with an arbitrary (non wq.db) REST API.
• The ajax() method should return a Promise that will resolve to the JSON object returned by the REST service.
• If the REST service is not compatible with wq.db, be sure to process the response into a compatible format.

wq/chartapp.js

A number of changes wq/chart.js and the wq/chartapp.js plugin make it possible to configure the main chart options via custom data-wq-* attributes. For example:

<svg data-wq-url="/api/data/timeseries.csv"
     data-wq-type="timeSeries"
     data-wq-point-cutoff="100"
     data-wq-time-format="%Y-%m-%d"
     data-wq-width="800"
     data-wq-height="600"
     data-wq-x="date"
     data-wq-y="value"
     data-wq-label-template="{{series_label}}"
     data-wq-point-label-template="Value on {{date}}" >
</svg>

wq.app 1.1.0

wq.app 1.1.0 brings better memory usage for the outbox, a new PyMiniRacer-based build system, and various other improvements.

• Better memory usage in outbox (#93). Specifically, File/Blob data is only loaded into memory when needed, rather than every time the outbox contents are listed. This is especially important when running in memory-constrained environments (like PhoneGap applications on iOS). Outbox items can be saved with an option specifying one of three storage modes:

mode	description	notes
(Default)	Store form data in outbox localForage key	Same as old usage
storage="store"	Separate form-specific localForage key	Default for form submissions containing files/blobs.
storage="temporary"	Don't persist form data at all	Default for login forms and all forms with data-background-sync=false; fixes #90

• Generate parent_url for items in outbox (#97 via @tomaszn)
• Hook to allow customization of list views with outbox items (#101 via @tomaszn)
• Leverage PyMiniRacer to run r.js (#92, #14, #46). The requirejs dependency is now available as a separate PyPI package. Thanks to @tomaszn for testing this update (#104).
• Preliminary ES6 support. Files containing ES6 code will be skipped by the optimizer and converted to ES5 via PyBabelJS. Support for actually optimizing ES6 code will come in a future wq.app release (see #83).

wq.app 1.0.0

wq.app 1.0.0 is finally here! This is the first stable release of wq.app 1.0, which is now ready for production use.

Changes since wq.app 1.0.0 RC2

• wq/model.js

  • Handle boolean filter fields (#82)

• wq/app.js

  • Include last context in postdelete URL render (#80)
  • Test and fix various EAV filter use cases (wq/wq.db#66)

• wq/map.js / wq/locate.js

  • Remove onshow in favor of better wq/app.js plugin support (d5f47f2)
  • Detect and utilize cordova-plugin-bluetooth-geolocation in wq/locate.js (1ad8d0d)
  • Other fixes (bf00503, 31ed208)

• Build tool

  • Add separate splash option to wq phonegap (b5d5788)
  • Incorporate default CSS from wq.start template (061afec)

• Incorporate Code of Conduct and Contributing Guidelines

Other changes since wq.app 0.8.2

• Changes in Alpha 1
  • Refactor patterns API (#38, wq/wq.db#35, #56)
  • Simplify plugin arguments (#58)
  • Test framework (#13)
• Changes in Alpha 2
  • Minor fix
• Changes in Beta 1
  • More wq/app.js plugin hooks and support for custom rendering modes (#30, #59)
  • Add wq/patterns.js and wq/mapserv.js
  • Various updates
• Changes in Beta 2
  • Flatten wq/map.js configuration
  • Improve handling of images, labels and foreign keys in wq/outbox.js
  • Update to Leaflet 1.0 and d3.js 4.2
  • Add wq/chartapp.js
  • Drop wq/appcache.js, wq/online.js, and wq/owl.js
  • Various plugin updates
• Changes in RC 1
  • New Pagination and Caching API (#47)
  • New wq icons and wq phonegap commands (wq/wq-django-template#4)
  • Improve support for deletion & offline editing (#70, #72, #77, #78, #80)
  • Various bug fixes & improvements
• Changes in RC 2
  • Minor bug fixes & improvements

wq.app 1.0.0 RC2

wq.app RC2 brings a number of minor bug fixes and improvements to RC1.

• Generate scss folder for swatch tests (#87 by @tomaszn)
• Update vendored Leaflet & leaflet plugins
• Added wq-status option to <progress> plugin (7eca458, see docs)
• Handle case when outbox syncs during a list view data fetch (45162e0)
• Various other fixes (see commit log)

wq.app 1.0.0 RC1

wq.app 1.0.0 RC1 brings enhanced support for PhoneGap/Cordova, an improved offline cache configuration API, and a number of other enhancements.

PhoneGap/Cordova support

• New wq icons command to automatically generate icons and splash screens for Android, iOS, and Windows (wq/wq-django-template#1, b3e1824, 7677e58, 0d8e196, 6438304, a3f84ab)
• New wq phonegap command to automatically generate values for config.xml, zip it up together with the built JavaScript application and generated icons, and upload everything to PhoneGap Build (wq/wq-django-template#4, 1d8698d, ).
• Various JavaScript improvements to improve PhoneGap compatibility (08f4764, a576254, c0f8e1d)

The updated wq django template automatically takes advantage of these new features.

New Configuration for Pagination/Offline Caching

The configuration options affecting pagination and offline caching (per_page, partial, max_local_pages, filter, and reversed) have been unified and replaced with a new cache setting. See Pagination and Caching for details (#47, 452b0d2).

Other Improvements & Fixes

Thanks to @tomaszn for reporting several of these issues.

Editing & Outbox (wq/app.js)

• Properly support deletion (#80, 4bf65be, 9cb2298)
• Ensure plugins run when loading items from outbox (#77, 68efcba)
• Make custom postsave URLs work for items still in outbox (#78, 95fb93a)
• Improve editing for nested forms (#72, f2ddbe8)
• Improve file support (#70, ea5835c, 8b7f577)
• Change default behavior from loadMissingAsHtml to loadMissingAsJson (02526bf)

Charting

• wq/pandas.js improve handling of blank values and field lookups (#73 via @ast0815, f484f5d)
• wq/chartapp.js fix field lookup (3d25c6f)

Maps

• wq/map.js more robust handling of wq/app.js pages (d5afe08, 68efcba)
• wq/mapserv.js add esri-feature layer type (557a668)
• wq/locate.js populate GeoJSON geometry field if present (da13a01)

Other

• wq/markdown.js syntax highlighting for server-rendered markdown (014b75d)

wq.app 1.0 beta 2

wq.app 1.0 beta 2 brings a lot of cleanup and a few new features. Note that this release is no longer compatible with Internet Explorer 8.

API Improvements

• Flatten the per-page configuration for wq/map.js to make it more managable. Instead of:

config.pages[page].map[mode].maps.main.layers = [...];

you can just do

config.pages[page].map = {
    'mode': mode,
    'map': 'main', // optional
    'layers': [...]
};

Also added the option to pre-register an oneach function (not unlike the pre-registered layer types) so it can be referenced from a JSON configuration object.

• Make it so wq/outbox.js items are accessible via foreign key references even before they are synced. The sync process now automatically determines the proper order to send records to the server and updates the references on the fly. To assign labels to items in the outbox (since the Python __str__ function is not available), a new label_template property is now supported on the configuration object. See the release notes for wq.db 1.0.0b3 for more information.
• Use a recursive in-place serializer for files saved in localForage (see localForage/localForage#603). Note that the new store uses a different naming convention and the contents of the old offline cache will not be automatically transferred to the new one.
• Support lookups for foreign keys within natural keys, and plugins on server-rendered non-list pages.

Third Party Libraries

• Update Leaflet from 0.7 to 1.0
• Update d3.js from 3.5 to 4.2
• Drop es5-shim and support for IE8
• Drop jquery.validate, proj4, proj4leaflet, and rbush (all rarely used in production wq applications)
• Add leaflet.wms and localforage-memoryStorageDriver
• Update most other libraries (except jQuery Mobile which will be updated in a future release)

For the full set of changes, compare the list in wq.app 1.0.0b1 vs. wq.app 1.0.0b2.

Other modules

• Update wq/markdown.js and wq/progress.js to work as wq/app.js plugins
• Add wq/chartapp.js, a wq/app.js plugin combining wq/chart.js and wq/pandas.js
• Drop wq/appcache.js, wq/online.js, and wq/owl.js

wq.app 1.0 beta

wq.app 1.0 beta brings a number of new features and bug fixes to wq.app 1.0 alpha 2.

wq/app.js

Plugin API

• Added two additional asynchronous plugin hooks. Returning a Promise from either hook will cause further processing to wait until the Promise is resolved.
  • [plugin].context(currentContext, routeInfo) is called just before rendering a page template. This makes it possible to perform arbitrary asynchronous data lookups before navigating to a new page (#30). Think Django's context processors, but on the client and asynchronous.
  • [plugin].onsave(outboxItem, serverResponse) is called just after a successful outbox submission (344e776). This makes it possible to perform additional work (e.g. custom model updates) before continuing.
• Made all plugin attributes optional, so you can register anonymous plugins - though they won't be configurable (#59).

Router / Template Modes

• Added support for arbitrary route modes beyond the default list, detail, and edit (de578f4). Custom modes have URLs of the form /items/123/custommode and will be rendered with templates named e.g. item_custommode.html. Modes should be specified as part of the page configuration, e.g.:

  {
      "name": "item",
      "url": "items",
      "list": "true",
  
     // Client+server modes
      "modes": ["list", "detail", "edit", "custommode"],
  
      // Server-only modes
      "server_modes": ["servercustommode"]
  }

  Both client+server and server-only modes can be specified. Client+server modes will be rendered on the client once wq/app.js initializes. Server-only modes are always rendered on the server, but can still take advantage of the plugin infrastructure ([plugin].run()). Since wq/app.js automatically loads URLs it doesn't recognize from the server, it is not necessary to register server-only modes if they don't require plugins.

• New wq/patterns.js plugin to support the common case of attaching multiple nested records to a single parent form (ee0b6a2).

• [parent]_label context variable for foreign keys (2a40fc7).

Other Improvements

• Added app.nav(url, changePageOptions) and app.refresh() to facilitate common page navigation tasks (ee0b6a2).
• Option to render postsave URLs as templates (a2b31d4)

wq/map.js

• Support multiple maps on the same page (34664cd).
• Switch default basemaps from MapQuest to Stamen, one of the few remaining free providers that doesn't require an API key (#68).
• Simplify integration of Esri basemaps via wq/mapserv.js (a2e4249).

wq/model.js

• Allow specifying the store backing the model as a string name (16a7b42). This makes it possible to configure custom model stores via JSON, and initialize them in JavaScript elsewhere.
• Deep-copy items retrieved from indexes to ensure modification doesn't have unexpected side effects (c4b7c34).

Other Fixes

• Ensure bar-shadow: false actually disables text shadows in SCSS themes (5bee3ba).
• Code style improvements

wq.app 1.0 alpha 2

Quick update to the wq.app 1.0 alpha release for better compatibility with wq.start 1.0.0a1.