Pop-up store demo using RedisTimeSeries, RedisGears and Redis plugins for Grafana

Introduction

The Pop-up store is using Redis Streams, RedisTimeSeries, RedisGears and Redis plugins to visualize data pipeline in Grafana.

• Node.js script adds random data to Customers and Orders streams
• RedisGears is using StreamReader to watch all queue: keys and adding Time-Series samples

# Add Time-Series
def tsAdd(x):
   xlen = execute('XLEN', x['key'])
   execute('TS.ADD', 'ts:len:'+x['key'], '*', xlen)
   execute('TS.ADD', 'ts:enqueue:' + x['key'], '*', x['value'])


# Stream Reader for any Queue
gb = GearsBuilder('StreamReader')
gb.countby(lambda x: x['key']).map(tsAdd)
gb.register(prefix='queue:*', duration=5000, batch=10000, trimStream=False)

• Another RedisGears script completes orders
  • adding data to queue:complete stream
  • deleting client's ordering
  • decreasing product amount
  • trimming Orders queue

# Complete order
def complete(x):
    execute('XADD', 'queue:complete', '*', 'order', x['id'],
            'customer', x['value']['customer'])
    execute('XDEL', 'queue:customers', x['value']['customer'])
    execute('DECR', 'product')


# Stream Reader for Orders queue
gb = GearsBuilder('StreamReader')
gb.map(complete)
gb.register(prefix='queue:orders', batch=3, trimStream=True)

• Grafana query streams and Time-Series keys every 5 seconds to display samples using Grafana plugins.

Demo

Demo is available on demo.volkovlabs.io:

• Redis Overview dashboard
• Pop-up Store dashboard

Requirements

• Docker to start Redis and Grafana.
• Node.js to run simulation script.

Start Redis with RedisTimeSeries, RedisGears modules installed and Grafana

npm run start

Register RedisGears functions

Install Readers to add Time-Series and complete orders

npm run register

Install ioredis module and start simulation

Script pop-up-store.js will add customers to stream queue:customers and their orders to the orders keys.

npm run simulation

Grafana Dashboards

Open Grafana Dashboard using browser http://localhost:3000

Redis-cli

To start redis-cli and look at the keys please run

npm run redis-cli