Merge pull request #10 from lukebuehler/l2

Milestone 3 - Layer 2 Support

README.md

@@ -1,7 +1,10 @@
 # azimuth-cli
-The azimuth-cli is "Bridge for the command line." It should allow ship owners and operators to do the same things that they can in [Bridge](https://bridge.urbit.org/) but via the command line, and in batch mode.
 
-The primary usage of the cli is to query and modify the [azimuth contracts](https://github.com/urbit/azimuth) on Ethereum.
+This is a command line tool to work with Urbit ID, which is the idenity layer behind [Urbit](https://urbit.org/) and is managed as a set of NFTs on the Ethereum blockchain.
+
+The azimuth-cli is "Bridge for the command line." It should allow Urbit ship owners and operators to do the same things that they can in [Bridge](https://bridge.urbit.org/) but via the command line, and in batch mode.
+
+The primary functionality of the cli is to query and modify the [azimuth contracts](https://github.com/urbit/azimuth) on Ethereum.
 
 ## Install
 
@@ -12,6 +15,8 @@ Due to an issues when building the azimuth-js package (as described [here](https
 Simply install the npm package globally:  
 `npm install -g azimuth-cli`
 
+**When Upgrading to the L2 version:** Make sure to delete the `cli-config.json` file in your home directory, usually in a folder called `.azimuth/`.
+
 ### Development or Manual Install
 1) Clone this [repo](https://github.com/lukebuehler/azimuth-cli)
 1) run `npm install`
@@ -24,11 +29,19 @@ To run the tests:
 
 
 ## Usage
+
+*Note: Using the CLI with a L2 roller is WIP, see [here for instructions](docs/roller.md).*
+
 After installing the npm package, just type `azimuth-cli` in the command line to see the options. You can also use `azi` for short.
 
-There are three main types of commands: `list`, `generate`, and `modify`. First, the `list` commands do not make any changes, they just print information to the command line. But `generate` commands usually save something to the current work directory; use this, for example, to generate HD wallets. The `modify` commands actually change the blockchain and usually require the private key of the address that owns the urbit point you are making a change to. The `modify` commands also save data to the work directory, such as Ethereum transaction receipts.
+There are three main types of commands: `get`, `generate`, and `modify`. First, the `get` commands do not make any changes, they just print information to the command line. But `generate` commands usually save something to the current work directory; use this, for example, to generate HD wallets. 
+
+The `modify` commands actually change the blockchain and usually require the private key of the address that owns the urbit point you are making a change to. The `modify` commands have two versions: `modify-l1` and `modify-l2`. This is because Urbit IDs can either be modified directly through an Ethereum smart contract called Azimuth, aka "L1", or via a Layer 2 solution that is cheaper, which uses a "roller" that gathers transactions that modify Urbit IDs and then submits them as one batch to Ethereum, and is called the "L2" solution. The `modify` commands also save data to the work directory, such as Ethereum or roller transaction receipts.
 
-More complicated operations--such as spawning ships, keying them, and transfering them to master tickets--cannot be executed in a single command. Multiple commands need to be called in order (see the example below).
+More complicated operations--such as spawning ships, keying them, and transfering them to master tickets--cannot be executed in a single command. Multiple commands need to be called in order (see the examples below).
+
+### Setting Up Your Own Layer 2 Roller
+If you want to modify large batches of points in one go--for example spawn 200 planets--then you need to [run your own roller](docs/roller.md). By default, the CLI points to the official roller run by Urbit. See the `--roller-provider` option in the CLI.
 
 ### Work Directory & Idempotent Commands
 Many commands, especially the `generate` and `modify` ones, need a work directory to fulfill their function. The reason for this is that a command may be called multiple times, but the end effect needs to always be the same. For example, if you call `azi generate spawn-list --count=10` multiple times, the resulting `spawn-list.txt` file will only be created once, and will not change in subsequent calls (unless the `--force` option is provided). The same goes for the `modify` commands.
@@ -42,35 +55,43 @@ The work directory is the current folder or can be supplied with the `--work-dir
 For the full documentation, please install the cli and explore the commands and sub-commands with the `--help` option.
 
 `aimuth-cli`
- * `list` - Prints azimuth data to the console.
+ * `get` - Retrieves data about points, rollers, and azimuth, and prints it to the console. By default, uses a L2 roller to get the information.
    * `children <point>` - Lists all child points of a certain Urbit point.
    * `owner <address>` - Lists all points owned by that Ethereum address.
    * `details <point>` - Prints details about the point.
-   * `gas-price` - Outputs the current Etherum gas prices. This is helpful if you want to provide a gas limit in the `modify` commands.
+   * `gas-price` - Outputs the current Etherum gas prices. This is helpful if you want to provide a gas limit in the `modify-l1` commands.
+   * `roller-info` - Prints details about the L2 roller.
+   * `roller-info` - Prints pending roller transactions.
  * `generate` - Generates various files that can be used in the `modify` commands.
    * `spawn-list <point>` - Creates a `spawn-list.txt` file that contains a number of points that can be spawned under the provided point.
    * `wallet` - Generates an HD wallet for each provided point and saves each wallet in JSON format in the current work directory. Use this especially if you plan to give the points away. Then, in subsequent commands, supply the `--use-wallet-files` option.
    * `network-keys` - Creates the network keyfile for each supplied point, and either creates a JSON file with the private and public network keys or uses the network keys from the walled files. 
    * `report` - Generates a CSV report for the provided points, containing patp, p, ticket, network keys, addresses, and transactions executed so far.
- * `modify` - Modifies the state of one or more points on the Ethereum blockchain (the azimuth contracts). For many of these commands to work, other files will have to have been generated with the `generate` commands.
+ * `modify-l1` - Modifies the state of one or more points on the Ethereum blockchain (the azimuth contracts). For many of these commands to work, other files will have to have been generated with the `generate` commands.
    * `spawn` - Spawns multiple points to the supplied address
    * `management-proxy` - Sets the management proxy address for the points.
    * `spawn-proxy` - Sets the spawn-proxy address for the points.
    * `network-key` - Sets the network keys for the points, which is required to be able to boot the Urbit.
    * `transfer` - Transfers the point to a target address or the wallet files. 
-
+ * `modify-l2` - Modifies the state of one or more points via a L2 roller. The roller then submits the changes to the L2 Ethereum contract. Any point modified via this command, needs to be on L2.
+   * `spawn` - Spawns multiple points to the supplied address. The galaxy or star that spawns needs to be on L2 or the spawn proxy needs to be on L2.
+   * `management-proxy` - Sets the management proxy address for the points.
+   * `network-key` - Sets the network keys for the points, which is required to be able to boot the Urbit.
+   * `transfer` - Transfers the point to a target address or the wallet files. 
 
 ### Examples
-#### Spawn, Set Network Keys, and Transfer to Master Ticket
+#### Spawn, Set Network Keys, and Transfer to Master Ticket on Azimuth (L1)
 This is an example of spawning planets and creating a master ticket for them. You would do this if you want to give some planets away to friends. It is similar to what you can do in Bridge, but we do it for 5 planets in one go. Generating a master ticket itself is not enough, though. Ownership needs to be transferred to the owner address that is associated with the master ticket. But for the master ticket to be usable, networking keys need to be set. Hence, we first spawn to a temporary address (usually the same as the owner or spawn-proxy of the star, here ~sardys), then set the keys, and only then move the planet to its own address--that of the HD wallet.
 
+The star you are spawning from needs to be on L1!
+
 ```
 # create a directory for your work
 mkdir spawn-planets
 cd spawn-planets
 
 # pick 5 random, unspawned planets under ~sardys and save them in a file
-azi generate spawn-list ~sardys --count=5 --pick=random
+azi generate spawn-list ~sardys --count=5 --pick=random --use-azimuth
 
 # now, generate HD wallet files based on the previous list
 azi generate wallet --points-file=spawn-list.txt
@@ -82,15 +103,35 @@ azi generate network-key --use-wallet-files
 
 # spawn the 5 planets that can be found in the wallet files, 
 # providing the PK of the wallet that owns ~sardys, or is the spawn proxy
-azi modify spawn --use-wallet-files --address=0xSardysOwnershipAddress --private-key=0x1234567890
+azi modify-l1 spawn --use-wallet-files --address=0xSardysOwnershipAddress --private-key=0x1234567890
 
 # set the network keys on the blockchain
-azi modify network-key --use-wallet-files --private-key=0x1234567890
+azi modify-l1 network-key --use-wallet-files --private-key=0x1234567890
 
 # transfer each planet ownership to the address of the wallet
-azi modify transfer --use-wallet-files --private-key=0x1234567890
+azi modify-l1 transfer --use-wallet-files --private-key=0x1234567890
 ```
 
+#### Spawn, Set Network Keys, and Transfer to Master Ticket on Azimuth (L2)
 
+This is the same example as above but using a roller to spawn the planets.
+
+The star you are spawning from needs to be on L2 or have the spawn proxy set to L2! Here is more info about the [Layer 2 solution](https://urbit.org/docs/azimuth/l2/layer2).
+
+```
+azi generate spawn-list ~sampel --count=2 --pick=random --use-roller
+
+azi generate wallet --points-file=spawn-list.txt
+
+azi generate network-key --use-wallet-files
+
+azi modify-l2 spawn --use-wallet-files --address=0xSpawnProxy --private-key=0xSpawnProxyKey
+
+azi modify-l2 management-proxy --use-wallet-files --address=0xManagementProxy --private-key=0xSpawnProxyKey
+
+azi modify-l2 network-key --use-wallet-files --private-key=0xSpawnProxyKey
+
+azi modify-l2 transfer --use-wallet-files --private-key=0xSpawnProxyKey 
+```
 
 

cli-config.json

@@ -1,5 +1,7 @@
 {
-  "ethProviderGanache": "'http://localhost:8545'",
+  "ethProviderGanache": "http://localhost:8545",
   "ethProviderRopsten": "https://ropsten.infura.io/v3/f23d3259814348dbb693726415858db8",
-  "ethProviderMainnet": "https://mainnet.infura.io/v3/f23d3259814348dbb693726415858db8"
+  "ethProviderMainnet": "https://mainnet.infura.io/v3/f23d3259814348dbb693726415858db8",
+  "rollerLocal": "http://localhost:8080/v1/roller",
+  "rollerUrbit": "https://roller.urbit.org/v1/roller"
 }

cli.js

@@ -26,6 +26,12 @@ function getUniversalOptions()
       choices: ['ganache', 'ropsten', 'mainnet'],
       type: 'string'
     },
+    'roller-provider':{
+      describe: 'What L2 roller provider to use.',
+      default: 'urbit',
+      choices: ['local', 'urbit'],
+      type: 'string'
+    },
     'config-file':{
       describe: 'What config file to use.',
       default: files.ensureDefaultConfigFilePath(),

cmds/generate_cmds/report.js

@@ -2,7 +2,7 @@ const ob = require('urbit-ob')
 const kg = require('urbit-key-generation');
 const ticket = require('up8-ticket');
 const _ = require('lodash')
-const {files, validate, eth, findPoints} = require('../../utils')
+const {files, validate, eth, findPoints, rollerApi} = require('../../utils')
 const ajs = require('azimuth-js')
 
 exports.command = 'report'
@@ -36,27 +36,47 @@ exports.builder = (yargs) =>{
     if (!argv.pointsFile && !argv.points && !argv.useWalletFiles) throw new Error('You must provide either --points-file, --points, or --use-wallet-files')
     return true
   });
+
+  yargs.option('use-roller',{
+    describe: 'Enforce using the roller (L2) for all data and do not allow fallback to azimuth (L1).',
+    type: 'boolean',
+    conflicts: 'use-azimuth'
+  });
+  yargs.option('use-azimuth',{
+    describe: 'Enforce using azimuth (L1) for all data and do not allow fallback to the roller (L2).',
+    type: 'boolean',
+    conflicts: 'use-roller'
+  });
 }
 
 exports.handler = async function (argv) 
 {
-  const workDir = files.ensureWorkDir(argv.workDir);
-  const ctx = await eth.createContext(argv);
+  const source = await rollerApi.selectDataSource(argv);
+  let ctx = null;
+  let rollerClient = null;
+  if(source == 'azimuth'){
+    ctx = await eth.createContext(argv);
+  }
+  else{
+    rollerClient = rollerApi.createClient(argv);
+  }
 
+  const workDir = files.ensureWorkDir(argv.workDir);
   const wallets = argv.useWalletFiles ? findPoints.getWallets(workDir) : null;
   const points = findPoints.getPoints(argv, workDir, wallets);
 
   const csvHeader = 
-    'patp,p,ship_type,master_ticket,network_keyfile,' +
-    'owner_address,proxy_address,' + //management_address is omitted because there is currently no API to retrieve the management proxy address
+    'patp,p,ship_type,parent_patp,master_ticket,network_keyfile,' +
+    'dominion,owner_address,spawn_proxy_address,management_proxy_address,' +
     'spawn_transaction,management_proxy_transaction,spawn_proxy_transaction,network_key_transaction,transfer_transaction';
   let csvLines = [csvHeader];
 
   console.log(`Will process ${points.length} points for the report.`);
   for (const p of points) {
     const patp = ob.patp(p);
+    const patpParent = ob.sein(patp);
     const shipType = ob.clan(patp);
-    let csvLine = `${patp},${p},${shipType},`;
+    let csvLine = `${patp},${p},${shipType},${patpParent},`;
 
     //see if we have a wallet to get the master from
     let masterTicket = '';
@@ -73,19 +93,41 @@ exports.handler = async function (argv)
     }
     csvLine += `${masterTicket},${networkKeyfileContents},`;
 
-    //get the addresses
-    const ownerAddress = await ajs.azimuth.getOwner(ctx.contracts, p);
-    const spawnProxyAddress = await ajs.azimuth.getSpawnProxy(ctx.contracts, p);
-    csvLine += `${ownerAddress},${spawnProxyAddress},`;
-
-    //try to get the transaction reciepts
-    let spawnTransaction = tryGetTransactionHash(patp, workDir, 'spawn');
-    let managementProxyTransaction = tryGetTransactionHash(patp, workDir, 'managementproxy');
-    let spawnProxyTransaction = tryGetTransactionHash(patp, workDir, 'spawnproxy');
-    let networkkeyTransaction = tryGetTransactionHash(patp, workDir, 'networkkey');
-    let transferTransaction = tryGetTransactionHash(patp, workDir, 'transfer');
-    csvLine += `${spawnTransaction},${managementProxyTransaction},${spawnProxyTransaction},${networkkeyTransaction},${transferTransaction}`;
+    //use azimuth
+    if(source == 'azimuth'){
+      //get the addresses
+      const dominion = 'L1'; //todo: try to get the dominion via ajs
+      const ownerAddress = await ajs.azimuth.getOwner(ctx.contracts, p);
+      const spawnProxyAddress = await ajs.azimuth.getSpawnProxy(ctx.contracts, p);
+      const managementProxyAddress = ''; //does not work yet: await ajs.azimuth.getManagementProxy(ctx.contracts, p);
+      csvLine += `${dominion},${ownerAddress},${spawnProxyAddress},${managementProxyAddress},`;
+
+      //try to get the transaction receipts
+      let spawnTransaction = tryGetTransactionHash(patp, workDir, 'spawn');
+      let managementProxyTransaction = tryGetTransactionHash(patp, workDir, 'managementproxy');
+      let spawnProxyTransaction = tryGetTransactionHash(patp, workDir, 'spawnproxy');
+      let networkkeyTransaction = tryGetTransactionHash(patp, workDir, 'networkkey');
+      let transferTransaction = tryGetTransactionHash(patp, workDir, 'transfer');
+      csvLine += `${spawnTransaction},${managementProxyTransaction},${spawnProxyTransaction},${networkkeyTransaction},${transferTransaction}`;
+    }
+    //L2 roller
+    else{
+      //get the addresses
+      const pointInfo = await rollerApi.getPoint(rollerClient, p);
+      const dominion = pointInfo.dominion;
+      const ownerAddress = pointInfo.ownership.owner.address;
+      const spawnProxyAddress = pointInfo.ownership.spawnProxy.address;
+      const managementProxyAddress = pointInfo.ownership.managementProxy.address;
+      csvLine += `${dominion},${ownerAddress},${spawnProxyAddress},${managementProxyAddress},`;
 
+      //try to get the transaction receipts
+      let spawnTransaction = tryGetTransactionHashL2(patp, workDir, 'spawn');
+      let managementProxyTransaction = tryGetTransactionHashL2(patp, workDir, 'setManagementProxy');
+      let spawnProxyTransaction = tryGetTransactionHashL2(patp, workDir, 'setSpawnProxy');
+      let networkkeyTransaction = tryGetTransactionHashL2(patp, workDir, 'configureKeys');
+      let transferTransaction = tryGetTransactionHashL2(patp, workDir, 'transferPoint');
+      csvLine += `${spawnTransaction},${managementProxyTransaction},${spawnProxyTransaction},${networkkeyTransaction},${transferTransaction}`;
+    }
     csvLines.push(csvLine);
   }
 
@@ -95,11 +137,19 @@ exports.handler = async function (argv)
 }
 
 function tryGetTransactionHash(patp, workDir, operationPostfix){
-  const transactionFile = `${patp.substring(1)}-reciept-${operationPostfix}.json`;
+  const transactionFile = `${patp.substring(1)}-receipt-${operationPostfix}.json`;
     if(files.fileExists(workDir, transactionFile)){
       return files.readJsonObject(workDir, transactionFile).transactionHash;
     }
     return '';
 }
 
+function tryGetTransactionHashL2(patp, workDir, method){
+  const transactionFile = `${patp.substring(1)}-receipt-L2-${method}.json`;
+    if(files.fileExists(workDir, transactionFile)){
+      return files.readJsonObject(workDir, transactionFile).hash;
+    }
+    return '';
+}
+
 const DEFAULT_REVISION = 1;

cmds/generate_cmds/spawn-list.js

@@ -1,8 +1,7 @@
 const ob = require('urbit-ob')
 const ajs = require('azimuth-js')
 const _ = require('lodash')
-
-const {files, validate, eth} = require('../../utils')
+const {files, validate, eth, rollerApi} = require('../../utils')
 
 exports.command = 'spawn-list <point>'
 exports.desc = 'Create a list of child points to spawn from <point>. If the file already exists, this command will be a no-op.'
@@ -32,21 +31,40 @@ exports.builder = (yargs) =>{
     default: 'random',
     type: 'string',
   });
+
+  yargs.option('use-roller',{
+    describe: 'Enforce using the roller (L2) for all data and do not allow fallback to azimuth (L1).',
+    type: 'boolean',
+    conflicts: 'use-azimuth'
+  });
+  yargs.option('use-azimuth',{
+    describe: 'Enforce using azimuth (L1) for all data and do not allow fallback to the roller (L2).',
+    type: 'boolean',
+    conflicts: 'use-roller'
+  });
 }
 
 exports.handler = async function (argv) 
 {
   const point = validate.point(argv.point, true);
+
   const workDir = files.ensureWorkDir(argv.workDir);
-
   if(files.fileExists(workDir, argv.output) && !argv.force)
   {
     console.log('Spawn list file already exists, will not recreate it.');
     return;
   }
 
-  const ctx = await eth.createContext(argv);
-  var childPoints = await ajs.azimuth.getUnspawnedChildren(ctx.contracts, point);
+  const source = await rollerApi.selectDataSource(argv);
+  let childPoints = [];
+  if(source == 'azimuth'){
+    const ctx = await eth.createContext(argv);
+    childPoints = await ajs.azimuth.getUnspawnedChildren(ctx.contracts, point);
+  }
+  else{
+    const rollerClient = rollerApi.createClient(argv);
+    childPoints = await rollerApi.getUnspawned(rollerClient, point);
+  }
 
   var spawnList = pickChildPoints(childPoints, argv.count, argv.pick);
   var spawnListPatp = _.map(spawnList, p => ob.patp(p));

cmds/get.js

@@ -0,0 +1,6 @@
+exports.command = 'get <command>'
+exports.desc = 'Retrieve information about Urbit points, L2 rollers, and Ethereum gas prices.'
+exports.builder = function (yargs) {
+  return yargs.commandDir('get_cmds')
+}
+exports.handler = function (argv) {}