Skip to content

Commit

Permalink
neuralang default broadcasting
Browse files Browse the repository at this point in the history
  • Loading branch information
maniospas committed May 28, 2024
1 parent cc092e2 commit 7ce9b64
Show file tree
Hide file tree
Showing 4 changed files with 22 additions and 12 deletions.
4 changes: 1 addition & 3 deletions JGNN/src/examples/nodeClassification/Scripting.java
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,7 @@ public static void main(String[] args) throws Exception {
.autosize(new EmptyTensor(dataset.samples().getSlice().size()));

ModelTraining trainer = new ModelTraining()
.setEpochs(300)
.setPatience(100)
.setOptimizer(new Adam(0.01))
.configFrom(modelBuilder)
.setVerbose(true)
.setLoss(new CategoricalCrossEntropy())
.setValidationLoss(new CategoricalCrossEntropy());
Expand Down
16 changes: 13 additions & 3 deletions JGNN/src/main/java/mklab/JGNN/adhoc/ModelBuilder.java
Original file line number Diff line number Diff line change
Expand Up @@ -941,23 +941,33 @@ else if(functions.containsKey(splt[2])) {
if(i<args.length) {
String config = args[i].substring(0, args[i].indexOf(":")).trim();
String value = args[i].substring(args[i].indexOf(":")+1).trim();
boolean isFinal = value.startsWith("!");
if(isFinal)
value = value.substring(1);
if(value.equals("extern")) {
if(!this.configurations.containsKey(config))
throw new RuntimeException("Required external config: "+config);
}
if(!this.configurations.containsKey(config))
if(!this.configurations.containsKey(config)) {
if(isFinal)
configStack.put(config, parseConfigValue(value));
this.config(config, parseConfigValue(value));
}
}
// these are parsed in the attempt to create an intermediate variable for the argument
if(i<splt.length-3) {
String config = splt[i+3].substring(0, splt[i+3].indexOf(":")).trim();
String value = splt[i+3].substring(splt[i+3].indexOf(":")+1).trim();
boolean isFinal = value.startsWith("!");
if(isFinal)
value = value.substring(1);
if(value.equals("extern")) {
if(!this.configurations.containsKey(config))
throw new RuntimeException("Required external config: "+config);
}
else
this.config(config, parseConfigValue(value));
if(isFinal)
configStack.put(config, parseConfigValue(value));
this.config(config, parseConfigValue(value));
}
}
List<String> tokens = extractTokens(functions.get(splt[2]));
Expand Down
2 changes: 1 addition & 1 deletion architectures.nn
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
fn classify(nodes, h, epochs: 3000, patience: 100, lr: 0.01) {
fn classify(nodes, h, epochs: !3000, patience: !100, lr: !0.01) {
return softmax(h[nodes], dim: "row");
}

Expand Down
12 changes: 7 additions & 5 deletions tutorials/Neuralang.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,16 @@ using a syntax inspired by Mojo. Use a Rust highlighter, which covers
all keywords. Below are examples of function declarations:

```rust
fn classify(nodes, h, epochs: 3000, patience: 100, lr: 0.01) {
fn classify(nodes, h, epochs: !3000, patience: !100, lr: !0.01) {
return softmax(h[nodes], dim: "row");
}
```

The classify function takes several parameters: nodes, which are the input nodes for classification; h, the feature matrix; epochs, which defaults to 3000 and represents the number of training epochs; patience, which defaults to 100 and denotes the early stopping patience; and lr, the learning rate, which defaults to 0.01. The function returns the softmax output for the specified nodes. Configuration defaults are indicated by a colon (:) in the function signatures. The same notation is used to set/overwrite them when calling functions, as we do for softmax to apply it row-wise. Configuration values have the priority:
The classify function takes two inputs: nodes are the input nodes for classification; h is the feature matrix. The function returns a softmax output for the specified nodes. It also considers several configuration values, whose defaults are indicated by a colon (:) in the function signatures. The same notation is used to set/overwrite them when calling functions, as we do for softmax to apply it row-wise. Think of them as keyword arguments. These defaults for the classify function are: epochs, which defaults to 3000 and represents the number of training epochs; patience, which defaults to 100 and denotes the early stopping patience; and lr the learning rate that defaults to 0.01.

Exclamation marks (!) before numbers broadcast them to all subsequent function calls as new defaults for the same configurations. Broadcasted configurations are retrievable from JGNN's Neuralang model builder, which is useful for Java integration later. Configuration values have the priority:
1. function call arguments
2. Java configurations
2. broacasted configurations (last value, includes configurations set by Java)
3. function signature defaults

```rust
Expand Down Expand Up @@ -70,6 +72,6 @@ ModelTraining trainer = new ModelTraining()
.setValidationLoss(new CategoricalCrossEntropy());
```

In this example, a dataset (Cora) is loaded, and its graph is prepared by adding self-loops (this is known as the renormalization trick) and normalizing symmetrically. A Neuralang instance is created, which is a ModelBuilder that can parse scripts as either file Paths or pure text. Constants like the adjacency matrix A and feature matrix h are set, along with variables (nodes) and configurations (classes, hidden). The model and its output is defined with a Neuralang statement. Finally, dimension names and sizes for ? found model declaration are filled by calling autosize. In the example we use empty tensors to avoid unecessary computations while determining the dimensions.
In the above example, a dataset (Cora) is loaded, and its graph is prepared by adding self-loops (the renormalization trick) and performing symmetric normalization. A Neuralang instance is ten created; this is a ModelBuilder that can parse scripts as either file Paths or pure text. Constants like the adjacency matrix A and feature matrix h are set, along with variables (nodes) and configurations (classes, hidden). The model and its output is defined with a Neuralang statement. Finally, dimension names and sizes for ? found model declaration are filled by calling autosize. In the example we use empty tensors to avoid unecessary computations while determining the dimensions.

A ModelTraining instance is then configured using parameters from the ModelBuilder, utilizing the configurations found in the classification method.
A ModelTraining instance is finally configured using parameters from the ModelBuilder, utilizing the configurations found in the classification method. Don't forget to broadcast configuration values that you need to access from Java code later.

0 comments on commit 7ce9b64

Please sign in to comment.