Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recipe: Dependency loop detection #240

Merged
merged 5 commits into from
Aug 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions hugo/content/docs/recipes/validation/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
title: Validation
weight: 150
---
336 changes: 336 additions & 0 deletions hugo/content/docs/recipes/validation/dependency-loops.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,336 @@
---
title: "Dependency Loops"
weight: 100
---

## What is the problem?
Lotes marked this conversation as resolved.
Show resolved Hide resolved

If you are building some composite data structures or have some computation flow, you might be interested whether the product, that you are generating, does contain any loops back to the already used product. We want to implement a validation that does this detection in the background and notifies us by highlighting the lines causing this problem.
The second part of the problem is the resolution of the dependencies. If there are no loops, we want to get a resolution of the dependencies in a loop-free order.

Examples for such dependency loops are:

* For data structures you could think of a structure in C that references itself (without using the pointer notation). This would lead to an infinitely expanding data type, which is practically not doable.

```c
typedef struct {
int value;
LinkedList next; //error, should be "MyStruct* next;", a pointer to next
} LinkedList;
```

* Or for control flows these loops can be interpreted as recursion detection, a function that calls itself (with any number of function calls to other functions in-between).

```c
void foo() {
bar();
}
void bar() {
answer42();
}
void answer42() {
bar(); //error, foo calls bar, bar calls answer42, answer42 calls foo
}
```

Regardless of what usecase you have, you might have an interest to detect those loops and get early feedback in the shape of a validation error.

The other point is the **resolution for loop-free dependencies**. Think of a net of package imports in
a programming language. You want to know the order in which you can import the packages without getting into trouble.

```plaintext
A -> B -> C
A -> C
C -> D

//resolution: A, B, C, D
```

Or think of a function call graph. You want to know the order in which you can build the functions such that every dependency was built before the dependent function.

```c
void answer42() {
printf("42\n");
}
void bar1() {
answer42();
}
void foo() {
bar1();
bar2();
}
void bar2() {
answer42();
}
```

Here the resolution would be: `answer42`, `bar1`, `bar2`, `foo`.

## How to solve it?

There are two approaches for a loop detection and the loop-free resolution depending on the nature of your situation.

### Simple nature

#### Simple detection

If you have a `1:n` relationship like the `super class`-to-`sub class` relation for classes, you can do it by simply walking along the parent route (or in this specific example the `super class`-route). Just keep in mind all visited places and if one parent is already in that list, you have detected a loop!

```java
public class A extends B {}
public class B extends C {}
public class C extends A {} //error
```

#### Simple resolution

Assuming that you have no loops back, you can resolve a list of dependencies.
You do a simple depth-first-search, starting with the parent visiting the children afterwards (recursively).

```java
public class A {} //add A
public class B extends A {} //add A -> B
public class C extends A {} //add A -> C
public class D extends C {} //add C -> D
public class E extends C {} //add C -> E

//resolution: A, (B or (C, (D or E)))
```

### Complex nature

#### Complex detection

If you have a `n:m` relationship like it is given for function calls (a function can be called by `m` function and can call `n` functions), you can solve the question for loops by creating a directed graph.

In this example the nodes are the set of all functions and function calls are stored as edges (for each call from function A to every function B).
The key algorithm is the search for the so-called strongly-connected components in the resulting graph.

Please use an existing solution for the algorithm (keep in mind the effort you can avoid and a quality you can gain)! The algorithm is able to output every loop with all its members of that loop. But you are free to make your own implementation.

#### Complex resolution
Lotes marked this conversation as resolved.
Show resolved Hide resolved

The directed graph approach can be processed further when there were no loops found:
Lotes marked this conversation as resolved.
Show resolved Hide resolved

With a "topological sort" you can get an order that respects all dependencies. Means more or less: You start with the node that has no dependencies, remove it, put it into your sorted list and do the same for the resulting graph again and again until all dependencies were resolved.

The topological sort (as well as the strongly-connected component search) is a standard algorithm in every good graph library.

## How to make it work in Langium?

Lotes marked this conversation as resolved.
Show resolved Hide resolved
In the following example we will resolve the dependencies for a complex nature of data.

Therefore we will take the `HelloWorld` example from the learning section and extend it with a validation that checks for greeting loops. Greeting loops are forbidden in this example. When `A` greets `B` and `B` greets `C`, then `C` must not greet `A`.

### Adapt the grammar

We will change the `HelloWorld` grammar, so that persons can greet each other. After that, we will introduce a validation in order to forbid "greeting loops".

```langium
grammar HelloWorld

entry Model:
(persons+=Person | greetings+=Greeting)*;

Person:
'person' name=ID;

Greeting:
greeter=[Person:ID] 'greets' greeted=[Person:ID] '!';

hidden terminal WS: /\s+/;
terminal ID: /[_a-zA-Z][\w_]*/;
````

After the change build your grammar with `npm run langium:generate`.

### Loop detection

Now we will add the validation. Here we will use the graph library ‚graphology‘. Please install these three packages (`graphology` contains the data structure, `graphology-components` contains the strongly-connected component search, `graphology-dag` contains the topological sort):

```bash
npm install graphology graphology-components graphology-dag
```

Open the `hello-world-validator.ts` and add another validator for `Model`. It is important to say that we do not create a check on the `Greeting` level, because we need the overview over all greetings. The complete overview is given for the `Model` AST node. It would be possible to just calculate cycles for a single greeting or person, but that is more complex and less performant!

```typescript
const checks: ValidationChecks<HelloWorldAstType> = {
Model: validator.checkGreetingCycles, // new!!!
Person: validator.checkPersonStartsWithCapital
};
```

And here is the implementation:

```typescript
checkGreetingCycles(model: Model, accept: ValidationAcceptor): void {
//arrange the graph
const graph = new DirectedGraph<{}, {greeting: Greeting}>();
model.persons.forEach(person => {
graph.addNode(person.name);
})
model.greetings.forEach(greeting => {
if(greeting.greeter.ref && greeting.greeted.ref && !graph.hasDirectedEdge(greeting.greeter.ref.name, greeting.greeted.ref.name)) {
graph.addEdge(greeting.greeter.ref.name, greeting.greeted.ref.name, {
greeting //we store the greeting for later reference in the validation message
});
}
});

//compute the components
const components = stronglyConnectedComponents(graph);

//evaluate result (filter out size-1-components)
const actualLoops = components.filter(c => c.length > 1);
for (const component of actualLoops) {
const set = new Set<string>(component);
//for each node in the component...
Lotes marked this conversation as resolved.
Show resolved Hide resolved
for (const from of set) {
//check whether the out edges...
for (const { target: to, attributes: { greeting } } of graph.outEdgeEntries(from)) {
Lotes marked this conversation as resolved.
Show resolved Hide resolved
//are within the component
if(set.has(to)) {
//if yes, set an error on the corresponding greeting
accept("error", "Greeting loop detected!", {
node: greeting
});
}
}
}
}
}
```

After finishing your validator, do not forget to build your project with `npm run build`.
So a `.hello` file like this one, would have 3 greetings with an error:

```plaintext
person Homer
person Marge
person Pinky
person Brain

Homer greets Marge! //error
Lotes marked this conversation as resolved.
Show resolved Hide resolved
Marge greets Brain! //error
Brain greets Homer! //error
Pinky greets Marge!
```

Here is the screenshot of VS Code with the error:

![Greeting loop errors](/assets/dependency-loops.png)

### Dependency resolution

The topological sort can be done like this:

```typescript
import { topologicalSort } from 'graphology-dag';

//resolvedOrder is an array of person names!
const resolvedOrder = topologicalSort(graph);
```

This will give you back an order of greeters. The rule would be like: `You can only greet if every greeting addressed to you was already spoken out.`
For a `.hello` file like this, we would get the order: `Homer`, `Brain`, `Pinky`, `Marge`.

```plaintext
person Homer
person Marge

person Pinky
person Brain

Homer greets Marge!
Brain greets Pinky!
Pinky greets Marge!
```

* `Homer` is not greeted by anyone, so he can start greeting `Marge`.
* `Marge` and `Pinky` are blocked by `Pinky` and `Brain`.
* `Brain` is the next and unblocks `Pinky`.
* After `Pinky` is done, `Marge` is unblocked as well.
* But `Marge` has no one to greet.
* So, we are done.

## Appendix

Lotes marked this conversation as resolved.
Show resolved Hide resolved
<details>
<summary>Full Implementation</summary>

```ts
import type { ValidationAcceptor, ValidationChecks } from 'langium';
import type { Greeting, HelloWorldAstType, Model } from './generated/ast.js';
import type { HelloWorldServices } from './hello-world-module.js';
import { DirectedGraph } from 'graphology';
import { stronglyConnectedComponents } from 'graphology-components';
import { topologicalSort } from 'graphology-dag';

/**
* Register custom validation checks.
*/
export function registerValidationChecks(services: HelloWorldServices) {
const registry = services.validation.ValidationRegistry;
const validator = services.validation.HelloWorldValidator;
const checks: ValidationChecks<HelloWorldAstType> = {
Model: validator.checkGreetingCycles,
//Not needed for this example
//Person: validator.checkPersonStartsWithCapital
};
registry.register(checks, validator);
}

/**
* Implementation of custom validations.
*/
export class HelloWorldValidator {
checkGreetingCycles(model: Model, accept: ValidationAcceptor): void {
//arrange the graph
const graph = new DirectedGraph<{}, {greeting: Greeting}>();
model.persons.forEach(person => {
graph.addNode(person.name);
})
model.greetings.forEach(greeting => {
if(greeting.greeter.ref && greeting.greeted.ref && !graph.hasDirectedEdge(greeting.greeter.ref.name, greeting.greeted.ref.name)) {
graph.addEdge(greeting.greeter.ref.name, greeting.greeted.ref.name, {
greeting
});
}
});

//compute the components
const components = stronglyConnectedComponents(graph);

//evaluate result (filter out size-1-components)
const actualLoops = components.filter(c => c.length > 1);
for (const component of actualLoops) {
const set = new Set<string>(component);
//for each node in the component...
for (const from of set) {
//check whether the out edges...
for (const { target: to, attributes: { greeting } } of graph.outEdgeEntries(from)) {
//are within the component
if(set.has(to)) {
//if yes, set an error on the corresponding greeting
accept("error", "Greeting loop detected!", {
node: greeting
});
}
}
}
}

//resolve all dependencies
if(actualLoops.length === 0) {
const resolvedOrder = topologicalSort(graph);
//this is done as a hint, just for demonstration purposes
accept('hint', "Please greet in the following greeter order: "+resolvedOrder.join(", "), {
node: model
});
}
}
}
```

</details>
Binary file added hugo/static/assets/dependency-loops.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading