-
Notifications
You must be signed in to change notification settings - Fork 2
/
extra.html
386 lines (341 loc) · 17 KB
/
extra.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
<!doctype html>
<meta charset="utf-8">
<script src="website/template.js"></script>
<script async src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-68SGYK1XFT"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'G-68SGYK1XFT');
</script>
<link rel="stylesheet" href="style.css">
<link async rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/dist/katex.min.css" integrity="sha384-zTROYFVGOfTw7JV7KUu8udsvW2fx4lWOsCEDqhBreBwlHI4ioVRtmIvEThzJHGET" crossorigin="anonymous">
<script defer src="https://cdn.jsdelivr.net/npm/[email protected]/dist/katex.min.js" integrity="sha384-GxNFqL3r9uRJQhR+47eDxuPoNE7yLftQM8LcxzgS4HT73tp970WS/wV5p8UzCOmb" crossorigin="anonymous"></script>
<script defer src="https://cdn.jsdelivr.net/npm/[email protected]/dist/contrib/auto-render.min.js" integrity="sha384-vZTG03m+2yp6N6BNi5iM4rW4oIwk5DfcNdFfxkk9ZWpDriOkXX8voJBFrAO7MpVl" crossorigin="anonymous"></script>
<script async>
document.addEventListener("DOMContentLoaded", function() {
renderMathInElement(document.body, {
// customised options
// • auto-render specific keys, e.g.:
delimiters: [
{left: '$$', right: '$$', display: true},
{left: '$', right: '$', display: false},
{left: '\\(', right: '\\)', display: false},
{left: '\\[', right: '\\]', display: true}
],
// • rendering keys, e.g.:
throwOnError : false
});
});
</script>
<script type="text/front-matter">
title: "Adversarial Takeover of Neural Cellular Automata extra resources"
description: "Description of the post"
authors:
- Lorenzo Cavuoti: https://github.com/LetteraUnica
- Francesco Sacco: https://github.com/Francesco215
- Ettore Randazzo: https://oteret.github.io
- Michael Levin: https://ase.tufts.edu/biology/labs/levin/
affiliations:
- University of Trieste: http://units.it
- University of Pisa: http://unipi.it
- Google: https://ai.google/
- Allen Discovery Center: https://allencenter.tufts.edu
</script>
<dt-article>
<h1>Adversarial Takeover of Neural Cellular Automata</h1>
<dt-byline></dt-byline>
<div style="position:relative; display:flex">
<div id="colab-hero-div">
<a href="https://colab.research.google.com/drive/1Jh7MZPxE_05-04neyX2XxYbvq1O457aG" class="colab-root" id="colab-hero">Try in a
<span class="colab-span">Notebook</span>
</a>
</div>
<div id="colab-hero-div">
<a href="https://github.com/LetteraUnica/neural_cellular_automata" class="colab-root" id="colab-hero">Go to the
<span class="git-span">repository</span>
</a>
</div>
<div id="colab-hero-div">
<a href="paper.pdf" class="colab-root" id="colab-hero">Read the article
</a>
</div>
</div>
<h2 style="font-size: 60px;" id="colab">Video Abstract</h2>
<div>
<iframe width="610" height="350" src="https://www.youtube.com/embed/wwu3T7KMcvI" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</div>
<h2 style="font-size: 40px;" id="nca-types">NCA types</h2>
<p>
There are three different types of NCA:
<ul>
<li>Growing: A NCA that grows into a final state and then decays, this represents growing and aging biological organisms.</li>
<li>Persistent: A NCA that grows into a final state and keeps it for an infinite amount of time,
some biological organisms like lobsters have this property.</li>
<li>Regenerating: A Persistent NCA that is able to regenerate damage, biological examples of these are planarians.</li>
</ul>
</p>
<figure class="l-body">
<video loop="" autoplay="" playsinline="" muted="" width="100%" height="100%">
<source src="website/videos/firework_grow_persist_regen.mp4" type="video/mp4" id="grow-persist-regen">
Your browser does not support the video tag.
</video>
<div style="position:relative; height:30px; display:flex">
<div style="width:33%; top:0px; text-align:center">Growing</div>
<div style="width:33%; top:0px; left:275px; text-align:center">Persistent</div>
<div style="width:33%; top:0px; left:560px; text-align:center">Regenerating</div>
</div>
<figcaption style="align-self: center; margin-top: 20px; margin-bottom: 25px; width: 100%" >
Video 1: Example of the three types of Neural Cellular Automata
</figcaption>
</figure>
<dt-byline></dt-byline>
<h2 style="font-size: 40px;">Experiments</h2>
<div class="body" style="margin-top: 70px;">
<h2 style="margin-top: 0em;" id="static-properties">Changing static properties</h2>
<p>Below we show the evolution of the lizard and bug Persistent NCA used in the static properties experiment</p>
<figure class="l-center">
<video controls="" muted="" width="100%" height="100%">
<source src="website/videos/static_properties/lizard_and_bug_persist.mp4" type="video/mp4" id="lizard-bug-persist">
Your browser does not support the video tag.
</video>
<figcaption style="align-self: center; margin-top: 20px; margin-bottom: 25px; width: 100%" >
Video 2: left: a persistent lizard, right: a persistent bug.
</figcaption>
</figure>
<p>Below, the left panel shows the evolution of the organism,
while the right panel shows where the cells are located, orange being the original cells and blue the adversaries.
</p>
<figure class="l-center">
<video controls="" muted="" width="100%" height="100%">
<source src="website/videos/static_properties/lizard_persist_to_red_square_2.mp4" type="video/mp4" id="red-lizard">
Your browser does not support the video tag.
</video>
<figcaption style="align-self: center; margin-top: 20px; margin-bottom: 25px; width: 100%" >
Video 3: red lizard experiment.
</figcaption>
</figure>
<br/>
<figure class="l-center">
<video controls="" muted="" width="100%" height="100%">
<source src="website/videos/static_properties/lizard_persist_to_no_tail_square_2.mp4" type="video/mp4" id="notail-lizard">
Your browser does not support the video tag.
</video>
<figcaption style="align-self: center; margin-top: 20px; margin-bottom: 25px; width: 100%" >
Video 4: no tail lizard experiment.
</figcaption>
</figure>
<br/>
<figure class="l-center">
<video controls="" muted="" width="100%" height="100%">
<source src="website/videos/static_properties/bug_to_butterfly_square_2.mp4" type="video/mp4" id="no-tail">
Your browser does not support the video tag.
</video>
<figcaption style="align-self: center; margin-top: 20px; margin-bottom: 25px; width: 100%" >
Video 5: bug to butterfly experiment.
</figcaption>
</figure>
<details>
<summary>
<div class="l-body" style="display: flex; width: 100%; margin-left: 20px; margin-top: -35px">
<div style="position: relative; margin-top:-px; width:60%">
<h3 style="margin-top: 0em;" id="static-properties-extras">Extras</h3>
</div>
</div>
</summary>
<p>
We found that adversaries trained to overtake the entire organism from a small 2x2 seed, oftentimes are robust to other
cell injection patterns, in the following we show examples of this behavior.
</p>
<p>
Cutting the tail of the lizard can also be done when starting from a spray pattern, where
1% of the original cells are randomly replaced by adversaries, or even starting from a single cell.
</p>
<figure class="l-center">
<video controls="" muted="" width="100%" height="100%">
<source src="website/videos/static_properties/lizard_persist_to_no_tail_1percent.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<figcaption style="align-self: center; margin-top: 20px; margin-bottom: 25px; width: 100%" >
Video 6: the adversaries start from a spray pattern in which 1% of the original cells are substituted.
</figcaption>
</figure>
<br/>
<figure class="l-center">
<video controls="" muted="" width="100%" height="100%">
<source src="website/videos/static_properties/lizard_persist_to_no_tail_square_1.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<figcaption style="align-self: center; margin-top: 20px; margin-bottom: 25px; width: 100%" >
Video 7: the adversaries start from a single cell.
</figcaption>
</figure>
<p>
We can turn a bug into a butterfly even if we start from spray patterns of different densities, like, 1% and 10%.
</p>
<figure class="l-center">
<video controls="" muted="" width="100%" height="100%">
<source src="website/videos/static_properties/bug_to_butterfly_1percent.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<figcaption style="align-self: center; margin-top: 20px; margin-bottom: 25px; width: 100%" >
Video 8: the adversaries start from a spray pattern with 1% of cells substituted.
</figcaption>
</figure>
<br/>
<figure class="l-center">
<video controls="" muted="" width="100%" height="100%">
<source src="website/videos/static_properties/bug_to_butterfly_10percent.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<figcaption style="align-self: center; margin-top: 20px; margin-bottom: 25px; width: 100%" >
Video 9: the adversaries start from a spray pattern with 10% of cells substituted.
</figcaption>
</figure>
<p>
While we said that the red lizard was the simplest one to train, it is also true that it is the adversary that is the least
robust of the three, we think this is due to the training, since it took so few number of
gradient updates to train, the adversaries didn't had the time to see a lot of different patterns.
</p>
</details>
</div>
<div class="body" style="margin-top: 70px;">
<h2 style="margin-top: 0em;" id="dynamic-properties">Changing dynamic properties</h2>
<p>Below we show the evolution of the three Growing NCA used in the dynamic properties experiment,
as you can see the NCA quickly decay after reaching the target state.
</p>
<figure class="l-center">
<video controls="" muted="" width="100%" height="100%">
<source src="website/videos/dynamic_properties/lizard_butterfly_firework_growing.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<figcaption style="align-self: center; margin-top: 20px; margin-bottom: 25px; width: 100%" >
Video 10: left: a growing lizard, center: a growing butterfly, right: a growing firework.
</figcaption>
</figure>
<br/>
<p>Below, the left panel shows the evolution of the organism,
while the right panel shows where the cells are located, orange being the original cells and blue the adversaries.
</p>
<figure class="l-center">
<video controls="" muted="" width="100%" height="100%">
<source src="website/videos/dynamic_properties/lizard_grow_to_persist_square_3.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<figcaption style="align-self: center; margin-top: 20px; margin-bottom: 25px; width: 100%" >
Video 11: lizard growing to persistent, the adversaries are injected in a 3x3 square seed.
</figcaption>
</figure>
<br/>
<figure class="l-center">
<video controls="" muted="" width="100%" height="100%">
<source src="website/videos/dynamic_properties/butterfly_grow_to_persist_3percent.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<figcaption style="align-self: center; margin-top: 20px; margin-bottom: 25px; width: 100%" >
Video 12: butterfly growing to persistent, 3% of the cells are substituted in a spray pattern.
</figcaption>
</figure>
<br/>
<figure class="l-center">
<video controls="" muted="" width="100%" height="100%">
<source src="website/videos/dynamic_properties/firework_grow_to_persist_50percent.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<figcaption style="align-self: center; margin-top: 20px; margin-bottom: 25px; width: 100%" >
Video 13: firework growing to persistent, 50% of cells are substituted in a spray pattern
</figcaption>
</figure>
<details>
<summary>
<div class="l-body" style="display: flex; width: 100%; margin-left: 20px; margin-top: -35px">
<div style="position: relative; margin-top:-px; width:60%">
<h3 style="margin-top: 0em;" id="dynamic-properties-extras">Extras</h3>
</div>
</div>
</summary>
<p>Also dynamic properties can show robustness to other injection patterns.</p>
<p>First, we have the lizard, which is able to turn into a persistent NCA even when we start from a spray pattern.</p>
<figure class="l-center">
<video controls="" muted="" width="100%" height="100%">
<source src="website/videos/dynamic_properties/lizard_grow_to_persist_1percent.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<figcaption style="align-self: center; margin-top: 20px; margin-bottom: 25px; width: 100%" >
Video 14: lizard growing to persistent, 1% of the cells are substituted in a spray pattern.
</figcaption>
</figure>
<p>
The butterfly, even though it was trained with 3% of cells substituted, works perfectly fine even with 1% of cells substituted.
</p>
<figure class="l-center">
<video controls="" muted="" width="100%" height="100%">
<source src="website/videos/dynamic_properties/butterfly_grow_to_persist_1percent.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<figcaption style="align-self: center; margin-top: 20px; margin-bottom: 25px; width: 100%" >
Video 15: butterfly growing to persistent, 1% of the cells are substituted in a spray pattern.
</figcaption>
<p>
These extra experiments, show how much the adversaries can adapt to out of distribution patterns,
this property is essential in biological situations, where changing the behaviour of individual cells cannot be done with perfect accuracy.
Having adversaries that work even for misplaced or random locations solves this problem at the root.
</p>
</details>
</div>
</dt-article>
<!--Bibliography-->
<script type="text/bibliography">
@article{mordvintsev2020growing,
author = {Mordvintsev, Alexander and Randazzo, Ettore and Niklasson, Eyvind and Levin, Michael},
title = {Growing Neural Cellular Automata},
journal = {Distill},
year = {2020},
url = {https://distill.pub/2020/growing-ca},
}
@article{randazzo2021adversarial,
author = {Randazzo, Ettore and Mordvintsev, Alexander and Niklasson, Eyvind and Levin, Michael},
title = {Adversarial Reprogramming of Neural Cellular Automata},
journal = {Distill},
year = {2021},
url = {https://distill.pub/selforg/2021/adversarial},
doi = {10.23915/distill.00027.004}
}
@article{handberg2008stem,
title={Stem cells and regeneration in planarians},
author={Handberg-Thorsager, Mette and Fernandez, Enrique and Salo, Emili},
journal={Front Biosci},
volume={13},
pages={6374--6394},
year={2008},
url ={https://www.fbscience.com/Landmark/articles/pdf/Landmark3160.pdf}
}
@article{TransmembraneVoltagePotential,
author = {Pai, Vaibhav P. and Aw, Sherry and Shomrat, Tal and Lemire, Joan M. and Levin, Michael},
title = {Transmembrane voltage potential controls embryonic eye patterning in Xenopus laevis},
journal = {Development},
volume = {139},
number = {2},
pages = {313-323},
year = {2012},
month = {01},
issn = {0950-1991},
doi = {10.1242/dev.073759},
url = {https://doi.org/10.1242/dev.073759},
eprint = {https://journals.biologists.com/dev/article-pdf/139/2/313/1160014/313.pdf}
}
</script>
<script src="https://d3js.org/d3.v4.js"></script>
<script src="website/grid_value.js" type="text/javascript"></script>
<script src="website/grid_yellow_value.js" type="text/javascript"></script>
<script src="website/grid.js" type="text/javascript"></script>
<script src="website/epoch_loss.js"></script>
<script src="website/epoch_loss_growing.js"></script>
<script>
window.addEventListener("resize", function(){
onresize1();
onresize2();
});
</script>