-
Notifications
You must be signed in to change notification settings - Fork 0
/
patchfile-4.16
14421 lines (14410 loc) · 509 KB
/
patchfile-4.16
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
diff -uNrw -uNr linux-base/README linux/README
--- linux-base/README 2020-02-19 14:32:34.000000000 +0100
+++ linux/README 1970-01-01 01:00:00.000000000 +0100
@@ -1,18 +0,0 @@
-Linux kernel
-============
-
-This file was moved to Documentation/admin-guide/README.rst
-
-Please notice that there are several guides for kernel developers and users.
-These guides can be rendered in a number of formats, like HTML and PDF.
-
-In order to build the documentation, use ``make htmldocs`` or
-``make pdfdocs``.
-
-There are various text files in the Documentation/ subdirectory,
-several of them using the Restructured Text markup notation.
-See Documentation/00-INDEX for a list of what is contained in each file.
-
-Please read the Documentation/process/changes.rst file, as it contains the
-requirements for building and running the kernel, and information about
-the problems which may result by upgrading your kernel.
diff -uNrw -uNr linux-base/README.md linux/README.md
--- linux-base/README.md 1970-01-01 01:00:00.000000000 +0100
+++ linux/README.md 2020-02-19 10:26:50.000000000 +0100
@@ -0,0 +1,50 @@
+Linux kernel
+============
+Actual linux documentation is available in `Documentation/admin-guide/README.rst`
+Commit hash of first commit 48c0dcc537 (needed to create a patch).
+
+#### Build requirements for ubuntu 18.04
+`sudo apt-get install git build-essential kernel-package fakeroot libncurses5-dev libssl-dev ccache bison flex`
+
+#### syscall specification:
+```c
+SYSCALL_NR 333
+
+void* vmcopy(void* src, unsigned long length);
+
+// use syscall defined in unistd.h to invoke the system call or,
+// use the header file (anker.h) defined in rapido/tests/include/
+
+void *copy = syscall(333, src, length);
+```
+#### Configure kernel with VMCOPY support
+1. Use `make menuconfig` to generate a `.config` file.
+2. Enable transparent hugepages by editing `.config` file.
+ * Set `CONFIG_TRANSPARENT_HUGEPAGE=y` and `CONFIG_TRANSPARENT_HUGEPAGE_MADVISE=y`.
+ * These options are necessary to run the tests specified under rapido/tests/src/
+3. Enable `vmcopy()` by setting `CONFIG_ANKER_VMCOPY=y`.
+4. Currently `TRANSPARENT_HUGEPAGE` must be enabled to use `vmcopy()`.
+
+#### Testing the kernel
+This kernel was developed using the testbed backed by `Qemu`. I have used a trimmed down version of [Rapido](https://github.com/rapido-linux/rapido) that provides a set of scripts to quicky generate `VM Image` and necessary modules using Dracut and boots the image using `Qemu`.
+
+#### Build instructions
+1. Generate relevant configure file as mentioned above.
+2. make -j
+3. INSTALL_MOD_PATH=./mods make modules_install
+
+#### Running tests
+1. `cd rapido/tests`
+2. `cmake -DCMAKE_CXX_FLAGS=-O2 .`
+3. `make -j4`
+4. `cd ..`
+5. `./cut_anker.sh`
+6. Boot the `VM` using `./vm.sh`.
+7. Tests are installed in path `/tests/` inside the VM.
+8. Use `shutdown` to powerdown the `VM`.
+
+#### Patching linux
+1. Create patch using `diff`.
+ * `diff -uNr linux.vanilla linux.new > patchfile`
+2. Apply patch using `patch`.
+ * `cd linux && patch -p1 < ../patchfile`
\ No newline at end of file
diff -uNrw -uNr linux-base/arch/x86/Makefile linux/arch/x86/Makefile
--- linux-base/arch/x86/Makefile 2020-02-19 14:32:34.000000000 +0100
+++ linux/arch/x86/Makefile 2020-02-19 10:26:50.000000000 +0100
@@ -271,7 +271,7 @@
libs-y += arch/x86/lib/
# See arch/x86/Kbuild for content of core part of the kernel
-core-y += arch/x86/
+core-y += arch/x86/ arch/x86/anker/
# drivers-y are linked after core-y
drivers-$(CONFIG_MATH_EMULATION) += arch/x86/math-emu/
diff -uNrw -uNr linux-base/arch/x86/anker/Makefile linux/arch/x86/anker/Makefile
--- linux-base/arch/x86/anker/Makefile 1970-01-01 01:00:00.000000000 +0100
+++ linux/arch/x86/anker/Makefile 2020-02-19 10:26:50.000000000 +0100
@@ -0,0 +1 @@
+obj-$(CONFIG_ANKER_VMCOPY) := anker.o
diff -uNrw -uNr linux-base/arch/x86/anker/anker.c linux/arch/x86/anker/anker.c
--- linux-base/arch/x86/anker/anker.c 1970-01-01 01:00:00.000000000 +0100
+++ linux/arch/x86/anker/anker.c 2020-02-19 14:36:18.000000000 +0100
@@ -0,0 +1,101 @@
+/**
+ * File : arch/x86/anker/anker.c
+ * Author : Ankur Sharma <[email protected]>
+ * Date : 09.07.2017
+ * Last Modified Date: 09.10.2017
+
+ * Copyright (c) 2018 Ankur Sharma <[email protected]>
+ */
+
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/mm_types.h>
+#include <linux/syscalls.h>
+
+#include <asm/tlbflush.h>
+#include <asm/syscalls.h>
+
+#include "../mm/internal.h"
+
+#define PAGE_ALIGNMENT ((1 << PAGE_SHIFT) - 1)
+
+/*
+ * SYSCALL_NR 333
+ * anker: vmcopy(void *, unsigned long);
+ */
+SYSCALL_DEFINE2(vmcopy, unsigned long, src_addr, unsigned long, len)
+{
+ struct mm_struct *mm = current->mm;
+ struct vm_area_struct *vma;
+ unsigned long ret = -EINVAL;
+ bool locked = false;
+ struct vm_userfaultfd_ctx uf = NULL_VM_UFFD_CTX;
+ unsigned long map_flags = 0;
+ unsigned long new_addr = 0, charged = 0;
+ unsigned long new_len = PAGE_ALIGN(len);
+ LIST_HEAD(uf_unmap_early);
+ LIST_HEAD(uf_unmap);
+
+ if (offset_in_page(src_addr))
+ return ret;
+
+ /*
+ * We allow a zero old-len as a special case
+ * for DOS-emu "duplicate shm area" thing. But
+ * a zero new-len is nonsensical.
+ */
+ if (!new_len)
+ return ret;
+
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
+ /*
+ * We weren't able to just expand or shrink the area,
+ * we need to create a new one and move it..
+ */
+ ret = -ENOMEM;
+
+ vma = find_vma(mm, src_addr);
+
+ if (!vma) goto out;
+
+ if (vma->vm_flags & VM_MAYSHARE)
+ map_flags |= MAP_SHARED;
+
+ new_addr = get_unmapped_area(vma->vm_file, 0, new_len,
+ vma->vm_pgoff +
+ ((src_addr - vma->vm_start) >> PAGE_SHIFT),
+ map_flags);
+
+ if (offset_in_page(new_addr))
+ {
+ ret = new_addr;
+ goto out;
+ }
+
+ for (; vma && charged < new_len; vma = vma->vm_next)
+ {
+ len = vma->vm_end - (src_addr + charged);
+ len = (new_len - charged < len) ? new_len - charged : len;
+
+ ret = anker_copy_vma(vma, src_addr + charged,
+ new_addr + charged, len, &locked,
+ &uf, &uf_unmap);
+
+ /* unmap new copy if anything failed */
+ if (ret == -ENOMEM) {
+ up_write(¤t->mm->mmap_sem);
+ vm_munmap(new_addr, new_addr + charged);
+ return ret;
+ }
+
+ charged += len;
+ }
+
+out:
+ up_write(¤t->mm->mmap_sem);
+ if (locked)
+ mm_populate(new_addr, new_len);
+ return ret;
+}
diff -uNrw -uNr linux-base/arch/x86/entry/syscalls/syscall_64.tbl linux/arch/x86/entry/syscalls/syscall_64.tbl
--- linux-base/arch/x86/entry/syscalls/syscall_64.tbl 2020-02-19 14:32:34.000000000 +0100
+++ linux/arch/x86/entry/syscalls/syscall_64.tbl 2020-02-19 10:26:50.000000000 +0100
@@ -339,6 +339,8 @@
330 common pkey_alloc sys_pkey_alloc
331 common pkey_free sys_pkey_free
332 common statx sys_statx
+333 64 vmcopy sys_vmcopy
+
#
# x32-specific system call numbers start at 512 to avoid cache impact
diff -uNrw -uNr linux-base/include/linux/huge_mm.h linux/include/linux/huge_mm.h
--- linux-base/include/linux/huge_mm.h 2020-02-19 14:32:34.000000000 +0100
+++ linux/include/linux/huge_mm.h 2020-02-19 10:26:50.000000000 +0100
@@ -43,6 +43,9 @@
extern bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr,
unsigned long new_addr, unsigned long old_end,
pmd_t *old_pmd, pmd_t *new_pmd, bool *need_flush);
+extern bool anker_copy_huge_pmd(struct vm_area_struct *vma, struct vm_area_struct *new_vma,
+ unsigned long old_addr,unsigned long new_addr, unsigned long old_end,
+ pmd_t *old_pmd, pmd_t *new_pmd, bool *need_flush);
extern int change_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd,
unsigned long addr, pgprot_t newprot,
int prot_numa);
diff -uNrw -uNr linux-base/include/linux/mm.h linux/include/linux/mm.h
--- linux-base/include/linux/mm.h 2020-02-19 14:32:34.000000000 +0100
+++ linux/include/linux/mm.h 2020-02-19 10:26:50.000000000 +0100
@@ -1312,6 +1312,9 @@
unsigned long end, unsigned long floor, unsigned long ceiling);
int copy_page_range(struct mm_struct *dst, struct mm_struct *src,
struct vm_area_struct *vma);
+extern unsigned long anker_copy_vma(struct vm_area_struct *vma,
+ unsigned long old_addr, unsigned long new_addr, unsigned long len,
+ bool *locked, struct vm_userfaultfd_ctx *uf, struct list_head *uf_unmap);
int follow_pte_pmd(struct mm_struct *mm, unsigned long address,
unsigned long *start, unsigned long *end,
pte_t **ptepp, pmd_t **pmdpp, spinlock_t **ptlp);
diff -uNrw -uNr linux-base/include/linux/syscalls.h linux/include/linux/syscalls.h
--- linux-base/include/linux/syscalls.h 2020-02-19 14:32:34.000000000 +0100
+++ linux/include/linux/syscalls.h 2020-02-19 10:26:50.000000000 +0100
@@ -940,5 +940,6 @@
asmlinkage long sys_pkey_free(int pkey);
asmlinkage long sys_statx(int dfd, const char __user *path, unsigned flags,
unsigned mask, struct statx __user *buffer);
-
+/* anker */
+asmlinkage long sys_vmcopy(unsigned long, unsigned long);
#endif
diff -uNrw -uNr linux-base/include/uapi/asm-generic/unistd.h linux/include/uapi/asm-generic/unistd.h
--- linux-base/include/uapi/asm-generic/unistd.h 2020-02-19 14:32:34.000000000 +0100
+++ linux/include/uapi/asm-generic/unistd.h 2020-02-19 10:26:50.000000000 +0100
@@ -732,9 +732,10 @@
__SYSCALL(__NR_pkey_free, sys_pkey_free)
#define __NR_statx 291
__SYSCALL(__NR_statx, sys_statx)
-
+#define __NR_vmcopy 333
+__SYSCALL(__NR_vmcopy, sys_vmcopy)
#undef __NR_syscalls
-#define __NR_syscalls 292
+#define __NR_syscalls 293
/*
* All syscalls below here should go away really,
diff -uNrw -uNr linux-base/init/Kconfig linux/init/Kconfig
--- linux-base/init/Kconfig 2020-02-19 14:32:34.000000000 +0100
+++ linux/init/Kconfig 2020-02-19 10:26:50.000000000 +0100
@@ -37,7 +37,6 @@
and put_task_stack() in save_thread_stack_tsk() and get_wchan().
menu "General setup"
-
config BROKEN
bool
@@ -97,8 +96,7 @@
release tree by looking for git tags that belong to the current
top of tree revision.
- A string of the format -gxxxxxxxx will be added to the localversion
- if a git-based tree is found. The string generated by this will be
+ A string of the format -gxxxxxxxx will be added to the localversion if a git-based tree is found. The string generated by this will be
appended after any matching localversion* files, and after the value
set in CONFIG_LOCALVERSION.
diff -uNrw -uNr linux-base/kernel/sys_ni.c linux/kernel/sys_ni.c
--- linux-base/kernel/sys_ni.c 2020-02-19 14:32:34.000000000 +0100
+++ linux/kernel/sys_ni.c 2020-02-19 10:26:50.000000000 +0100
@@ -259,3 +259,6 @@
cond_syscall(sys_pkey_mprotect);
cond_syscall(sys_pkey_alloc);
cond_syscall(sys_pkey_free);
+
+/* anker: vmcopy ~ defined in mm/mremap.c */
+cond_syscall(sys_vmcopy);
diff -uNrw -uNr linux-base/mm/Kconfig linux/mm/Kconfig
--- linux-base/mm/Kconfig 2020-02-19 14:32:34.000000000 +0100
+++ linux/mm/Kconfig 2020-02-19 10:26:50.000000000 +0100
@@ -422,6 +422,13 @@
benefit.
endchoice
+config ANKER_VMCOPY
+ bool "vmcopy() support"
+ depends on TRANSPARENT_HUGEPAGE
+ default n
+ help
+ Use kernel based virtual memory copy support.
+
config ARCH_WANTS_THP_SWAP
def_bool n
diff -uNrw -uNr linux-base/mm/huge_memory.c linux/mm/huge_memory.c
--- linux-base/mm/huge_memory.c 2020-02-19 14:32:34.000000000 +0100
+++ linux/mm/huge_memory.c 2020-02-19 10:26:50.000000000 +0100
@@ -1836,6 +1836,77 @@
return false;
}
+bool anker_copy_huge_pmd(struct vm_area_struct *vma, struct vm_area_struct *new_vma,
+ unsigned long old_addr, unsigned long new_addr, unsigned long old_end,
+ pmd_t *old_pmd, pmd_t *new_pmd, bool *need_flush)
+{
+ spinlock_t *old_ptl, *new_ptl;
+ pmd_t pmd;
+ struct page *src_page;
+ struct mm_struct *mm = vma->vm_mm;
+ bool force_flush = false;
+
+ if ((old_addr & ~HPAGE_PMD_MASK) ||
+ (new_addr & ~HPAGE_PMD_MASK) ||
+ old_end - old_addr < HPAGE_PMD_SIZE)
+ return false;
+
+ /*
+ * The destination pmd shouldn't be established, free_pgtables()
+ * should have release it.
+ */
+ if (WARN_ON(!pmd_none(*new_pmd))) {
+ VM_BUG_ON(pmd_trans_huge(*new_pmd));
+ return false;
+ }
+
+ /*
+ * We don't have to worry about the ordering of src and dst
+ * ptlocks because exclusive mmap_sem prevents deadlock.
+ */
+ old_ptl = __pmd_trans_huge_lock(old_pmd, vma);
+ if (old_ptl) {
+ new_ptl = pmd_lockptr(mm, new_pmd);
+ if (new_ptl != old_ptl)
+ spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING);
+ pmd = *old_pmd;
+ if (pmd_present(pmd) && pmd_dirty(pmd))
+ force_flush = true;
+ VM_BUG_ON(!pmd_none(*new_pmd));
+
+ if (is_cow_mapping(vma->vm_flags)) {
+ pmdp_set_wrprotect(mm, old_addr, old_pmd);
+ pmd = pmd_wrprotect(pmd);
+ }
+
+ if (vma->vm_flags & VM_SHARED)
+ pmd = pmd_mkclean(pmd);
+
+ pmd = pmd_mkold(pmd);
+
+ src_page = pmd_page(pmd);
+
+ if (src_page) {
+ get_page(src_page);
+ page_add_anon_rmap(src_page, new_vma, new_addr, true);
+ }
+
+ // add_mm_counter(mm, MM_ANONPAGES, HPAGE_PMD_NR);
+ set_pmd_at(mm, new_addr, new_pmd, pmd);
+
+ if (new_ptl != old_ptl)
+ spin_unlock(new_ptl);
+ if (force_flush)
+ flush_tlb_range(vma, old_addr, old_addr + PMD_SIZE);
+ else
+ *need_flush = true;
+ spin_unlock(old_ptl);
+ return true;
+ }
+ return false;
+}
+
+
/*
* Returns
* - 0 if PMD could not be locked
diff -uNrw -uNr linux-base/mm/mremap.c linux/mm/mremap.c
--- linux-base/mm/mremap.c 2020-02-19 14:32:34.000000000 +0100
+++ linux/mm/mremap.c 2020-02-19 10:26:50.000000000 +0100
@@ -643,3 +643,235 @@
userfaultfd_unmap_complete(mm, &uf_unmap);
return ret;
}
+
+/*
+ * AnKer: copy ptes to new virtual address.
+ */
+static void anker_copy_ptes(struct vm_area_struct *vma, pmd_t *old_pmd,
+ unsigned long old_addr, unsigned long old_end,
+ struct vm_area_struct *new_vma, pmd_t *new_pmd,
+ unsigned long new_addr, bool need_rmap_locks, bool *need_flush)
+{
+ struct mm_struct *mm = vma->vm_mm;
+ pte_t *old_pte, *new_pte, pte;
+ spinlock_t *old_ptl, *new_ptl;
+ bool force_flush = false;
+ struct page *page;
+ unsigned long len = old_end - old_addr;
+
+ if (need_rmap_locks)
+ take_rmap_locks(vma);
+
+ /*
+ * We don't have to worry about the ordering of src and dst
+ * pte locks because exclusive mmap_sem prevents deadlock.
+ */
+ old_pte = pte_offset_map_lock(mm, old_pmd, old_addr, &old_ptl);
+ new_pte = pte_offset_map(new_pmd, new_addr);
+ new_ptl = pte_lockptr(mm, new_pmd);
+
+ if (new_ptl != old_ptl)
+ spin_lock_nested(new_ptl, SINGLE_DEPTH_NESTING);
+
+ flush_tlb_batched_pending(vma->vm_mm);
+ arch_enter_lazy_mmu_mode();
+
+ for (; old_addr < old_end; old_pte++, old_addr += PAGE_SIZE,
+ new_pte++, new_addr += PAGE_SIZE) {
+ if (pte_none(*old_pte))
+ continue;
+
+ pte = *old_pte;
+
+ /*
+ * If we are remapping a dirty PTE, make sure
+ * to flush TLB before we drop the PTL for the
+ * old PTE or we may race with page_mkclean().
+ *
+ * This check has to be done after we removed the
+ * old PTE from page tables or another thread may
+ * dirty it after the check and before the removal.
+ */
+
+ if (pte_present(pte) && pte_dirty(pte))
+ force_flush = true;
+
+ /*
+ * If it's a COW mapping, write protect both parent and child
+ *
+ */
+ if (is_cow_mapping(vma->vm_flags)) {
+ ptep_set_wrprotect(mm, old_addr, old_pte);
+ pte = pte_wrprotect(pte);
+ }
+
+ /*
+ * If its a shared mapping, mark it clean in the child
+ */
+ if (vma->vm_flags & VM_SHARED)
+ pte = pte_mkclean(pte);
+
+ pte = pte_mkold(pte);
+
+ page = vm_normal_page(vma, old_addr, pte);
+
+ if (page) {
+ get_page(page);
+ inc_mm_counter(mm, mm_counter(page));
+ if (vma_is_anonymous(new_vma))
+ page_add_anon_rmap(page, new_vma, new_addr, false);
+ else
+ page_add_file_rmap(page, false);
+ }
+
+ // set the pte at new address
+ set_pte_at(mm, new_addr, new_pte, pte);
+ }
+
+ arch_leave_lazy_mmu_mode();
+
+ if (new_ptl != old_ptl)
+ spin_unlock(new_ptl);
+ pte_unmap(new_pte - 1);
+
+ if (force_flush)
+ flush_tlb_range(vma, old_end - len, old_end);
+ else
+ *need_flush = true;
+
+ pte_unmap_unlock(old_pte - 1, old_ptl);
+
+ if (need_rmap_locks)
+ drop_rmap_locks(vma);
+}
+
+/*
+ * AnKer: copy the page tables backing the given VMA
+ */
+unsigned long anker_copy_page_tables(struct vm_area_struct *vma,
+ struct vm_area_struct *new_vma, unsigned long old_addr,
+ unsigned long new_addr, unsigned long len, bool need_rmap_locks)
+{
+ unsigned long extent, next, old_end;
+ pmd_t *old_pmd, *new_pmd;
+ bool need_flush = false;
+ unsigned long mmun_start; /* For mmu_notifiers */
+ unsigned long mmun_end; /* For mmu_notifiers */
+
+ old_end = old_addr + len;
+ flush_cache_range(vma, old_addr, old_end);
+
+ mmun_start = old_addr;
+ mmun_end = old_end;
+ mmu_notifier_invalidate_range_start(vma->vm_mm, mmun_start, mmun_end);
+
+ for (; old_addr < old_end; old_addr += extent, new_addr += extent) {
+ cond_resched();
+
+ next = (old_addr + PMD_SIZE) & PMD_MASK;
+
+ /* even if next overflowed, extent below will be ok */
+ extent = next - old_addr;
+
+ if (extent > old_end - old_addr)
+ extent = old_end - old_addr;
+
+ old_pmd = get_old_pmd(vma->vm_mm, old_addr);
+
+ if (!old_pmd)
+ continue;
+
+ new_pmd = alloc_new_pmd(vma->vm_mm, vma, new_addr);
+
+ if (!new_pmd)
+ break;
+
+ if (is_swap_pmd(*old_pmd) || pmd_trans_huge(*old_pmd)) {
+ if (extent == HPAGE_PMD_SIZE) {
+ bool moved = false;
+ /* See comment in move_ptes() */
+ if (need_rmap_locks)
+ take_rmap_locks(vma);
+
+ moved = anker_copy_huge_pmd(vma, new_vma, old_addr, new_addr,
+ old_end, old_pmd, new_pmd, &need_flush);
+
+ if (need_rmap_locks)
+ drop_rmap_locks(vma);
+
+ if (moved)
+ continue;
+ }
+
+ split_huge_pmd(vma, old_pmd, old_addr);
+
+ if (pmd_trans_unstable(old_pmd))
+ continue;
+ }
+
+ if (pte_alloc(new_vma->vm_mm, new_pmd, new_addr))
+ break;
+
+ next = (new_addr + PMD_SIZE) & PMD_MASK;
+
+ if (extent > next - new_addr)
+ extent = next - new_addr;
+
+ if (extent > LATENCY_LIMIT)
+ extent = LATENCY_LIMIT;
+
+ anker_copy_ptes(vma, old_pmd, old_addr, old_addr + extent, new_vma,
+ new_pmd, new_addr, need_rmap_locks, &need_flush);
+ }
+
+ if (need_flush)
+ flush_tlb_range(vma, old_end-len, old_addr);
+
+ mmu_notifier_invalidate_range_end(vma->vm_mm, mmun_start, mmun_end);
+
+ return len + old_addr - old_end; /* how much done */
+}
+
+/*
+ * AnKer: copy all VMAs backing the given virtual memory area.
+ * */
+unsigned long anker_copy_vma(struct vm_area_struct *vma,
+ unsigned long old_addr, unsigned long new_addr, unsigned long len,
+ bool *locked, struct vm_userfaultfd_ctx *uf, struct list_head *uf_unmap)
+{
+ struct mm_struct *mm = vma->vm_mm;
+ struct vm_area_struct *new_vma;
+ unsigned long vm_flags = vma->vm_flags;
+ unsigned long new_pgoff;
+ unsigned long moved_len;
+ bool need_rmap_locks;
+
+ /*
+ * We'd prefer to avoid failure later on in do_munmap:
+ * which may split one vma into three before unmapping.
+ */
+ if (mm->map_count >= sysctl_max_map_count - 3)
+ return -ENOMEM;
+
+ new_pgoff = vma->vm_pgoff + ((old_addr - vma->vm_start) >> PAGE_SHIFT);
+ new_vma = copy_vma(&vma, new_addr, len, new_pgoff, &need_rmap_locks);
+
+ if (!new_vma)
+ return -ENOMEM;
+
+ moved_len = anker_copy_page_tables(vma, new_vma, old_addr, new_addr,
+ len, need_rmap_locks);
+
+ if (moved_len < len) {
+ return -ENOMEM;
+ }
+
+ vm_stat_account(mm, vma->vm_flags, len >> PAGE_SHIFT);
+
+ if (vm_flags & VM_LOCKED) {
+ mm->locked_vm += len >> PAGE_SHIFT;
+ *locked = true;
+ }
+
+ return new_addr;
+}
diff -uNrw -uNr linux-base/rapido/COPYING linux/rapido/COPYING
--- linux-base/rapido/COPYING 1970-01-01 01:00:00.000000000 +0100
+++ linux/rapido/COPYING 2020-02-19 10:00:19.000000000 +0100
@@ -0,0 +1,674 @@
+----------------------------------------------------------------------
+
+ GNU LESSER GENERAL PUBLIC LICENSE
+ Version 2.1, February 1999
+
+ Copyright (C) 1991, 1999 Free Software Foundation, Inc.
+ 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+[This is the first released version of the Lesser GPL. It also counts
+ as the successor of the GNU Library Public License, version 2, hence
+ the version number 2.1.]
+
+ Preamble
+
+ The licenses for most software are designed to take away your
+freedom to share and change it. By contrast, the GNU General Public
+Licenses are intended to guarantee your freedom to share and change
+free software--to make sure the software is free for all its users.
+
+ This license, the Lesser General Public License, applies to some
+specially designated software packages--typically libraries--of the
+Free Software Foundation and other authors who decide to use it. You
+can use it too, but we suggest you first think carefully about whether
+this license or the ordinary General Public License is the better
+strategy to use in any particular case, based on the explanations below.
+
+ When we speak of free software, we are referring to freedom of use,
+not price. Our General Public Licenses are designed to make sure that
+you have the freedom to distribute copies of free software (and charge
+for this service if you wish); that you receive source code or can get
+it if you want it; that you can change the software and use pieces of
+it in new free programs; and that you are informed that you can do
+these things.
+
+ To protect your rights, we need to make restrictions that forbid
+distributors to deny you these rights or to ask you to surrender these
+rights. These restrictions translate to certain responsibilities for
+you if you distribute copies of the library or if you modify it.
+
+ For example, if you distribute copies of the library, whether gratis
+or for a fee, you must give the recipients all the rights that we gave
+you. You must make sure that they, too, receive or can get the source
+code. If you link other code with the library, you must provide
+complete object files to the recipients, so that they can relink them
+with the library after making changes to the library and recompiling
+it. And you must show them these terms so they know their rights.
+
+ We protect your rights with a two-step method: (1) we copyright the
+library, and (2) we offer you this license, which gives you legal
+permission to copy, distribute and/or modify the library.
+
+ To protect each distributor, we want to make it very clear that
+there is no warranty for the free library. Also, if the library is
+modified by someone else and passed on, the recipients should know
+that what they have is not the original version, so that the original
+author's reputation will not be affected by problems that might be
+introduced by others.
+
+ Finally, software patents pose a constant threat to the existence of
+any free program. We wish to make sure that a company cannot
+effectively restrict the users of a free program by obtaining a
+restrictive license from a patent holder. Therefore, we insist that
+any patent license obtained for a version of the library must be
+consistent with the full freedom of use specified in this license.
+
+ Most GNU software, including some libraries, is covered by the
+ordinary GNU General Public License. This license, the GNU Lesser
+General Public License, applies to certain designated libraries, and
+is quite different from the ordinary General Public License. We use
+this license for certain libraries in order to permit linking those
+libraries into non-free programs.
+
+ When a program is linked with a library, whether statically or using
+a shared library, the combination of the two is legally speaking a
+combined work, a derivative of the original library. The ordinary
+General Public License therefore permits such linking only if the
+entire combination fits its criteria of freedom. The Lesser General
+Public License permits more lax criteria for linking other code with
+the library.
+
+ We call this license the "Lesser" General Public License because it
+does Less to protect the user's freedom than the ordinary General
+Public License. It also provides other free software developers Less
+of an advantage over competing non-free programs. These disadvantages
+are the reason we use the ordinary General Public License for many
+libraries. However, the Lesser license provides advantages in certain
+special circumstances.
+
+ For example, on rare occasions, there may be a special need to
+encourage the widest possible use of a certain library, so that it becomes
+a de-facto standard. To achieve this, non-free programs must be
+allowed to use the library. A more frequent case is that a free
+library does the same job as widely used non-free libraries. In this
+case, there is little to gain by limiting the free library to free
+software only, so we use the Lesser General Public License.
+
+ In other cases, permission to use a particular library in non-free
+programs enables a greater number of people to use a large body of
+free software. For example, permission to use the GNU C Library in
+non-free programs enables many more people to use the whole GNU
+operating system, as well as its variant, the GNU/Linux operating
+system.
+
+ Although the Lesser General Public License is Less protective of the
+users' freedom, it does ensure that the user of a program that is
+linked with the Library has the freedom and the wherewithal to run
+that program using a modified version of the Library.
+
+ The precise terms and conditions for copying, distribution and
+modification follow. Pay close attention to the difference between a
+"work based on the library" and a "work that uses the library". The
+former contains code derived from the library, whereas the latter must
+be combined with the library in order to run.
+
+ GNU LESSER GENERAL PUBLIC LICENSE
+ TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+ 0. This License Agreement applies to any software library or other
+program which contains a notice placed by the copyright holder or
+other authorized party saying it may be distributed under the terms of
+this Lesser General Public License (also called "this License").
+Each licensee is addressed as "you".
+
+ A "library" means a collection of software functions and/or data
+prepared so as to be conveniently linked with application programs
+(which use some of those functions and data) to form executables.
+
+ The "Library", below, refers to any such software library or work
+which has been distributed under these terms. A "work based on the
+Library" means either the Library or any derivative work under
+copyright law: that is to say, a work containing the Library or a
+portion of it, either verbatim or with modifications and/or translated
+straightforwardly into another language. (Hereinafter, translation is
+included without limitation in the term "modification".)
+
+ "Source code" for a work means the preferred form of the work for
+making modifications to it. For a library, complete source code means
+all the source code for all modules it contains, plus any associated
+interface definition files, plus the scripts used to control compilation
+and installation of the library.
+
+ Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope. The act of
+running a program using the Library is not restricted, and output from
+such a program is covered only if its contents constitute a work based
+on the Library (independent of the use of the Library in a tool for
+writing it). Whether that is true depends on what the Library does
+and what the program that uses the Library does.
+
+ 1. You may copy and distribute verbatim copies of the Library's
+complete source code as you receive it, in any medium, provided that
+you conspicuously and appropriately publish on each copy an
+appropriate copyright notice and disclaimer of warranty; keep intact
+all the notices that refer to this License and to the absence of any
+warranty; and distribute a copy of this License along with the
+Library.
+
+ You may charge a fee for the physical act of transferring a copy,
+and you may at your option offer warranty protection in exchange for a
+fee.
+
+ 2. You may modify your copy or copies of the Library or any portion
+of it, thus forming a work based on the Library, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+ a) The modified work must itself be a software library.
+
+ b) You must cause the files modified to carry prominent notices
+ stating that you changed the files and the date of any change.
+
+ c) You must cause the whole of the work to be licensed at no
+ charge to all third parties under the terms of this License.
+
+ d) If a facility in the modified Library refers to a function or a
+ table of data to be supplied by an application program that uses
+ the facility, other than as an argument passed when the facility
+ is invoked, then you must make a good faith effort to ensure that,
+ in the event an application does not supply such function or
+ table, the facility still operates, and performs whatever part of
+ its purpose remains meaningful.
+
+ (For example, a function in a library to compute square roots has
+ a purpose that is entirely well-defined independent of the
+ application. Therefore, Subsection 2d requires that any
+ application-supplied function or table used by this function must
+ be optional: if the application does not supply it, the square
+ root function must still compute square roots.)
+
+These requirements apply to the modified work as a whole. If
+identifiable sections of that work are not derived from the Library,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works. But when you
+distribute the same sections as part of a whole which is a work based
+on the Library, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote
+it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Library.
+
+In addition, mere aggregation of another work not based on the Library
+with the Library (or with a work based on the Library) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+ 3. You may opt to apply the terms of the ordinary GNU General Public
+License instead of this License to a given copy of the Library. To do
+this, you must alter all the notices that refer to this License, so
+that they refer to the ordinary GNU General Public License, version 2,
+instead of to this License. (If a newer version than version 2 of the
+ordinary GNU General Public License has appeared, then you can specify
+that version instead if you wish.) Do not make any other change in
+these notices.
+
+ Once this change is made in a given copy, it is irreversible for
+that copy, so the ordinary GNU General Public License applies to all
+subsequent copies and derivative works made from that copy.
+
+ This option is useful when you wish to copy part of the code of
+the Library into a program that is not a library.
+
+ 4. You may copy and distribute the Library (or a portion or
+derivative of it, under Section 2) in object code or executable form
+under the terms of Sections 1 and 2 above provided that you accompany
+it with the complete corresponding machine-readable source code, which
+must be distributed under the terms of Sections 1 and 2 above on a
+medium customarily used for software interchange.
+
+ If distribution of object code is made by offering access to copy
+from a designated place, then offering equivalent access to copy the
+source code from the same place satisfies the requirement to
+distribute the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+ 5. A program that contains no derivative of any portion of the
+Library, but is designed to work with the Library by being compiled or
+linked with it, is called a "work that uses the Library". Such a
+work, in isolation, is not a derivative work of the Library, and
+therefore falls outside the scope of this License.
+
+ However, linking a "work that uses the Library" with the Library
+creates an executable that is a derivative of the Library (because it
+contains portions of the Library), rather than a "work that uses the
+library". The executable is therefore covered by this License.
+Section 6 states terms for distribution of such executables.
+
+ When a "work that uses the Library" uses material from a header file
+that is part of the Library, the object code for the work may be a
+derivative work of the Library even though the source code is not.
+Whether this is true is especially significant if the work can be
+linked without the Library, or if the work is itself a library. The
+threshold for this to be true is not precisely defined by law.
+
+ If such an object file uses only numerical parameters, data
+structure layouts and accessors, and small macros and small inline
+functions (ten lines or less in length), then the use of the object
+file is unrestricted, regardless of whether it is legally a derivative
+work. (Executables containing this object code plus portions of the
+Library will still fall under Section 6.)
+
+ Otherwise, if the work is a derivative of the Library, you may
+distribute the object code for the work under the terms of Section 6.
+Any executables containing that work also fall under Section 6,
+whether or not they are linked directly with the Library itself.
+
+ 6. As an exception to the Sections above, you may also combine or
+link a "work that uses the Library" with the Library to produce a
+work containing portions of the Library, and distribute that work
+under terms of your choice, provided that the terms permit
+modification of the work for the customer's own use and reverse
+engineering for debugging such modifications.
+
+ You must give prominent notice with each copy of the work that the
+Library is used in it and that the Library and its use are covered by
+this License. You must supply a copy of this License. If the work
+during execution displays copyright notices, you must include the
+copyright notice for the Library among them, as well as a reference
+directing the user to the copy of this License. Also, you must do one
+of these things:
+
+ a) Accompany the work with the complete corresponding
+ machine-readable source code for the Library including whatever
+ changes were used in the work (which must be distributed under
+ Sections 1 and 2 above); and, if the work is an executable linked
+ with the Library, with the complete machine-readable "work that
+ uses the Library", as object code and/or source code, so that the
+ user can modify the Library and then relink to produce a modified
+ executable containing the modified Library. (It is understood
+ that the user who changes the contents of definitions files in the
+ Library will not necessarily be able to recompile the application
+ to use the modified definitions.)
+
+ b) Use a suitable shared library mechanism for linking with the
+ Library. A suitable mechanism is one that (1) uses at run time a
+ copy of the library already present on the user's computer system,
+ rather than copying library functions into the executable, and (2)
+ will operate properly with a modified version of the library, if
+ the user installs one, as long as the modified version is
+ interface-compatible with the version that the work was made with.
+
+ c) Accompany the work with a written offer, valid for at
+ least three years, to give the same user the materials
+ specified in Subsection 6a, above, for a charge no more
+ than the cost of performing this distribution.
+
+ d) If distribution of the work is made by offering access to copy
+ from a designated place, offer equivalent access to copy the above
+ specified materials from the same place.
+
+ e) Verify that the user has already received a copy of these
+ materials or that you have already sent this user a copy.
+
+ For an executable, the required form of the "work that uses the
+Library" must include any data and utility programs needed for
+reproducing the executable from it. However, as a special exception,
+the materials to be distributed need not include anything that is
+normally distributed (in either source or binary form) with the major
+components (compiler, kernel, and so on) of the operating system on
+which the executable runs, unless that component itself accompanies
+the executable.
+
+ It may happen that this requirement contradicts the license
+restrictions of other proprietary libraries that do not normally
+accompany the operating system. Such a contradiction means you cannot
+use both them and the Library together in an executable that you
+distribute.
+
+ 7. You may place library facilities that are a work based on the
+Library side-by-side in a single library together with other library
+facilities not covered by this License, and distribute such a combined
+library, provided that the separate distribution of the work based on
+the Library and of the other library facilities is otherwise
+permitted, and provided that you do these two things:
+
+ a) Accompany the combined library with a copy of the same work
+ based on the Library, uncombined with any other library
+ facilities. This must be distributed under the terms of the
+ Sections above.
+
+ b) Give prominent notice with the combined library of the fact
+ that part of it is a work based on the Library, and explaining
+ where to find the accompanying uncombined form of the same work.
+
+ 8. You may not copy, modify, sublicense, link with, or distribute
+the Library except as expressly provided under this License. Any
+attempt otherwise to copy, modify, sublicense, link with, or
+distribute the Library is void, and will automatically terminate your
+rights under this License. However, parties who have received copies,
+or rights, from you under this License will not have their licenses
+terminated so long as such parties remain in full compliance.
+
+ 9. You are not required to accept this License, since you have not
+signed it. However, nothing else grants you permission to modify or
+distribute the Library or its derivative works. These actions are
+prohibited by law if you do not accept this License. Therefore, by
+modifying or distributing the Library (or any work based on the
+Library), you indicate your acceptance of this License to do so, and