Skip to content

Meeting Notes Rolling Log

Jump to bottom
George Karlos edited this page Apr 27, 2021 · 26 revisions

21/4/2021

What we have done:

  • Matthijs: Paper progress is going well. KubeEdge: A new bug appeared, I am waiting for the devs to answer a github issue.
  • Giulia: Worked on Fogify, will do a presentation and demo (we were out of time, so the demo will be in a next session).
  • George: Exhausted the literature on in-network aggregation. Deployed DAIET (Sapio'17) artifacts but only the simple tuple-sending example. Issues with the MapReduce one (hope to resolve soon).

Notes:

  • KubeEdge: Some new problems arose, so there is delay in successfully deploying it.
  • Fogify: Uses Docker Swarm to manage Docker containers, with each Docker container being a cloud node, an edge node or an IoT node.
  • An idea of how to combine of KubeEdge and Fogify: Use fogify to simulate an edge environment (we don't have real Raspberry Pi's and 5G networks), and run KubeEdge on top of that. Problem: Kubernetes can't run in a Docker container, and Fogify is based on docker containers so that seems difficult to combine.
  • https://edge-net.org/index.html

Planning:

  • Matthijs: Continue working on KubeEdge. Read on how KubeEdge works: What are the scheduling policies, docker overhead etc. Present this.
  • Giulia: Continue with Fogify. Try to prepare more complex demo, cassandra deployment on fogify and bare docker containers on fogify. How to combine KubeEdge and Fogify.
  • George: Understand the P4 programs of the more advanced INC papers (e.g. SwitchML). Try to justify arbitrary aggregation functions on the switch - Do we only care about addition (gradients all-reduce)? +Outcome of meeing with Lin.
  • Read EdgeSys (and ICPE) papers and search for interesting papers

EuroSys papers:

EdgeSys papers:

  • .

ICPE papers:

7/4/2021

What we have done:

  • Matthijs: Started working with KubeEdge, but it is quite complicated to deploy it, this needs more time.
  • Giulia: Worked on deploying Fogify, is still working on it.
  • George: Learning p4, reading on in-network aggregation

Notes:

  • KubeEdge: See https://kubeedge.io/en/docs/kubeedge/ for the KubeEdge architecture. Main idea:
    • 1 central controller in the cloud, "cloudcore", that manages all edge nodes, "edgecore"
    • 1 controller per edge site, "edgecore", that runs an kubernetes deployment (so it can do RM&S all nodes in that edge site)
    • There is no direct communication between edgecores, that has to go through the cloudcore
    • There are still many questions: What is the exact interaction between cloud and edges? What kind of hardware can edgecores run on?
  • Fogify: It seems like you can only deploy 1 application per edge node. You can't model an edge node that runs multiple applications at the same time.

Planning:

  • Matthijs: Deploy KubeEdge with an example application. Give a short presentation on how KubeEdge works.
    • Run it on the cluster we have running at the VU, in containers.
  • Giulia: Deploy Fogify. Email the authors if needed. Give a short presentation on Fogify.
  • George: Look at in-network aggregation. How does it work, maybe deploy a framework that does this.

24/3/2021

What we have done:

  • Matthijs: Worked on the edge resource management reference architecture paper, mainly on the reading list and a visualization of the reference architecture.
  • Giulia: Took some time off in the last two weeks. Deployed Cassandra. Paper reading.
  • George: Mainly reading papers, gathering more knowledge on edge computing and related fields. Focused on papers about orbital edge computing, soft container loading / migration, programmable networks.

General notes:

  • We need to get an infrastructure up for doing experiments, this should be a priority for everyone. For now we will look into kubeedge and Fogify as a first step, afterwards we can look into deploying applications and further customizing the setup.

Planning:

  • Matthijs: Deploy kubeedge, and look into Fogify (mainly how these two systems can cooperate). Continue working on the reference architecture paper, rewrite the introduction and background, and make the contributions more concrete.
  • Giulia: Deploy Fogify, run YCSB on Cassandra. How can Fogify be used to simulate network traffic? Write a C++ benchmark against Cassandra API and understand what is happening. Understand YCSB different workload classes.
    • Giulia: get familiar with Cassandra, based on the paper draft for NSDI understand which parts are relevant for a benchmark
    • Giulia - you have a pending question from our meeting: is edge more heterogeneous than the cloud? Collect evidence and data.
  • George: Come up with a concrete research goal. Some potential promising research fields include soft container loading, programmable networks, meta-programming and network modeling.

10/3/2021

What we have done:

  • Matthijs: Worked on edge resource management reference architecture paper
  • Matthijs / Giulia: EuroSys artifact evaluation
  • George: Exploring, reading, finding an interesting research direction

Notes:

  • We need to make an experimental setup for all edge papers, especially for the NSDI paper that we plan on writing. We can't make much progress in the next 2 weeks as Giulia is taking some time off and Matthijs is writing a paper
  • Frogify paper (https://ucy-linc-lab.github.io/fogify/) has an interesting and open source test setup
  • Current idea for the system: 3 layers (sensor, edge, cloud), model network traffic between these layers in terms of bandwidth, latency and jitter using tc. This is important to model different network topologies.

Planning:

  • Matthijs: Continue writing the reference architecture paper. Try to finish it before summer.
  • Giulia: Takes some time off.

24/2/2021

What we have done in the last three weeks:

  • George: Just started his PhD, so he is still exploring / reading papers
  • Giulia: Working on a edge / storage vision paper, submission is this Friday. Started with a simple application setup for the edge using Cassandra
  • Matthijs: Created a simple stateless machine learning pipeline for the edge. Started working on the reference architecture for resource management at the edge paper

We need to create an infrastructure on which we can run our edge experiments:

  • Find a 'killer app' that all 3 phd students can use (Something with a shared state like an online game?)
  • Simulate real world network delay between endpoints using tc
    • Need some trace or data about exact behaviour of real network traffic
    • This is probably difficult to find for 5g, so focus on doing this for 4g for the time being
  • Create Usenix and ACM latex templates for offline latex compilation, and add them to github (instead of using overleaf)

Notes on the current edge application infrastructure

  • Giulia: Try to avoid using epbf unless it is really needed for the next paper
  • Matthijs: Try using either firecracker or kubeedge as a resource management / scheduling framework for the edge. These frameworks already have a lot of functionalities that you want.

To do for next meeting (10/3/2021):

  • George: Determine what to do for a first paper
  • Giulia: Finish writing the edge vision paper and submit it. Continue with setting up an infrastructure for stateful edge applications using Cassandra
  • Matthijs: Continue working on the reference architecture paper, finish the introduction / background and create a selection of resource managers to focus on.

27/1/2021

Applications for the edge:

  • Machine learning (object detection, face recognition etc)
  • Federated learning
  • Simple linear algebra kernels / benchmarks
  • More: Stream analysis, video analytics, genome sequencing, online gaming

What hardware / infrastructure is needed:

  • Raspberry Pi as a general purpose edge node
  • Need to simulate network traffic to the DAS (4g/5g simulation)
  • Need to throttle the Raspberry Pi, to simulate slower IoT hardware
  • How to accurately measure energy usage of these devices?

Tools:

  • Important to invest in tools for evaluation of all future edge papers
  • Can be anything: Network (traffic shaping), infrastructure, applications, emulators etc.
  • Make a list on github with the state-of-the-practice on edge computing
  • Scan evaluation section of edge computing paper

Paper ideas:

  • Benchmarking at the edge
    • Spec group worked on creating such a benchmark, but it was dropped due to there not being enough interested people
    • Do work in our current group for a few months, then maybe organize a workshop to bring in more people to this research area
    • Take a look at cloud benchmarks: How can we transfer it to the edge. Holistic overview: Only reporting energy usage or performance won't help, need to combine metrics to create a clear picture.
  • Reference architecture for edge RM&S (Matthijs)
  • Evaluating consistency protocols based on latency of used technology (Giulia)

Next meeting:

  • Pick an edge application, research what kind of infrastructure / tools are needed to deploy this application, and deploy it. What part of it is edge specific compared to normal cloud solutions?
    • Machine learning, stateless (Matthijs)
    • Federated learning, self-driving cars, stateful (Giulia)
  • Paper plan: What is the story, give a small pitch (+ write a paper introduction)

21/1/2021

  • Finish github wiki setup
  • Paper planning for 2021

13/1/2021

  • What kind of paper do you want to write this year? Which venue?
  • Setup Github organization + logo

Next meeting:

  • What applications do we want to focus on?
  • What hardware / infrastructure do we need?
Clone this wiki locally