examples/node-regression-with-subgraph-and-graph-sample.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "69f4a041",
   "metadata": {},
   "source": [
    "# Node Regression with Subgraph and Graph Sample projections"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "958fd77b",
   "metadata": {
    "colab_type": "text"
   },
   "source": [
    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/neo4j/graph-data-science-client/blob/main/examples/node-regression-with-subgraph-and-graph-sample.ipynb\">\n",
    "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
    "</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba81764b",
   "metadata": {},
   "source": [
    "This Jupyter notebook is hosted [here](https://github.com/neo4j/graph-data-science-client/blob/main/examples/node-regression-with-subgraph-and-graph-sample.ipynb) in the Neo4j Graph Data Science Client Github repository.\n",
    "\n",
    "For a video presentation of a former version of this notebook, see the talk [Fundamentals of Neo4j Graph Data Science Series 2.x – Pipelines and More](https://youtu.be/7hx56qtf80Q?t=1759) that was given at the NODES 2022 conference.\n",
    "\n",
    "The notebook exemplifies using a Node Regression pipeline.\n",
    "It also contains many examples of using\n",
    "\n",
    "- Convenience objects\n",
    "- Subgraph projection\n",
    "- Graph sample projection\n",
    "\n",
    "It is written in pure Python, to showcase the GDS Python Client's ability to abstract away from Cypher queries."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f55decc2",
   "metadata": {},
   "source": [
    "## The dataset\n",
    "\n",
    "Our input graph represents Wikipedia pages on particular topics, and how they link to each other:\n",
    "\n",
    "- Chameleons\n",
    "- Squirrels\n",
    "- Crocodiles\n",
    "\n",
    "The features are presences of certain informative nouns in the text of the page.\n",
    "The target is the average monthly traffic of the page.\n",
    "\n",
    "The dataset was first published in _Multi-scale Attributed Node Embedding_ by B. Rozemberczki, C. Allen and R. Sarkar, [eprint 1909.13021](https://arxiv.org/abs/1909.13021).\n",
    "The version hosted here was taken from [SNAP](https://snap.stanford.edu/data/wikipedia-article-networks.html) on 2022-11-14."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c69ea6d",
   "metadata": {},
   "source": [
    "## Pre-requisites\n",
    "\n",
    "In order to run this pipeline, you must have a running Neo4j DBMS with a recent version of the Neo4j Graph Data Science plugin installed.\n",
    "These requirements are satisfied if you have an AuraDS instance active and running."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19e3a37d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# First, we must install the GDS Python Client\n",
    "%pip install graphdatascience"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a939971e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Then, we connect to our Neo4j DBMS hosting the Graph Data Science library\n",
    "from graphdatascience import GraphDataScience\n",
    "\n",
    "# Get Neo4j DB URI, credentials and name from environment if applicable\n",
    "NEO4J_URI = os.environ.get(\"NEO4J_URI\", \"bolt://localhost:7687\")\n",
    "NEO4J_AUTH = None\n",
    "NEO4J_DB = os.environ.get(\"NEO4J_DB\", \"neo4j\")\n",
    "if os.environ.get(\"NEO4J_USER\") and os.environ.get(\"NEO4J_PASSWORD\"):\n",
    "    NEO4J_AUTH = (\n",
    "        os.environ.get(\"NEO4J_USER\"),\n",
    "        os.environ.get(\"NEO4J_PASSWORD\"),\n",
    "    )\n",
    "gds = GraphDataScience(NEO4J_URI, auth=NEO4J_AUTH, database=NEO4J_DB)\n",
    "\n",
    "# Test our connection and print the Graph Data Science library version\n",
    "print(gds.server_version())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "df96d470",
   "metadata": {
    "tags": [
     "verify-version"
    ]
   },
   "outputs": [],
   "source": [
    "from graphdatascience.server_version.server_version import ServerVersion\n",
    "\n",
    "assert gds.server_version() >= ServerVersion(2, 1, 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e94d9704",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Importing the dataset\n",
    "\n",
    "# The dataset is sourced from this GitHub repository\n",
    "baseUrl = (\n",
    "    \"https://raw.githubusercontent.com/neo4j/graph-data-science-client/main/examples/datasets/wikipedia-animals-pages\"\n",
    ")\n",
    "\n",
    "# Constraints to speed up importing\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    CREATE CONSTRAINT chameleons\n",
    "    FOR (c:Chameleon)\n",
    "    REQUIRE c.id IS NODE KEY\n",
    "\"\"\"\n",
    ")\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    CREATE CONSTRAINT crocodiles\n",
    "    FOR (c:Crocodile)\n",
    "    REQUIRE c.id IS NODE KEY\n",
    "\"\"\"\n",
    ")\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    CREATE CONSTRAINT squirrels\n",
    "    FOR (s:Squirrel)\n",
    "    REQUIRE s.id IS NODE KEY\n",
    "\"\"\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a746f07",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create nodes and relationships\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    LOAD CSV WITH HEADERS FROM $baseUrl + '/chameleon/musae_chameleon_edges.csv' AS row\n",
    "    MERGE (c1:Chameleon {id: row.id1})\n",
    "    MERGE (c2:Chameleon {id: row.id2})\n",
    "    MERGE (c1)-[:LINK]->(c2)\n",
    "\"\"\",\n",
    "    {\"baseUrl\": baseUrl},\n",
    ")\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    LOAD CSV WITH HEADERS FROM $baseUrl + '/crocodile/musae_crocodile_edges.csv' AS row\n",
    "    MERGE (c1:Crocodile {id: row.id1})\n",
    "    MERGE (c2:Crocodile {id: row.id2})\n",
    "    MERGE (c1)-[:LINK]->(c2)\n",
    "\"\"\",\n",
    "    {\"baseUrl\": baseUrl},\n",
    ")\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    LOAD CSV WITH HEADERS FROM $baseUrl + '/squirrel/musae_squirrel_edges.csv' AS row\n",
    "    MERGE (s1:Squirrel {id: row.id1})\n",
    "    MERGE (s2:Squirrel {id: row.id2})\n",
    "    MERGE (s1)-[:LINK]->(s2)\n",
    "\"\"\",\n",
    "    {\"baseUrl\": baseUrl},\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92340c47",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create target properties\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    LOAD CSV WITH HEADERS FROM $baseUrl + '/chameleon/musae_chameleon_target.csv' AS row\n",
    "    MATCH (c:Chameleon {id: row.id})\n",
    "    SET c.target = toInteger(row.target)\n",
    "\"\"\",\n",
    "    {\"baseUrl\": baseUrl},\n",
    ")\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    LOAD CSV WITH HEADERS FROM $baseUrl + '/crocodile/musae_crocodile_target.csv' AS row\n",
    "    MATCH (c:Crocodile {id: row.id})\n",
    "    SET c.target = toInteger(row.target)\n",
    "\"\"\",\n",
    "    {\"baseUrl\": baseUrl},\n",
    ")\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    LOAD CSV WITH HEADERS FROM $baseUrl + '/squirrel/musae_squirrel_target.csv' AS row\n",
    "    MATCH (s:Squirrel {id: row.id})\n",
    "    SET s.target = toInteger(row.target)\n",
    "\"\"\",\n",
    "    {\"baseUrl\": baseUrl},\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "021dd665",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create feature vectors\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    LOAD CSV WITH HEADERS FROM $baseUrl + '/chameleon/musae_chameleon_features.csv' AS row\n",
    "    MATCH (c:Chameleon {id: row.id})\n",
    "    WITH c, split(row.features, '|') AS features\n",
    "    SET c.features = features\n",
    "\"\"\",\n",
    "    {\"baseUrl\": baseUrl},\n",
    ")\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    LOAD CSV WITH HEADERS FROM $baseUrl + '/crocodile/musae_crocodile_features.csv' AS row\n",
    "    MATCH (c:Crocodile {id: row.id})\n",
    "    WITH c, split(row.features, '|') AS features\n",
    "    SET c.features = features\n",
    "\"\"\",\n",
    "    {\"baseUrl\": baseUrl},\n",
    ")\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    LOAD CSV WITH HEADERS FROM $baseUrl + '/squirrel/musae_squirrel_features.csv' AS row\n",
    "    MATCH (c:Squirrel {id: row.id})\n",
    "    WITH c, split(row.features, '|') AS features\n",
    "    SET c.features = features\n",
    "\"\"\",\n",
    "    {\"baseUrl\": baseUrl},\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86194e76",
   "metadata": {},
   "source": [
    "## Preparing the dataset for the pipeline\n",
    "\n",
    "In order to use the dataset, we must prepare the features in a format that the model supports and can work well with.\n",
    "In their raw form, the features are ids of particular words, and therefore are not suitable as input to linear regression.\n",
    "\n",
    "To overcome this, we will use a one-hot encoding.\n",
    "This will produce features that work well for linear regression.\n",
    "We begin by learning the dictionaries of nouns across the node sets.\n",
    "We create a node to host the dictionary, then we use it to one-hot encode all feature vectors.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6c6001be",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Construct one-hot dictionaries\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    MATCH (s:Chameleon)\n",
    "    WITH s.features AS features\n",
    "    UNWIND features AS feature\n",
    "    WITH feature\n",
    "      ORDER BY feature ASC\n",
    "    WITH collect(distinct feature) AS orderedTotality\n",
    "    CREATE (:Feature {animal: 'chameleon', totality: orderedTotality})\n",
    "    RETURN orderedTotality\n",
    "\"\"\"\n",
    ")\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    MATCH (s:Crocodile)\n",
    "    WITH s.features AS features\n",
    "    UNWIND features AS feature\n",
    "    WITH feature\n",
    "      ORDER BY feature ASC\n",
    "    WITH collect(distinct feature) AS orderedTotality\n",
    "    CREATE (:Feature {animal: 'crocodile', totality: orderedTotality})\n",
    "    RETURN orderedTotality\n",
    "\"\"\"\n",
    ")\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    MATCH (s:Squirrel)\n",
    "    WITH s.features AS features\n",
    "    UNWIND features AS feature\n",
    "    WITH feature\n",
    "      ORDER BY feature ASC\n",
    "    WITH collect(distinct feature) AS orderedTotality\n",
    "    CREATE (:Feature {animal: 'squirrel', totality: orderedTotality})\n",
    "    RETURN orderedTotality\n",
    "\"\"\"\n",
    ")\n",
    "\n",
    "# Do one-hot encoding\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    MATCH (f:Feature {animal: 'chameleon'})\n",
    "    MATCH (c:Chameleon)\n",
    "    SET c.features_one_hot = gds.alpha.ml.oneHotEncoding(f.totality, c.features)\n",
    "\"\"\"\n",
    ")\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    MATCH (f:Feature {animal: 'crocodile'})\n",
    "    MATCH (c:Crocodile)\n",
    "    SET c.features_one_hot = gds.alpha.ml.oneHotEncoding(f.totality, c.features)\n",
    "\"\"\"\n",
    ")\n",
    "gds.run_cypher(\n",
    "    \"\"\"\n",
    "    MATCH (f:Feature {animal: 'squirrel'})\n",
    "    MATCH (c:Squirrel)\n",
    "    SET c.features_one_hot = gds.alpha.ml.oneHotEncoding(f.totality, c.features)\n",
    "\"\"\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2b954555",
   "metadata": {},
   "outputs": [],
   "source": [
    "# First, let's project our graph into the GDS Graph Catalog\n",
    "# We will use a native projection to begin with\n",
    "G_animals, projection_result = gds.graph.project(\n",
    "    \"wiki_animals\",\n",
    "    [\"Chameleon\", \"Squirrel\", \"Crocodile\"],\n",
    "    {\"LINK\": {\"orientation\": \"UNDIRECTED\"}},\n",
    "    nodeProperties=[\"features_one_hot\", \"target\"],\n",
    ")\n",
    "print(projection_result[[\"graphName\", \"nodeCount\", \"relationshipCount\"]])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a923b13",
   "metadata": {},
   "source": [
    "## Connectivity\n",
    "\n",
    "In graph analysis, it is common to operate only over _connected_ graphs.\n",
    "That is, graphs that consist of only a single _component_.\n",
    "The reason for this is that in most cases, information does not flow where there are no connections.\n",
    "\n",
    "The fastest way to determine the number of components in our graph is to use the WCC (Weakly Connected Components) algorithm."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e4c1526a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We use the WCC algorithm to see how many components we have\n",
    "wcc_result = gds.wcc.mutate(G_animals, mutateProperty=\"wcc_component\")\n",
    "\n",
    "print(wcc_result[[\"computeMillis\", \"componentCount\"]])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8e3e454",
   "metadata": {},
   "source": [
    "## Component separation\n",
    "\n",
    "Learning that our graph consists of three components, we will next separate the components into separate graphs.\n",
    "We will use the `subgraph` projection to accomplish this.\n",
    "We will create one subgraph for each of the components."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "62d65285",
   "metadata": {},
   "outputs": [],
   "source": [
    "# First, we stream the component ids\n",
    "components = gds.graph.nodeProperty.stream(G_animals, \"wcc_component\")\n",
    "\n",
    "# Second, we compute the unique component ids\n",
    "component_ids = components[\"propertyValue\"].unique()\n",
    "\n",
    "# Third, we project a subgraph for each component\n",
    "component_graphs = [\n",
    "    gds.beta.graph.project.subgraph(\n",
    "        f\"animals_component_{component_id}\",\n",
    "        G_animals,\n",
    "        f\"n.wcc_component = {component_id}\",\n",
    "        \"*\",\n",
    "    )[0]\n",
    "    for component_id in component_ids\n",
    "]\n",
    "\n",
    "# Lastly, we map the node labels in the graphs to the graph\n",
    "graph_components_by_labels = {str(G_component.node_labels()): G_component for G_component in component_graphs}\n",
    "\n",
    "print({k: v.name() for k, v in graph_components_by_labels.items()})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cbd63cd3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now, we are only interested in the Chameleon graph,\n",
    "# so we will drop the other graphs and define a better variable for the one we keep\n",
    "graph_components_by_labels[str([\"Crocodile\"])].drop()\n",
    "graph_components_by_labels[str([\"Squirrel\"])].drop()\n",
    "G_chameleon = graph_components_by_labels[str([\"Chameleon\"])]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e7e07ca7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# With the graph object G_chameleon, we can inspect some statistics\n",
    "print(\"#nodes: \" + str(G_chameleon.node_count()))\n",
    "print(\"#relationships: \" + str(G_chameleon.relationship_count()))\n",
    "print(\"Degree distribution\")\n",
    "print(\"=\" * 25)\n",
    "print(G_chameleon.degree_distribution().sort_index())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7139dcde",
   "metadata": {},
   "source": [
    "## Now, let's construct a training pipeline!\n",
    "\n",
    "We will create a Node Regression pipeline, and then\n",
    "\n",
    "1. configure the splitting\n",
    "2. add model candidates\n",
    "3. configure auto-tuning\n",
    "4. add node property steps\n",
    "6. select model features\n",
    "\n",
    "The pipeline lives in the Pipeline Catalog, and we are operating it through the Pipeline object, for maximum convenience.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2df00db8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now, let's construct a training pipeline!\n",
    "chameleons_nr_training = gds.nr_pipe(\"node_regression_pipeline__Chameleons\")\n",
    "\n",
    "# We configure the splitting\n",
    "chameleons_nr_training.configureSplit(validationFolds=5, testFraction=0.2)\n",
    "\n",
    "# We add a set of model candidates\n",
    "# A linear regression model with the learningRate parameter in a search space\n",
    "chameleons_nr_training.addLinearRegression(\n",
    "    penalty=1e-5,\n",
    "    patience=3,\n",
    "    tolerance=1e-5,\n",
    "    minEpochs=20,\n",
    "    maxEpochs=500,\n",
    "    learningRate={\"range\": [100, 1000]},  # We let the auto-tuner find a good value\n",
    ")\n",
    "# Let's try a few different models\n",
    "chameleons_nr_training.configureAutoTuning(maxTrials=10)\n",
    "\n",
    "# Our input feature dimension is 3132\n",
    "# We can reduce the dimension to speed up training using a FastRP node embedding\n",
    "chameleons_nr_training.addNodeProperty(\n",
    "    \"fastRP\",\n",
    "    embeddingDimension=256,\n",
    "    propertyRatio=0.8,\n",
    "    featureProperties=[\"features_one_hot\"],\n",
    "    mutateProperty=\"frp_embedding\",\n",
    "    randomSeed=420,\n",
    ")\n",
    "\n",
    "# And finally we select what features the model should be using\n",
    "# We rely on the FastRP embedding solely, because it encapsulates the one-hot encoded source features\n",
    "chameleons_nr_training.selectFeatures(\"frp_embedding\")\n",
    "\n",
    "# The training pipeline is now fully configured and ready to be run!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cab6a293",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We use the training pipeline to train a model\n",
    "nc_model, train_result = chameleons_nr_training.train(\n",
    "    G_chameleon,  # First, we use the entire Chameleon graph\n",
    "    modelName=\"chameleon_nr_model\",\n",
    "    targetNodeLabels=[\"Chameleon\"],\n",
    "    targetProperty=\"target\",\n",
    "    metrics=[\"MEAN_SQUARED_ERROR\", \"MEAN_ABSOLUTE_ERROR\"],\n",
    "    randomSeed=420,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4342baec",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Winning model parameters: \\n\\t\\t\" + str(train_result[\"modelInfo\"][\"bestParameters\"]))\n",
    "print()\n",
    "print(\"MEAN_SQUARED_ERROR      test score: \" + str(train_result[\"modelInfo\"][\"metrics\"][\"MEAN_SQUARED_ERROR\"][\"test\"]))\n",
    "print(\"MEAN_ABSOLUTE_ERROR     test score: \" + str(train_result[\"modelInfo\"][\"metrics\"][\"MEAN_ABSOLUTE_ERROR\"][\"test\"]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a0dc05e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's sample the graph to see if we can get a similarly good model\n",
    "G_chameleon_sample, _ = gds.alpha.graph.sample.rwr(\n",
    "    \"cham_sample\",\n",
    "    G_chameleon,\n",
    "    samplingRatio=0.30,  # We'll use 30% of the graph\n",
    ")\n",
    "\n",
    "# Now we can use the same training pipeline to train another model, but faster!\n",
    "nc_model_sample, train_result_sample = chameleons_nr_training.train(\n",
    "    G_chameleon_sample,\n",
    "    modelName=\"chameleon_nr_model_sample\",\n",
    "    targetNodeLabels=[\"Chameleon\"],\n",
    "    targetProperty=\"target\",\n",
    "    metrics=[\"MEAN_SQUARED_ERROR\", \"MEAN_ABSOLUTE_ERROR\"],\n",
    "    randomSeed=420,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "db7eabfa",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Winning model parameters: \\n\\t\\t\" + str(train_result_sample[\"modelInfo\"][\"bestParameters\"]))\n",
    "print()\n",
    "print(\n",
    "    \"MEAN_SQUARED_ERROR      test score: \"\n",
    "    + str(train_result_sample[\"modelInfo\"][\"metrics\"][\"MEAN_SQUARED_ERROR\"][\"test\"])\n",
    ")\n",
    "print(\n",
    "    \"MEAN_ABSOLUTE_ERROR     test score: \"\n",
    "    + str(train_result_sample[\"modelInfo\"][\"metrics\"][\"MEAN_ABSOLUTE_ERROR\"][\"test\"])\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9677ac5c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's see what our models predict\n",
    "\n",
    "# The speed-trained model on 24% training data (30% sample - 20% test set)\n",
    "predicted_targets_sample = nc_model_sample.predict_stream(G_chameleon)\n",
    "# The fully trained model on 80% training data (20% test set)\n",
    "predicted_targets_full = nc_model.predict_stream(G_chameleon)\n",
    "\n",
    "# The original training data for comparison\n",
    "real_targets = gds.graph.nodeProperty.stream(G_chameleon, \"target\")\n",
    "\n",
    "# Merging the data frames\n",
    "merged_full = real_targets.merge(predicted_targets_full, left_on=\"nodeId\", right_on=\"nodeId\")\n",
    "merged_all = merged_full.merge(predicted_targets_sample, left_on=\"nodeId\", right_on=\"nodeId\")\n",
    "\n",
    "# Look at the last 10 rows\n",
    "print(merged_all.tail(10))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "237c6875",
   "metadata": {},
   "source": [
    "## And we are done!\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}