routing: Add source-based routing algorithm (#30)

* hw(pkg): Rename `Consumption` routing to `SourceRouting`

* hw(router): Implement `SourceRouting` by consumption

* hw(chimney): Implement `SourceRouting`

* tb: Add new `map_i` port to all chimneys

* pkg: Define axi_ch enum with specific number of bits

Previously an integer of size 32-bit was inferred which is unnecessary loarge

* hw(routing): Support both table & address map

* hw(chimney): Compute route for response packet in case of `SourceRouting`

* pkg: Define the preliminary output type of the route computation

* floogen(endpoint): Add additional helper functions

* floogen(routing): Enable sorting by endpoint ID

* floogen(routing): Add source-based routing algorithm

* floogen(tpl): Adapt templates

* hw(routing): Use two-stage address decoding for source-based routing

* hw(chimney): Adapt to support source-based routing

* floogen(test): Refactoring

* floogen(utils): Modify helper functions

* hw(tb): Refactor port of chimneys

* floogen(examples): Add mesh example with source-based routing

* hw(pkg): Regenerate package sources

* floogen(test): Fix unit tests

* hw(chimney): Fix multiple-driven assignments

* floogen: Linting

* hw(chimney): Linting

* floogen: Small fix

* hw(pkg): Set number of address rules to 1 by default

* hw(routing): Fix address translation bug

* hw(chimney): Give `route_comp` modules a proper name

* floogen(tpl): Fix chimney ports

* floogen(routing): Fix bit width of XY coordinates

* floogen(tpl): Small fixes in templates

* floogen(routing): Render the route table in reverse

* doc: Update CHANGELOG

* tb: Fix compilation errors with routing algorithms

* lint: Make linter happy

CHANGELOG.md

@@ -4,6 +4,19 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## Unreleased
+
+### Added
+
+- Support for source-based routing algorithm in routers, chimnyes and `floogen`. The route is encoded in the header as a `route_t` field, and each router consumes a couple of bits to determine the output ports. In the chimney, a two-stage encoder was added to first determine the destination ID of the request, and then retrive the pre-computed route to that destination from a table. The `floogen` configuration was extended to support the new routing algorithm, and it will also generate the necessary tables for the chimneys.
+
+### Changed
+
+- `floo_route_comp` now supports source-based routing, and can output both destination ID and a route to the destination.
+- The chimneys have an additional port `route_table_i` to receive the pre-computed routing table that is generated by `floogen`.
+- System address map was renamed from `AddrMap` to `Sam`.
+- The destination field in the flit header have a new type `dst_t` which is either set to `route_t` for the new source-based routing algorithm, and `id_t` for all the other routing algorithms.
+
 ## [0.4.0] - 2024-02-07
 
 ### Added

floogen/examples/occamy_mesh_src.yml

@@ -0,0 +1,122 @@
+# Copyright 2023 ETH Zurich and University of Bologna.
+# Licensed under the Apache License, Version 2.0, see LICENSE for details.
+# SPDX-License-Identifier: Apache-2.0
+
+name: occamy_mesh
+description: "Occamy mesh configuration for FlooGen with source-based routing"
+
+routing:
+  route_algo: "SRC"
+  use_id_table: true
+
+protocols:
+  - name: "narrow"
+    type: "AXI4"
+    direction: "manager"
+    data_width: 64
+    addr_width: 48
+    id_width: 4
+    user_width: 1
+  - name: "narrow"
+    type: "AXI4"
+    direction: "subordinate"
+    data_width: 64
+    addr_width: 48
+    id_width: 2
+    user_width: 1
+  - name: "wide"
+    type: "AXI4"
+    direction: "manager"
+    data_width: 512
+    addr_width: 48
+    id_width: 3
+    user_width: 1
+  - name: "wide"
+    type: "AXI4"
+    direction: "subordinate"
+    data_width: 512
+    addr_width: 48
+    id_width: 1
+    user_width: 1
+
+endpoints:
+  - name: "cluster"
+    array: [3, 8]
+    addr_range:
+      base: 0x0000_1000_0000
+      size: 0x0000_0004_0000
+    mgr_port_protocol:
+      - "narrow"
+      - "wide"
+    sbr_port_protocol:
+      - "narrow"
+      - "wide"
+  - name: "hbm"
+    array: [8]
+    addr_range:
+      base: 0x0000_8000_0000
+      size: 0x0000_4000_0000
+    sbr_port_protocol:
+      - "narrow"
+      - "wide"
+  - name: "serial_link"
+    array: [3]
+    addr_range:
+      base: 0x0100_0000_0000
+      size: 0x0010_000_0000
+    mgr_port_protocol:
+      - "narrow"
+      - "wide"
+    sbr_port_protocol:
+      - "narrow"
+      - "wide"
+  - name: "cva6"
+    mgr_port_protocol:
+      - "narrow"
+  - name: "peripherals"
+    addr_range:
+      start: 0x0000_0000_0000
+      end: 0x0000_0fff_ffff
+    mgr_port_protocol:
+      - "narrow"
+    sbr_port_protocol:
+      - "narrow"
+
+routers:
+  - name: "router"
+    array: [3, 8]
+
+connections:
+  - src: "cluster"
+    dst: "router"
+    src_range:
+    - [0, 2]
+    - [0, 7]
+    dst_range:
+    - [0, 2]
+    - [0, 7]
+    bidirectional: true
+  - src: "hbm"
+    dst: "router"
+    src_range:
+    - [0, 7]
+    dst_range:
+    - [0, 0]
+    - [0, 7]
+    bidirectional: true
+  - src: "serial_link"
+    dst: "router"
+    src_range:
+    - [0, 2]
+    dst_range:
+    - [0, 2]
+    - [0, 0]
+    bidirectional: true
+  - src: "cva6"
+    dst: "router"
+    dst_idx: [0, 7]
+    bidirectional: true
+  - src: "peripherals"
+    dst: "router"
+    dst_idx: [1, 7]
+    bidirectional: true

floogen/model/endpoint.py

@@ -61,6 +61,14 @@ def is_mgr(self) -> bool:
         """Return true if the endpoint is a manager."""
         return self.mgr_port_protocol is not None
 
+    def is_only_sbr(self) -> bool:
+        """Return true if the endpoint is only a subordinate."""
+        return self.is_sbr() and not self.is_mgr()
+
+    def is_only_mgr(self) -> bool:
+        """Return true if the endpoint is only a manager."""
+        return self.is_mgr() and not self.is_sbr()
+
     def get_ni_name(self, name: str) -> str:
         """Return the name of the NI."""
         return name.replace(self.name, f"{self.name}_ni")

floogen/model/link.py

@@ -9,7 +9,7 @@
 from abc import ABC, abstractmethod
 from pydantic import BaseModel
 
-from floogen.utils import snake_to_camel, sv_struct_typedef
+from floogen.utils import snake_to_camel, sv_struct_typedef, clog2
 
 
 class Link(BaseModel, ABC):
@@ -40,14 +40,15 @@ def render_ports(self):
     @classmethod
     def render_enum_decl(cls):
         """Render the enum declaration of the link."""
-        string = "typedef enum {"
         i = 0
+        string = ""
         for _, mapping in cls.channel_mapping.items():
             for ch_type, axi_chs in mapping.items():
                 for axi_ch in axi_chs:
                     name = f"{ch_type}_{axi_ch}"
                     string += f"{snake_to_camel(name)} = {i},\n"
                     i += 1
+        string = f"typedef enum logic [{clog2(i+1)-1}:0]{{" + string
         string += f"NumAxiChannels = {i}\n}} axi_ch_e;\n"
         return string
 

floogen/model/network.py

@@ -14,7 +14,7 @@
 from mako.lookup import Template
 from pydantic import BaseModel, ConfigDict, field_validator, model_validator
 
-from floogen.model.routing import Routing, RouteAlgo, RoutingRule, RoutingTable
+from floogen.model.routing import Routing, RouteAlgo, RouteMapRule, RouteRule, RouteMap, RouteTable
 from floogen.model.routing import Coord, SimpleId, AddrRange
 from floogen.model.graph import Graph
 from floogen.model.endpoint import EndpointDesc, Endpoint
@@ -293,7 +293,7 @@ def compile_ids(self):
                     if node.id_offset is not None:
                         node_id += node.id_offset
                     self.graph.nodes[node_name]["id"] = node_id
-            case RouteAlgo.ID:
+            case RouteAlgo.ID | RouteAlgo.SRC:
                 for ep_name, ep in self.graph.get_ep_nodes(with_name=True):
                     node_id = SimpleId(id=self.graph.create_unique_ep_id(ep_name))
                     if ep.id_offset is not None:
@@ -348,13 +348,13 @@ def compile_routers(self):
                     }
                     self.graph.set_node_obj(rt_name, NarrowWideXYRouter(**router_dict))
 
-                case RouteAlgo.ID:
+                case RouteAlgo.ID | RouteAlgo.SRC:
                     router_dict = {
                         "name": rt_name,
                         "incoming": self.graph.get_edges_to(rt_name),
                         "outgoing": self.graph.get_edges_from(rt_name),
                         "degree": len(set(self.graph.neighbors(rt_name))),
-                        "routing": self.routing.model_copy(),
+                        "route_algo": self.routing.route_algo,
                     }
                     self.graph.set_node_obj(rt_name, NarrowWideRouter(**router_dict))
 
@@ -464,39 +464,44 @@ def compile_nis(self):
             self.graph.set_node_obj(ni_name, NarrowWideAxiNI(**ni_dict))
 
     def gen_routing_info(self):
-        """Generate the routing table for the network."""
+        """Wrapper function to generate all the routing info for the network,
+        for a specific routing algorithm."""
         self.routing.num_endpoints = len(self.graph.get_ni_nodes())
         self.routing.num_id_bits = clog2(len(self.graph.get_ni_nodes()))
         match self.routing.route_algo:
             case RouteAlgo.XY:
-                self.gen_xy_routing_info()
+                for info, value in self.gen_xy_routing_info().items():
+                    setattr(self.routing, info, value)
             case RouteAlgo.ID:
-                self.gen_routing_tables()
+                self.gen_router_tables()
+            case RouteAlgo.SRC:
+                self.gen_routes()
             case _:
-                raise NotImplementedError("Only XY and IdTable routing is supported yet")
-
-        if self.routing.use_id_table:
-            self.gen_address_table()
+                raise NotImplementedError(
+                    f"Routing algorithm {self.routing.route_algo} is not supported yet"
+                )
+        self.routing.sam = self.gen_sam()
+        # Provide the routing info to the network interfaces
+        for ni in self.graph.get_ni_nodes():
+            ni.routing = self.routing
 
-    def gen_routing_tables(self):
+    def gen_router_tables(self):
         """Generate the routing table for the network."""
-        for router in self.graph.get_rt_nodes():
+        for rt in self.graph.get_rt_nodes():
             routing_table = []
             ni_sbr_nodes = [ni for ni in self.graph.get_ni_nodes() if ni.is_sbr()]
             for ni in ni_sbr_nodes:
-                shortest_path = nx.shortest_path(self.graph, router.name, ni.name)
-                out_edge = (router.name, shortest_path[1])
+                shortest_path = nx.shortest_path(self.graph, rt.name, ni.name)
+                out_edge = (rt.name, shortest_path[1])
                 out_link = self.graph.get_edge_obj(out_edge)
-                out_idx = router.outgoing.index(out_link)
+                out_idx = rt.outgoing.index(out_link)
                 dest = SimpleId(id=out_idx)
                 addr_range = AddrRange(start=ni.id.id, size=1)
-                routing_table.append(RoutingRule(dest=dest, addr_range=addr_range))
+                routing_table.append(RouteMapRule(dest=dest, addr_range=addr_range, desc=ni.name))
 
             # Add routing table to the router
-            routing_table = RoutingTable(rules=routing_table)
-            routing_table.trim()
-            router.routing = self.routing.model_copy()
-            router.routing.table = routing_table
+            rt.table = RouteMap(name=rt.name + "_map", rules=routing_table)
+            rt.table.trim()
 
     def gen_xy_routing_info(self):
         """Generate the XY routing info for the network."""
@@ -507,27 +512,57 @@ def gen_xy_routing_info(self):
         max_x = max(ni.id.x for ni in ni_nodes)
         max_y = max(ni.id.y for ni in ni_nodes)
         max_address = max(ni.addr_range.end for ni in ni_sbr_nodes)
-        self.routing.num_x_bits = clog2(max_x - min_x)
-        self.routing.num_y_bits = clog2(max_y - min_y)
-        self.routing.addr_offset_bits = clog2(max_address)
-        self.routing.id_offset = Coord(x=min_x, y=min_y)
-        for ni in self.graph.get_ni_nodes():
-            ni.routing = self.routing.model_copy()
-        for rt in self.graph.get_rt_nodes():
-            rt.id = rt.id - self.routing.id_offset
-
-    def gen_address_table(self):
-        """Generate the address table for the network."""
+        xy_routing_info = {}
+        xy_routing_info["num_x_bits"] = clog2(max_x - min_x + 1)
+        xy_routing_info["num_y_bits"] = clog2(max_y - min_y + 1)
+        xy_routing_info["addr_offset_bits"] = clog2(max_address)
+        xy_routing_info["id_offset"] = Coord(x=min_x, y=min_y)
+        return xy_routing_info
+
+    def gen_routes(self):
+        """Generates the routes for source-based routing."""
+        self.routing.num_route_bits = 0
+        for ni_src in self.graph.get_ni_nodes():
+            routes = []
+            for ni_dst in self.graph.get_ni_nodes():
+                # Skip if source and destination are the same
+                # and for manager-manager and subordinate-subordinate
+                # connections
+                if (
+                    ni_src.name == ni_dst.name
+                    or (ni_src.is_only_mgr() and ni_dst.is_only_mgr())
+                    or (ni_src.is_only_sbr() and ni_dst.is_only_sbr())
+                ):
+                    continue
+                route = nx.shortest_path(self.graph, ni_src.name, ni_dst.name)
+                max_route_bits = 0
+                port_lst = []
+                for i in range(1, len(route) - 1):
+                    out_edge = (route[i], route[i + 1])
+                    out_link = self.graph.get_edge_obj(out_edge)
+                    rt = self.graph.get_node_obj(route[i])
+                    out_port = rt.outgoing.index(out_link)
+                    num_port_bits = clog2(len(rt.outgoing))
+                    port_lst.append((out_port, num_port_bits))
+                    max_route_bits += num_port_bits
+                rule = RouteRule(route=port_lst, id=ni_dst.id, desc=f"-> {ni_dst.name}")
+                routes.append(rule)
+                self.routing.num_route_bits = max(self.routing.num_route_bits, max_route_bits)
+            ni_src.table = RouteTable(name=ni_src.name + "_table", routes=routes)
+
+    def gen_sam(self):
+        """Generate the system address map, which is used by the network interfaces
+        to determine the destination of a packet based on the address."""
         addr_table = []
         ni_sbr_nodes = [ni for ni in self.graph.get_ni_nodes() if ni.is_sbr()]
         for ni in ni_sbr_nodes:
             dest = ni.id
+            if self.routing.id_offset is not None:
+                dest += self.routing.id_offset
             addr_range = ni.addr_range
-            addr_rule = RoutingRule(dest=dest, addr_range=addr_range)
+            addr_rule = RouteMapRule(dest=dest, addr_range=addr_range, desc=ni.name)
             addr_table.append(addr_rule)
-        self.routing.table = RoutingTable(rules=addr_table)
-        for ni in self.graph.get_ni_nodes():
-            ni.routing.table = self.routing.table.model_copy()
+        return RouteMap(name="sam", rules=addr_table)
 
     def render_ports(self):
         """Render the ports in the generated code."""
@@ -581,9 +616,7 @@ def render_link_cfg(self):
         else:
             axi_type, link_type = "axi", NarrowLink
 
-        return axi_type, self.tpl_pkg.render(
-            name=axi_type, noc=self, link=link_type
-        )
+        return axi_type, self.tpl_pkg.render(name=axi_type, noc=self, link=link_type)
 
     def visualize(self, savefig=True, filename: pathlib.Path = "network.png"):
         """Visualize the network graph."""

floogen/model/network_interface.py

@@ -11,7 +11,7 @@
 from pydantic import BaseModel
 from mako.lookup import Template
 
-from floogen.model.routing import Id, AddrRange, Routing
+from floogen.model.routing import Id, AddrRange, Routing, RouteMap
 from floogen.model.protocol import AXI4
 from floogen.model.link import NarrowWideLink
 from floogen.model.endpoint import EndpointDesc
@@ -24,6 +24,7 @@ class NetworkInterface(BaseModel):
     endpoint: EndpointDesc
     description: str = ""
     routing: Routing
+    table: Optional[RouteMap] = None
     id: Optional[Id] = None
     arr_idx: Optional[Id] = None
     addr_range: Optional[AddrRange] = None
@@ -36,6 +37,14 @@ def is_mgr(self) -> bool:
         """Return true if the network interface is a manager."""
         return self.endpoint.is_mgr()
 
+    def is_only_sbr(self) -> bool:
+        """Return true if the network interface is only a subordinate."""
+        return self.endpoint.is_sbr() and not self.endpoint.is_mgr()
+
+    def is_only_mgr(self) -> bool:
+        """Return true if the network interface is only a manager."""
+        return self.endpoint.is_mgr() and not self.endpoint.is_sbr()
+
 
 class NarrowWideAxiNI(NetworkInterface):
     """ " NarrowWideNI class to describe a narrow-wide network interface."""