Pubky Nexus

Pubky Nexus is the central bridge connecting Pubky homeservers with Pubky-App’s social clients. By aggregating events from homeservers into a rich social graph, Nexus transforms decentralized interactions into a high-performance, fully featured social-media-like API. It's designed to support Social-Semantic-Graph (SSG) inference, and more.

🌟 Key Features

• Real-time Social Graph Aggregation: Nexus ingests events from multiple Pubky homeservers, generating a structured social graph in real time.
• Full-Content Indexing: Nexus serves content directly, improving latency and user experience. Clients do not need to locate homeservers to retrieve content unless they wish to perform content attestation. We also envision a light-weight Nexus mode that merely point clients to homeserver locations using pubky uris.
• High Performance & Scalability: Built in Rust, Nexus is optimized for speed and efficiency, handling complex social queries across distributed systems with minimal latency.
• Powerful Social Semantic Graph: Nexus supports SSG-based interactions, fostering secure and trusted connections between users.
• Graph-Enhanced Search & Recommendations: Nexus leverages Neo4j to offer deep insights, relationships, and recommendations based on the social graph.
• Flexible Caching Layer: A Redis cache accelerates access to common queries and minimizes database load, ensuring a smooth user experience. Most requests can be served in less than 1 ms at constant time complexity with respect number of users.
• Rich Observability: Easily explore the indexed data using Redis Insight and visualize the graph database with Neo4j Browser.

🌐 Accessing the API

⚠️ Warning: The API is currently unstable. We are using the /v0 route prefix while the API undergoes active development and changes. Expect potential breaking changes as we work toward stability.

Nexus provides a REST API, accessible via Swagger UI:

• Staging API (latest): https://nexus.staging.pubky.app/swagger-ui/
• Production API (current): https://nexus.pubky.app/swagger-ui/

You can explore available endpoints, test queries, and view schema definitions directly within Swagger.

🏗️ Architecture Overview

Nexus is composed of several core components:

• service.rs: The REST API server for handling client requests, querying databases, and returning responses to the Pubky-App frontend.
• watcher.rs: The event aggregator that listens to homeserver events, translating them into social graph updates within the Nexus databases.
• lib.rs: A library crate containing common functionalities shared by service and watcher, including database connectors, models, and queries.

Data Flow

1. Event Ingestion: The watcher ingests events from Pubky homeservers and indexes them into our social graph.
2. Indexing and Caching: Relationships, common queries, and graph data are cached in Redis for high-speed access. Complex queries use Neo4j.
3. API Responses: The service server reads from these indexes to respond to client queries efficiently.

Nexus graph schema.

⚙️ Setting Up the Development Environment

To get started with Nexus, first set up the required databases: Neo4j and Redis.

1. Configure Databases

1. Clone the repository and navigate to the project directory.

2. Copy the environment template and set up the Docker environment:

   cd docker
   cp .env-sample .env
   docker-compose up -d

3. Populate the Neo4j database with initial data:

   docker exec neo4j bash /db-graph/run-queries.sh

Once the Neo4j graph database is seeded with data, the next step is to populate the Redis database by running the nexus-service

If the Redis cache is empty, the nexus-service will handle it automatically. If not follow the steps of warning section

4. Run the Nexus service:

   cargo run

5. Access Redis and Neo4j UIs:

   • Redis UI: http://localhost:8001/redis-stack/browser
   • Neo4J UI: http://localhost:7474/browser/

🚀 Contributing

To contribute to Nexus, follow these steps:

1. Fork the Repository and create a feature branch.
2. Write Tests: Ensure new features and changes are tested and benchmarked.
3. Submit a Pull Request and provide a description of the changes.

Running Tests

To run all tests:

cargo test

To test specific modules or features:

cargo test watcher:users
cargo test test_homeserver_user_event

Benchmarking:

cargo bench --bench user get_user_view_by_id

⚠️ Troubleshooting

If tests or the development environment seem out of sync, follow these steps to reset:

1. Reset Neo4j:

   docker exec neo4j bash -c "cypher-shell -u neo4j -p 12345678 'MATCH (n) DETACH DELETE n;'"
   docker exec neo4j bash /db-graph/run-queries.sh

2. Re-index Redis Cache:

   REINDEX=true cargo run

🌐 Useful Links

• Swagger API:
  • Staging: https://nexus.staging.pubky.app/swagger-ui/
  • Production: https://nexus.pubky.app/swagger-ui/
• Local Redis Insight: http://localhost:8001/redis-stack/browser
• Local Neo4J Browser: http://localhost:7474/browser/